Data Engineer / Data Architect (Consultant)
About the Role
IANS is looking for a Data Engineer / Data Architect Consultant to build the foundational data architecture that powers all of our products as we become a data-first company. You'll design and implement scalable data systems across Azure SQL Server, PostgreSQL, Elasticsearch, Salesforce integrations, and Azure cloud services — creating the infrastructure that enables initiatives like content metadata enrichment, new data-driven product offerings (e.g., Vendor Intelligence), analytics, and AI/ML workloads. This is about building the foundation once and building it right, so it scales across everything we do. This is a full-time, ongoing consulting engagement — we hire contractors to keep them as long as we can.
What You'll Do
- Design and build a scalable, org-wide data architecture that serves as the backbone for all current and future products.
- Design, build, and maintain data pipelines and ETL/ELT processes across Azure SQL Server, PostgreSQL, Elasticsearch, and third-party sources (Salesforce, APIs).
- Evaluate and support a potential migration from MSSQL to PostgreSQL, including feasibility analysis, schema translation, deadlock remediation, performance benchmarking, and migration planning.
- Build and maintain a centralized data warehouse or data platform to support analytics, reporting, AI/ML workflows, and our broader data-as-a-service
- Architect metadata models, taxonomies, and tagging systems that enable content enrichment across products (e.g., tagging content with vendors, team size, revenue, industry).
- Collaborate with engineering, product, and AI teams to define data models and ensure clean, consistent data flows across systems.
- Implement data quality checks, monitoring, and alerting to maintain data integrity across all data products.
- Document data architecture, lineage, and dictionary standards for the engineering team.
- Support and improve existing Azure SQL databases, Redis caching layers, and Elasticsearch clusters.
- Identify and resolve data bottlenecks, query performance issues, and infrastructure gaps.
Success in This Role
- 30 days: Understand the current data landscape — databases, pipelines, integrations, and pain points. Deliver a data architecture assessment with prioritized recommendations.
- 60 days: Key pipelines improved or rebuilt. Foundational data platform architecture designed and in progress. PostgreSQL evaluation underway with initial benchmarking against MSSQL. Data warehouse foundations in place.
- 90 days: Reliable, documented data infrastructure supporting product, analytics, and AI workloads. Metadata and enrichment patterns established for product teams to build on. Monitoring and data quality processes running. Clear roadmap for data-as-a-service expansion.
What We're Looking For
- 4–7 years of experience in data engineering, data architecture, or a related role.
- Advanced or fluent spoken English.
- Strong expertise in Microsoft SQL Server — schema design, stored procedures, views, performance tuning, and query optimization.
- Experience with PostgreSQL — schema design, migrations, performance tuning, and tooling (or demonstrated ability to evaluate and adopt it).
- Hands-on experience building and managing ETL/ELT pipelines (FiveTran, SSIS, Python, or similar).
- Solid experience with Azure cloud services — Azure SQL, Azure Functions, Azure Redis Cache, and related data tooling.
- Experience designing metadata models, taxonomies, or tagging systems for content enrichment or data products.
- Working knowledge of Elasticsearch for search indexing and data retrieval.
- Experience designing and maintaining data warehouses or analytical data stores.
- Familiarity with data integration from Salesforce or other CRM/SaaS platforms via APIs or connectors.
- Strong documentation, communication, and problem-solving skills.
- Experience designing and implementing database sharding and partitioning strategies to support high-throughput, horizontally scalable applications.
- Self-motivation and ability to work effectively in a small, collaborative remote team.
Nice to Have
- Experience with Python for data processing and automation.
- Familiarity with js/Express backend services and how they interact with data layers.
- Experience supporting AI/ML data pipelines (training data preparation, vector databases, embeddings).
- Knowledge of MongoDB, n8n, or other NoSQL databases.
- Exposure to GitHub Actions CI/CD workflows.
- Experience with data governance, cataloging, or compliance frameworks.
- Background in BI/reporting tools (Power BI, Looker, Tableau or similar).
Details
- Location: Remote — Bogotá, Colombia
- Hours: Up to 40/week
- Duration: 12 months to start with likely extension based on impact
- Type: Consulting engagement
About IANS Research
IANS Research is an information security advisory and consulting firm, serving Fortune-class information security teams and professionals with in-depth insights and decision support regarding their most pressing technical and strategic challenges. IANS provides access to information security experts who address and solve our clients' challenges as they arise. We help security teams achieve technical excellence and improve engagement with the organization to drive security's impact deeper into the company.