Data Engineer

Position: Data Engineer

Role Overview:

• Collaborate with Architects to develop high-quality data pipelines for research, operations, and public web applications.

• Support current and future reporting and analytics solutions.

• Apply expertise in data architecture, data engineering, and reporting.

Responsibilities:

• Architect, design, and build scalable, robust, and efficient data pipelines using Azure Data Factory and other cloud-native tools.

• Establish best practices for data ingestion, transformation, storage, and access, optimizing for performance and cost-efficiency.

• Design and implement complex data models, schemas, and partitioning strategies for analytical and operational workloads.

• Champion modern data modeling strategies such as medallion architecture.

• Define and enforce frameworks for data quality, lineage, and observability.

• Enhance data platform infrastructure including CI/CD, monitoring, cost optimization, and failover strategies.

• Optimize performance of SQL Server and PostgreSQL databases and cloud services by identifying and resolving bottlenecks.

• Collaborate with product, analytics, and business teams to translate business goals into data architecture and pipeline solutions.

• Serve as the technical subject matter expert for data platforms and initiatives.

• Identify automation opportunities, streamline operations, and promote data democratization.

• Maintain comprehensive technical documentation, architecture diagrams, and operational playbooks.

• Ensure architectural consistency and enforce data security across all services and platforms.

• Work with Architects to model analytical data models, data warehouses, pipelines, and ETL processes.

Skills & Qualifications:

• Bachelor’s degree in Computer Science, Information Systems, Information Technology, or equivalent experience.

• 8+ years of hands-on experience in data engineering or data platform development with a record of delivering production-grade solutions.

• Expertise in Azure Data Factory, SQL Server, and PostgreSQL for data ingestion, transformation, and storage.

• Strong knowledge of data warehousing principles, dimensional modeling, and scalable ETL/ELT pipeline development.

• Experience with cloud services, preferably Azure, and data security protocols.

• Proficiency in Python and/or Scala focused on building maintainable and testable data solutions.

• Excellent problem-solving skills and attention to detail.

• Proven leadership ability and experience delivering quality data solutions.

• Strong communication and collaboration skills.

• Familiarity with machine learning methodologies and data science models.

• Azure or other cloud services certifications are a plus.