Senior Data Engineer

Important Information:

• Years of Experience: 8+ years overall in data engineering and data platforms

• Job Mode: Full-time

• Work Mode: Remote 

Job Summary:
We are looking for a Senior Data Engineer with deep expertise in building, optimizing, and managing data pipelines and modern cloud-based data platforms. This role involves designing scalable, secure, and high-performance data ecosystems for global clients while collaborating closely with data scientists, BI teams, and architects. The ideal candidate has experience in big data processing, ETL/ELT pipelines, and cloud-native services, particularly on Azure OpenAI, within a nearshore delivery model.

Responsibilities and Duties:

• Design, build, and optimize ETL/ELT processes and data pipelines for structured and unstructured data.

• Develop and maintain data lakes, warehouses, and analytics platforms in the cloud.

• Ensure data quality, governance, and security compliance across environments.

• Translate business and technical requirements into scalable data solutions.

• Implement best practices for data storage, partitioning, and performance tuning.

• Work with Azure cloud-native services (Data Factory, Synapse, Databricks) for ingestion, transformation, and orchestration.

• Build real-time and batch data processing solutions using big data frameworks.

• Partner with analysts and data scientists to support AI/ML and reporting initiatives.

• Monitor and troubleshoot data pipelines, workflows, and infrastructure issues.

• Contribute to automation, CI/CD pipelines, and Infrastructure-as-Code practices for data platforms.

Qualifications and Skills:

• 8+ years of professional experience in data engineering, architecture, or ETL development.

• Strong programming skills in Python, Java, or Scala for data processing.

• Advanced SQL expertise and proficiency in relational databases (PostgreSQL, MySQL, Oracle, SQL Server).

• Hands-on experience with big data frameworks (Apache Spark, Hadoop, Databricks).

• Strong knowledge of Azure data services (Data Factory, Synapse, Databricks).

• Experience with workflow orchestration tools (Airflow, Luigi, Prefect).

• Familiarity with NoSQL databases (MongoDB, Cassandra, DynamoDB).

• Solid understanding of data modeling, partitioning, and query optimization.

• Strong communication skills in English (written and verbal).

Role-specific Requirements:

• Experience with streaming platforms (Kafka, Flink, Spark Streaming).

• Familiarity with MLOps / AI pipelines to support data science workflows.

• Knowledge of data governance and regulatory compliance (GDPR, HIPAA).

• Prior experience in Agile/Scrum delivery environments.

• Experience supporting US/European clients in nearshore delivery models.

• Professional certifications in cloud or data engineering (e.g., Azure Data Engineer Associate, AWS Big Data Specialty, GCP Data Engineer).

Technologies:

• Programming: Python, Java, Scala

• Databases: PostgreSQL, MySQL, Oracle, SQL Server, MongoDB, Cassandra, DynamoDB

• Cloud Platforms: Azure (Data Factory, Synapse, Databricks)

• Big Data: Apache Spark, Hadoop, Databricks

• Workflow: Airflow, Luigi, Prefect

• Streaming: Kafka, Flink, Spark Streaming

Skillset Competencies:

• Data pipeline design & optimization

• Cloud-native data platform development (Azure focus)

• Big data processing (batch & real-time)

• ETL/ELT development & orchestration

• Data quality, governance, and compliance

• CI/CD automation & Infrastructure-as-Code

• Collaboration with global BI and AI/ML teams

About Encora
Encora is the preferred digital engineering and modernization partner of some of the world’s leading enterprises and digital native companies. With over 9,000 experts in 47+ offices and innovation labs worldwide, Encora’s technology practices include Product Engineering & Development, Cloud Services, Quality Engineering, DevSecOps, Data & Analytics, Digital Experience, Cybersecurity, and AI & LLM Engineering.

At Encora, we hire professionals based solely on their skills and qualifications, and do not discriminate based on age, disability, religion, gender, sexual orientation, socioeconomic status, or nationality.