We are seeking a hands-on Data Engineer with strong Databricks and PySpark experience to build scalable data pipelines and analytics applications within a modern data platform.
This role focuses on modernizing legacy ETL and reporting systems (Teradata, Informatica, Tableau) into Databricks-native pipelines, dashboards, and Python-based data applications. The ideal candidate operates across both data engineering and lightweight application development.
Responsibilities
Design, develop, and maintain scalable data pipelines using PySpark on Databricks
Build analytics data models and transformation workflows for enterprise reporting and analytics
Migrate legacy ETL workloads from platforms such as Informatica and Teradata to Databricks
Develop Databricks-native dashboards and analytics applications to replace traditional BI tools
Build lightweight Python-based data applications (e.g., FastAPI) to expose and interact with data
Integrate Databricks pipelines with APIs and application services
Implement Slowly Changing Dimensions (SCD) and dimensional data modeling techniques
Develop reusable data engineering frameworks and standardized pipelines
Optimize Spark workloads for performance, scalability, and cost efficiency
Collaborate with analytics and business teams to deliver user-facing data solutions
Leverage AI-assisted coding tools (e.g., Copilot, ChatGPT) to improve development productivity
Contribute to best practices for modern data engineering and analytics application development
Required Experience
5+ years of hands-on experience building data pipelines using PySpark in production environments
Strong experience with the Databricks platform (workspaces, clusters, Jobs & Workflows, Unity Catalog)
Experience building analytics dashboards within Databricks (Databricks SQL)
Proven experience designing and building scalable ETL/ELT data pipelines
Strong Python development skills, including building REST APIs or data services
Experience building or supporting data-driven applications (not just traditional ETL pipelines)
Solid understanding of data modeling, including dimensional modeling and transformation patterns
Experience using AI-assisted development tools (e.g., Copilot, ChatGPT) in engineering workflows
Exposure to LLM integration or an AI-powered data application
Familiarity with cloud platforms (AWS, Azure, or GCP)
Preferred Experience
Experience migrating from legacy platforms: Informatica → Databricks / Spark; Teradata → cloud-native data platforms; Tableau (or similar) → Databricks-native dashboards
Experience with FastAPI or similar Python frameworks for data applications
Exposure to CI/CD pipelines for data engineering workflows
Understanding of microservices architecture and scalable application design
Experience in Healthcare Payor domain (e.g., claims processing, member data, provider data, eligibility, or billing systems)
McLaren Strategic Solutions is a leading-edge global technology consulting firm, addressing critical challenges across industries such as retail, financial services, and healthcare. Integrating a powerful ecosystem of platforms with capital-efficient execution, McLaren specializes in digital transformation to help businesses optimize operations, accelerate revenue, and achieve scalable outcomes. McLaren’s expertise spans the development of customer-centric applications, modernizing systems for cost-effectiveness and security, and leveraging cloud scalability for future-ready architectures. With a deep commitment to operational excellence, McLaren provides comprehensive managed services, including application maintenance, cybersecurity, platform solutions, and AI-optimized operations, ensuring seamless, secure, and efficient performance.
From supply chain automation to compliance and analytics, McLaren drives measurable impact: improving workforce productivity, reducing inventory costs, and cutting technology ownership expenses. With its emphasis on automation and zero business downtime, McLaren facilitates seamless migrations from legacy systems to modern platforms, enabling organizations to harness the full potential of digital transformation. Backed by strategic partnerships and a proven delivery model, McLaren empowers clients to innovate, modernize, and achieve lasting success in today’s digital economy.
McLaren is a certified minority owned business through the NMSDC and has a mission to place more people from non-traditional backgrounds into sustainable technology careers. Through partnerships with non-profit technology programs in underserved communities and Veteran organizations, candidates transition from tech training programs into real IT careers at McLaren. Our unique recruitment policy allows us to create exceptional teams, bringing a broad spectrum of experience to our company and creating anything but a traditional consulting firm.
Doran Jones Inc. is proud to be part of the McLaren Strategic Ventures Group.