Consultant / Sr Consultant - Data Engineer (Databricks)
About Fresh Gravity:
Founded in 2015 and rapidly growing, Fresh Gravity (www.freshgravity.com) is a business and technology consulting company at the cutting-edge of digital transformation. We drive digital success for our clients by helping them adopt transformative technologies that make them nimble, adaptive and responsive to their rapidly-changing business needs. Our unparalleled digital transformation expertise combines business strategy prowess with digital technologies know-how. Our expertise includes Data Management, Artificial Intelligence, Data Science & Analytics, and API Management & Integration.
In a short time, we have crafted an exceptional team who have delivered impactful projects for some of the largest corporations in the world. We are on a mission to solve the most complex business problems for our clients using the most exciting new technologies. And we are looking for top talent to join us in our quest.
Fresh Gravity’s team members are authorities in their field, but know how to have fun, too. We’re building an inspiring, open organization you’ll take pride in. We challenge ourselves to grow – every day. We create value for our clients and partners – every day. We promise rich opportunities for you to succeed, to shine, to exceed even your own expectations.
We are thoughtful. We are engaged. We are relentless. We are Fresh Gravity.
Fresh Gravity is an equal opportunity employer.
Requirements
• Databricks Platform Proficiency: Deep understanding of the Databricks Lakehouse Platform, including its Medallion architecture, workspace, and capabilities.
• Apache Spark: Expertise in using Apache Spark for data processing, including Spark SQL, DataFrames, and RDDs.
• Data Engineering with Delta Lake: Knowledge of Delta Lake for managing data lakes, including features like ACID transactions, schema enforcement, and time travel.
• ETL Processes: Experience with Extract, Transform, Load (ETL) processes using Databricks and other tools to integrate and transform data from various sources.
• Performance tuning and optimization in SPARK
• Knowledge to integrate with JDBC, SFTP, REST API and Cloud storage account based source systems
• Knowledge of workload patterns , designing metadata driven framework for workloads
• Programming and Scripting: Proficiency in programming languages such as Python and SQL for data manipulation and pipeline development.
• Data Governance and Security: Understanding of data governance best practices and security measures to ensure data integrity and compliance. • Usage of UNITY Catalog , external and internal tables
• Data Analysis: Ability to analyze large datasets to extract meaningful insights and support datadriven decision-making.
• Problem-Solving: Strong analytical skills to troubleshoot and resolve data-related issues efficiently.
• Data Quality and Unit Testing: Ensuring data accuracy and integrity through rigorous testing and validation processes.
• Continuous Learning: Staying updated with the latest trends and advancements in data engineering and Databricks technologies.
Skills & Experience we value
• 2 – 8 years of experienced professionals.
• Python Scripting
Benefits
In addition to a competitive package, we promise rich opportunities for you to succeed, to shine, to exceed even your own expectations. In keeping with Fresh Gravity’s challenger ethos, we have developed the 5Dimensions (5D) benefits program. This program recognizes the multiple dimensions within each of us and seek to provide opportunities for deep development across these dimensions. Enrich Myself; Enhance My Client; Build my Company, Nurture My Family; and Better Humanity.