Senior Data Engineer

About the role

This is a hands-on role for a Senior Data Engineer sitting within the Data Products team. In this role, you will have an enormous impact on the design, implementation, and maintenance of existing and greenfield customer-facing data products. Data is key to Auctane’s strategy. We work at a scale pace and with the latest architecture patterns and tech. We process thousands of events per second, and our massive dataset keeps growing at a staggering pace. We keep improving our data platform and data engineering stack to accommodate growth, enable novel solutions, and provide the best service to our customers. We have a flat and open engineering culture where data & evidence beat opinion and hierarchy, backed by honest discussions. We passionately believe in forming autonomous, cross-functional teams who are empowered to deliver our ambitious strategy. Energy and passion for our business and customers are a part of the Auctane culture – and we love working with like-minded people.

What will you be doing?

• Contribute to the design, build, and operational management of our data pipelines and analytics solution on top of proven AWS data technologies like S3, Athena, Lambda, and Kinesis.
• Collaborate with the Product Owner and other stakeholders to implement the Data Products strategy.
• Develop frameworks and solutions that enable us to acquire, process, monitor and extract value from our massive dataset.
• Drive the design and architecture of Auctane’s data product.
• Contribute directly to the implementation and operations of our systems.
• Be an advocate of data quality and observability principles and use state-of-the-art technologies like Airflow, dbt, and Spark to process data and get our datasets just right.
• Foster engineering excellence by delivering highly reliable software and data pipelines using Software Engineering best practices like automation, version control, continuous integration/continuous delivery, testing, security, etc.

What are we looking for?

• At least 5 years of proven experience in data engineering roles
• A strong Data Engineering background in a data warehouse or data lake architecture
• Experience working in AWS/GCP cloud infrastructure.
• Experience developing and supporting robust, automated, and reliable data pipelines in Python and SQL.
• Mastered Python and SQL skills.
• Experience with data processing frameworks like Spark, Athena, or Pandas.
• Knowledge of Data Orchestration solutions, preferably Airflow.
• Knowledge of relational database design as well as Snowflake/Redshift or similar

What will make you stand out?

• Production experience working with very large datasets.
• Experience with AWS and big data cloud technologies like EMR, Athena, Glue, and EC2.
• Direct experience with Infrastructure as Code tools (e.g., Terraform, CloudFormation)
• Knowledge and direct experience in using business intelligence and analytics tools (Tableau, Looker, Power BI, etc.)
• A desire to constantly challenge the norm Proven track record of promoting and maintaining high engineering standards.
• Fast learner with a keen interest in evaluating and adopting new technologies: Apache Iceberg, Dagster, DBT, GreatExpectations, Prefect, Snowflake, etc.
• Understanding of core ML concepts like supervised and unsupervised learning, overfitting, and cross-validation as well as LLM concepts like fine-tuning, etc.

Tech stack

Python SQL Amazon Web Services C# .NET

What do we offer?