Sr. Data Engineer
About Us:
We are an innovative company revolutionising retail checkout experiences by replacing traditional barcodes with cutting-edge Computer Vision technology. Our platform enables seamless, faster, and smarter checkout processes, enhancing the shopping experience for both retailers and consumers. We're growing rapidly and are looking for an experienced Android/Cross-Platform App Developer to join our team and help us build the future of retail technology.
Role Overview:
We are looking for a highly skilled Senior Data Engineer to join our growing data team. You will play a key role in building and optimizing data infrastructure, pipelines, and tooling that empower analytics, machine learning, and product development. This role requires strong experience in cloud-native data engineering, a passion for scalable systems, and the ability to work independently with minimal guidance.
Key Responsibilities:
Design, build, and maintain scalable data pipelines and ETL/ELT workflows using tools like Kestra, or Prefect.
Architect and manage cloud-based data infrastructure (e.g., Snowflake, MySQL, LanceDB ).
Implement and enforce data quality, lineage, and governance best practices.
Collaborate with analytics, BI, and product teams to define data models that support reporting, experimentation, and operational use cases.
Optimize query performance, storage costs, and data reliability across platforms.
Own data ingestion from internal and external systems via APIs, CDC, or streaming (e.g., Kafka, MQTT).
Automate data validation, testing, and monitoring frameworks to ensure trust in data.
Contribute to infrastructure-as-code and deployment processes using CI/CD pipelines and version control (e.g., Git).
Capable of working independently with minimal supervision from leadership.
Skills and Qualifications:
5+ years of experience as a data engineer or software engineer working with large-scale data systems.
Strong expertise in SQL, Python, and modern data transformation frameworks.
Experience building and maintaining production-level ETL/ELT pipelines.
Hands-on experience with cloud data warehouses (e.g., Snowflake, RedPanda Cloud).
Proficiency with workflow orchestration tools such as Airflow, Kestra, or Prefect.
Familiarity with infrastructure-as-code and CI/CD practices.
Strong understanding of data modeling techniques (dimensional modeling, normalization, denormalization).
Conduct data analysis to uncover trends and identify gaps aligned with product team insights and objectives.
Experience with cloud platforms such as AWS and Azure for data infrastructure and services.
Ability to work independently and drive projects forward with minimal guidance.
Nice to Have:
Experience working with streaming data technologies (RedPanda).
Knowledge of data security, privacy, and compliance practices (e.g., GDPR, HIPAA).
Background in DevOps for data, including containerization (Docker/Kubernetes) and observability tools.
Prior experience in a Retail or e-commerce data environment.
Software Qualifications:
Languages: Python, SQL, Rust
Data Warehousing: Snowflake, MySQL
ETL/ELT Orchestration Tools: Kestra, Prefect
Version Control & CI/CD: Git, GitHub Actions
Orchestration & Infrastructure: Docker, Kubernetes, Redpanda, Cloudflare
Monitoring: OpenobserveAI, Keep
Why Join Us?:
Be part of a forward-thinking company shaping the future of retail technology.
Work with a dynamic and collaborative team that values creativity and innovation.
Opportunity to contribute to cutting-edge projects and enhance your skills.
Competitive salary and benefits package.
Flexible work environment with opportunities for career growth.