Senior Machine Learning Engineer

About Buildkite

At Buildkite, our mission is to unblock every developer on the planet. We’ve rethought how software delivery should work and have built a platform that is fast, reliable, secure, and able to scale to the needs of the most demanding high-growth tech companies globally including Airbnb, Shopify, Canva, PagerDuty, Lyft, and Pinterest.

Job Overview

As the first Machine Learning Engineer on the Buildkite Test Engine team at Buildkite, you’ll own the end-to-end ML lifecycle — from partnering with customers to design meaningful products, to designing, deploying, and monitoring production-grade models at scale. Your work will directly reduce developer friction and improve confidence in software delivery for high-growth, engineering-led organisations. You’ll build robust inference and retraining pipelines, and ensure models are observable, cost-efficient, and impactful. This is a high-autonomy position where you’ll shape how machine learning differentiates Test Engine and meaningfully improves the developer experience.

🔧 About the Team

Test Engine helps to make test suites cheaper, faster and more reliable, reducing friction in developers' day to day work and making their working lives happier and more productive. We do this by identifying and quarantining flaky tests, providing tools to parallelise your suites to minimise build time, as well as showing trends over time of your suite and providing the ability to deep dive into what happens when an individual test executes.

🚀 What You’ll Do

• Talk directly to Buildkite customers to deeply understand their problems and work together on coming up with AI/ML product solutions
• Design, build and evaluate models, and different strategies to apply and use them
• Help drive architectural decisions around the data stack, with a focus on scalability, cost efficiency, and maintainability
• Instrument production models with observability metrics: latency, throughput, prediction drift, and data quality at inference time
• Shape product direction, help define where machine learning meaningfully differentiates our product — and where simpler approaches suffice.

🎨What You Bring

• 5+ years of experience in working with machine learning models, particularly feature engineering, hyperparameter tuning, model serving, monitoring, and automated retraining pipelines
• Familiarity with the developer tooling space, contemporary testing practices and software engineering best practice
• Strong communication skills, with a focus on telling data driven stories and bringing a data shaped lens to engineering product design
• Ability to work autonomously in a fast-paced, low-process environment where you’ll need to navigate ambiguity and drive impactful decisions

Key Technical Stacks:

• ML infrastructure and libraries (XGBoost for example)
• Deep proficiency in SQL and Python
• Strong experience with distributed data processing frameworks such as Spark, PySpark, or Flink
• Experience with streaming data infrastructure (Kafka, Kinesis, pub/sub, event hubs, or similar)
• Hands-on experience building and operating ML infrastructure — model registries, feature stores, serving layers, or similar
• Comfort working in cloud environments (AWS) and with containerised workloads (Docker, Kubernetes) and AWS Sagemaker

✨ Why Join Buildkite

At Buildkite, we value kindness, autonomy, and collaboration. You’ll be part of a remote-first company where your work can make a meaningful impact – empowering engineers worldwide to build and deliver better software faster.

• Competitive compensation and benefits package
• Flexible, remote-first culture
• Opportunities for professional growth, leadership, and technical ownership
• Work alongside talented, passionate engineers building world-class developer tools
• A collaborative, inclusive, and innovative culture where your ideas make a real impact