Software Engineer - Large Scale Inference
We're building the company which will de-risk the largest infrastructure build-out in history.
When people finance GPU clusters, the datacenters housing them, and the infrastructure powering them, they need "offtake" - meaning someone has signed a contract to lease the cluster for a period of time before its even built.
Financing a GPU cluster is inherently risky, since margins are thin and volumes are huge. Lenders don't want to take on the risk that cluster developers can't repay their loan, and cluster developers really don't want to risk not selling their cluster. As a result, risk is offloaded to the customer using fixed-price long-term contracts.
If you don't mitigate this customer risk, there's a bubble. This isn't SaaS anymore - application layer companies sign multi-year contracts for computer and inference, but sell to customers on monthly subscriptions. If you mess up a purchase, it's game over: a minor shift in your revenue growth rate might mean the difference between profit or bankruptcy. But what if companies could exit their contract by selling it back to the market?
Otherwise, as AI scales, compute only becomes available to folks who can effectively take on that risk. A 2-person startup in a San Francisco Victorian can't realistically sign a 5-year take or pay contract on $100m supercomputers. But they may be able to buy the month of liquidity that someone else sold back.
So that's what we make: a liquid market for GPU offtake.
About the Role
As an engineer working on Large Scale Inference you will build and scale software systems that accept, distribute, and purchase compute for inference workloads. You will build systems to dynamically deploy and serve inference on our cloud infrastructure depending on customer demand. You will play a key role in building out and defining the inference products at SFC. You will work closely with a small, competent team driving high impact revenue generating changes. You will have the opportunity to deliver the highest quality, most affordable inference products at scale.
On a day-to-day basis, you might…
Quantify, optimize, and visualize the unit economics of inference
Design solutions to maximize compute utilization
Create automated compute purchasing software to optimally fulfill inference job demand
Design, build and deploy systems to dynamically scale inference for customer needs
This role may be a good fit for you if…
You enjoy the craftsmanship of software
You’re a thoughtful high-agency engineer
Have a deep appreciation for reliable fault tolerant systems
Are a strong communicator and work well with others
Bonus points if you…
Have previously worked at a startup
Have interest in market dynamics
Have experience in cloud environments
Have experience with vLLM
Are comfortable working in Rust
Have some Kubernetes or terraform experience
Benefits
Generous equity grant
Team members are offered a competitive salary along with equity in the company
Visa Sponsorships
Yes, we sponsor visas and work permits
Retirement matching
We match 401(k) plans up to 4%
Medical, dental & vision
We offer competitive medical, dental, vision insurance for employees and dependents and cover 100% of premiums
Time off
We offer unlimited paid time off as well as 10+ observed holidays
Parental leave
We offer biological, adoptive, and foster parents paid time off to spend quality time with family
Daily lunch
We cover lunch daily for employees
Unlimited office book budget
You can buy as many books for the office as you want
The San Francisco Compute Company is committed to maintaining a workplace free from discrimination and harassment.
We make employment decisions based on business needs, job requirements, and individual qualifications, without regard to race, color, religion, belief, national origin, social or ethical origin, age, physical, mental, or sensory disability, sexual orientation, gender identity or expression, marital status, civil union or domestic partnership status, past or present military service, HIV status, family medical history or genetic information, family or parental status including pregnancy, or any other status protected by law.
We welcome the opportunity to consider qualified applicants with prior arrest or conviction records. Our commitment to diversity includes hiring talented individuals regardless of their criminal history, in accordance with local, state, and federal laws, including San Francisco’s Fair Chance Ordinance and California’s ban-the-box laws.
If you require reasonable accommodation for any reason, please reach out to us at hiring@sfcompute.com
