Technical Product Manager – AI Neocloud Software Stack
$180k - $220k USD
About Gruve
Gruve is an innovative software services startup dedicated to transforming enterprises to AI powerhouses. We specialize in cybersecurity, customer experience, cloud infrastructure, and advanced technologies such as Large Language Models (LLMs). Our mission is to assist our customers in their business strategies utilizing their data to make more intelligent decisions. As a well-funded early-stage startup, Gruve offers a dynamic environment with strong customer and partner networks.
About the Role
Gruve is seeking a Technical Product Manager (Inbound) with experience in AI neocloud environments and modern inference software stacks. In this role, you will shape inference-related product capabilities by collaborating closely with engineering teams. You will translate customer and partner feedback into actionable priorities, guide technical trade-offs, and ensure product decisions are well understood across teams. While you won’t be implementing systems yourself, you’ll need a deep technical understanding to ask the right questions and earn the trust of engineers.
Key Responsibilities
- Own inbound product responsibilities related to AI inference software.
- Define and refine product requirements for model serving, routing, and performance.
- Translate customer and partner feedback into actionable product priorities.
- Partner closely with engineering to guide roadmap decisions and trade-offs.
- Balance latency, throughput, cost, and usability in product decisions.
- Ensure product decisions are documented, communicated, and well understood.
Basic Qualifications
- 2+ years of product management experience in AI neoclouds, inference platforms, or GPU-centric infrastructure products.
- Experience working closely with engineering teams on highly technical systems.
- Proven experience owning inbound product responsibilities.
- Understanding of inference stack components such as:
-
-
vLLM, SGLang, TensorRT-LLM, or similar.
-
Continuous batching and KV cache behavior.
-
Prefill vs. decode phases.
-
Model routing and request scheduling.
-
Quantization (INT8, INT4, FP8).
-
GPU utilization, memory constraints, and autoscaling.
-
OpenAI-compatible APIs, SDKs, SLAs, and observability.
-
-
Bachelor’s degree in Computer Science, Engineering, or related technical field (or equivalent practical experience).
Preferred Qualifications
- Advanced degree in a technical field.
- Hands-on experience with AI infrastructure, inference platforms, or software systems.
- Strong customer-centric mindset, technical fluency, and collaborative approach.
- Excellent communication skills, able to ask clarifying questions and earn trust with engineering teams.
Salary Range
$180,000 - $220,000 + Benefits
This is a full-time, on-site position based at Gruve’s office in Redwood City, CA.
Why Gruve
At Gruve, we foster a culture of innovation, collaboration, and continuous learning. We are committed to building a diverse and inclusive workplace where everyone can thrive and contribute their best work. If you’re passionate about technology and eager to make an impact, we’d love to hear from you.
Gruve is an equal opportunity employer. We welcome applicants from all backgrounds and thank all who apply; however, only those selected for an interview will be contacted.