Overview In this role you will architect and lead the development of a state-of-the-art inference platform that delivers low-latency, scalable online ML results for advertising. You will mentor engineers, drive innovations at scale, and shape Roku's ML serving capabilities across hardware, software, and models. You'll stay ahead of inference frameworks and hardware acceleration while collaborating across teams to optimize system reliability and performance. This is a mission-driven opportunity to impact the advertising ecosystem at a global scale.
Compensation / Benefits- health insurance
- equity awards
- life insurance
- disability benefits
- parential leave
- wellness benefits
Responsibilities- Design and lead development of a state-of-the-art inference platform
- Oversee monitoring, observability, and tooling to ensure performance, reliability, and scalability of online inference services
- Identify and resolve performance bottlenecks and inefficiencies across end-to-end systems
- Stay abreast of advances in inference frameworks, ML hardware acceleration, and distributed systems and apply relevant innovations
Key requirements- 10+ years building and deploying large-scale distributed systems
- 5+ years in a leadership or technical lead role
- Strong programming skills in high-performance languages
- Deep understanding of inference frameworks and ML system deployment
- Proven experience optimizing performance for large-scale ML systems (SOTA model optimizations, hardware-software co-design, GPU acceleration, HPC)
- Excellent communication and collaboration skills
- Experience leading teams on high-throughput, low-latency ML serving systems
- Experience collaborating with and leading globally distributed, cross-functional teams
- Contributions to open-source ML or systems projects
- Excellent communication
- Collaboration and teamwork
- Leadership and mentorship
- ML serving / inference frameworks
- Distributed systems
- ML hardware acceleration
