Overview In this role you will build and own the host-side stack for MatX's custom silicon platform, enabling memory management, DMA, streams, and Python bindings. You will define and evolve the compiler-runtime contracts and the kernel ABI, while delivering a scalable LLM inference serving stack and robust cluster orchestration. You will shape the host-side tools and profiling interfaces used by ML engineers, targeting strong runtime overhead and serving throughput. This is a hands-on, systems-focused opportunity to bridge hardware and software for high-performance AI workloads.
Compensation / Benefits- 4 weeks PTO (accrued) + 12 company holidays
- health, dental, vision, life insurance for employees and dependents
- HSA and FSA options
- 401K with up to 5% company contribution
- Professional Development Budget ($1500/year)
- remote work flexibility up to 3 weeks
Responsibilities- Build the host-side interface library (memory management, DMA, streams, events, sync primitives) on top of which compiler-emitted programs run
- Own and extend the executable/contract format, including versioning and data layouts for weights/quantization
- Design the custom-kernel ABI and host-side marshaling layer to connect Python tensors to the device (DLPack, buffer protocol, numpy)
- Develop Python bindings via PyO3 with a C-ABI bridge for downstream consumers
- Build the LLM inference serving stack (paged KV cache, batching, scheduling, token streaming) and underpinning cluster orchestration
- Implement host interconnect topology, failure-detection, and clean teardown for cross-rack resilience
- Define chip-exposed profiling/debugging interfaces (perf counters, traces) to meet runtime and throughput targets
Key requirements- Strong experience in a systems programming language (Rust, C, C++, or Go) including memory management and ABI work
- Production experience with Python interop layers (PyO3, ctypes, pybind11, or equivalent)
- Experience designing and maintaining API/ABI contracts with versioning and breaking-change discipline
- Hands-on with accelerator programming models (CUDA, ROCm, oneAPI Level Zero, TPU) and understanding of device memory, async execution, and kernel launch
- ML-systems literacy and familiarity with training/inference loops and tensor layouts
- Rust, C, C++, or Go
- Python interop (PyO3, pybind11, ctypes)
- API/ABI versioning and contract design
- CUDA/ROCm/oneAPI Level Zero/TPU or similar accelerator models
- ML-systems familiarity: training and inference loops, tensor layouts
- DLPack, numpy, Python bindings
