Specialty
The role
Cohere, an enterprise AI company, is hiring a Senior ML Systems Engineer to build and maintain the training framework behind its large language models. The role covers distributed training, performance and tooling for GPU clusters, and can be remote or office-based.
What you will do
- Build and own the training framework used for large-scale LLM training.
- Design distributed training abstractions such as data, tensor and pipeline parallelism, FSDP/ZeRO, memory management and checkpointing.
- Improve training throughput and stability on multi-node GPU clusters.
- Develop tooling for monitoring, logging, debugging and developer ergonomics.
- Work with infra teams on cluster, container and hardware setups, and fix performance bottlenecks.
- Build systems that make large runs reproducible and debuggable.
What they ask for
- Strong engineering experience in large-scale distributed training or HPC systems.
- Deep familiarity with JAX internals, distributed training libraries or custom kernels and fused ops.
- Experience with multi-node orchestration such as Slurm, Ray or Kubernetes.
- Ability to debug performance issues across CUDA/NCCL, networking, IO and data pipelines.
- Experience with containers (Docker, Singularity/Apptainer).
- Record of building tools that speed up ML developers.
- Nice to have: LLM training experience and contributions to PyTorch, JAX or DeepSpeed.
What they offer
- Weekly lunch stipend
- Health and dental benefits with a mental health budget
- RRSP matching, 401(k) or pension
- Parental leave top-up for up to 6 months
- 6 weeks paid vacation
- $500 home office stipend and co-working benefit
This is our own short reading of the posting. The full description, and anything that has changed since, is on Cohere's careers page behind the apply button.
