Specialty
Who is hiring is shown to signed-in users. Create a free account to see the employer and apply.
The role
The company is hiring a Senior Staff Software Engineer for its AI Platform team, which supplies tools and infrastructure to hundreds of AI and ML engineers. The role shapes a company-wide evaluation platform for GenAI and ML systems. Open to US-based applicants only.
What you will do
- Find modernization opportunities and set the roadmap for a company-wide AI/ML evaluation system
- Define cross-team technical strategy and drive adoption of shared evaluation capabilities
- Make architectural decisions that improve reliability and usability of the evaluation platform
- Set evaluation methods that balance automated metrics, expert judgment, human feedback and safety
- Lead complex cross-team workstreams and resolve technical tradeoffs
What they ask for
- 3+ years designing and operating evaluation systems for GenAI and ML, including benchmarks, golden datasets and model-based grading
- Strong Python skills and solid grasp of distributed systems
- Familiarity with evaluation libraries such as AutoEvals, OpenEvals and Ragas
- 5+ years building platforms for experimentation, model versioning, orchestration and production monitoring
- Sound judgment on GenAI risks such as bias, privacy and security
- Based in the US; office visits 1-2 times per quarter
What they offer
- Eligible for equity in addition to base salary
- Remote working with occasional in-person gatherings
This is our own short reading of the posting. The full description, and anything that has changed since, is on the employer's careers page behind the apply button.