Back to All Job Posts
Research Engineer
About Sherlocks.ai
Sherlocks.ai is your SRE teammate, handling alerts, conducting RCAs, and planning long-term stability projects. We aim to resolve incidents faster and prevent future outages. Our founding team has deep expertise in scaling startups and AI-driven ventures.
What will you be working on?
- Benchmarking existing agents: Compare Sherlocks’ current agents with other available models, including open-source alternatives, to understand their strengths and gaps.
- Diagnosing issues: Identify which agents lead to suboptimal responses, analyze why those failures happen, and form hypotheses about patterns and root causes.
- Running quick experiments: Test fixes or improvements on sample cases to validate hypotheses and enhance agent performance.
- Extending agent capabilities: Assist in implementing new or improved agents aimed at boosting the accuracy and efficiency of Sherlocks’ incident management.
- Exploring new tools and solutions: Evaluate emerging tools and platforms, and run small proofs of concept to gradually offload or enhance in-house systems with scalable and enterprise-grade alternatives.
Must-have skills
- Python
- LangChain or LLM-Ops experience
- Prompt and eval design experience