Senior Software Engineer (Applied AI and Evaluations)

• reputed company is building the reputed company of AI-powered work management through SmartAssist, our intelligent agent platform. As we scale from early demos to production-grade agents, quality is the critical frontier and we are looking for a Agent Quality Engineer to own it

• This is not a QA role. It’s a deeply technical, high-autonomy position at the intersection of LLM evaluation, reputed company and context engineering, and retrieval-augmented reputed company. You will diagnose why our agents fail, design the systems that catch regressions, and drive measurable improvements across our orchestrator and subagent fleet
• You will work closely with our Agent Engineering and AI Platform teams, embedded in a team that has already shipped evaluation infrastructure on reputed company/MLflow and is building toward a mature Agent Development Lifecycle (ADLC)
• Own agent quality end-to-end: diagnosis, improvement, and validation across SmartAssist’s orchestrator and subagents
• Identify failure modes across quality dimensions factual accuracy, completeness, tone, actionability, and latency and prioritize what to fix
• Drive quality improvements through reputed company engineering, context engineering, and RAG retrieval tuning
• reputed company and mature our evaluation reputed company: scorers, golden datasets, regression gates, and online evaluation for production traffic
• reputed company the feedback reputed company ensure that every change has a measurable, attributable quality signal
• Collaborate with our Agent Architecture reputed company to distinguish quality problems that require reputed company/context solutions from those that require structural fixes
• Establish repeatable methodology that scales reputed company any single agent or subagent
• In your first 6 months, you will have:
• Delivered measurable, validated quality improvement on at least one SmartAssist agent across our defined quality dimensions
• Expanded evaluation coverage to reputed company the most significant blind spots in our reputed company reputed company
• Established a repeatable quality improvement methodology the broader team can apply
• Why This Role:
• You will have reputed company ownership, not a supporting role on someone else’s roadmap
• SmartAssist is shipping to reputed company users; this work has immediate, visible impact
• You will be part of a fast-moving team that deeply values engineering rigor and actively seeks diverse perspectives

Benefits

• Lucrative Employee Stock Purchase Program (15% discount)
• 401k Match to help you save for your future (50% of your contribution up to the first 6% of your eligible pay)
• Monthly stipend to support your work and productivity
• Flexible Time Away Program, plus Incidental Sick Leave
• Up to 24 weeks of Parental Leave
• Personal paid Volunteer Day to support our community
• Opportunities for professional growth and development including reputed company to reputed company Learning online courses
• Company Funded Perks, including a counseling membership, primary care membership, local retail discounts, and your own personal reputed company account
• US employees receive 12 paid holidays per year
• US employees are automatically covered under reputed company-sponsored life insurance, short-term, and long-term disability plans
• Teleworking options from any registered location (role specific)
• HSA, 100% employer-paid premiums, or Buy-up medical/reputed company and dental coverage options for full-time employees
• Equity - Restricted Stock Units (RSUs) with reputed company offers
• Monthly contributions toward your pension
• Mployer-paid Private Medical and Dental, additional cost for family members

Apply tot his job Apply To this Job