Microsoft
Senior Data Product Manager, Outlook
Jul 2021 - Present
Evals platform, AI agents and the telemetry ecosystem for Outlook and Copilot.
Highlights
- Own the evals platform initiative across Outlook, partnering with 30+ teams inside and outside the org to build and run online and offline evals for AI scenario owners (Summarize, Draft, and others), surfacing quality metrics to hill-climb agent quality toward task-completion and quality targets.
- Designed and built CEAD (Copilot Evaluation Agent for Data), an AI agent that guides teams end to end — from defining the evaluation objective to generating synthetic data to assembling the complete eval set — feeding Microsoft's internal SEVAL platform, where jobs run daily in production.
- Use AI-native practices daily to build product artifacts: generating eval sets and synthetic data with AI tools, building DSAT eval sets from real user feedback, and connecting multiple data sources into metrics and dashboards — while embedding responsible AI concepts into the eval harness.
- Mentor and coach teams on evals as a first-class discipline, leading Outlook's offline-evals office hours to spread an eval-first culture and codify a repeatable playbook that scenario teams reuse.
- Lead Outlook's telemetry ecosystem — defining success and setting multi-geography data platform strategy, ensuring consistent, reliable, low-latency signals across seven clients at worldwide scale.
- Led the modernization and unification of High-Value Actions (HVAs), balancing customer experience and engineering constraints and reducing data processing, production, and storage overhead by 86%.
- Sold new measurement frameworks and standards to 20+ partner teams, driving org-wide adoption of unified telemetry governance and a shared roadmap for Consumer Trust and Copilot metrics.
- Built internal tools for data discoverability, reliability, and root-cause analysis across 2,000+ telemetry signals, accelerating engineering velocity for client and service teams.