Engineering Analyst, Responsible AI Testing

Minimum qualifications:

• Bachelor's degree or equivalent practical experience.
  
• 5 years of experience in data analysis, including identifying trends, generating summary statistics, and drawing insights from quantitative and qualitative data.
  
• 5 years of experience managing projects and defining project scope, goals, and deliverables.
  

Preferred qualifications:

• Master's degree in a quantitative discipline.
  
• 5 years of experience or familiarity with one or more of the following languages: SQL, R, Python, or C++.
• Experience working in Trust and Safety or Responsible AI.
• Experience with operational process design and workflow optimization.
• Strong understanding of AI/ML evaluation metrics (e.g., precision, recall, F1) and LLM limitations.
• Comfortable working with data analytics, interpreting ML model performance metrics, and understanding technical architecture to identify safety gaps.

About the job:

Trust & Safety team members are tasked with identifying and taking on the biggest problems that challenge the safety and integrity of our products. They use technical know-how, excellent problem-solving skills, user insights, and proactive communication to protect users and our partners from abuse across Google products like Search, Maps, Gmail, and Google Ads. On this team, you're a big-picture thinker and strategic team-player with a passion for doing what’s right. You work globally and cross-functionally with Google engineers and product managers to identify and fight abuse and fraud cases at Google speed - with urgency. And you take pride in knowing that every day you are working hard to promote trust in Google and ensuring the highest levels of user safety.

At Google we work hard to earn our users’ trust every day. Trust & Safety is Google’s team of abuse fighting and user trust experts working daily to make the internet a safer place. We partner with teams across Google to deliver bold solutions in abuse areas such as malware, spam and account hijacking. A team of Analysts, Policy Specialists, Engineers, and Program Managers, we work to reduce risk and fight abuse across all of Google’s products, protecting our users, advertisers, and publishers across the globe in over 40 languages.

Responsibilities:

• Manage the full lifecycle of technical requests (e.g., bugs, feature requests) for the team. Combine program management with technical analysis (e.g., root-cause debugging, data-driven scoping) to allocate resources effectively.
  
• Drive the strategic roadmap for RAI Sustainability by identifying high-impact opportunities through data analysis. Translate business needs into technical requirements for Engineering partners and track delivery milestones.
  
• Serve as the central communication hub for RAI Testing Sustainability. Use quantitative evidence to influence senior stakeholders in Engineering and T&S, ensuring alignment on launch criteria and risk mitigation strategies.
  
• Lead the technical evaluation of new testing capabilities (e.g., measuring PromptGen quality, Autorater precision/recall), ensuring that program decisions are grounded in rigorous statistical validity.
• Engineer closed-loop feedback systems where testing insights directly inform model development. Monitor KPIs to ensure these feedback loops are reducing time-to-mitigation for AI risks.