Job Information
Mphasis Corporation Sr Quality Analyst with AI/ML and LLM testing in NEW YORK, New York
Role SummaryThis role requires deep understanding of AI fundamentals, handson experience with LLM testing, guardrail design, data sanitization,and evaluation of agent trajectories. The tester will work closely with AI engineers, data scientists, and product teams to ensure safe, reliable, and highquality AI behavior.58+ years in QA, with at least 23 years in AI/ML or LLM testing.Strong understanding of AI fundamentals: embeddings, vector stores, RAG, LLMs, agentic frameworks.Handson experience testing: * LLM prompts and outputs * Agentic flows (e.g., LangChain, AutoGen, Semantic Kernel, AIP agents) * Model evaluation metricsExperience writing guardrails, safety rules, and evaluation prompts.Knowledge of data sanitization, PII detection, and data governance.Familiarity with pipeline validation and MLOps workflows.Ability to evaluate model performance using LowPick Score or similar ranking metrics.Strong Python skills for test automation and evaluation scripting.SkillsEqual opportunity Employer:Mphasis is an equal opportunity/affirmative action employer. We provide equal employment opportunities to applicants and existing associates and evaluate qualified candidates without regard to race, gender, national origin, ancestry, age, color, religious creed, marital status, genetic information, sexual orientation, gender identity, gender expression, sex (including pregnancy, breast feeding and related medical conditions), mental or physical disability, medical conditions military and veteran status or any other status or condition protected by applicable federal, state, or local laws, governmental regulations and executive orders.