OneMain Financial Jobs

Intern, AI Observability and Evaluation Job ID 2026-13495

of Openings

1 Job Locations Remote - U.S. Category Engineering and Testing Overview This internship supports the Engineering team building Hyland's contentcentric AI services powered by Agents and large language models. The team uses Arize as its observability platform to trace agent actions, evaluate LLM behavior, and improve the reliability of AIdriven features. The intern will focus on understanding, analyzing, and designing improvements around the observability data collected through Arize-not primarily coding, but research, experimentation, and workflow design. This role is ideal for someone curious about AI systems, evaluation frameworks, and how observability shapes product quality. Responsibilities Gain access to and onboard into the AI Platform and Arize observability environment. Learn how agents are instrumented and how traces, spans, and evaluations are collected. Map how observability data flows into monitoring, alerting, LLMasajudge evaluations, and episodic memory. Analyze existing monitoring and evaluation workflows to identify gaps or improvement opportunities. Explore ways to integrate Arize data into the evaluation pipeline for regression testing and model validation. Investigate and propose improvements to LLMasajudge workflows for quality control and alerting. Define strategies for sampling, trace selection, and processing (e.g., determining trace percentages). Prototype concepts such as customerfacing monitoring insights or configurable observability settings. Produce tangible outcomes such as workflow improvements, prototypes, or proofofconcept designs. Collaborate closely with ML1 and ML2 teams, with technical mentorship from Ralph and organizational support from Gabe. Adapt to evolving priorities in a fastmoving AI environment and proactively identify areas where observability can improve reliability. Adapt to evolving priorities in a fastmoving AI environment and proactively identify areas where observability can improve reliability. Basic Qualifications Currently enrolled in an educational institution Proficiency with Microsoft Office software products Attentive to department needs as demonstrated by rapid and high-quality responsiveness to requests Excellent interpersonal skills; able to maintain solid rapport with team members as well as maintain professionalism with those outside of department Excellent oral and written communications skills that demonstrate a professional demeanor and the ability to interact with others with discretion and tact Keen attention to detail Capable of identifying and completing tasks independently, with a sense of urgency and ownership Demonstrated success at maintaining high personal work standards Demonstrated ability to handle sensitive information with discretion and tact Or an equivalent combination of education and experience sufficient to successfully perform the principal duties of the position. Interest in AI systems, LLMs, agent architectures, or AI observability. Familiarity with concepts like tracing, evaluations, monitoring, or model validation (coursework or projects count). Ability to analyze complex systems and communicate findings clearly. Curiosity about how AI products are built, tested, and improved in production environments. Comfort working in ambiguous, fastchanging environments. Strong initiative and willingness to explore openended problems. Basic technical literacy (Python, data analysis, or ML fundamentals) is helpful but not required. Equal Opportunity Employer -- minorities/females/veterans/individuals with disabilities/sexual