Job Information
UNIVERSITY OF TEXAS AT AUSTIN Lead Data Engineer in Austin, Texas
Technical Leadership:Design, architect, and deliver production-grade, scalable data pipelines and AI-ready data platforms using Databricks,AWScloud-native services and modern data engineering frameworks.Lead end-to-end implementation oflakehousedata pipelines, ensuring performance, reliability, and cost efficiency.Championindustrybest practices for data engineering.Conduct andparticipatein peer code reviews tomaintaincode quality and consistency across the team.Proactivelyidentifyand resolve bottlenecks in data ingestion, transformation, and orchestration processes using Databricks Delta Live Tables, Spark optimizationtechniques, and workflow automation.Implement systems for data quality, observability, governance, and compliance using tools such as Unity Catalog, Delta Lake, and data validation frameworks.Lead technical knowledge-sharing sessions on topics such as AI/ML integration, datalakehousearchitecture, and emerging data technologies.Project Management:Define project milestones, timelines, and deliverables for data and AI initiatives, ensuringtimelyand high-quality outcomes.Collaborate with both internal and external stakeholderssuch as data architects, system architects, business users, Agile teammembers, and other D2I internal groups.Manage project priorities, sprint planning, and team workloads while balancing innovation with delivery.Communicate risks, dependencies, and resource constraints effectively, and develop mitigation plans for on-time project delivery.Team Management and Leadership:Supervise and mentor a team of Data Engineers (25 individuals) working on cloud, Databricks, and AI pipeline initiatives.Foster a culture of continuous learning, experimentation, and technical excellence, encouraging engineers to explore AI and automation use cases.Participate in recruiting, onboarding, and developing data engineering talent with strong Databricks and AI skillsets.Conduct performance reviews, set development goals, and create individualized growth plans for team members.Encourage collaboration across Data, AI/ML, Analytics, and Infrastructure teams to drive cross-functional success.Communication:Provide regular updates on project progress, technical challenges, andprojectmilestones to both technical and business stakeholders.Translate complex technical concepts related to Databricks, AI, and data architecture into clear narratives for non-technical audiences.Foster a transparent communication culture and provide actionable feedback to promote a growth mindset.Ensure all data engineering processes, architectures, and standards are well-documented for reuse, governance, andknowledgecontinuity.Innovation and Other Duties:Stay current with advancements in AI, data engineering, and Databricks ecosystem, evaluating new tools and frameworks for potential adoption.Pilot and promote innovative solutions such as AI-assisted data quality checks, data observability automation, and intelligent pipeline optimization.Perform other duties as assigned, contributing to the organizations data-driven and AI-enabled transformation.