Job Information
CTG Lead Data Platform Engineer (Airflow / Astro & AWS) in Atlanta, Georgia
CTG is seeking to fill a Lead Data Platform Engineer (Airflow / Astro & AWS) position for our client.
Location: Atlanta, GA
Duration: 9 months
Duties:
Lead L2/L3 application support for enterprise Astronomer (Astro)-managed Apache Airflow environments
Own Incident, Problem, and Change Management processes for the data orchestration platform
Perform advanced root cause analysis (RCA) for pipeline failures, scheduler issues, and infrastructure bottlenecks
Improve DAG reliability, SLA adherence, and reduce MTTR across data pipelines
Establish operational playbooks, runbooks, and support standards
Design and maintain dynamic DAG frameworks (Factory-based) to enable scalable pipeline onboarding
Support complex ETL/ELT workflows across data lakes (S3), data warehouses (Snowflake, Redshift), and streaming/batch pipelines
Ensure data quality validation, reconciliation, and SLA tracking across workloads
Collaborate with Data Engineering teams to optimize performance and cost efficiency
Support metadata management, data lineage, and governance integrations
Develop production-grade Python DAGs, operators, and plugins
Build reusable, configuration-driven orchestration frameworks
Automate deployment, provisioning, and pipeline lifecycle management
Enforce coding standards, version control, and CI/CD best practices
Manage Astro deployments in AWS (EKS, S3, RDS, IAM, CloudWatch, VPC)
Troubleshoot and optimize Kubernetes-based Airflow clusters
Perform capacity planning and resource tuning for schedulers and workers
Implement Infrastructure-as-Code (Terraform or CloudFormation)
Ensure high availability, disaster recovery, and cloud cost optimization
Design and implement observability dashboards for DAG health, SLA compliance, task failures, and resource utilization
Integrate monitoring tools such as CloudWatch, Prometheus, and Grafana
Implement proactive alerting and automated remediation workflows
Ensure compliance with data governance, audit, and regulatory requirements
Implement access controls, encryption standards, and secure data handling practices
Partner with Data Governance and Security teams
Lead and mentor a team of Data Platform and Application Support engineers
Collaborate with cross-functional teams including Data Engineering, Analytics, and Cloud Infrastructure
Provide executive-level reporting on platform KPIs (uptime, SLA adherence, incident trends)
Drive continuous improvement and platform modernization initiatives
Skills:
Strong expertise in Astronomer (Astro) and Apache Airflow
Advanced Python programming and automation skills
Deep knowledge of ETL/ELT processes and modern data architectures
Experience with AWS services including EKS, S3, RDS, IAM, and CloudWatch
Hands-on Kubernetes experience (EKS preferred)
Experience building observability and monitoring solutions (Prometheus, Grafana, CloudWatch)
Strong understanding of data lakes, data warehousing, and distributed data processing
Familiarity with Infrastructure-as-Code (Terraform, CloudFormation)
Knowledge of CI/CD pipelines and DevOps practices
Strong analytical, troubleshooting, and problem-solving skills
Excellent leadership, communication, and stakeholder management abilities
Experience:
7+ years of experience in Data Platform, Data Engineering, or Application Support environments
2+ years of hands-on experience with Astronomer (Astro) Airflow
Proven experience with dynamic DAG development (Factory-based frameworks)
Experience supporting enterprise-scale data orchestration platforms
Background in cloud-native data platforms and distributed systems
Experience in regulated industries (e.g., Financial Services) preferred
Exposure to MLOps orchestration workflows is a plus
Education:
- Bachelor’s degree in Computer Science, Data Engineering, Information Systems, or related field
Excellent verbal and written English communication skills and the ability to interact professionally with a diverse group are required.
CTG does not accept unsolicited resumes from headhunters, recruitment agencies, or fee based recruitment services for this role.
To Apply:
To be considered, please apply directly to this requisition using the link provided. For additional information, please contact Laura Dominguez at Laura.Dominguez@ctg.com . Kindly forward this to any other interested parties. Thank you!
About CTG
CTG, a Cegeka company, delivers IT and business solutions that enhance clients’ digital agility, empowering them to seize new opportunities and overcome any challenge. Backed by more than 60 years’ experience and a commitment to being a reliable, results-driven partner, we work shoulder to shoulder with clients to shape digital together. Our vision is to be an indispensable partner to our clients and the preferred career destination for digital and technology experts. With more than 9,000 team members in over 15 countries, we combine global expertise with local insight to deliver innovative solutions. We operate across the Americas, Europe, and India, working with over 3,000 clients in many of today's highest-growth industries.
Together, we shape what’s next—working shoulder to shoulder to deliver impactful solutions for our clients and society. Our culture is built by the people who work at CTG, the values we hold, and the actions we take. It's a living, breathing thing that is renewed every day through the ways we engage with each other, our clients, and our communities. At CTG, you’ll find a workplace where you are encouraged to grow, supported in your ambitions, and empowered to shape your own career journey. For more information, visit www.ctg.com .
CTG will consider for employment all qualified applicants including those with criminal histories in a manner consistent with the requirements of all applicable local, state, and federal laws.
CTG is an Equal Opportunity Employer. CTG will assure equal opportunity and consideration to all applicants and employees in recruitment, selection, placement, training, benefits, compensation, promotion, transfer, and release of individuals without regard to race, creed, religion, color, national origin, sex, sexual orientation, gender identity and gender expression, age, disability, marital or veteran status, citizenship status, or any other discriminatory factors as required by law. CTG is fully committed to promoting employment opportunities for members of protected classes.