5 years building production-grade ETL pipelines on AWS, orchestrating 1M+ record/day workflows with Apache Airflow, and integrating LLM-based automation into data platforms.
William O'Neil India · Bengaluru, Karnataka
Data Engineer with 5 years of experience designing and owning production-grade ETL pipelines on AWS. Specialized in Apache Airflow orchestration, PySpark and AWS Glue transformations, and analytical data modeling for large-scale workloads (1M+ records/day). Currently serving as interim lead for a team of 4 engineers, building LLM-based automation (OpenAI, LangGraph, Pinecone) into the data platform for failure detection and self-service analytics.
Lovely Professional University, Phagwara, Punjab
P.C. High School, Samastipur
Holy Mission High School, Darbhanga
Amazon Web Services
William O'Neil India, Bengaluru
Excelmax Technologies / Unlimiteye, Bengaluru
SHL (Saville and Holdsworth Ltd), Gurugram
ETL Pipeline
Python-powered extraction pipeline that automates web data ingestion, cleans messy content, and prepares analytics-ready datasets for reporting and downstream workflows.
Backend + Data
Robust Django + Celery + Redis workflow platform that processes orders and moves structured business data reliably through connected pipeline stages with async task queuing.
AI + Vision
Image analytics stack for object detection, face recognition, and emotion analysis — demonstrating expertise in unstructured data pipelines for advanced AI-driven analytics use cases.
Feel free to reach out — I'd love to connect and explore opportunities in data engineering, ETL automation, and cloud data platforms.