Skip to content

Nagendra Reddy Koppula

Data Engineer • Northlake, Texas • k*********@gmail.com • 216****555 • linkedin.com/•••••

Professional Summary

Data Engineer with 5+ years of experience building resilient batch and streaming data platforms. Expert in cloud-native ETL/ELT, Spark-based processing, and warehouse optimization to accelerate analytics and production ML workflows. Proven track record implementing CI/CD and infrastructure-as-code to improve reliability, reduce costs, and scale multi-terabyte pipelines.

Technical Skills

Programming Languages: Python,Scala,ETL
Databases: SQL,dbt,MySQL,PostgreSQL,SQL Server,MongoDB
Cloud and DevOps: Azure Data Lake Storage,Azure Synapse Analytics,Amazon S3,Amazon Redshift,Terraform,Docker,Azure DevOps,GitHub Actions,Cloud,Data warehousing,Data Modeling,SageMaker),Data-bricks,Snowflake
Data and Analytics: Apache Spark,PySpark,Hadoop,Azure Machine Learning,MLflow,Power BI,Tableau
Tools and Methodologies: Apache Airflow,Azure Data Factory,AWS Glue,Git,Dataproc,Apache Beam
Streaming & Messaging: Kafka,Azure Event Hubs
Libraries: Pandas,NumPy,PySpark,Matplotlib,Seaborn
MLOps and AI/ML Tools: Feature Engineering Pipelines,ML-ready data pipelines,TensorFlow,PyTorch,Scikit-learn,XGBoost,AutoML

Work Experience

Startech Networks Inc
Texas
Data Engineer
Jul 2024 – Current
Built cloud data platforms and ML-ready pipelines at a technology/networking company; supported analytics, model feature stores, and downstream model workflows.
Tech Stack: Azure Data Factory, Databricks, PySpark, Azure Data Lake Storage, Azure Event Hubs, Azure Synapse Analytics, Azure Machine Learning, MLflow, Azure Stream Analytics, Terraform, Azure DevOps, GitHub Actions, Power BI, Python, SQL
  • Designed end-to-end ingestion and processing pipelines using Azure Data Factory and Databricks (PySpark) to move telemetry and business data from ADLS Gen2 and Event Hubs into Azure Synapse, enabling analytics and ML feature stores.
  • Implemented scalable ETL/ELT frameworks in Python and SQL with incremental loads, schema evolution, and automated data quality checks, improving batch processing performance by 35% across 20+ multi-terabyte tables.
  • Optimized Synapse data warehouse using distribution strategies, partitioning, and query tuning to reduce median analytical query times by 40%, resulting in $8K monthly infrastructure cost savings.
  • Partnered with data science to productionize ML data pipelines using Azure Machine Learning and MLflow; built real-time feature engineering paths with Azure Stream Analytics and integrated model outputs into Power BI dashboards.
  • Introduced CI/CD and IaC using Azure DevOps, Terraform, and GitHub Actions to automate deployments and environment provisioning, cutting deployment time to 15 minutes and improving pipeline reliability to 99.5%.
  • Mentored and onboarded 3 junior data engineers on ETL design patterns, dimensional modeling, and performance tuning, increasing team delivery velocity and reducing critical incident recovery time.
Subaru of America
Cherry Hills, New Jersey
Data Engineer
Aug 2023 – May 2024
Delivered data engineering for automotive customer and risk analytics; developed pipelines that supported real-time customer insights and ML-based risk analytics.
Tech Stack: Azure Data Factory, Databricks, PySpark, Azure Data Lake Storage, Azure Event Hubs, Azure Synapse Analytics, Microsoft Purview, Power BI, Azure Cosmos DB, Azure SQL Database, Azure DevOps, MLflow, Python, SQL
  • Architected and optimized ETL pipelines with Azure Data Factory and Databricks to process ~50,000 daily transactions from ADLS Gen2, reducing end-to-end processing time by 40% through PySpark automation.
  • Built a customer insights platform leveraging Event Hubs and Databricks for near real-time processing using Python and PySpark; integrated outputs into Power BI to deliver a 25% lift in key engagement metrics.
  • Led implementation of data governance with Purview and Synapse, establishing metadata, lineage, and automated validation in Data Factory and Databricks to ensure regulatory compliance and data accuracy.
  • Engineered ML-ready pipelines to support risk analytics using PySpark and MLflow; containerized scoring with Azure Container Instances and orchestrated runs via Data Factory, cutting credit risk data processing time by 60%.
  • Orchestrated multi-source integration using Azure Cosmos DB and Azure SQL Database and enforced CI/CD using Azure DevOps and Git to automate testing and deployment, reducing data inconsistencies by 35% across systems.
  • Performed query tuning and table design in Synapse and Databricks to accelerate dashboard refreshes and reduce report latency for business stakeholders.
BeyondScale
Hyderabad, India
Associate Data Engineer
Nov 2020 – Dec 2022
Worked at a technology services firm on cloud migrations and serverless ETL for insurance clients, consolidating data for analytics and fraud detection.
Tech Stack: AWS S3, Amazon Redshift, AWS Glue, AWS Lambda, IAM, VPC, AWS CodePipeline, Git, Python, SQL, PySpark
  • Led on-premises to AWS data warehouse migration, consolidating 500GB+ of insurance data into Amazon S3 and Redshift with star schema designs, cutting reporting query times by 50%.
  • Built serverless ETL pipelines using AWS Glue (PySpark) and Lambda to automate claims ingestion and validation, reducing average processing time from 4 hours to 15 minutes.
  • Designed dimensional models and optimized Redshift fact/dimension tables to support fraud detection analytics, improving fraud identification response by 20% through faster queries.
  • Implemented event-driven ingestion using S3 event notifications, Glue jobs, and Lambda triggers to reduce data latency from 6 hours to 1 hour and eliminate manual uploads.
  • Enforced data security and compliance for insurance workloads using S3 encryption (AES-256), IAM role-based access, and VPC isolation to meet HIPAA and SOC2 requirements.
  • Established CI/CD for ETL using AWS CodePipeline and Git to automate testing and deployment of Glue jobs, reducing deployment cycles by 70% and minimizing manual errors.

Projects

Emotion Detection using DistilBERT | Jan 2024 – May 2024
Tools Used: Python, Data Preprocessing, TensorFlow, DistilBERT, MLflow
  • Prepared and standardized an emotion-labeled text dataset by cleaning raw text, normalizing labels, and validating inputs to ensure consistent model training.
  • Built a reusable Python preprocessing pipeline to produce feature-consistent inputs for both training and inference, enabling reproducible ML workflows.
  • Trained a lightweight DistilBERT classification model to validate dataset quality and preprocessing, demonstrating multi-class emotion prediction capability.
  • Developed a standalone prediction script that reused training preprocessing logic for real-time inference on new text inputs.
Twitter Data Pipeline | Aug 2023 – Dec 2023
Tools Used: Python, Apache Airflow, AWS EC2, Amazon S3, ETL
  • Designed a batch pipeline to ingest Twitter API streams and external datasets, handling schema differences and normalizing records for analytics.
  • Orchestrated ingestion and transformation workflows using Apache Airflow on an AWS EC2 host to provide scheduled, reliable data movement.
  • Implemented Python ETL logic to clean and enrich tweets, storing processed datasets in organized S3 paths to support downstream analysis.
Oil–Energy Cost and Production Prediction | Aug 2023 – Dec 2023
Tools Used: Python, Time Series, Feature Engineering, Scikit-learn
  • Explored industrial time-series data to assess quality and performed cleaning to address missing values and outliers common to sensor datasets.
  • Executed correlation analysis and feature selection to prepare inputs for regression and tree-based models, demonstrating the impact of preprocessing on forecast accuracy.
  • Evaluated baseline ML models to benchmark prediction performance and identified preprocessing strategies that improved model stability.

Education

Kent State University
Master of Science in Computer Science • Kent, OH • Jan 2022 – May 2024
Bharath Institute of Higher Education and Research (BIHER)
Bachelor of Technology • Chennai, India • Jun 2018 – Apr 2022

Certifications

Microsoft Fabric Data Engineer Associate — Microsoft • Dec 2025

Powered by Drivetube · Create your own profile at drivetube.ai