Skip to content

Rahul Sharma

Data Engineer • Dallas, TX • r***********@email.com • 555****567 • linkedin.com/••••• • drivetube.ai/•••••

Professional Summary

Data Engineer with 5 years of experience building scalable ETL pipelines, cloud data platforms, and data warehouses. Strong expertise in Python, SQL, Spark, AWS, Snowflake, Airflow, Kafka, and data modeling. Proven record optimizing performance, reducing costs, and delivering reliable data for analytics and reporting.

Technical Skills

Programming Languages: Python
Databases: SQL,Snowflake,PostgreSQL,MySQL,SQL Server,MongoDB
Cloud and DevOps: AWS,Amazon S3,AWS Glue,Amazon Redshift,AWS CloudWatch,Jenkins,Docker,Terraform,CI,CD
Testing: Dimensional Modeling,Data Quality Frameworks,Monitoring & Alerts
Data and Analytics: Apache Spark,PySpark,Spark Streaming,Hadoop
Tools and Methodologies: Git,Databricks,Snowflake SnowPro concepts
Streaming & Messaging: Apache Kafka
Orchestration & ETL: Apache Airflow,dbt

Work Experience

ABC Financial Services
Dallas, TX
Data Engineer
01/2023 – Present
Financial services firm; built data platforms and pipelines supporting analytics, risk models, and reporting for business stakeholders.
Tech Stack: Python, PySpark, Apache Spark, Apache Airflow, AWS S3, Snowflake, Git, Jenkins, AWS CloudWatch
  • Designed and implemented production ETL pipelines processing 500M+ records monthly using Python, PySpark and Airflow, ingesting data into AWS S3 and Snowflake for analytics and reporting.
  • Optimized Spark jobs, partitioning and resource configuration to reduce end-to-end data processing time by 45% and lower compute consumption on clusters.
  • Built Snowflake dimensional models and materialized views to support BI dashboards, improving query performance for analysts and reporting teams.
  • Automated ingestion from 20+ external APIs into S3 and Snowflake using Python and Airflow operators with incremental loading patterns to enable near-daily data refreshes.
  • Implemented CI/CD pipelines for data platform deployments using Git and Jenkins, enabling repeatable releases, schema migrations and faster rollback.
  • Established monitoring and data quality checks with Airflow sensors and CloudWatch alerts to detect failures and improve data reliability for downstream risk and reporting processes.
TechNova Solutions
Dallas, TX
Data Engineer
06/2021 – 12/2022
Technology consulting company delivering cloud data platforms and analytics solutions to enterprise clients.
Tech Stack: AWS Glue, PySpark, Apache Kafka, Apache Spark, Amazon Redshift, AWS S3, Terraform, CI, CD
  • Designed end-to-end ETL workflows using AWS Glue and PySpark to ingest and transform structured and semi-structured client datasets into Redshift and S3.
  • Built Apache Kafka streaming pipelines processing 100K+ events per hour and integrated Spark Streaming for near-real-time enrichment and delivery to analytics stores.
  • Developed Amazon Redshift data warehouse schemas with optimized distribution and sort keys, improving analytic query performance and reducing BI latency.
  • Led cloud cost optimization initiatives including job tuning, partitioning and instance rightsizing, reducing monthly processing costs by 30%.
  • Implemented data validation and schema evolution strategies within ETL workflows to reduce pipeline failures and ensure backward compatibility.
  • Automated infrastructure provisioning and deployments with Terraform and CI/CD pipelines to accelerate environment setup and delivery cadence.
Infosys Ltd
Hyderabad, India
Data Engineer
07/2019 – 05/2021
Global IT services and consulting company; supported banking clients by delivering ETL solutions, cloud migrations and data quality frameworks.
Tech Stack: Python, SQL, Apache Spark, AWS, Apache Airflow, Git, Jenkins, PostgreSQL, MySQL
  • Delivered ETL solutions for banking clients using Python and SQL to consolidate transactional and customer datasets for regulatory and business reporting.
  • Migrated legacy ETL workloads to AWS and re-architected batch jobs for Spark to improve scalability, maintainability and processing throughput.
  • Implemented data quality and monitoring frameworks with automated checks and alerts to improve accuracy of data used in regulatory reports and analytics.
  • Collaborated with data analysts and business stakeholders to design star schemas and dimensional models that enabled faster ad-hoc analysis and standardized reports.
  • Optimized SQL queries and implemented table partitioning and indexing strategies to reduce query times on large transactional tables.
  • Supported deployments and operational runbooks using Git and Jenkins, documented procedures and performed production troubleshooting to maintain reliable pipelines.

Projects

Enterprise Data Lake Modernization
Tools Used: AWS S3, AWS Glue, Apache Spark, Snowflake, Python
  • Built an AWS data lake architecture ingesting and cataloging 2TB+ of daily data to centralize enterprise datasets for analytics.
  • Implemented ETL pipelines and transformations with Spark and Glue, enabling normalized and query-ready tables in Snowflake.
  • Reduced reporting latency from 24 hours to under 2 hours by improving ingestion cadence, partitioning and queryable datasets.
Real-Time Customer Analytics Platform
Tools Used: Apache Kafka, Apache Spark, PySpark, Databricks
  • Designed Kafka-based event ingestion and Spark Streaming pipelines to process 5M+ events daily for near-real-time customer analytics.
  • Implemented streaming enrichment and aggregation layers to feed dashboards and personalization services.
  • Maintained high pipeline reliability and alerting to ensure consistent event delivery for analytics consumers.

Education

Jawaharlal Nehru Technological University Hyderabad
Bachelor of Technology in Computer Science • Hyderabad, India • 2015 – 2019

Certifications

AWS Certified Data Engineer Associate — AWS
AWS Certified Solutions Architect Associate — AWS
Snowflake SnowPro Core Certification — Snowflake
Databricks Certified Data Engineer Associate — Databricks

Powered by Drivetube · Create your own profile at drivetube.ai