Skip to content

Venkatesh Koya

Data Engineer • USA • drivetube.ai/•••••

Professional Summary

Data Engineer with 4 years of experience designing and delivering production-grade data platforms, distributed data pipelines, and real-time streaming systems across AWS and Azure environments. Proficient in Apache Spark, PySpark, Databricks, Kafka, Airflow, and Snowflake with a strong track record of building scalable ETL and ELT architectures, cloud data lake solutions, and Delta Lake-based data lakehouses that power enterprise analytics. Adept at migrating legacy ETL systems to modern cloud-native frameworks, optimizing Spark job performance, and driving data reliability and consistency across complex multi-source environments.

Technical Skills

Programming Language: Python,Scala,Shell Scripting,SQL
Databases: Snowflake,Microsoft SQL Server,Azure SQL Database,Amazon Redshift,Azure Synapse Analytics,Google BigQuery
Cloud Platforms: AWS,S3,AWS Glue,Lambda,AWS Athena,Kinesis
Version Control & Development Tools: Git
DevOps & Infrastructure: Jenkins,Docker,Kubernetes,Terraform,GitHub Actions
Data Engineering & Processing: Delta Lake,EMR,Azure Databricks,Apache Spark,PySpark,Apache Hadoop,Apache Hive,HDFS,Data Lake,Data Lakehouse,Apache Airflow
Data Analysis & Visualization: Tableau,Power BI
Data Integration & ETL: ADF,ELT,dbt,Apache NiFi

Work Experience

Fiserv
USA
Data Engineer
Jul 2025 – Present
Fiserv is a global fintech and payments provider; role built cloud data lakes and streaming pipelines that powered large-scale analytics on financial transaction data.
Tech Stack: PySpark, AWS Glue, Databricks, AWS S3, AWS Athena, Amazon Redshift, Airflow, Kafka, Spark Streaming, dbt, Tableau
  • Designed and deployed scalable ETL pipelines using PySpark on AWS Glue and Databricks, enabling efficient large-scale batch data processing for enterprise workloads.
  • Architected and maintained a cloud data lake using AWS S3, Glue Data Catalog, Athena, and Redshift to support reliable analytics ingestion and querying.
  • Developed and managed Airflow DAGs to orchestrate complex scheduled and dependency-driven data pipelines, improving reliability and observability.
  • Built near real-time streaming data pipelines leveraging Kafka and Spark Streaming to process and deliver time-sensitive financial data.
  • Led migration of legacy MapReduce-based ETL workloads to a PySpark-based distributed processing framework, reducing processing time and operational overhead.
  • Developed dbt models to standardize data transformations, enforce business logic consistency, and improve data quality across downstream pipelines.
Truist Financial
USA
Data Engineer
May 2024 – Jun 2025
Truist is a major U.S. bank; engineered AWS-based batch and streaming platforms to drive real-time analytics and reporting on high-volume banking datasets.
Tech Stack: PySpark, AWS EMR, Kafka, AWS Kinesis, Spark Streaming, Delta Lake, AWS Glue, Airflow, Git, Docker, GitHub Actions, SQL Server, Hive, AWS S3, Amazon Redshift
  • Built and maintained large-scale data processing pipelines using PySpark on AWS EMR to handle high-volume financial datasets with low latency.
  • Designed and implemented batch and streaming ingestion pipelines using Kafka, AWS Kinesis, and Spark Streaming to support real-time analytics use cases.
  • Architected a Delta Lake-based unified data lakehouse to seamlessly support both batch and streaming workloads with ACID-compliant data management.
  • Automated ETL workflows using AWS Glue and Airflow, increasing pipeline reliability, reducing manual intervention, and enhancing maintainability.
  • Tuned Spark job performance through advanced partitioning strategies, data caching, and execution plan optimization, resulting in significant reduction in processing time.
  • Implemented CI/CD pipelines using Git, Docker, and GitHub Actions to automate deployment and improve code delivery consistency.
Infosys
Systems Engineer
Nov 2021 – Aug 2023
Infosys, a global IT services firm, supported enterprise clients by maintaining SQL Server ETL systems and migrating applications to Azure for resilient, high-availability operations.
Tech Stack: SQL Server, SSIS, Azure App Services, Azure SQL Database, Azure Key Vault, SQL, Shell Scripting
  • Supported enterprise applications and production ETL systems, maintaining high availability and minimizing downtime across critical business processes.
  • Developed and optimized SQL Server queries and database schemas, improving query performance and overall application responsiveness.
  • Automated ETL workflows using SSIS packages and SQL scripting, reducing manual data processing efforts and improving pipeline throughput.
  • Performed database tuning activities including indexing, query plan analysis, and performance profiling to enhance system efficiency.
  • Implemented Azure cloud services including Azure App Services, Azure SQL Database, and Azure Key Vault to support secure and scalable application deployments.
  • Managed incident resolution, production support, and release activities, ensuring minimal business disruption and adherence to SLA commitments.

Education

Northwest Missouri State University
Master of Science – Applied Computer Science
JNTU Kakinada
Bachelor of Technology – Electronics and Communication Engineering

Powered by Drivetube · Create your own profile at drivetube.ai

Everything on Drivetube

Six products, two of them free forever. Start wherever you are.

Read the docs →
  • Your free digital resume

    at drivetube.ai/in/your-name: one true standard resume with a Hiring Snapshot (visa status, expected salary, notice period, work preference, relocation), an ATS-ready PDF download and a single shareable link. Free forever; interview requests come from verified employers and your contact details stay masked until you accept.

    Documentation
  • Verified jobs, posted in the last 3 days

    verified openings crawled ATS-by-ATS from 160,000+ real company career pages across 50+ ATS platforms. Shows the true posting date from the source ATS — not when a listing was indexed — and deletes every general listing 3 days after it was actually posted. No ghost jobs, no ad-sponsored listings, no staffing reposts, no account needed.

    Documentation
  • Written by senior career writers

    human-written, ATS-optimised resumes by senior career writers, from ₹499.99 / $25.99. Available in every country, written to the destination country's own standard — a US resume, UK CV, German Lebenslauf and Indian resume are genuinely different documents. A paid service, separate from the free Drivetube Profile.

    Documentation
  • Drivetube OneFrom $39.99 / 3 mo

    Daily matched jobs, tailored resumes, or we apply for you

    Drivetube's paid service. Community sends daily Jobsheets of fresh jobs matched to your profile on WhatsApp and email, with the Drivetube Community ($39.99 for 3 months or $69.99 for 6 months; ₹999 or ₹1,899 in India). Jobhunt is one-time application credits with Community included: one credit tailors one resume and its cover letter (100 for $99 with 3 months of Community, 250 for $199 with 6 months). Let Us Apply is an add-on to Jobhunt: a human expert applies 250 times for you on your credits ($0.60 each, $150). Everything lives on one dashboard at drivetube.ai/drivetubeone.

    Documentation
  • For employers — no job postings, no applications

    hiring with no job postings and no applications. Paste your real job description and AI matches it against candidates' true standard resumes, returning ranked candidates with a match %, matched and missing skills and written reasoning. Free tier included; employers pay, candidates never do.

    Documentation
  • Documentation

    every product explained in full, with feature-by-feature comparisons against the job boards, AI apply tools, resume services and hiring platforms people actually use.

  • Playbooks

    tactical job-search guides. Each post is one named tactic with the situation it applies to, the exact steps, a copy-paste message and honest failure modes. Free, no account.

The job board covers the United States, India, United Kingdom, Canada, Europe and Australia, and Resume Writing Services are available in every country.

Pricing is identical on the web, iOS and Android — no app-store markup. Monthly Community membership is purchased in the iOS app only; the website sells annual plans. The Android app has no in-app purchase, so Android members subscribe on drivetube.ai and then sign in to the app with full access. Membership follows your account, not your device.