Pavan Kumar Gorla
Professional Summary
Data Engineer with 5+ years of experience designing and delivering scalable, secure data pipelines, cloud data platforms, and published data products across financial services (asset management, payment processing, retail banking) using AWS, Snowflake, Databricks, Python, PySpark, and SQL, with earlier delivery experience on Azure. Hands-on building domain-owned data products on an AWS lakehouse applying Data Mesh principles. Strong AWS depth across S3, Glue, Lambda, Step Functions, EventBridge, EMR, Athena, Redshift, Lake Formation, IAM, KMS, CloudWatch, SNS, and SQS. Experienced automating infrastructure via CI/CD and IaC (CloudFormation, Terraform, GitHub Actions). Skilled building metadata-driven, incremental-load pipelines with watermarking, CDC, and SCD Type 1/2. Hands-on with Snowflake (Snowpipe, Streams, Tasks, Time Travel, Zero-Copy Cloning) and Spark/PySpark over Parquet, Delta Lake, and Apache Iceberg. Experienced in data quality, reconciliation, event-driven architectures, data security/governance, dimensional modeling, and delivering Power BI/Tableau reporting while collaborating cross-functionally in Agile/Scrum.
Technical Skills
Work Experience
- Built metadata-driven, incremental-load pipelines into the AWS lakehouse using Python, PySpark, AWS Glue, and the Glue Data Catalog, applying watermarking, CDC, and SCD Type 1/2 to keep data products current without full reloads.
- Developed and tuned PySpark jobs on AWS Glue, EMR, and Databricks over Parquet, Apache Iceberg, and Delta Lake tables, optimizing with partitioning, predicate pushdown, and adaptive query execution to hold refresh SLAs.
- Authored AWS CloudFormation templates driving CI/CD to provision AWS-native resources, including Lambda-backed custom resources for provisioning steps not natively supported by CloudFormation.
- Engineered event-driven workloads across multiple AWS regions using Lambda, Step Functions, and EventBridge, increasing reliability and business continuity during a regional AWS outage.
- Implemented Python and SQL data quality validation, reconciliation, completeness, freshness, and schema-drift checks that gate publication so downstream consumers only subscribe to certified datasets.
- Delivered curated marts, secure views, and semantic layers in Athena and Redshift, powering Power BI and Tableau reporting for analytical partners and business stakeholders.
- Developed metadata-driven Python ETL frameworks to extract, transform, validate, and load payment transaction, merchant, and account data into Snowflake, reducing build effort for each new source.
- Built out Snowflake environments end to end (databases, schemas, tables, stages, file formats, virtual warehouses) supporting high-volume card authorization and settlement analytics.
- Automated continuous and incremental ingestion of settlement and reconciliation files using Snowpipe, Streams, Tasks, and Python, and used Sqoop to sync high-volume tables into HDFS and Hive.
- Designed Spark and PySpark applications on Hadoop, HDFS, and Databricks to process high-volume daily transaction data, optimizing with partitioning, caching, and broadcast joins.
- Built SQL-based data quality frameworks to detect duplicate, null, orphan, and out-of-balance records, and implemented reconciliation routines comparing transaction counts and amounts source to target.
- Enforced Snowflake security using RBAC, secure views, dynamic masking of cardholder and PII fields, and row access policies, supporting PCI-aligned handling of sensitive payment data.
- Analyzed large-scale transaction time-series and customer correspondence data using SQL and Excel to surface trends supporting account-activity monitoring and service-request classification.
- Performed data cleaning, wrangling, and exploratory analysis with Python (Pandas, NumPy) and SQL across MySQL, SQL Server, and Oracle banking datasets.
- Built Tableau and Power BI dashboards summarizing unusual-activity trends and customer complaint and inquiry classification metrics for operations and risk stakeholders.
- Applied regression, clustering, and TF-IDF text analysis to support transaction anomaly detection and customer-inquiry categorization, tracking model performance KPIs (precision, recall, F1-score) for stakeholder review.
- Partnered with data engineers and domain experts to gather requirements, document reporting specifications, and support early-stage pipeline design using PySpark on Azure Databricks.
- Fulfilled ad hoc data pull and reporting requests for cross-functional operations and risk teams across high-volume banking datasets.
Education
Powered by Drivetube · Create your own profile at drivetube.ai
Everything on Drivetube
Six products, two of them free forever. Start wherever you are.
Your free digital resume
at drivetube.ai/in/your-name: one true standard resume with a Hiring Snapshot (visa status, expected salary, notice period, work preference, relocation), an ATS-ready PDF download and a single shareable link. Free forever; interview requests come from verified employers and your contact details stay masked until you accept.
Documentation- Free Job BoardFree
Verified jobs, posted in the last 3 days
verified openings crawled ATS-by-ATS from 115,000+ real company career pages across 54 ATS platforms. Shows the true posting date from the source ATS — not when a listing was indexed — and deletes every general listing 3 days after it was actually posted. No ghost jobs, no ad-sponsored listings, no staffing reposts, no account needed.
Documentation - Resume Writing ServicesFrom $25.99
Written by senior career writers
human-written, ATS-optimised resumes by senior career writers, from ₹499.99 / $25.99. Available in every country, written to the destination country's own standard — a US resume, UK CV, German Lebenslauf and Indian resume are genuinely different documents. A paid service, separate from the free Drivetube Profile.
Documentation - Drivetube OneFrom $39.99 / 3 mo
Daily matched jobs, tailored resumes, or we apply for you
Drivetube's paid service. Community sends daily Jobsheets of fresh jobs matched to your profile on WhatsApp and email, with the Drivetube Community ($39.99 for 3 months or $69.99 for 6 months; ₹999 or ₹1,899 in India). Jobhunt is one-time application credits with Community included: one credit tailors one resume and its cover letter (100 for $99 with 3 months of Community, 250 for $199 with 6 months). Let Us Apply is an add-on to Jobhunt: a human expert applies 250 times for you on your credits ($0.60 each, $150). Everything lives on one dashboard at drivetube.ai/drivetubeone.
Documentation - Drivetube HireFree tier
For employers — no job postings, no applications
hiring with no job postings and no applications. Paste your real job description and AI matches it against candidates' true standard resumes, returning ranked candidates with a match %, matched and missing skills and written reasoning. Free tier included; employers pay, candidates never do.
Documentation
- Documentation
every product explained in full, with feature-by-feature comparisons against the job boards, AI apply tools, resume services and hiring platforms people actually use.
- Playbooks
tactical job-search guides. Each post is one named tactic with the situation it applies to, the exact steps, a copy-paste message and honest failure modes. Free, no account.
The job board covers the United States, India, United Kingdom, Canada, Europe and Australia, and Resume Writing Services are available in every country.
Pricing is identical on the web, iOS and Android — no app-store markup. Monthly Community membership is purchased in the iOS app only; the website sells annual plans. The Android app has no in-app purchase, so Android members subscribe on drivetube.ai and then sign in to the app with full access. Membership follows your account, not your device.