Lakshmi Narasimha
Professional Summary
Data Engineer (AI/ML) with 4+ years of experience designing and shipping production-grade data pipelines, real-time lakehouse platforms, and ML systems. Experienced across telecom, healthcare, financial services, and manufacturing; skilled with Azure Databricks, Apache Spark, Delta Lake/Iceberg, Snowflake, Collibra, MLflow, RAG/FAISS pipelines, Terraform, Kubernetes, and CI/CD to deliver governed, analytics-ready data and deployed ML services.
Technical Skills
Work Experience
- Designed and implemented a production lakehouse on Azure Databricks using Apache Iceberg for ACID transactions, schema evolution, and time-travel querying — replaced an aging batch warehouse and reduced data availability latency from 24 hours to under 45 minutes.
- Built distributed Spark (PySpark, Spark SQL) transformation workflows processing 100M+ records daily across retail and pharmacy datasets; applied partitioning, broadcast joins, and caching to meet performance SLAs under high volume.
- Led end-to-end development and deployment of a binary classification churn model (98% ROC-AUC) including feature engineering, hyperparameter tuning, MLflow experiment tracking and model registry, and FastAPI-based REST serving for live dashboard consumption.
- Implemented a retrieval-augmented generation (RAG) search pipeline: chunked and embedded internal documentation, stored vectors in FAISS, and integrated LLM APIs to deliver instant Q&A for operations teams, replacing 30+ minutes of manual search.
- Integrated model explainability (SHAP, LIME) and automated data quality checks (Great Expectations) to meet compliance and audit requirements across feature and serving layers.
- Instrumented pipeline observability and alerting using Amazon CloudWatch custom metrics and freshness dashboards; standardized deployments with Docker and GitHub Actions CI/CD across dev, staging, and production.
- Engineered enterprise ELT pipelines in Python, SQL, and Talend processing 20M+ records per batch; automated workflows with Azure Data Factory and Prefect to reduce manual batch effort by 50% across 20+ managed workflows.
- Designed a reusable REST API-based source onboarding framework to standardize ingestion patterns for third-party vendors and internal teams, reducing average onboarding time from weeks to days.
- Implemented enterprise data governance using Collibra for business glossary/policy management and Alation for cataloging and lineage, providing audit teams a single governed view across 20+ pipelines.
- Built feature engineering pipelines in Python (pandas, NumPy) and SQL to produce model-ready datasets; ensured reproducibility and versioning for downstream ML training runs.
- Provisioned cloud infrastructure with Terraform using environment-specific configurations; containerized ETL workloads for Kubernetes and integrated pipelines into GitLab and Jenkins CI/CD for zero-touch deployments.
- Implemented Azure Monitor dashboards and alerting across pipelines, cutting average troubleshooting time by ~40% by surfacing anomalies before downstream consumers were impacted.
- Built Python and SQL ETL pipelines extracting from Oracle, MySQL, and flat files into Amazon S3 as Parquet to create a schema-consistent landing zone for downstream Snowflake models.
- Designed Snowflake warehouse (fact/dimension tables, surrogate key strategy, incremental loads) enabling BI teams to migrate from ad hoc queries to structured self-service reporting.
- Orchestrated batch jobs using Apache Airflow DAGs with dependency management, retry logic, and SLA alerting to replace undocumented manual scripts and eliminate on-call failures.
- Integrated AWS Glue for centralized data cataloging and schema discovery; developed serverless preprocessing with AWS Lambda for lightweight event-driven transforms.
- Performed query profiling and optimized Snowflake SQL using clustering keys, materialized views, and result caching to reduce reporting query times on core dashboards.
- Established data landing, transformation, and publishing patterns with documentation and versioning to ensure model-ready datasets for analytics consumers.
- Developed Informatica PowerCenter mappings and sessions to extract SAP S/4HANA transactional data (via ABAP programs) into staging, integration HUB, and data mart layers, delivering audit-documented data for manufacturing analytics.
- Implemented complex Informatica transformations including unconnected lookups, aggregators, and routers to handle merging, deduplication, and conditional routing across 10+ source feeds.
- Built reusable mapplets for Row ID generation, batch control flags, and standard audit columns used across mapping teams to reduce development duplication and accelerate delivery.
- Automated job scheduling using Workflow Manager with Command and Decision tasks, created ControlM dependency diagrams in Visio, and authored UNIX shell scripts for parameterization and environment switching.
- Led requirement analysis with business and technical stakeholders to define ETL specifications, error handling, and validation rules; produced documentation that became the team standard.
- Generated data validation reports during system and UAT testing, performed root-cause analysis on ETL defects, and maintained traceability (daily load logs, documentation) to ensure reliable data mart delivery.
Education
Powered by Drivetube · Create your own profile at drivetube.ai
Everything on Drivetube
Six products, two of them free forever. Start wherever you are.
Your free digital resume
at drivetube.ai/in/your-name: one true standard resume with a Hiring Snapshot (visa status, expected salary, notice period, work preference, relocation), an ATS-ready PDF download and a single shareable link. Free forever; interview requests come from verified employers and your contact details stay masked until you accept.
Documentation- Free Job BoardFree
Verified jobs, posted in the last 3 days
verified openings crawled ATS-by-ATS from 115,000+ real company career pages across 54 ATS platforms. Shows the true posting date from the source ATS — not when a listing was indexed — and deletes every general listing 3 days after it was actually posted. No ghost jobs, no ad-sponsored listings, no staffing reposts, no account needed.
Documentation - Resume Writing ServicesFrom $25.99
Written by senior career writers
human-written, ATS-optimised resumes by senior career writers, from ₹499.99 / $25.99. Available in every country, written to the destination country's own standard — a US resume, UK CV, German Lebenslauf and Indian resume are genuinely different documents. A paid service, separate from the free Drivetube Profile.
Documentation - Drivetube OneFrom $39.99 / 3 mo
Daily matched jobs, tailored resumes, or we apply for you
Drivetube's paid service. Community sends daily Jobsheets of fresh jobs matched to your profile on WhatsApp and email, with the Drivetube Community ($39.99 for 3 months or $69.99 for 6 months; ₹999 or ₹1,899 in India). Jobhunt is one-time application credits with Community included: one credit tailors one resume and its cover letter (100 for $99 with 3 months of Community, 250 for $199 with 6 months). Let Us Apply is an add-on to Jobhunt: a human expert applies 250 times for you on your credits ($0.60 each, $150). Everything lives on one dashboard at drivetube.ai/drivetubeone.
Documentation - Drivetube HireFree tier
For employers — no job postings, no applications
hiring with no job postings and no applications. Paste your real job description and AI matches it against candidates' true standard resumes, returning ranked candidates with a match %, matched and missing skills and written reasoning. Free tier included; employers pay, candidates never do.
Documentation
- Documentation
every product explained in full, with feature-by-feature comparisons against the job boards, AI apply tools, resume services and hiring platforms people actually use.
- Playbooks
tactical job-search guides. Each post is one named tactic with the situation it applies to, the exact steps, a copy-paste message and honest failure modes. Free, no account.
The job board covers the United States, India, United Kingdom, Canada, Europe and Australia, and Resume Writing Services are available in every country.
Pricing is identical on the web, iOS and Android — no app-store markup. Monthly Community membership is purchased in the iOS app only; the website sells annual plans. The Android app has no in-app purchase, so Android members subscribe on drivetube.ai and then sign in to the app with full access. Membership follows your account, not your device.