Lakshmi Narasimha
Professional Summary
Data Engineer (AI/ML) with 4+ years of experience designing and shipping production-grade data pipelines, real-time lakehouse platforms, and ML systems. Experienced across telecom, healthcare, financial services, and manufacturing; skilled with Azure Databricks, Apache Spark, Delta Lake/Iceberg, Snowflake, Collibra, MLflow, RAG/FAISS pipelines, Terraform, Kubernetes, and CI/CD to deliver governed, analytics-ready data and deployed ML services.
Technical Skills
Work Experience
- Designed and implemented a production lakehouse on Azure Databricks using Apache Iceberg for ACID transactions, schema evolution, and time-travel querying — replaced an aging batch warehouse and reduced data availability latency from 24 hours to under 45 minutes.
- Built distributed Spark (PySpark, Spark SQL) transformation workflows processing 100M+ records daily across retail and pharmacy datasets; applied partitioning, broadcast joins, and caching to meet performance SLAs under high volume.
- Led end-to-end development and deployment of a binary classification churn model (98% ROC-AUC) including feature engineering, hyperparameter tuning, MLflow experiment tracking and model registry, and FastAPI-based REST serving for live dashboard consumption.
- Implemented a retrieval-augmented generation (RAG) search pipeline: chunked and embedded internal documentation, stored vectors in FAISS, and integrated LLM APIs to deliver instant Q&A for operations teams, replacing 30+ minutes of manual search.
- Integrated model explainability (SHAP, LIME) and automated data quality checks (Great Expectations) to meet compliance and audit requirements across feature and serving layers.
- Instrumented pipeline observability and alerting using Amazon CloudWatch custom metrics and freshness dashboards; standardized deployments with Docker and GitHub Actions CI/CD across dev, staging, and production.
- Engineered enterprise ELT pipelines in Python, SQL, and Talend processing 20M+ records per batch; automated workflows with Azure Data Factory and Prefect to reduce manual batch effort by 50% across 20+ managed workflows.
- Designed a reusable REST API-based source onboarding framework to standardize ingestion patterns for third-party vendors and internal teams, reducing average onboarding time from weeks to days.
- Implemented enterprise data governance using Collibra for business glossary/policy management and Alation for cataloging and lineage, providing audit teams a single governed view across 20+ pipelines.
- Built feature engineering pipelines in Python (pandas, NumPy) and SQL to produce model-ready datasets; ensured reproducibility and versioning for downstream ML training runs.
- Provisioned cloud infrastructure with Terraform using environment-specific configurations; containerized ETL workloads for Kubernetes and integrated pipelines into GitLab and Jenkins CI/CD for zero-touch deployments.
- Implemented Azure Monitor dashboards and alerting across pipelines, cutting average troubleshooting time by ~40% by surfacing anomalies before downstream consumers were impacted.
- Built Python and SQL ETL pipelines extracting from Oracle, MySQL, and flat files into Amazon S3 as Parquet to create a schema-consistent landing zone for downstream Snowflake models.
- Designed Snowflake warehouse (fact/dimension tables, surrogate key strategy, incremental loads) enabling BI teams to migrate from ad hoc queries to structured self-service reporting.
- Orchestrated batch jobs using Apache Airflow DAGs with dependency management, retry logic, and SLA alerting to replace undocumented manual scripts and eliminate on-call failures.
- Integrated AWS Glue for centralized data cataloging and schema discovery; developed serverless preprocessing with AWS Lambda for lightweight event-driven transforms.
- Performed query profiling and optimized Snowflake SQL using clustering keys, materialized views, and result caching to reduce reporting query times on core dashboards.
- Established data landing, transformation, and publishing patterns with documentation and versioning to ensure model-ready datasets for analytics consumers.
- Developed Informatica PowerCenter mappings and sessions to extract SAP S/4HANA transactional data (via ABAP programs) into staging, integration HUB, and data mart layers, delivering audit-documented data for manufacturing analytics.
- Implemented complex Informatica transformations including unconnected lookups, aggregators, and routers to handle merging, deduplication, and conditional routing across 10+ source feeds.
- Built reusable mapplets for Row ID generation, batch control flags, and standard audit columns used across mapping teams to reduce development duplication and accelerate delivery.
- Automated job scheduling using Workflow Manager with Command and Decision tasks, created ControlM dependency diagrams in Visio, and authored UNIX shell scripts for parameterization and environment switching.
- Led requirement analysis with business and technical stakeholders to define ETL specifications, error handling, and validation rules; produced documentation that became the team standard.
- Generated data validation reports during system and UAT testing, performed root-cause analysis on ETL defects, and maintained traceability (daily load logs, documentation) to ensure reliable data mart delivery.
Education
Powered by Drivetube · Create your own profile at drivetube.ai
Everything on Drivetube
Six products, two of them free forever. Start wherever you are.
Your free digital resume
at drivetube.ai/in/your-name: one true standard resume with a Hiring Snapshot (visa status, expected salary, notice period, work preference, relocation), an ATS-ready PDF download and a single shareable link. Free forever; interview requests come from verified employers and your contact details stay masked until you accept.
Documentation- Free Job BoardFree
Verified jobs, posted in the last 3 days
verified openings crawled ATS-by-ATS from 100,000+ real company career pages across 35 ATS platforms. Shows the true posting date from the source ATS — not when a listing was indexed — and deletes every general listing 3 days after it was actually posted. No ghost jobs, no ad-sponsored listings, no staffing reposts, no account needed.
Documentation - Job Hunt ProgramFrom $199.99
We run your job hunt for you
managed job hunting, a one-time purchase from $199.99. JobScout matches verified roles to your real experience band, Blend AI writes a uniquely tailored resume and cover letter for every application, and the Autofill extension fills the form — or Let Us Apply submits it for you.
Documentation - Resume Writing ServicesFrom $25.99
Written by senior career writers
human-written, ATS-optimised resumes by senior career writers, from ₹499.99 / $25.99. Available in every country, written to the destination country's own standard — a US resume, UK CV, German Lebenslauf and Indian resume are genuinely different documents. A paid service, separate from the free Drivetube Profile.
Documentation - Community MembershipFrom $49.99/yr
AI tools, gated filters and a $10,000+ library
from $49.99/year (₹1,999/year in India), or $4.99/month in the iOS app. Unlocks the gated job-board filters — visa sponsorship, security clearance, workplace and application time — plus Job-Scout AI matching, Resume Report AI, Interview AI prep sheets, Recruiter Outreach AI sent from your own Gmail, Apply or Skip triage, a daily market feed and a $10,000+ library including 23 ATS-validated resume templates.
Documentation - Drivetube HireFree tier
For employers — no job postings, no applications
hiring with no job postings and no applications. Paste your real job description and AI matches it against candidates' true standard resumes, returning ranked candidates with a match %, matched and missing skills and written reasoning. Free tier included; employers pay, candidates never do.
Documentation
- Documentation
every product explained in full, with feature-by-feature comparisons against the job boards, AI apply tools, resume services and hiring platforms people actually use.
- Playbooks
tactical job-search guides. Each post is one named tactic with the situation it applies to, the exact steps, a copy-paste message and honest failure modes. Free, no account.
The job board covers the United States, India, United Kingdom, Canada, Europe and Australia, and Resume Writing Services are available in every country.
Pricing is identical on the web, iOS and Android — no app-store markup. Monthly Community membership is purchased in the iOS app only; the website sells annual plans. The Android app has no in-app purchase, so Android members subscribe on drivetube.ai and then sign in to the app with full access. Membership follows your account, not your device.