Durga Priya Vanam

Herndon

Summary

Accomplished Big Data Engineer with expertise in AWS and Python, previously at Freddie Mac. Enhanced data pipeline efficiency and reliability, achieving a 25% cost reduction. Proficient in Spark and SQL, with strong collaboration skills. Adept at transforming complex requirements into scalable solutions, driving innovation and performance in data engineering projects.

Overview

years of professional experience

Work History

Big Data Engineer

Freddie Mac

McLean

02.2023 - Current

Designed and operated a high-throughput, fault-tolerant data pipeline processing large-scale datasets using PySpark, Python, and AWS cloud ETL services.
Developed scalable data ingestion and processing workflows using AWS Glue, Lambda, Step Functions, S3, and DynamoDB.
Implemented partitioning, bucketing, and performance-optimized Hive/Spark SQL queries to support fast data access at scale.
Established monitoring and alerting using CloudWatch, Splunk, and Elastic, improving observability and reducing MTTR for data incidents.
Operated and maintained AWS EKS-based infrastructure, leveraging Splunk logging and monitoring to proactively identify issues, improve observability, and support reliable data platform operations.
Optimized AWS EKS infrastructure costs by ~25–30% through resource right-sizing, pod autoscaling, and log retention tuning, while maintaining system reliability and performance.
Developed and maintained unit tests, regression tests, and integration tests to ensure data quality and pipeline reliability.
Participated in peer code reviews, offering constructive feedback to uphold coding standards and promote team learning.

Data Engineer

Accenture

Hyderabad

11.2018 - 07.2021

Developed and optimized Python-based data processing scripts and Hive/SQL transformations for complex ETL workflows, enhancing large-scale data processing capabilities.
Led migration of data from legacy systems and NAS storage into centralized data platforms, resulting in a 25% improvement in query performance and data accessibility.
Played a key role in designing and implementing enterprise-scale EDW and ODS platforms, supporting high-volume analytical and reporting workloads.
Built reusable Python and shell-based automation to orchestrate data ingestion, validation, and transformation tasks, increasing pipeline reliability and operational efficiency.
Authored complex Hive queries and SQL stored procedures to support downstream analytics and business intelligence use cases.
Implemented CI/CD pipelines using Git and Bitbucket to automate build and deployment of data workflows.
Collaborated with cross-functional client teams, including data analysts and application engineers, to translate business requirements into scalable data engineering solutions.

Education

Master of Science - Computer Science

University of Missouri

Kansas City, MO

12.2022

Bachelor of Science - Electrical Engineering

Kakatiya University College of Engineering & Technology

India

01.2018

Skills

Programming: Python
Big Data Technologies: Apache Spark, Spark-SQL, Hadoop, Hive
Database: SQL & NoSQL (SQL Server, DB2, MongoDB)
Job Scheduling: Control-M, Autosys
Reporting Tools: Tableau
Data modeling

ETL Tools: Talend, Attunity, IICS
Agile: Jira
Cloud Services: AWS (S3, EC2, DynamoDB, Glue, CloudWatch, Step Functions, Lambda, DataSync, API Gateway)
Logging and Monitoring: Kibana, Elasticsearch, Splunk
Data Warehouses: Snowflake

Timeline