ABHILASH RAJAN
Data Engineer | Project Delivery Specialist (AWS & Snowflake)
Richmond
Summary
Accomplished Senior Data Engineer with over 11 years of experience in designing, building, and optimizing scalable data pipelines that integrate diverse data sources into analytics platforms using AWS cloud-native technologies. Expertise in managing small to medium projects end-to-end, standardizing reusable ETL processes with minimal customization, and leveraging AI-assisted tools such as Amazon Bedrock and OpenAI to enhance ETL optimization and pipeline intelligence. Extensive hands-on experience with AWS Glue, Lambda, EMR, PySpark, Python, Java, Snowflake, and Redshift for both batch processing and serverless architectures. Strong foundation in CI/CD practices utilizing Terraform and Jenkins, along with a commitment to security protocols and Agile methodologies, ensures the delivery of high-impact data solutions that empower informed business decisions.
Overview
11
11
years of professional experience
1
1
Certification
3
3
Languages
Work History
Lead Data Engineer
Bitwise Solutions Inc
11.2024 - Current
- Designed, developed, and unit-tested highly scalable AWS Glue ETL applications using PySpark and Python to integrate data from multiple sources into Snowflake analytics platform, creating standardized, reusable pipelines with minimal customization.
- Leveraged AI agents (AWS Bedrock and OpenAI) to generate synthetic data, automate documentation and code quality checks, and enhance data cataloguing and pipeline intelligence within Snowflake workflows.
- Built Java-based components and utilities supporting robust ETL pipelines and seamless integration with Snowflake data warehouse.
- Led migration of legacy datasets to AWS cloud, performed package/library compatibility upgrades, and standardized processes for reusability across upcoming use cases.
- Migrated internal scheduling and monitoring processes from AROW to REDWOOD, replicated complex workflows, tested in QA/PROD/PARALLEL environments, and maintained production Lambdas and scripts.
- Automated manual team processes with Python and PySpark tools, significantly improving efficiency and enabling self-service data operations.
- Evaluated and integrated AWS Bedrock for automated documentation and AI-enhanced process insights during migrations, supporting future GenAI applications in production.
- Client Location: VA, USA
Lead AWS Data Engineer
Bitwise Solutions Pvt. Ltd.
04.2021 - 10.2024
- Led POC research and evaluation of AWS services (Glue, Lambda, EMR, Step Functions) for migrating legacy data replication; gained stakeholder consensus on a generic, standardized architecture reusable across multiple use cases.
- Engineered end-to-end serverless data pipelines using AWS Lambda, Glue, stored procedures, and metadata-driven processing in Step Functions, delivering automated extraction, transformation, and loading from source to target (Redshift/Postgres).
- Developed RESTful APIs to trigger orchestrated workflows and integrated scheduling with Stonebranch and EventBridge for reliable, highly available batch data pipelines.
- Implemented Infrastructure-as-Code with Terraform and full CI/CD pipelines using Jenkins and Git for consistent, repeatable deployments and security governance.
- Conducted unit, integration, and system testing; provided production support, performed root-cause analysis, and optimized performance using CloudWatch and X-Ray to ensure robustness and error handling.
- Mentored junior engineers on AWS data engineering best practices, cloud architecture, and Agile processes; facilitated daily stand-ups, sprint planning, and stakeholder demos.
- Designed robust metadata management and security controls (IAM roles, encryption, PII handling) to protect business-sensitive data while maintaining scalability.
- Designed, developed, and performance-tuned Ab Initio ETL graphs (GDE) for large-scale data processing; created parameterized generic graphs using partitioning, lookups, in-memory joins, and rollups to standardize logic and accelerate delivery.
Senior Software Developer
Atos Syntel Pvt Ltd
01.2015 - 04.2021
- Created Python-based automation tools and web applications (Django/Flask) with MySQL and Oracle databases to eliminate manual processes, including a defect-reporting tool that saved 462 hours annually.
- Developed reusable Python libraries, parsers, and data-analysis solutions using Pandas and Matplotlib to process large datasets and deliver graphical insights.
- Designed, deployed, and supported 300+ production code changes while maintaining high security standards and resolving downstream issues.
- Built web-automation scripts with Selenium and Pyppeteer and implemented comprehensive exception handling for robust, maintainable solutions.
- Collaborated in Agile teams, participating in stand-ups, demos, and requirements sessions with clients and quantitative analysts.
Education
Bachelor of Electrical and Electronics Engineering -
Gyan Ganga College of Technology, RGPV
Bhopal, MP, India01.2014
Higher Secondary Certificate (H.S.C.) - undefined
CBSE Board
India01.2010
Secondary School Certificate (S.S.C.) - undefined
CBSE Board
India01.2008
Skills
- Cloud analytics expertise
- Data pipeline development
- Python and Pyspark programming
- Database management: Snowflake, Redshift, Postgres, Oracle, MySQL, DynamoDB
Certification
Certified Scrum Master (CSM) – Scrum Alliance
Timeline
Lead Data Engineer
Bitwise Solutions Inc
11.2024 - Current
Lead AWS Data Engineer
Bitwise Solutions Pvt. Ltd.
04.2021 - 10.2024
Senior Software Developer
Atos Syntel Pvt Ltd
01.2015 - 04.2021
Higher Secondary Certificate (H.S.C.) - undefined
CBSE Board
Secondary School Certificate (S.S.C.) - undefined
CBSE Board
Bachelor of Electrical and Electronics Engineering -
Gyan Ganga College of Technology, RGPV