Pallav Vyas
# [email protected] +91-6377699799 ï linkedin/pallavvyas § github/pallavvyas
Work Experience
ATRIUM INSIGHTS LLP
Data Engineer Consultant | July 2023 - Present
• Developed an in-depth understanding of Snowflake’s architecture, specializing in minimizing credit usage
during development phases through strategic optimization techniques.
• Led the development of Snowflake-based data marts using DBT, overseeing complex transformations to
streamline the data pipeline.
• Designed the Medallion Architecture to enhance data quality and created DBT Tests to rigorously validate
the entire data pipeline, reducing error rates by 25% across all levels.
• Tested 500 TB of raw source data using PyTest and automated the whole testing process to ensure
accuracy and reliability of data transformations.
• Conducted a comprehensive knowledge transfer session for the client’s technical team, covering the entire
code base and emphasizing its reusability principles.
• Conducted internal knowledge sharing sessions on DBT, Snowflake, and Git, facilitating team proficiency
in leveraging new technologies for enhanced project execution and collaboration.
Associate Data Engineer Consultant | July 2022 - June 2023
• Automated extraction and loading of data from client APIs into Snowflake tables using Python scripts,
optimizing efficiency and significantly reducing manual workload.
• Orchestrated the entire data pipeline using Apache Airflow, resulting in a 20-hour per week reduction in
processing time and enhanced operational accuracy.
• Converted significant functionality from PL/SQL to Snowflake’s Stored Procedures, enhancing efficiency
and integration.
• Engineered a scalable ELT pipeline for real-time data processing by leveraging Airbyte and DBT; increased
data processing efficiency by 40%.
• Curated Data Vault structures, including Hubs, Links, and Satellites, and utilized AutomateDV to automate
SQL generation, significantly reducing processing time and improving efficiency.
• Maintained open and effective communication with Product Owners to ensure seamless project alignment
and successful deliveries.
Data Engineer Intern | Jan 2022 - June 2022
• Mastered DBT concepts and implemented complex transformations using it, involving comprehensive
sales data and employee time tracking.
• Created built-in tests to ensure comprehensive coverage of large-scale datasets, ensuring data integrity
and accuracy.
Technical Skills
Languages: Python, SQL
Technologies: Snowflake, DBT(Data Build Tool), Airbyte, Apache Airflow, ELT, ETL, FiveTran
Tools: Git, VS Code
Project
HEALTHSURE Snowflake, DBT, FiveTran, Git
• Designed and implemented an ELT pipeline for the healthcare industry using DBT and FiveTran, integrating
advanced Snowflake features to ensure sensitive data masking and compliance with privacy regulations.
Education
BTech Computer Science Engineering CGPA: 8.4 | 2018 - 2022
Swami Keshvanand Institute of Technology
Coursework: Object Oriented Programming; Database Management System; Operating Systems; Data
Structures and Algorithms;
