Welcome, I am

Nishita Pendyala

Data Engineer

Building enterprise-scale data solutions that drive innovation

Get In Touch View My Work
Nishita Pendyala

About Me

I am a Data Engineer with 5+ years of experience designing and implementing enterprise-scale data solutions across telecommunications, retail, and technology sectors. My expertise spans cloud platforms (AWS, Azure), modern data warehousing technologies (Snowflake, Teradata, Redshift), and end-to-end ETL/ELT pipeline development.

I specialize in building automated data pipelines that process millions of records daily with exceptional accuracy, designing fault-tolerant architectures, and implementing comprehensive data governance frameworks. I'm passionate about optimizing cloud infrastructure, implementing CI/CD automation, and ensuring regulatory compliance.

5+
Years Experience
50+
ETL Workflows Built
99.9%
Data Accuracy
85%
Anomaly Reduction

Technical Skills

SQL Technologies

Teradata
SQL Server
Oracle DB
Snowflake
Amazon Redshift
Azure Synapse

Languages

SQL
Python
PL/SQL
Unix Shell Scripting
DAX & M Query

ETL Tools

Informatica PowerCenter
AWS Glue
PySpark
Apache Spark
Power BI Dataflows

Big Data & Storage

Databricks
Elasticsearch
Great Expectations
Collibra

IDE & Tools

Databricks
Elasticsearch
Great Expectations
JDE ERP
Control-M

Version Control

Git
GitHub Actions
U-Release
U-Deploy
JIRA

Cloud Technologies

AWS S3, Lambda, Glue
Step Functions, CloudWatch
KMS, SageMaker, CodePipeline
CloudFormation
Azure Synapse, ML, Key Vault

Visualization & Analytics

Power BI
Advanced Dashboards
DAX Calculations
Row-Level Security

Data Governance & Security

Data Governance
Encryption (KMS, Key Vault)
Data Lineage
Regulatory Compliance

Professional Experience

Data Engineer at Lowe's

  • Designed and implemented a fully automated end-to-end invoice processing pipeline with 300% increase in daily processing capacity (5-30 to 90-100 invoices/day) using AWS (S3, Lambda, Glue) and Informatica PowerCenter.
  • Integrated supplier data from secure SFTP servers into JDE ERP system (Tables F0411, F0911) with automated validation and transformation.
  • Engineered event-driven AWS Lambda functions for automated data validation, transformation, and error handling with comprehensive retry mechanisms.
  • Implemented fault-tolerant data pipelines using AWS Step Functions to orchestrate workflows across multiple services with robust error management.
  • Engineered advanced Python scripts with Pandas and AWS Boto3 to process complex multi-header invoice structures efficiently.
  • Leveraged AWS KMS for encryption and secure handling of sensitive financial data, ensuring compliance with data protection standards.
  • Built modern CI/CD pipelines using GitHub Actions and AWS CodePipeline with Infrastructure as Code (IaC) using AWS CloudFormation.
  • Developed comprehensive monitoring solutions using AWS CloudWatch, Elasticsearch, and AWS SNS for real-time alerting and proactive issue resolution.
  • Spearheaded migration of legacy ETL workflows to modern cloud-based architecture with AWS Glue and Snowflake, reducing infrastructure costs significantly.

Data Engineer at Verizon

  • Architected enterprise-scale telecom data lake migration across AWS and Azure platforms for network performance monitoring and reporting.
  • Implemented secure Snowflake reporting solutions for telecom analytics, ensuring scalability and cost-efficiency at enterprise scale.
  • Developed advanced Power BI dashboards for critical telecom metrics including network monitoring, Call Data Record (CDR) processing, and customer churn analytics.
  • Implemented comprehensive data governance frameworks using Collibra with accurate data lineage, compliance tracking, and regulatory adherence.
  • Engineered robust ETL processes using PySpark and Power BI Dataflows for complex telecom dataset transformations and analysis.
  • Developed sophisticated DAX calculations and created advanced T-SQL/M Query transformations for complex telecom data preparation.
  • Implemented stringent row-level security in Power BI to protect sensitive subscriber information (SPI) across reporting platforms.
  • Migrated telecom analytical workloads between Azure Synapse Analytics and Amazon Redshift, achieving 40% reduction in computational resource utilization.
  • Developed machine learning pipelines for customer churn prediction using AWS SageMaker and Azure ML, integrated with Power BI analytics.
  • Implemented cross-platform encryption key management across Azure Key Vault and AWS KMS for enterprise data security.
  • Automated data quality validation using Great Expectations framework, reducing data anomalies by 85% in complex telecom datasets.
  • Optimized Spark configurations and SQL queries across multiple telecom data platforms for enhanced performance and cost efficiency.

Data Engineer at Cisco – Tech Mahindra

  • Led end-to-end ETL development lifecycle for enterprise data pipelines, including requirement gathering, technical design, and implementation using Informatica PowerCenter and Teradata SQL.
  • Designed and developed 50+ complex ETL workflows processing 10+ million records daily with 99.9% accuracy maintained through comprehensive validation frameworks.
  • Developed Python scripts for data validation, quality checks, and automated monitoring, reducing manual validation time by 60%.
  • Implemented data profiling and reconciliation frameworks to ensure data integrity across source and target systems with daily Control-M dry runs.
  • Optimized complex SQL queries and Informatica mappings, improving ETL performance by 40% and reducing overnight batch processing time.
  • Orchestrated systematic code deployment across dev/test/prod environments using Git repositories, U-Release, and U-Deploy for seamless multi-environment deployments.
  • Implemented robust version control practices with Git, including branching strategies and meticulous code review ensuring zero production incidents.
  • Monitored ETL job execution using Control-M, proactively identifying and resolving job failures, dependencies, and performance bottlenecks.
  • Participated in production support rotation providing timely resolution for critical data issues with minimal business impact.
  • Mentored junior team members on ETL best practices, Informatica development standards, and Python scripting techniques.
  • Developed and maintained Unix shell scripts for file processing, data archival, and automated job scheduling integration.
  • Implemented data security and masking techniques to protect sensitive information in non-production environments ensuring compliance with data privacy regulations.
  • Contributed to continuous improvement initiatives, reducing manual effort by 45% through automation solutions and process optimization.

Let's Connect