Data Engineer with Azure

Job Title: Azure Data Engineer (Azure to AWS Migration) Spark, Kafka & Databricks
Location: Dallas TX
Job Type: Contract
Work Arrangement: 5 Days Onsite
Domain: Telecom
Interview : 2-3 rounds, Final round in person required
Note : US Citizenship Required for this role

Job Summary
We are seeking an experienced Azure Data Engineer with strong expertise in Apache Spark, Apache Kafka, Azure Databricks , and cloud migration projects , specifically migrating data platforms from Microsoft Azure to AWS . The ideal candidate will have extensive experience building scalable data pipelines, streaming solutions, and modern data lake architectures while driving cloud transformation initiatives.
This role requires hands-on experience with Azure services, Databricks, Spark optimization, Kafka event streaming, and AWS data services to support enterprise-scale migration efforts.

• Design, develop, and maintain scalable batch and real-time data pipelines.
• Lead migration of enterprise data platforms from Microsoft Azure to Amazon Web Services (AWS) .
• Develop and optimize ETL/ELT pipelines using Azure Databricks and Apache Spark .
• Build and maintain real-time streaming applications using Apache Kafka .
• Migrate data lakes, data warehouses, and analytics workloads from Azure to AWS.
• Optimize Spark jobs for performance, scalability, and cost efficiency.
• Work closely with solution architects to design cloud-native AWS data architectures.
• Implement CI/CD pipelines for data engineering deployments.
• Ensure data quality, governance, monitoring, and security throughout migration activities.
• Collaborate with business stakeholders, data scientists, and application teams to deliver reliable data solutions.
• Troubleshoot production issues and optimize distributed data processing environments.

• Microsoft Azure
• Amazon Web Services (AWS)
• Azure to AWS Migration experience (Mandatory)

• 8 years of Data Engineering experience.
• 4 years of hands-on experience with Azure Databricks .
• Strong expertise in Apache Spark and PySpark .
• Hands-on experience with Apache Kafka and streaming architectures.
• Proven experience in Azure to AWS cloud migration projects .
• Experience designing enterprise-scale data lake solutions.
• Strong SQL and Python programming skills.
• Experience with Delta Lake and modern Lakehouse architecture.
• Knowledge of CI/CD and Infrastructure as Code.
• Excellent communication and stakeholder management skills.

• AWS Certified Data Engineer Associate or AWS Certified Solutions Architect.
• Microsoft Azure Data Engineer Associate (DP-203).
• Databricks Certified Data Engineer Associate/Professional.
• Experience with Snowflake, Iceberg, or Apache Hudi.
• Experience with Airflow or other orchestration tools.