Senior Data Engineer

Job Description

• Job title: Senior Data Engineer
• Experience: 5-20 Years
• Location: Glendale, USA
• Job Type: Contract, On-site

Must Haves:

• Candidates must be willing to work on W2, no C2C.
• 5+ years of data engineering experience specifically developing large-scale data pipelines.
• Databricks and Python expertise.

Qualifications:

• 5+ years of data engineering experience developing large data pipelines
• Proficiency in at least one major programming language.
• Expertise on databricks and Python.
• Hands-on production environment experience with distributed processing systems such as Spark
• Hands-on production experience with data pipeline orchestration systems such as Airflow for creating and maintaining data pipelines
• Experience with at least one major Massively Parallel Processing (MPP) or cloud database technology (Snowflake, Databricks, Big Query).
• Experience in developing APIs with GraphQL
• Advanced understanding of OLTP vs OLAP environment.
• Strong background in at least one of the following: distributed data processing or software engineering of data services, or data modeling

Key Responsibilities:

• Contribute to maintaining, updating, and expanding existing Core Data platform data pipelines.
• Build and maintain APIs to expose data to downstream applications.
• Develop real-time streaming data pipelines.
• Tech stack includes Airflow, Spark, Databricks, Delta Lake, and Snowflake.
• Collaborate with product managers, architects, and other engineers to drive the success of the Core Data platform.
• Contribute to developing and documenting both internal and external standards and best practices for pipeline configurations, naming conventions, and more.
• Ensure high operational efficiency and quality of the Core Data platform datasets to ensure our solutions meet SLAs and project reliability and accuracy to all our stakeholders.