Data Engineer - Databricks, Scala

Role: Data Engineer - Databricks, Scala
Skills: Digital: Scala, Digital: Databricks Experience Required: 8-10 Years

Databricks, Scala- Data Engineer

Must have atleast 8 years of extensive Data engineering experience in Databricks, ETL/ELT using data pipelines, SQL/Procesurr experience

Must have 8 years of hands on development proficiency in implementing Databricks solutions using scala and spark with data frames and notebooks/sql

Must have 8 years of hands on development and performance tuning/ enhancement experience in scala/pyspark and sql

Must have strong experience with Databricks, including developing and optimizing spark jobs, data transformations, and data processing workflows

Must have experience and a good understanding on the below topics

Databricks delta lake storage

Unity catalog

Autoloader

Lakehouse architecture

Medallion architecture

Databricks serverless options

Dataset and data frame

Databricks clusters configuration and sizing

Databricks job orchestration

Spark structured streaming