Databricks Architect

Databricks Architect - 14 years
Malvern PA
Senior Databricks Architect with strong experience in designing and implementing enterprise-scale Lakehouse architectures on cloud platforms. Expertise in Databricks, Spark, PySpark, Delta Lake, data modeling, and real-time data processing.

Databricks Architect, Databricks Lakehouse, Apache Spark, PySpark, Delta Lake, Unity Catalog, Structured Streaming, Kafka, Data Modeling, ETL, ELT, AWS, S3, Redshift, Glue, Airflow, Terraform, CI/CD, Data Governance, Lakehouse Architecture, Performance Tuning

• What is the Databricks Lakehouse architecture, and how is it different from a traditional Data Warehouse?
• What is Delta Lake, and what benefits do ACID transactions provide in Databricks?
• How have you optimized Spark/Databricks jobs for performance and cost?
• Can you explain the difference between Spark transformations and actions?
• What is Unity Catalog, and how have you used it for data governance and security?
• How would you design a real-time data ingestion pipeline in Databricks?
• Look for: Kafka, Structured Streaming, Delta Live Tables.
• What is your experience with AWS services integrated with Databricks?
• Look for: S3, Glue Catalog, IAM, Redshift, Lambda.