Databricks Architect - Tarrytown, NY [Hybrid]

Job Details:

Position: Databricks Architect
Location: Tarrytown, NY
Work Arrangement: Hybrid — 3 days onsite per week
Contract Duration: 1 year

Role Summary

We are looking for a strong Databricks Architect to lead enterprise-scale Lakehouse architecture and implementation on AWS. The role involves designing scalable data solutions, modernizing existing data platforms, establishing governance standards, and leading architecture discussions with customers.

Key Responsibilities

• Lead enterprise-scale Databricks Lakehouse architecture and implementation on AWS.
• Drive migration of EMR, NiFi, Redshift, and legacy ETL workloads to Databricks.
• Evaluate newer Databricks capabilities, including Serverless, Lakehouse Federation, Genie, Databricks Apps, Lakebase, and AI/ML.
• Architect scalable batch, streaming, CDC, and real-time data solutions.
• Design reusable frameworks using Delta Lake, PySpark, Auto Loader, DLT, and Databricks Workflows.
• Define governance, security, lineage, and data-quality standards using Unity Catalog.
• Establish best practices for performance, scalability, and cost optimization.
• Lead customer workshops, architecture reviews, and solution assessments.
• Define target-state architecture, migration roadmaps, and reusable patterns.

Required Skills & Qualifications

• 15+ years of overall Data & Analytics experience.
• 4+ years of hands-on Databricks experience as specified in the JD.
• Strong experience with Databricks, Delta Lake, Unity Catalog, DLT, and Auto Loader.
• Proficiency in PySpark, Spark SQL, Python, and Structured Streaming.
• AWS experience with S3, IAM, Glue, Redshift, EMR, and Lambda.
• Experience with Airflow, CDC, and enterprise data migration.
• Knowledge of data governance, security, and observability.
• Strong customer-facing, solution architecture, and technical leadership skills.

Preferred Qualifications

• Healthcare or pharmaceutical domain experience is preferred but not mandatory.
• Experience with enterprise-scale data platform modernization and migration is beneficial.