Databricks Data Engineer

Position Overview

As a Databricks Data Engineer, you will build the pipelines that move critical data products from the client’s enterprise data warehouse and operational systems into Databricks and stand up curated datasets used for Power BI, Databricks AI/BI Dashboards, and custom applications.

Key Responsibilities

• Build and maintain ingestion and transformation pipelines in Databricks using PySpark and Databricks SQL.
• Migrate datasets and ETL logic from legacy enterprise data platforms to Delta Lake on Databricks, validating parity with source data.
• Implement Bronze / Silver / Gold medallion patterns with lineage and access controls in Unity Catalog.
• Use Databricks Autoloader, Lakeflow Connect, and Structured Streaming for incremental ingestion where appropriate.
• Collaborate with architects, analysts, developers, and government stakeholders to deliver scalable and governed data products.
• Support data quality, performance optimization, and platform modernization initiatives.

Required Qualifications

• U.S. Citizen.
• Ability to obtain and maintain a U.S. Government suitability determination.
• Hands-on data engineering experience on Databricks.
• Strong PySpark, Python, and SQL skills.
• Experience with Delta Lake, medallion architecture, Autoloader and/or Structured Streaming, and Unity Catalog.
• Experience designing, developing, and supporting production data pipelines and data integration solutions.
• Strong problem-solving and analytical skills.

Preferred Qualifications

• Active Public Trust or other federal security clearance.
• Databricks Data Engineer Associate or Professional certification.
• Experience migrating workloads from legacy databases or data warehouses to Databricks.
• Experience supporting U.S. Federal Government clients.
• Experience working in cloud-native data environments.

Location

• Fully Remote (U.S.-based)
• Analyst, Consultant, and Senior Consultant levels available