Design and lead Databricks Lakehouse solutions, optimize data pipelines, implement Delta Lake practices, and integrate cloud services.
Company Description
Role: Databricks Architect
Duration: 12 months
Location: Troy, MI- Remote
Minimum exp.- 14-15 yrs.
Role Overview
We are looking for a Databricks Architect to design and lead modern Lakehouse data platforms using Databricks. The role focuses on building scalable, high-performance data pipelines and enabling analytics and AI use cases on cloud-native data platforms.
Key Responsibilities
- Architect and implement Databricks Lakehouse solutions for large-scale data platforms
- Design and optimize batch & streaming data pipelines using Apache Spark (PySpark/SQL)
- Implement Delta Lake best practices (ACID, schema enforcement, time travel, performance tuning)
- Build and manage Databricks jobs, workflows, notebooks, and clusters
- Enable data governance using Unity Catalog (access control, lineage)
- Integrate Databricks with cloud data services (ADLS / S3, ADF, Synapse, etc.)
- Support analytics, BI, and AI/ML workloads (MLflow exposure is a plus)
- Lead solution design discussions and mentor data engineering teams
Must-Have Skills
- 10+ years in data engineering / data architecture
- 5+ years of strong hands-on experience with Databricks
- Expert in Apache Spark, PySpark, SQL
- Strong experience with Delta Lake & Lakehouse architecture
- Cloud experience on Azure Databricks / AWS Databricks
- Proven experience in designing high-volume, scalable data pipelines
Good-to-Have
- Unity Catalog, MLflow, Databricks Workflows
- Streaming experience (Kafka / Event Hubs)
- CI/CD for Databricks (Azure DevOps / GitHub)
All your information will be kept confidential according to EEO guidelines.
Top Skills
Spark
Aws Databricks
Azure Databricks
Azure Devops
Ci/Cd
Databricks
Delta Lake
Event Hubs
Git
Kafka
Mlflow
Pyspark
SQL
Unity Catalog
Similar Jobs
Cloud • Computer Vision • Information Technology • Sales • Security • Cybersecurity
The Business Systems Analyst will enhance partner interactions through Salesforce optimization, gather requirements, design reporting solutions, and manage stakeholder relationships.
Top Skills:
SalesforceTableau
Cloud • Computer Vision • Information Technology • Sales • Security • Cybersecurity
As a Full Stack Software Engineer, you will develop tools for cloud management, build APIs and UIs, and maintain cloud systems.
Top Skills:
AWSCi/CdCloudFormationGoGrpcJavaScriptKubernetesLinuxReactRestful ApisTerraformTypescript
Legal Tech • Software • Generative AI
Lead a team of Account Executives in the Mid-Market segment, focusing on driving performance, scaling sales processes, and talent development.
Top Skills:
Ai PlatformB2B Saas
What you need to know about the Colorado Tech Scene
With a business-friendly climate and research universities like CU Boulder and Colorado State, Colorado has made a name for itself as a startup ecosystem. The state boasts a skilled workforce and high quality of life thanks to its affordable housing, vibrant cultural scene and unparalleled opportunities for outdoor recreation. Colorado is also home to the National Renewable Energy Laboratory, helping cement its status as a hub for renewable energy innovation.
Key Facts About Colorado Tech
- Number of Tech Workers: 260,000; 8.5% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Lockheed Martin, Century Link, Comcast, BAE Systems, Level 3
- Key Industries: Software, artificial intelligence, aerospace, e-commerce, fintech, healthtech
- Funding Landscape: $4.9 billion in VC funding in 2024 (Pitchbook)
- Notable Investors: Access Venture Partners, Ridgeline Ventures, Techstars, Blackhorn Ventures
- Research Centers and Universities: Colorado School of Mines, University of Colorado Boulder, University of Denver, Colorado State University, Mesa Laboratory, Space Science Institute, National Center for Atmospheric Research, National Renewable Energy Laboratory, Gottlieb Institute
