v4c.ai Logo

v4c.ai

Lead Data Engineer, Databricks (Remote, US)

Posted 2 Hours Ago
Remote
Hiring Remotely in United States
Senior level
Remote
Hiring Remotely in United States
Senior level
Lead customer-facing data engineering engagements from architecture through production handover. Build and optimize batch and streaming Lakehouse pipelines on Databricks using PySpark, SQL, and Delta Lake. Establish Medallion architecture conventions, implement data quality and automated testing, optimize Spark performance and platform costs, and mentor engineers. Partner with architects and explain technical trade-offs to customers.
The summary above was generated by AI
About V4C

V4C is a pure-play Databricks partner, and in under two years we have earned 850+ Databricks certifications across 325+ customers and been named Databricks' 2026 Partner of the Year (C&SI Emerging).

As a Lead Data Engineer, you will lead the build of production Lakehouse pipelines on customer engagements, working directly with customers and often Databricks' own field teams. Every technical hire here gets a certification roadmap built for them and taught in-house by Databricks-approved trainers, plus a dedicated Employee Success Manager whose only responsibility is your experience at V4C.

The Role

Lead Data Engineers are the technical backbone of V4C delivery. You lead the engineering workstream on an engagement — translating an architecture into working pipelines, setting the patterns the team follows, and owning the quality of what ships. You will work across batch and streaming, on messy enterprise source systems, under real deadlines, and in front of the customer explaining what you built and why. The difference from a Data Engineer is ownership: you are accountable for a workstream and the engineers on it.

Responsibilities

  • Lead engineering on customer engagements, from design through production handover
  • Build and optimize batch and streaming pipelines using PySpark, SQL, and Delta Lake on Databricks
  • Implement Medallion architecture across cloud ecosystems and set the layer conventions the team follows
  • Own performance and cost: benchmark Spark jobs, tune SQL, right-size compute, and explain the trade-offs to the customer
  • Implement data quality and validation as code, including Lakeflow expectations and automated testing in CI
  • Mentor mid-level and junior engineers through code review, pairing, and design feedback
  • Partner with the Data Architect on design decisions, and escalate when implementation reveals the design will not hold

Requirements
  • 7+ years of data engineering plus experience leading projects or teams
  • Strong proficiency in Python, SQL, Apache Spark, and Delta Lake
  • Production experience building both batch and streaming pipelines at enterprise scale
  • Hands-on experience with AWS, Azure, or GCP and its data services
  • Experience with data modeling, Git, and CI/CD for data workloads
  • Demonstrated experience optimizing Spark performance and managing platform cost
  • Strong client communication skills, including the ability to explain technical trade-offs to a non-technical stakeholder

Preferences

  • Healthcare or life sciences experience preferred but not required
  • Existing Databricks Certified Data Engineer Associate or Professional certification
  • Hands-on experience with Unity Catalog, Lakeflow, or Databricks Asset Bundles
  • Experience with dbt, Airflow, or equivalent orchestration and transformation tooling
  • Experience migrating a legacy ETL estate to a modern cloud platform
  • Consulting or client-facing engineering background

Benefits
  • Unlimited PTO
  • Health Care Plan (Medical, Dental & Vision)
  • Retirement Plan (401k, IRA)
  • Life Insurance (Basic, Voluntary & AD&D)
  • Family Leave (Maternity, Paternity)
  • Short Term & Long Term Disability
  • Training & Development

Similar Jobs

An Hour Ago
Remote or Hybrid
United States
246K-329K Annually
Senior level
246K-329K Annually
Senior level
Artificial Intelligence • Big Data • Software • Analytics • Business Intelligence • Big Data Analytics
Own and improve Hex’s production infrastructure across cloud computing, networking, databases, CI/CD, observability, security, and single-tenant environments. Projects may include Kubernetes and AWS operations, infrastructure automation, PostgreSQL scaling, deployment systems, cost optimization, workload isolation, and customer-facing connectivity features. The role requires end-to-end ownership, on-call participation, technical writing, and the ability to turn ambiguous infrastructure problems into shippable solutions.
Top Skills: AWSCi/CdDatadogDnsDynamoDBEksElasticacheGoIamKubernetesLancedbMskNode.jsPostgresPrivatelinkPythonRdsRedisRustTerraformTlsTypescriptVpc
An Hour Ago
Remote or Hybrid
200K-500K Annually
Expert/Leader
200K-500K Annually
Expert/Leader
Artificial Intelligence • Big Data • Cloud • Information Technology • Machine Learning • Software
Lead and scale Nexthink’s U.S. federal sales organization, owning pipeline, forecasting, quota attainment, strategic agency opportunities, partner and contract-vehicle strategy, and federal go-to-market execution. Recruit and coach sales leaders and account executives, build repeatable sales processes, navigate procurement and FedRAMP requirements, and collaborate cross-functionally to expand SaaS adoption across civilian, DoD, and intelligence agencies.
Top Skills: ApmCloud-Native PlatformsEndpoint CybersecurityFedrampGsa ScheduleItsmMeddpiccObservabilitySaaSVdi
An Hour Ago
Remote or Hybrid
CA, USA
100K-148K Annually
Senior level
100K-148K Annually
Senior level
Gaming
Designs and implements live game content, events, systems, and features for a mobile game. Responsibilities include balancing events using mathematical models, analyzing player feedback and metrics, improving design tools and pipelines, collaborating across disciplines, documenting user experiences, prototyping new event types, and leading creative brainstorming. The role requires strong systems, narrative, content, game-balancing, communication, and mobile gaming expertise, plus Unity experience and either two shipped games or five years of Live Ops experience.
Top Skills: Unity

What you need to know about the Colorado Tech Scene

With a business-friendly climate and research universities like CU Boulder and Colorado State, Colorado has made a name for itself as a startup ecosystem. The state boasts a skilled workforce and high quality of life thanks to its affordable housing, vibrant cultural scene and unparalleled opportunities for outdoor recreation. Colorado is also home to the National Renewable Energy Laboratory, helping cement its status as a hub for renewable energy innovation.

Key Facts About Colorado Tech

  • Number of Tech Workers: 260,000; 8.5% of overall workforce (2024 CompTIA survey)
  • Major Tech Employers: Lockheed Martin, Century Link, Comcast, BAE Systems, Level 3
  • Key Industries: Software, artificial intelligence, aerospace, e-commerce, fintech, healthtech
  • Funding Landscape: $4.9 billion in VC funding in 2024 (Pitchbook)
  • Notable Investors: Access Venture Partners, Ridgeline Ventures, Techstars, Blackhorn Ventures
  • Research Centers and Universities: Colorado School of Mines, University of Colorado Boulder, University of Denver, Colorado State University, Mesa Laboratory, Space Science Institute, National Center for Atmospheric Research, National Renewable Energy Laboratory, Gottlieb Institute

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account