Railroad19
Jobs
Senior Data Engineer (GCP • Python • Iceberg • Delta Lake • Kafka • Snowflake • Databricks)
Railroad19
Senior Data Engineer (GCP • Python • Iceberg • Delta Lake • Kafka • Snowflake • Databricks)
Design and build modern GCP lakehouse architectures, including Python and Spark pipelines, BigQuery ingestion, Kafka CDC, Delta Lake and Iceberg UniForm integrations, Delta Sharing endpoints, Snowflake and Databricks connectivity, data lineage, governed access controls, and semantic-layer alignment. Collaborate cross-functionally, troubleshoot full-stack issues, and deliver reusable data-sharing adapters and end-to-end data features.
Railroad19, Inc is seeking a Senior Data Engineer with deep, hands‑on experience in building modern lakehouse architectures on GCP. This role focuses on designing, developing in Python & Spark, and delivering reusable data‑sharing adapters that connect BigQuery‑backed data products to Snowflake and Databricks using Iceberg UniForm and Delta Sharing.
About Railroad19:
At Railroad19, Inc, we develop customized software solutions and provide software development services. We’re a specialized team of developers and architects. As such, we only bring an “A” team to the table, through hard work and a desire to lead the industry — this is our company culture — this is what sets Railroad19 apart.
As a Railroad19 employee, you will be part of a company that values your work and gives you the tools you need to succeed. Our headquarters is in Saratoga Springs, New York, but this position is 100% remote. Railroad19 provides competitive compensation and excellent benefits, including Medical/Dental/Vision/Pet Insurance, Paid Time Off, and 401 (k).
NO 1099, C2C, Corp-to-Corp; only full-time employment.
NO Agencies.
Core Responsibilities:
- Design and implement the UniForm write layer (Delta + Iceberg dual metadata).
- Build GCS → BigQuery ingestion pipelines for structured operational datasets.
- Develop and implement in Python and Spark.
- Implement Kafka-based CDC patterns for real-time and near-real-time ingestion.
- Develop data lineage, dependency tracking, and modular adapter code.
- Configure Snowflake Horizon external tables for zero-copy reads.
- All data hub tables are to be written once using Delta Lake with Iceberg UniForm enabled… readable by all target consumers without conversion.
- Implement and certify Delta Sharing endpoints for Databricks consumers.
- Build governed access layer components: RBAC, connector registry entries, tenant-scoped authorization.
- Align semantic layer models with LookML and KPI catalog definitions.
- Collaborate with cross‑functional teams to deliver end‑to‑end features.
- Troubleshoot issues across the full stack and contribute to code quality.
Skills/Experience:
- 6+ years of proven enterprise-level experience in Python & Spark
- Advanced experience in GCP BigQuery
- Strong working knowledge of Apache Iceberg, Delta Lake, Iceberg UniForm
- Experience with Delta Sharing; Kafka / CDC pipelines
- Specific work experience in Snowflake Horizon Catalog; Databricks Unity Catalog
- Solid experience with Data lake architecture & ingestion pipeline design
- Excellent Communication skills and the ability to work cohesively with multiple teams.
Preferred Experience – Nice to Have
- Prior delivery in enterprise SaaS, media, or advertising technology.
- Active daily use of AI-assisted development tools (Claude Code preferred).
Similar Jobs
Aerospace • Artificial Intelligence • Machine Learning • Robotics • Software • Defense Technology
Leads Shield AI’s strategic autonomy capture strategy for PACOM, developing multi-year capture plans, building relationships with requirements and acquisition officials, and influencing defense policy, requirements, and solicitations. The role coordinates technical and operational teams to secure strategic program wins and expand long-term revenue and mission impact across autonomy product portfolios.
Top Skills:
Artificial IntelligenceAutonomy SoftwareSynthetic Reality Technologies
Consumer Web • Healthtech • Professional Services • Social Impact • Software
Lead full-cycle technical recruiting for engineering teams: partner with hiring managers, design sourcing strategies, run searches from intake to offer, use data to diagnose and improve searches, and ensure a structured, inclusive candidate experience while building high-quality pipelines.
Aerospace • Information Technology • Software • Cybersecurity • Design • Defense • Manufacturing
Lead certification efforts to ensure aircraft software meets FAA/DoD/NASA safety standards. Prepare certification artifacts, collaborate with authorities, mentor engineers, resolve complex engineering issues, and track safety and certification milestones.
Top Skills:
AgileCodevueConfiguration ManagementGitJIRARtca Do-178
What you need to know about the Colorado Tech Scene
With a business-friendly climate and research universities like CU Boulder and Colorado State, Colorado has made a name for itself as a startup ecosystem. The state boasts a skilled workforce and high quality of life thanks to its affordable housing, vibrant cultural scene and unparalleled opportunities for outdoor recreation. Colorado is also home to the National Renewable Energy Laboratory, helping cement its status as a hub for renewable energy innovation.
Key Facts About Colorado Tech
- Number of Tech Workers: 260,000; 8.5% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Lockheed Martin, Century Link, Comcast, BAE Systems, Level 3
- Key Industries: Software, artificial intelligence, aerospace, e-commerce, fintech, healthtech
- Funding Landscape: $4.9 billion in VC funding in 2024 (Pitchbook)
- Notable Investors: Access Venture Partners, Ridgeline Ventures, Techstars, Blackhorn Ventures
- Research Centers and Universities: Colorado School of Mines, University of Colorado Boulder, University of Denver, Colorado State University, Mesa Laboratory, Space Science Institute, National Center for Atmospheric Research, National Renewable Energy Laboratory, Gottlieb Institute
