Enhance a Python-based ingestion framework for operational metadata and build connectors for major systems in data and ML stacks.
DataHub is an AI & Data Context Platform adopted by over 3,000 enterprises, including Apple, CVS Health, Netflix, and Visa. Innovated jointly with a thriving open-source community of 13,000+ members, DataHub's metadata graph provides in-depth context of AI and data assets with best-in-class scalability and extensibility.
The company's enterprise SaaS offering, DataHub Cloud, delivers a fully managed solution with AI-powered discovery, observability, and governance capabilities. Organizations rely on DataHub solutions to accelerate time-to-value from their data investments, ensure AI system reliability, and implement unified governance, enabling AI & data to work together and bring order to data chaos.
In this role, you will
- Enhance the Python-based ingestion framework to support ingesting usage statistics, lineage, and operational metadata from systems like Snowflake, Redshift, Kafka, & more!
- Build connectors for major systems in the modern data and ML stacks
- Enable the ingestion framework to run in a cloud native environment
Requirements:
- Minimum 4 years of engineering experience
- Expertise in Python
- Familiarity with tools in the modern data and ML ecosystem
- Knowledge of distributed systems
- Ability to design for scale and fault tolerance
Benefits
- Competitive salary
- Equity
- Medical, dental, and vision insurance (99% coverage for employees, 65% coverage for dependents; USA-based employees)
- Carrot Fertility Program (USA-based employees)
- Work from home and monthly co-working space budget
Top Skills
Kafka
Python
Redshift
Snowflake
Similar Jobs
AdTech • Cloud • Digital Media • Information Technology • News + Entertainment • App development
Develop high availability applications for the Golf platform, collaborating with cross-functional teams while maintaining best practices in software development.
Top Skills:
Amazon Web ServicesC#DockerElasticGitKubernetesMongodbMs SqlTfs
Blockchain • eCommerce • Fintech • Payments • Software • Financial Services • Cryptocurrency
As a Security Software Engineer, you will implement identity and access projects, improve foundational systems, and ensure secure communication across Block's infrastructure.
Top Skills:
AuthenticationAuthorizationIdentity ManagementSecurity Software Engineering
Cloud • Enterprise Web • Sales • Software • Transportation
Join a remote team as a Software Engineer to develop transportation management software, focused on user problem-solving and rapid feature deployment.
Top Skills:
ReactRuby On Rails
What you need to know about the Colorado Tech Scene
With a business-friendly climate and research universities like CU Boulder and Colorado State, Colorado has made a name for itself as a startup ecosystem. The state boasts a skilled workforce and high quality of life thanks to its affordable housing, vibrant cultural scene and unparalleled opportunities for outdoor recreation. Colorado is also home to the National Renewable Energy Laboratory, helping cement its status as a hub for renewable energy innovation.
Key Facts About Colorado Tech
- Number of Tech Workers: 260,000; 8.5% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Lockheed Martin, Century Link, Comcast, BAE Systems, Level 3
- Key Industries: Software, artificial intelligence, aerospace, e-commerce, fintech, healthtech
- Funding Landscape: $4.9 billion in VC funding in 2024 (Pitchbook)
- Notable Investors: Access Venture Partners, Ridgeline Ventures, Techstars, Blackhorn Ventures
- Research Centers and Universities: Colorado School of Mines, University of Colorado Boulder, University of Denver, Colorado State University, Mesa Laboratory, Space Science Institute, National Center for Atmospheric Research, National Renewable Energy Laboratory, Gottlieb Institute
