Lead design and deployment of ML systems for enterprise search, retrieval, recommendations, and AI agents. Build scalable data/ML pipelines, evaluation frameworks, and KPIs. Apply statistics, causal inference, experimentation, and analysis to guide product strategy. Prototype and productionize LLM-based features, mentor junior staff, and collaborate cross-functionally.
Description
Chaos Labs builds infrastructure for organizations adopting AI at scale. Our products help organizations understand their AI work, capture the knowledge it generates, and turn everyday AI activity into intelligence they own.
Reporting directly to the Chief Science Officer we are looking for a talented Senior AI Data Scientist to join our team and help shape the future of AI.
Responsibilities
- Design and build machine learning systems that power enterprise search, retrieval, recommendations, and AI agents.
- Develop evaluation frameworks and metrics to measure the quality, performance, and business impact of LLM-powered applications.
- Build scalable data and ML pipelines that process and analyze large-scale AI interaction data.
- Apply statistical modeling, experimentation, and causal inference to guide product development and strategic decision-making.
- Collaborate closely with Product, Engineering, and Design to translate research and insights into production features.
- Analyze product usage and customer behavior to identify opportunities for improving AI adoption, productivity, and user experience.
- Prototype, evaluate, and deploy new AI capabilities using foundation models, embeddings, and retrieval-augmented generation (RAG) techniques.
- Define key performance indicators and build analytical frameworks that inform product strategy and company-wide decisions.
- Contribute to the technical direction of our AI platform by identifying new opportunities to leverage machine learning and generative AI.
- Mentor junior team members and help establish best practices in data science, experimentation, and machine learning across the organization.
- 5+ years of experience in Data Science, Applied Machine Learning, or a quantitative research role (or 3+ years with a PhD).
- Degree in Computer Science, Statistics, Mathematics, Machine Learning, Economics, Physics, or another highly quantitative field.
- Strong proficiency in Python and SQL, with experience building scalable data and ML pipelines.
- Deep understanding of statistics, experimentation, causal inference, and predictive modeling.
- Experience developing and deploying machine learning models in production.
- Experience working with large-scale datasets and modern data infrastructure.
- Familiarity with LLMs, embeddings, retrieval systems, RAG, or AI agents.
- Strong analytical thinking with the ability to translate ambiguous problems into measurable solutions.
- Excellent communication skills and the ability to work cross-functionally with engineering, product, and leadership.
- Comfortable operating in a fast-paced, high-ownership startup environment.
Preferred Qualifications
- Based in NYC with the ability to work from our Brooklyn office, or the ability to work remotely, with travel up to 30% of the year to collaborate in person with the team.
- Experience building or evaluating LLM-powered products or AI applications.
- Background in search, recommendation systems, information retrieval, or knowledge graphs.
- Experience designing A/B tests, offline evaluations, and ML benchmarking frameworks.
- Familiarity with vector databases, embedding models, and agent orchestration frameworks.
- Contributions to open-source ML projects or publications in machine learning, NLP, or related fields.
- Experience in B2B SaaS, developer tools, or enterprise AI products.
- Based in NYC and able to work from our Brooklyn office, or remote within the U.S. with the ability to travel up to 30% for team collaboration.
Benefits:
- Paid Time Off – 21 vacation days + 7 sick days + 8 observed U.S. company holidays
- Health Coverage – 100% employer-paid options (medical, dental & vision) for you and your dependents
- FSA / HSA Options – depending on selected health insurance plan
- Wellness Programs – OneMedical, Teladoc, Talkspace, and EAP
- 401(k) - with a 100% company match on the first 6% contributed
- Career Growth – opportunities in a rapidly expanding, global technology company with personalized professional development
- Pre-tax Commuter Benefits
- Compensation & Equity – competitive package aligned with growth and merit
- Salary and Seniority level commensurate with experience (range: $ 179K - 217K)
Similar Jobs
Healthtech • Biotech • Pharmaceutical • Manufacturing
Design and implement GenAI, ML, and NLP solutions for clinical trial operations: fine-tune LLMs, harmonize trial data, build enrollment forecasting and cost-optimization tools, simulate patient enrollment, and communicate technical findings to stakeholders.
Top Skills:
DspyEhrFinetuningGenerative AiInsurance ClaimsLangchainLlmsMlopsNlpPythonRegistry DataSQL
Software
Design and deploy ML systems for enterprise search, recommendations, retrieval, and AI agents. Build scalable data/ML pipelines, evaluation frameworks, and metrics. Apply statistics, experimentation, and causal inference to guide product decisions. Collaborate cross-functionally, prototype LLM-based capabilities (embeddings, RAG), and mentor junior team members.
Top Skills:
Agent Orchestration FrameworksEmbeddingsFoundation ModelsLlmsPythonRetrieval SystemsRetrieval-Augmented Generation (Rag)SQLVector Databases
Digital Media • Fintech • Information Technology • Machine Learning • Financial Services • Cybersecurity • Automation
Design, develop, and deploy production-grade generative AI and agentic systems (LLMs, RAG, AI agents). Build document intelligence, retrieval pipelines, vector search, and monitoring frameworks. Collaborate with risk, compliance, product, and engineering to deliver scalable, governed GenAI solutions in regulated environments.
Top Skills:
AirflowAWSAws BedrockAzure AiCi/CdDatabricksEmbeddingsHugging FaceLangchainLanggraphMlflowOcrPrompt EngineeringPysparkPythonPyTorchRetrieval-Augmented Generation (Rag)SagemakerScikit-LearnSnowflakeSnowparkSparkSQLTensorFlowVector DatabasesVllm
What you need to know about the Colorado Tech Scene
With a business-friendly climate and research universities like CU Boulder and Colorado State, Colorado has made a name for itself as a startup ecosystem. The state boasts a skilled workforce and high quality of life thanks to its affordable housing, vibrant cultural scene and unparalleled opportunities for outdoor recreation. Colorado is also home to the National Renewable Energy Laboratory, helping cement its status as a hub for renewable energy innovation.
Key Facts About Colorado Tech
- Number of Tech Workers: 260,000; 8.5% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Lockheed Martin, Century Link, Comcast, BAE Systems, Level 3
- Key Industries: Software, artificial intelligence, aerospace, e-commerce, fintech, healthtech
- Funding Landscape: $4.9 billion in VC funding in 2024 (Pitchbook)
- Notable Investors: Access Venture Partners, Ridgeline Ventures, Techstars, Blackhorn Ventures
- Research Centers and Universities: Colorado School of Mines, University of Colorado Boulder, University of Denver, Colorado State University, Mesa Laboratory, Space Science Institute, National Center for Atmospheric Research, National Renewable Energy Laboratory, Gottlieb Institute
