Design task-specific, objective evaluation frameworks for data science deliverables. Assess AI- and human-generated analyses, models, dashboards, experiments, and recommendations using reproducible scoring. Produce detailed written justifications, refine methodologies with reviewer feedback, and collaborate with cross-functional teams to improve AI training and evaluation standards.
This role is for one of our clients
Compensation: $120-$170 per hour
Join a leading AI research organization working on the next generation of intelligent systems. We are seeking experienced Data Science Experts to help evaluate how effectively AI performs real-world data science tasks. Rather than building models or delivering business analyses yourself, you will define what high-quality work looks like by creating robust evaluation frameworks and assessing AI-generated outputs with structured, evidence-based feedback.
This is an excellent opportunity to apply your industry expertise to improve advanced AI systems while influencing how they reason through complex analytical problems.
RequirementsKey Responsibilities
- Design detailed, task-specific evaluation criteria for real-world data science deliverables, including:
- Data analyses
- Predictive and statistical models
- Dashboards and visualizations
- Experiment results
- Executive recommendations and business insights
- Evaluate AI-generated and human-produced work using objective scoring frameworks.
- Provide detailed written justifications for each evaluation, ensuring consistency and technical accuracy.
- Apply sound analytical judgment to produce reproducible and defensible assessments.
- Incorporate structured reviewer feedback and continuously refine evaluation methodologies.
- Collaborate with cross-functional teams to improve AI training quality and evaluation standards.
- 5+ years of professional experience in Data Science within an industry environment.
- Strong background in business operations, product analytics, growth analytics, or similar data science functions at technology-driven organizations.
- Deep expertise in:
- Experimental design and A/B testing
- Statistical analysis
- SQL and Python
- Metric development and performance measurement
- Translating analytical findings into business recommendations
- Excellent written communication skills with the ability to clearly explain technical reasoning.
- Exceptional attention to detail and the ability to evaluate work consistently against structured rubrics.
- Comfortable receiving feedback and calibrating evaluation standards with peer reviewers.
- Experience contributing to AI evaluation, model training, human feedback, or data annotation initiatives.
- Familiarity with large language models (LLMs), AI-assisted analytics, or machine learning workflows.
- Experience reviewing technical work or mentoring analytical teams is a plus.
- Submit your resume or relevant technical background to begin the application process.
- Qualified candidates may be invited to complete a short technical assessment or provide additional information relevant to their expertise.
- Contribute to the development of next-generation AI systems used for real-world analytical reasoning.
- Apply your professional data science expertise to high-impact AI evaluation projects.
- Work remotely with flexible scheduling.
- Collaborate with a network of experienced technical professionals helping define the future of AI quality and reasoning.
We welcome applicants from all backgrounds and are committed to providing equal opportunities and reasonable accommodations throughout the recruitment process.
Contract Details- Independent contractor engagement.
- Fully remote with flexible working hours.
- Project duration may be extended or concluded based on project requirements and performance.
- Weekly payments based on approved work completed.
- Please note that H1-B and STEM OPT sponsorship is not available for this engagement.
Similar Jobs
Artificial Intelligence • HR Tech • Professional Services • Software
Design realistic, multi-step data analysis benchmark tasks and create reproducible reference analyses. Evaluate AI-generated analyses for correctness, statistical validity, and interpretation. Collaborate with researchers to improve benchmark quality and document methodologies in Jupyter/Colab notebooks.
Top Skills:
GitGoogle ColabJupyter NotebooksNumpyPandasPythonScikit-LearnScipy
.jpg){kind=logo}
Healthtech • Social Impact • Software • Telehealth
Build and deploy AI-powered tools for Rula’s business and clinical teams by observing workflows, rapidly delivering production software, measuring usage and impact, and enabling teams to operate tools independently. The role requires privacy-by-design practices, rigorous AI evaluation, customer-centric discovery, and integration with complex legacy systems. Staff engineers may optimize LLM costs and quality while creating reusable building blocks for future teams.
Top Skills:
Artificial IntelligenceBilling SystemsCoding AgentsCrm SystemsLarge Language ModelsScheduling Systems
Healthtech • Social Impact • Software • Telehealth
Manage provider credentialing and payer enrollment processes, including scoping requirements, submitting applications, guiding providers, documenting workflows, and supporting new payer launches. Collaborate cross-functionally to improve provider operations and educate credentialing associates on new processes and payer requirements. The role requires healthcare credentialing experience, payer enrollment expertise, process documentation skills, and proficiency with spreadsheets, CRM, and ticketing platforms.
Top Skills:
AirtableCaqhCertifyCmsExcelGoogle SheetsNcqaNppesSalesforceZendesk
What you need to know about the Colorado Tech Scene
With a business-friendly climate and research universities like CU Boulder and Colorado State, Colorado has made a name for itself as a startup ecosystem. The state boasts a skilled workforce and high quality of life thanks to its affordable housing, vibrant cultural scene and unparalleled opportunities for outdoor recreation. Colorado is also home to the National Renewable Energy Laboratory, helping cement its status as a hub for renewable energy innovation.
Key Facts About Colorado Tech
- Number of Tech Workers: 260,000; 8.5% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Lockheed Martin, Century Link, Comcast, BAE Systems, Level 3
- Key Industries: Software, artificial intelligence, aerospace, e-commerce, fintech, healthtech
- Funding Landscape: $4.9 billion in VC funding in 2024 (Pitchbook)
- Notable Investors: Access Venture Partners, Ridgeline Ventures, Techstars, Blackhorn Ventures
- Research Centers and Universities: Colorado School of Mines, University of Colorado Boulder, University of Denver, Colorado State University, Mesa Laboratory, Space Science Institute, National Center for Atmospheric Research, National Renewable Energy Laboratory, Gottlieb Institute
