Leads large-scale AI and software-engineering data programs, owning delivery scope, timelines, quality, throughput, contributor performance, and cost. Designs workflows, quality-control systems, metrics, and capacity plans for coding datasets, agentic trajectories, benchmarks, and evaluations. Manages distributed contributor and reviewer networks, partners with talent teams, communicates with AI-lab customers, analyzes operational data, and converts successful processes into reusable playbooks and evaluation assets.
About Gramian
Gramian Consultancy is a boutique consultancy specializing in IT professional services and engineering talent solutions. With a strong background in software engineering and leadership, we help companies build high-performing teams by matching them with professionals who truly fit their needs.
About the Role
We are looking for an experienced Technical Program Operations Lead to own the production systems behind large-scale AI and software-engineering data programs. You will turn complex research requirements into predictable delivery across quality, throughput, contributor performance, timelines, and cost.
Programs may include coding datasets, repository-level tasks, agentic trajectories, reinforcement-learning environments, benchmarks, code reviews, and rubric-based evaluations. This is an operations leadership role with a strong technical bar, requiring the ability to inspect code, understand tests, analyze quality signals, and improve workflows at scale.
Key Responsibilities
- Own end-to-end program delivery across scope, timelines, quality, throughput, contributor performance, and cost.
- Design and manage workflows for coding datasets, agentic trajectories, RL environments, benchmarks, and rubric-based evaluations.
- Identify operational bottlenecks and improve workflows through better instructions, sequencing, incentives, review systems, and capacity planning.
- Define contributor requirements and partner with talent teams to source, assess, onboard, train, and ramp distributed software engineers.
- Build team-lead and reviewer structures for programs involving 100–1,000+ contributors.
- Own quality-control systems and analyze datasets to identify trends, systematic errors, and root causes.
- Act as a primary customer contact for AI labs, communicating progress, risks, quality trends, and recovery plans.
- Translate research objectives into practical task specifications and challenge requirements when they may not produce the intended evaluation signal.
- Use Python, SQL, or similar tools to automate quality sampling, defect analysis, throughput reporting, and operational reviews.
- Convert successful workflows into reusable playbooks, quality controls, evaluation assets, and contributor-management systems.
- Share operational learnings and mentor other program leads.
Requirements
- Proven experience leading complex, multi-stakeholder programs in software engineering, technical program management, consulting, finance, startups, operations, or a similar environment.
- Strong analytical and problem-solving skills, including the ability to identify bottlenecks, define meaningful metrics, and improve production performance.
- Experience managing distributed teams, contributor networks, marketplaces, or large-scale technical operations.
- Strong customer-facing communication skills, including managing expectations, communicating risks, and building long-term client relationships.
- Ability to read and review code, understand test suites, and independently assess technical work.
- Working knowledge of at least one programming language such as Python, TypeScript, Java, or Go.
- Experience using data and operational metrics to monitor quality, throughput, performance, and delivery.
- Ability to operate effectively in environments where research requirements and priorities evolve quickly.
Similar Jobs
Software • Hospitality
Provides friendly and efficient food and beverage service to hotel guests. Responsibilities include explaining menu items and promotions, taking and entering orders into the POS system, delivering food and drinks, maintaining a stocked and organized service station, and following health, safety, sanitation, and alcohol-awareness regulations.
Top Skills:
Point-Of-Sale (Pos) System
Software • Hospitality
Manages daily health club, fitness center, and spa operations within a hotel. Leads, trains, schedules, and supports team members; maintains guest service standards; assists with marketing, programming, merchandising, inventory, and financial tracking; and ensures compliance with health, safety, sanitation, and alcohol awareness regulations.
Software • Hospitality
Provides administrative support to the General Manager and department managers, including word processing, data entry, filing, copying, correspondence, telephone and email communication, guest interactions, travel bookings, and special project assistance.
What you need to know about the Colorado Tech Scene
With a business-friendly climate and research universities like CU Boulder and Colorado State, Colorado has made a name for itself as a startup ecosystem. The state boasts a skilled workforce and high quality of life thanks to its affordable housing, vibrant cultural scene and unparalleled opportunities for outdoor recreation. Colorado is also home to the National Renewable Energy Laboratory, helping cement its status as a hub for renewable energy innovation.
Key Facts About Colorado Tech
- Number of Tech Workers: 260,000; 8.5% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Lockheed Martin, Century Link, Comcast, BAE Systems, Level 3
- Key Industries: Software, artificial intelligence, aerospace, e-commerce, fintech, healthtech
- Funding Landscape: $4.9 billion in VC funding in 2024 (Pitchbook)
- Notable Investors: Access Venture Partners, Ridgeline Ventures, Techstars, Blackhorn Ventures
- Research Centers and Universities: Colorado School of Mines, University of Colorado Boulder, University of Denver, Colorado State University, Mesa Laboratory, Space Science Institute, National Center for Atmospheric Research, National Renewable Energy Laboratory, Gottlieb Institute
