Leads delivery of production LLM and AI systems, including multi-provider integrations, RAG, vector databases, agents, MCP tool use, guardrails, evaluation, observability, latency optimization, and cost management. Establishes organizational AI engineering standards, troubleshoots critical features hands-on, manages offshore delivery, reviews designs and code, and reports risks and progress to engineering leadership.
Key Responsibilities
- Own delivery of LLM API integration and SDK patterns used across applications.
- Set organizational guidance on which LLM to use for what use case and drive delivery of multi-LLM scenarios.
- Define standards for advanced prompt engineering and context window management.
- Own delivery of RAG systems, including vector database selection/topology and knowledgebase design.
- Drive delivery of AI agent and multi-agent systems and tool-use/MCP integration patterns.
- Own guardrails delivery (safety, compliance, PII handling in prompts/outputs) — critical in a financial-services context.
- Define evaluation frameworks and real-time eval strategy; set standards for AI testing in CI/CD.
- Own latency profiling, AI observability, and cost tracking/management for LLM-backed systems.
- Run day-to-day delivery of the offshore development team: sprint commitments, code/design review, real-time unblocking, and hands-on work on critical-path AI features.
- Report delivery status, risks, and blockers to engineering leadership.
Must-Have Qualifications
- 6+ years in software engineering, with 2+ years as a tech lead owning end-to-end delivery of LLM/AI-powered systems (not a pure design/review architect role).
- Proven track record of shipping AI-powered features on committed timelines, including hands-on troubleshooting under delivery pressure.
- Strong, hands-on Python skills at an architectural/systems level.
- Proven experience architecting LLM API integrations and SDK-level abstractions across multiple providers.
- Demonstrated judgment on model selection (cost, latency, capability trade-offs) across use cases.
- Deep expertise in prompt engineering and context window management at scale.
- Proven design experience with RAG systems, including vector database architecture and knowledgebase design.
- Experience architecting AI agents/multi-agent systems and tool-use patterns (MCP or equivalent).
- Strong understanding of guardrails design — content safety, PII protection, compliance controls for AI outputs.
- Experience defining evaluation frameworks and integrating AI testing into CI/CD.
- Proven ability to design for latency, observability, and cost management of AI systems in production.
- Financial-services or regulated-industry experience strongly preferred given compliance/guardrail stakes.
- Strong stakeholder communication; able to directly manage day-to-day delivery of an offshore team (standups, unblocking, sprint accountability).
Nice-to-Have Qualifications
- Direct experience with specific frameworks (LangChain, LlamaIndex, Semantic Kernel, or equivalent).
- Experience with AWS Bedrock or comparable managed LLM platforms.
- Contributions to or deep familiarity with MCP (Model Context Protocol) implementations.
- Experience building internal LLM gateways.
- Familiarity with responsible-AI/model-risk-management frameworks used in financial services.
Compensation, Benefits and Duration
Minimum Compensation: USD 56,000
Maximum Compensation: USD 196,000
Compensation is based on actual experience and qualifications of the candidate. The above is a reasonable and a good faith estimate for the role.
Medical, vision, and dental benefits, 401k retirement plan, variable pay/incentives, paid time off, and paid holidays are available for full-time employees.
This position is not available for independent contractors
No applications will be considered if received more than 120 days after the date of this post
Similar Jobs
Artificial Intelligence • Software
The AI Engineer will develop scalable AI systems, optimize LLMs, maintain data workflows, and ensure system resilience while collaborating on innovative solutions.
Top Skills:
AIAirflowAnthropic ModelsAutogenAWSAzureChromadbCrew.AiDaskDelta LakeDockerFastapiGCPGitGrafanaKubeflowKubernetesLangchainLanggraphLlmsMilvusMlOpenaiPgvectorPineconePrometheusRayTorchVector Databases
AdTech • Agency • Artificial Intelligence • Digital Media • Marketing Tech • Social Media • PropTech
Manage strategically complex client relationships and portfolios, owning retention, expansion, and cross-channel digital marketing strategy. Lead quarterly business presentations, deliver data-backed recommendations, coordinate internal teams, and act as a Fiona platform expert. Drive departmental initiatives, process improvements, product feedback, client onboarding, and mentorship for Account Managers and Coordinators. Success is measured through retention, client satisfaction, response times, reporting accuracy, expansion, revenue growth, and team development.
Top Skills:
AIDisplay AdvertisingEmail MarketingFionaGeofencingNative AdvertisingOrganic SocialPaid SearchPaid SocialReputation ManagementSeo
Cloud • Fintech • Information Technology • Machine Learning • Software
Support Xero’s partner sales channel by guiding accounting and bookkeeping practices through onboarding, data migration, software implementation, and activation. Deliver one-on-one and group training on Xero and related products, manage stakeholder expectations, schedule virtual and on-site sessions, maintain Salesforce records, and create self-service resources. The role requires frequent travel to partner practices and strong accounting, communication, presentation, and relationship-building skills.
Top Skills:
Advance TrackCloud Accounting PlatformsGmtHubdocJet ConvertMelioSaaSSalesforce CRMSyftXeroXero Payroll
What you need to know about the Colorado Tech Scene
With a business-friendly climate and research universities like CU Boulder and Colorado State, Colorado has made a name for itself as a startup ecosystem. The state boasts a skilled workforce and high quality of life thanks to its affordable housing, vibrant cultural scene and unparalleled opportunities for outdoor recreation. Colorado is also home to the National Renewable Energy Laboratory, helping cement its status as a hub for renewable energy innovation.
Key Facts About Colorado Tech
- Number of Tech Workers: 260,000; 8.5% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Lockheed Martin, Century Link, Comcast, BAE Systems, Level 3
- Key Industries: Software, artificial intelligence, aerospace, e-commerce, fintech, healthtech
- Funding Landscape: $4.9 billion in VC funding in 2024 (Pitchbook)
- Notable Investors: Access Venture Partners, Ridgeline Ventures, Techstars, Blackhorn Ventures
- Research Centers and Universities: Colorado School of Mines, University of Colorado Boulder, University of Denver, Colorado State University, Mesa Laboratory, Space Science Institute, National Center for Atmospheric Research, National Renewable Energy Laboratory, Gottlieb Institute
