Who You Are
Be Curious
We ask questions, seek to understand others, and stay curious about the problems we solve. Curiosity fuels creativity and innovation - and helps us build better solutions.
Customer First
We put people at the center of what we do - listening with care, acting with empathy, and going the extra mile to support those we serve. Customer First means understanding the needs of anyone we impact - whether that’s a customer, a coworker, or an end user - and doing right by them.
Own the Outcome
We take initiative and do what it takes to deliver results. With resourcefulness and resolve, we overcome obstacles, follow through on our commitments, and hold ourselves accountable - because each of us is empowered to make an impact.
About Alchemer
Founded in 2006, Alchemer is a powerful survey and data insights platform that empowers business professionals to make informed decisions. As SaaS application software, it offers user-friendly data collection tools for understanding customers, markets, and employees in real time and communicating this information across an organization. It provides data insights in over 205 countries, with 50K new surveys created and 5M responses collected every week.
Alchemer has tremendous opportunity to continue this growth, based on current market size and the potential for more sophisticated product positioning and a robust sales and marketing engine. Details on Alchemer’s products and services can be found on our website (www.Alchemer.com).
What You Will Do
As Principal AI Architect, you will set the technical direction of our Generative and Agentic AI portfolio and build the hardest pieces yourself. This is a hands-on, individual contributor role for an architect who has spent the last several years shipping modern AI systems to production.
Alchemer's acquisition of Chatmeter brought a live, proven AI engine with 32+ LLM endpoints, 100M+ vector embeddings, semantic search, real-time feedback intelligence, and full cost observability, all in production. Your mandate is to unify it, extend it across multiple products, and evolve it into an industry-leading agentic platform.
The hard problems you will own: a unified AI entry point across five products, an agent runtime that coordinates across product boundaries, a retrieval layer at a scale most architects never encounter, and a Trust Layer that makes it all observable, compliant, and governable.
You will partner closely with Product, Engineering, and Data leaders. You will not manage the team and you will set the technical bar. This role has board-level visibility.
How You Will Spend Your Days
Strategic Leadership
- Shape and execute the vision for embedding Generative and Agentic AI across the platform — from natural language understanding and feedback analytics to multi-agent workflows that drive closed-loop customer action.
- Define and own the multi-year AI architecture, sequencing the agentic roadmap and aligning it with long-term business goals.
- Make the build / fine-tune / buy decisions on models, frameworks, and tooling, and own the rationale.
- Champion responsible and ethical AI: evaluation, bias mitigation, transparency, data governance, privacy, and regulatory compliance.
Hands-On Engineering
- Build, not just diagram. We expect you to be writing production code on the hardest parts — multi-agent orchestration, retrieval, tool and function-calling layers, and evaluation harnesses — every week.
- Own the agent runtime: planning, tool use, memory, routing, fallbacks, and cost and latency budgets.
- Own the retrieval layer at scale: hybrid search, chunking and embedding strategy, reranking, freshness, and multi-tenant scoping over large volumes of structured and unstructured data.
- Architect and build the Integration Gateway — the unified entry point routing all AI traffic across products, with per-tenant cost controls, smart routing, circuit breakers, and the connector layer that lets agents act inside products.
- Stand up evaluation and observability end to end — offline eval sets, online eval, regression gates in CI, traces, and cost dashboards. No agent ships without an eval.
Collaboration & Influence
- Set the patterns the rest of Engineering uses to build AI features — reference implementations, internal SDKs, prompt and tool-use conventions, and evaluation templates.
- Partner with Product, Engineering, and Data teams to integrate AI seamlessly into existing platform capabilities.
- Coach senior engineers and ML practitioners on agentic patterns, retrieval design, evaluation discipline, and production readiness.
- Present AI architecture and roadmaps to executive leadership and serve as a credible technical voice with customers and partners.
Engineering Ideals & Environment
- We operate a polyglot, distributed environment that prioritizes architectural rigor over specific toolsets. You will work within a production-grade stack that leverages:
- AI & ML: Foundation-model APIs from major providers, LLM tracing and evaluation harnesses (e.g., Langfuse), and sophisticated model serving layers (e.g., BentoML), supported by internal prompt-engineering tooling and reusable AI building blocks.
- Architecture: Distributed systems primarily built on JVM and Python services, with React and React Native on the client.
- Data Layer: Large-scale retrieval engines including vector stores (e.g., Qdrant), NoSQL databases, and Lakehouse architectures (Databricks-class) for structured and unstructured signals at enterprise scale.
- Modern Best Practices: We are committed to — and continuously moving toward — engineering best practices. You will lead the charge in establishing unified CI/CD patterns for AI, including automated evaluation gates and regression testing across our various codebases.
What you bring to the role
Leadership & Experience
- 10+ years building production software, with recent hands-on experience shipping Generative and Agentic AI systems to real users at meaningful scale.
- Demonstrable ownership of at least one production multi-agent or tool-using system — planning, tool and function calling, memory, routing, fallback strategies, and cost and latency control.
- Deep, hands-on experience with retrieval-augmented generation at scale: hybrid retrieval, chunking and embedding strategy, reranking, multi-tenant scoping, freshness, and retrieval evaluation.
- Strong understanding of NLP fundamentals: tokenization, embeddings, language modeling, and text generation.
- B2B SaaS experience with multi-tenant data, customer-configurable workflows, and enterprise security expectations (SOC 2, GDPR, HIPAA-adjacent).
Technical Expertise
- Strong Python comfort with at least one modern agent framework (LangGraph, LlamaIndex, CrewAI, AutoGen, OpenAI Agents SDK, or equivalent).
- Production fluency with foundation-model APIs (OpenAI, Anthropic, Bedrock, Vertex), vector databases, and LLM observability and evaluation tooling.
- Experience with modern AI/ML and Lakehouse platforms (Databricks, SageMaker, Azure ML, or Vertex AI) and data warehouses (Snowflake, Redshift, or BigQuery).
- Real evaluation discipline: golden sets, LLM-as-judge with calibration, online eval and feedback loops, and regression gates in CI.
- Working knowledge of SQL and NoSQL databases, vector stores, and streaming and batch processing (Kafka, Spark, or equivalent).
- Familiarity with DevOps and MLOps: Git, CI/CD, infrastructure as code, and containerization.
Culture & Communication
- Excellent written and verbal communication; able to translate complex AI concepts to non-technical audiences.
- Passion for coaching and mentoring engineers and ML practitioners.
- Excels in a dynamic, fast-paced environment with evolving priorities.
Nice to Have
- Fine-tuning and model adaptation (PEFT/LoRA, distillation, small-model routing) where it produced measurable wins.
- Background in customer experience, feedback analytics, reputation, or unstructured-text analytics.
- Open-source contributions, conference talks, or published work on Generative or Agentic AI.
- Formal degree (BS or MS) in Computer Science, Mathematics, Statistics, or a related technical field.
Compensation for this position includes an annual base salary of $180,000 to $220,000, with eligibility to participate in our Employee Participation Unit program.
Alchemer provides a rich benefits package that includes health and disability coverages, an incredible 401(k) option that includes a per-payroll match and immediate vesting, unlimited time off and twelve paid company holidays.
Alchemer is taking applications for this position from candidates who are legally authorized to work in the US and do not require now, or in the future, immigration sponsorship for work authorization.
At Alchemer, we're committed to attracting, developing, and promoting a diverse workforce, and infusing DEI throughout all of our internal practices. By ensuring that everyone is able to bring a diversity of talents to work, we're increasingly capable of living out our mission in helping our clients. We encourage people from underrepresented backgrounds and all walks of life to apply.
Don’t meet all the requirements? Studies have shown that women and people of color are less likely to apply to jobs unless they meet every single qualification listed. Don’t let that stop you! We encourage you to apply if you have some of the requirements listed. You may bring qualities that we’re looking for in other areas. Building a diverse, inclusive, and customer centric workplace is important to us at Alchemer. If you’re excited about this role and your experience doesn’t perfectly align, we encourage you to apply anyway. Let’s talk!
Alchemer Louisville, Colorado, USA Office
168 Centennial Parkway, Unit 250, Louisville, CO, United States, 80027
Similar Jobs
What you need to know about the Colorado Tech Scene
Key Facts About Colorado Tech
- Number of Tech Workers: 260,000; 8.5% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Lockheed Martin, Century Link, Comcast, BAE Systems, Level 3
- Key Industries: Software, artificial intelligence, aerospace, e-commerce, fintech, healthtech
- Funding Landscape: $4.9 billion in VC funding in 2024 (Pitchbook)
- Notable Investors: Access Venture Partners, Ridgeline Ventures, Techstars, Blackhorn Ventures
- Research Centers and Universities: Colorado School of Mines, University of Colorado Boulder, University of Denver, Colorado State University, Mesa Laboratory, Space Science Institute, National Center for Atmospheric Research, National Renewable Energy Laboratory, Gottlieb Institute
