Discovered Labs Logo

Discovered Labs

Senior Full Stack Engineer

Reposted Yesterday
Remote
Hiring Remotely in United States
Senior level
Remote
Hiring Remotely in United States
Senior level
Build and productionize LLM-backed agents and workflows: create eval harnesses, observability, provenance and human-review surfaces, extend agent capabilities (web/CMS, OAuth), implement pipeline orchestration, and ship scoring/ranking models and reusable modules to scale across clients.
The summary above was generated by AI
About Discovered Labs

At Discovered Labs we work with $10M - $50M ARR companies to help them get more leads, users and customers from Google, Bing and AI assistants such as ChatGPT, Claude and Perplexity.

We approach marketing the way engineers approach systems: data in, insights out, feedback loops everywhere. Every decision traces back to measurable outcomes. Every workflow is designed to eliminate manual bottlenecks and compound over time.

High-level overview of our approach:

Data-driven automation: We treat marketing programs like products. We instrument everything, automate the repetitive, and focus human effort on high-leverage problems.

First principles thinking: We don't copy what others do. We understand the underlying mechanics of how search and AI systems work, then build solutions from that foundation.

Full-stack ownership: SEO and AEO rarely work as isolated tasks. We work across the entire funnel and multiple surface areas to ensure we own the outcome and clients win.

The Agency OS

We're building the first agency OS: a system where the agency's work is the software, enabling clients to gain an unfair advantage.

As Karpathy puts it, Software 3.0 automates what humans can verify. Verifiability, not capability, decides which work can safely be handed over. So every eval we write expands the set of work we can trust an agent with, and the harness isn't overhead around the product, it's what makes the product possible. Dex Horthy covers the operational half: the agents that survive contact with customers are well-engineered software with human judgement placed deliberately, and disciplined loops beat vibes.

What that looks like here: expert orchestrators, not operators. One strategist supervising a fleet of agents running real client workflows at scale, with the system surfacing what genuinely needs a human decision and staying quiet about what doesn't. Feedback loops everywhere. Every correction an expert makes becomes an eval case, and the swarm is better the next cycle.

The Team

You'll join the Automations team at Discovered Labs. We build the agent swarm: a growing fleet of agents and workflows that do real client work end to end, and the engineering that makes them trustworthy enough to let loose on it.

Your mandate is both halves of that. Expand what the swarm can do, and prove that what it does is right. That means applying hardcore engineering to a famously slippery problem: making non-deterministic systems reliable, measurable and provable. Evals that fail a build. Traces that explain a decision. Guardrails that hold when a model doesn't. Agents that know what they don't know and say so.

You'll work alongside the Automations lead, and with our AI & Data team who own the platform and tooling underneath you. You won't be building orchestration from scratch or fighting for infrastructure. It's there, and it's good. Your job is to take it the last mile.

Every agent you make provable is an agent we can put in front of a client. That's the whole game, and the ceiling on how fast this company grows.

We're a deeply technical team building the SpaceX of the AEO & SEO space. You'll work alongside engineers who have built fraud engines powering Stripe, shipped AI code review at CodeRabbit, built at Amazon, developed self-driving car systems at Aurora, and conducted AI research at Stanford. We don't have layers of management. You'll work directly with founders who can go deep on architecture, code, and product.

This Role

You'll join the Automations team and ship agent-powered features end to end. A Slack command or a UI, through the API, out to the agent, and back in front of an expert who can act on it.

We have live 24/7 agents, data pipelines and workflow infrastructure. You'll use that infra rather than run it. Your job is to turn it into things our team and our clients actually use.

That means evals that catch regressions before they ship, observability that explains why an agent did what it did, provenance that traces every claim back to its source, and interfaces that let a non-engineer SEO strategist run and review the work. You'll build the harnesses and eval suites that gate agent changes in CI, the lineage and step-trace tooling that makes agent reasoning inspectable, and the surfaces the team actually uses. Then you'll close the loop: instrument the output, feed the signal back, and make the agents measurably better every cycle.

You'll also extend what agents can reach. Today they work over APIs and our own data. Next they operate real websites along withsession handling, and navigating interfaces that were never designed for machines.

The hard problem is reliability and legibility. An agent that's right 80% of the time and can't tell you which 80% is worthless. An agent that's right 95% of the time, shows its working, and flags its own uncertainty is a product.

You report to the CTO and work alongside the Automations lead, with our AI & Data team owning the platform beneath you. You own your evals, your CI, and your monitoring.

What You'll Do

Agent eval harnesses. Golden suites, deterministic checks, and online scoring that gate every agent change in CI. A regression should fail a build, not a client report.

Agent observability. Step traces, token and cost accounting, run-level SLOs, and failure taxonomies. You'll know the difference between "the agent ran" and "the agent was right."

Data lineage and provenance. Every claim an agent makes should be traceable to its source. Which inputs, which tool calls, which reasoning steps produced this output.

End-to-end agents with human review. Agents that complete real work start to finish, with a curated review surface so an expert can approve, correct, or reject, and so that correction becomes training signal.

Interfaces for non-engineers. Dashboards and controls that let the SEO team run, inspect and trust agent work without asking an engineer.

Expanding what agents can do. Agents that open pull requests against real repositories, authenticate through OAuth, and operate real sites and CMSs through the browser. Every new capability is a new class of work the swarm can take on.

Closing the loop. Turn signals we already collect, such as AI perception and citation data, into concrete client value: strategy recommendations, prioritised actions, and measurable outcomes.

Algorithms and scoring tools. Not everything should be a prompt. You'll implement the scoring and ranking tools behind what we recommend: internal linking and semantic relevance, Reddit opportunity scoring, content and citation quality. You won't own the underlying research, but you will build the tools and get them working in production.

Shared, composable modules. Reusable capability blocks that both agents and workflows compose, so an improvement lands everywhere at once instead of being reimplemented per agent.

Scaling across clients. Take a workflow that works for one client and make it run reliably for many, with per-client configuration, isolation, and failure that stays contained.

The Ideal Person for This Role

A builder who ships. You care about getting working systems into production, not endless planning or polish. You've built AI systems people actually rely on.

An operator, not just an architect. You don't just design systems, you run them. You find satisfaction in making things reliable, not just making them work once in a demo.

An owner. You take responsibility for outcomes, not just tasks. When an agent silently produces bad output, you catch it, fix it, and build the eval that stops it recurring.

Maniacal about detail. This is the one that matters most here. AI has made it trivially cheap to produce a large volume of plausible-looking work, and most of that work is slop. We are building the opposite: output that holds up when a client reads it line by line. If you'd rather ship one thing that is provably right than ten that are probably fine, this is your team. If you cannon volume and let the reviewer sort it out, it isn't.

Sceptical of your own output. You assume the model is wrong until measured. You've been burned by a demo that worked and a production run that didn't, and you build accordingly.

Humble and curious. You acknowledge what you don't know, ask good questions, and genuinely want to learn. You take feedback as a gift, not a threat.

A first-principles thinker. You understand why things work, not just how. You can go five levels deep on eval design, prompt architecture, and where to put the human in the loop.

Always improving. You're not satisfied with "good enough." You actively seek ways to get better at your craft and make systems better over time.

Requirements

4+ years in software engineering, with meaningful recent time on LLM-backed or ML-backed production systems.

Product taste. You have opinions about what good feels like. You'd rather ship one thing an expert opens every day than five they click once and never return to.

Python, React, Typescript and strong systems fundamentals. You write production services, not notebooks.

LLM application engineering in production. Agents, tool use, structured outputs, retrieval, prompt architecture. You've shipped something real that used them and stayed up.

Testing non-deterministic systems. You write evals for the features you ship, on the harness provided. You've thought hard about what "correct" means when there are many correct answers.

Observability for AI systems. Tracing, run inspection, cost and token accounting. Langfuse, LangSmith, Braintrust, OpenTelemetry or equivalent.

Debugging non-determinism. You can diagnose why an agent failed on run 47 of 100 and turn that into a permanent check.

Pipeline orchestration. Airflow, Dagster, Temporal or similar. Retries, idempotency, partial failure, resumption.

Third-party API integration. Auth flows, rate limits, pagination, breaking changes. Not just calling endpoints, but handling the full operational reality.

Ship to production yourself. Containers, CI/CD, deployment, credential handling. You own your code all the way out, using the platform and tooling our AI & Data team provides rather than building or running it.

Product sense for expert users. You've built tooling that domain experts, not engineers, use daily. You know that an unexplained AI output is an unusable one.

Collaborative. You'll define contracts with the engineers who own the infra beneath you. You document decisions, write clear specs, and communicate tradeoffs in writing.

Preferred Qualifications
  • Browser automation or authenticated web agents (Playwright, Puppeteer, computer-use models)

  • Applied ML: ranking, scoring, classification, or recommendation systems in production

  • Python, React, Typescript

  • Prior experience at a fast-moving startup

What's in It for You
  • Fully remote position

  • Work directly with the CTO on high-impact projects

  • High ownership and autonomy. No micromanagement.

  • First-hand exposure to cutting-edge AI and search technology

  • Your work will directly impact well-known (10M+ ARR) companies' performance

  • Join a fast-growing company at the intersection of AI and marketing

Our Hiring Process
  1. Application

  2. Technical Deep Dive

  3. Reference Checks

Similar Jobs

4 Days Ago
Remote
United States
113K-181K Annually
Senior level
113K-181K Annually
Senior level
Cloud • Fintech • Food • Information Technology • Software • Hospitality
Develop and deliver production-ready web experiences for marketing using React, Gatsby, and Tailwind. Build API integrations, maintain a lightweight codebase and design-system components, ensure accessibility, performance, SEO, and multi-region support, participate in code reviews, and help shape tooling and best practices across the team.
Top Skills: Ai-Assisted WorkflowsCi/CdContentfulCSSGatsbyGitGraphQLHTMLJavaScriptNode.jsReactRest ApiServerlessTailwind Css
7 Days Ago
Easy Apply
Remote
United States
Easy Apply
145K-200K Annually
Senior level
145K-200K Annually
Senior level
Cannabis • eCommerce • Enterprise Web • Logistics • Payments • Software • Database
Build and own complex frontend features using Vue 3 and TypeScript while developing the Python services and APIs that power them. Improve performance, accessibility, testing, CI/CD, and observability; collaborate with product and design; mentor engineers and apply AI-assisted development tools responsibly to accelerate productivity and maintain quality and security.
Top Skills: ClaudeCopilotCSSCursorDockerEvent-Driven ApisGraphQLHTMLJavaScriptKubernetesPiniaPlaywrightPytestPythonRelational DatabasesRestTypescriptVitestVue 3Vue Composition ApiVue Test UtilsVuex
7 Days Ago
Easy Apply
Remote
United States
Easy Apply
145K-200K Annually
Senior level
145K-200K Annually
Senior level
Cannabis • eCommerce • Enterprise Web • Logistics • Payments • Software • Database
Design and implement scalable backend services for payment and transaction workflows using Java. Own end-to-end features from data model and APIs to Angular frontends, improve reliability and performance, refactor legacy systems, troubleshoot production issues, and collaborate with product and design. Leverage AI-assisted development tools responsibly and share best practices across the team.
Top Skills: AngularAPIsClaudeCursorDockerGithub CopilotJavaJvm FrameworksKubernetesReactRelational DatabasesTypescriptVue

What you need to know about the Colorado Tech Scene

With a business-friendly climate and research universities like CU Boulder and Colorado State, Colorado has made a name for itself as a startup ecosystem. The state boasts a skilled workforce and high quality of life thanks to its affordable housing, vibrant cultural scene and unparalleled opportunities for outdoor recreation. Colorado is also home to the National Renewable Energy Laboratory, helping cement its status as a hub for renewable energy innovation.

Key Facts About Colorado Tech

  • Number of Tech Workers: 260,000; 8.5% of overall workforce (2024 CompTIA survey)
  • Major Tech Employers: Lockheed Martin, Century Link, Comcast, BAE Systems, Level 3
  • Key Industries: Software, artificial intelligence, aerospace, e-commerce, fintech, healthtech
  • Funding Landscape: $4.9 billion in VC funding in 2024 (Pitchbook)
  • Notable Investors: Access Venture Partners, Ridgeline Ventures, Techstars, Blackhorn Ventures
  • Research Centers and Universities: Colorado School of Mines, University of Colorado Boulder, University of Denver, Colorado State University, Mesa Laboratory, Space Science Institute, National Center for Atmospheric Research, National Renewable Energy Laboratory, Gottlieb Institute

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account