Handshake Logo

Handshake

AI Red Teamer (Remote)

Posted 5 Days Ago
Remote
Hiring Remotely in USA
32-95 Hourly
Entry level
Remote
Hiring Remotely in USA
32-95 Hourly
Entry level
Stress-test large language models by crafting adversarial prompts and multi-turn scenarios that expose safety, bias, hallucination, prompt injection, and guardrail vulnerabilities. Evaluate model responses using harm taxonomies and severity rubrics, document findings, refine testing methods, and collaborate with engineers, data scientists, and researchers to improve AI safety and robustness. The role involves regular exposure to disturbing harmful content.
The summary above was generated by AI

California eligibility: We are unable to hire candidates residing in California for this role.

AI Red Teamer (LLM Generalist)

Location: Remote (USA)

Employment type: W-2, hourly, employed through TCWGlobal

Hours: 40 hours per week

About the Role

As an AI Red Teamer, you will stress-test large language models by intentionally trying to break them. Rather than checking whether an answer is correct, you will design creative, adversarial prompts that expose vulnerabilities: unsafe content, bias, broken guardrails, hallucinations, prompt injection weaknesses, and unexpected behaviors. Your work directly supports AI safety and model robustness for leading research labs.

This is a generalist red teaming role. You will probe models across the full spectrum of risk categories, including content safety, CBRN (chemical, biological, radiological, nuclear), cybersecurity, persuasion and influence operations, child safety, self-harm, over-companionship, and regulatory compliance. Red teaming may span text, image, voice, and agentic model capabilities depending on project needs.

This role requires creativity, curiosity, and an ability to think like an adversary while operating with strong ethical judgment.

Day-to-Day Responsibilities
  • Craft creative prompts and multi-turn scenarios to stress-test AI guardrails across diverse risk categories

  • Discover ways around safety filters, restrictions, and defenses using jailbreak, evasion, and prompt injection techniques

  • Explore edge cases to provoke disallowed, harmful, or incorrect outputs

  • Evaluate and score model responses against structured harm taxonomies and severity rubrics

  • Document experiments clearly, including what you tried, why you tried it, and what it revealed

  • Review and refine adversarial prompts generated by other team members

  • Contribute to harm taxonomy development, calibration exercises, and inter-rater reliability work

  • Collaborate with engineers, data scientists, and researchers to share findings and strengthen defenses

  • Work with potentially disturbing content on a regular basis (see Content Warning below)

  • Stay current on jailbreaks, attack methods, and evolving model behaviors

Desired Capabilities

Core

  • Strong hands-on experience using multiple LLMs (ChatGPT, Claude, Gemini, open-source models, etc.)

  • Intuition for crafting adversarial prompts; familiarity with jailbreak or evasion techniques is a strong plus

  • Creative, adversarial problem-solving skills

  • Clear and thoughtful written communication

  • Strong ethical judgment and the ability to separate adversarial thinking from personal values

  • Self-directed, collaborative, and comfortable in feedback-heavy environments

  • Curiosity, persistence, and comfort with frequent failure in experimentation

Nice to Have

  • Familiarity with Python or other scripting languages

  • Experience working with LLM APIs or evaluation tooling

  • Comfort with structured data annotation and rubric-based scoring

  • Prior work in trust and safety, content moderation, QA, or security research

  • Subject matter expertise in any high-risk domain (cybersecurity, chemistry, biology, medicine, law, finance, etc.)

You Will Thrive Here If
  • You treat every model response as a hypothesis to challenge

  • You can switch between creative free-association and rigorous documentation in the same session

  • You go deep into unusual interests (fandoms, niche internet cultures, gaming exploits, Wikipedia rabbit holes, etc.)

  • You come from a creative background: writing, visual art, improv, puzzle design, or similar

  • You are energized by finding the thing nobody else thought to try

  • You are genuinely passionate about AI and follow the space closely

Content Warning

This role involves regular and deliberate exposure to harmful content. You will encounter and intentionally generate content involving violence, self-harm, hate speech, sexually explicit material, child safety scenarios, and other categories of harmful output as part of structured adversarial testing. Candidates must be able to engage with this material professionally and sustainably. Support resources are available.

About Handshake AI

Handshake AI partners with leading AI research labs to make models safer and more robust. Our red teaming operations help identify vulnerabilities before they reach users, contributing directly to the responsible development of frontier AI systems.

 

Handshake Denver, Colorado, USA Office

Denver, United States

Similar Jobs

5 Days Ago
Remote
USA
65-125 Hourly
Entry level
65-125 Hourly
Entry level
Edtech • Enterprise Web • HR Tech • Software
Evaluate AI models for cybersecurity risks by crafting adversarial prompts and multi-turn attack chains. Assess model-generated malware, exploit code, phishing content, privilege-escalation scripts, and other offensive tooling for functional correctness and real-world exploitability. Score responses using harm taxonomies, document technical findings, develop evaluation frameworks, and collaborate with AI researchers, red teamers, and policy teams to improve model safety.
Top Skills: BashCC++ChatgptClaudeCloud SecurityContainer SecurityCvesGeminiInfrastructure As CodeJavaScriptLlm ApisMitre Att&CkOwaspPowershellPython
One Month Ago
Remote
USA
65-158 Hourly
Entry level
65-158 Hourly
Entry level
Edtech • Enterprise Web • HR Tech • Software
Evaluate AI models for dangerous CBRNE-related outputs, designing adversarial prompts, multi-turn attack chains, and threat models. Assess technical accuracy and real-world risk, distinguish open-source information from harmful uplift, score responses using severity rubrics, and document failures. Collaborate with AI researchers and policy teams to improve model safeguards and evaluation frameworks while following strict ethical and operational security protocols.
Top Skills: ChatgptClaudeEvaluation ToolingGeminiLlm ApisOpen-Source ModelsPython
An Hour Ago
In-Office or Remote
202K-318K Annually
Expert/Leader
202K-318K Annually
Expert/Leader
Cloud • Information Technology • Productivity • Security • Software • App development • Automation
Lead Atlassian’s Agent Platform engineering team, combining people management with hands-on technical leadership. Set direction for applied ML, agent workflows, retrieval, evaluation, observability, reliability, security, privacy, and scalable platform foundations. Hire, coach, and develop engineers; establish planning and execution practices; write production code; and align roadmaps across product, engineering, ML, data, and platform stakeholders to deliver reliable AI experiences.
Top Skills: Agentic SystemsAi/Ml SystemsApplied MlContext RetrievalEvaluation SystemsObservabilityPlatform EngineeringProduction Operations

What you need to know about the Colorado Tech Scene

With a business-friendly climate and research universities like CU Boulder and Colorado State, Colorado has made a name for itself as a startup ecosystem. The state boasts a skilled workforce and high quality of life thanks to its affordable housing, vibrant cultural scene and unparalleled opportunities for outdoor recreation. Colorado is also home to the National Renewable Energy Laboratory, helping cement its status as a hub for renewable energy innovation.

Key Facts About Colorado Tech

  • Number of Tech Workers: 260,000; 8.5% of overall workforce (2024 CompTIA survey)
  • Major Tech Employers: Lockheed Martin, Century Link, Comcast, BAE Systems, Level 3
  • Key Industries: Software, artificial intelligence, aerospace, e-commerce, fintech, healthtech
  • Funding Landscape: $4.9 billion in VC funding in 2024 (Pitchbook)
  • Notable Investors: Access Venture Partners, Ridgeline Ventures, Techstars, Blackhorn Ventures
  • Research Centers and Universities: Colorado School of Mines, University of Colorado Boulder, University of Denver, Colorado State University, Mesa Laboratory, Space Science Institute, National Center for Atmospheric Research, National Renewable Energy Laboratory, Gottlieb Institute

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account