Remote Application Developer – AI Agent Evaluation

About the role

This fully remote position focuses on designing, evaluating, and improving autonomous AI agents that operate across health, education, daily‑life, and other real‑world domains. You will contribute to the development of agentic AI systems by providing expert human feedback and helping train large language models for complex, multi‑step workflows.

Key responsibilities

• Write clear evaluation rubrics with objective pass/fail criteria for AI agents.
• Debug agent execution traces to identify failure patterns.
• Stress‑test agents against edge cases, prompt injection, and tool misuse.
• Assess production‑grade modular software architecture and multi‑turn system interactions.
• Provide high‑density technical feedback to improve LLM training.

Required profile

• Proven experience in backend engineering, AI automation, or complex systems integration.
• Track record of building and maintaining production‑grade software with modular separation of services.
• Hands‑on experience delivering solutions in live, non‑mocked environments.
• Ability to work independently on flexible, task‑based assignments.

Required skills

• Strong command of at least two major programming languages (e.g., Python, JavaScript, Go, Java).
• Experience with SQL databases.
• Familiarity with integrating agents with live APIs such as Supabase or Gmail.
• Knowledge of persistent state, session‑tracking patterns, and security concerns like privacy leaks or prompt injection.

What we offer

• Hourly compensation ranging from $30 to $50, based on experience and task complexity.
• Weekly payments via supported payout platforms (e.g., PayPal, AirTM).
• Fully remote work with flexible scheduling.