Your agent can now get human feedback on Prolific

Integrate feedback from real, verified humans faster with the Prolific CLI

Access the CLI

Read API docs

Trusted by leading engineering teams

Use cases

How teams do more with the CLI

Programmatic workflows across one human network.

Embed human evaluation

Run pre-release evaluation and reproduce across model versions. Specify a cohort by filter, aggregate responses against a threshold and pass or fail on the result.

Human-in-the-loop checkpoints

Agents call Prolific mid-run for human review, escalation or disambiguation - filtered down to the right participants for the task. Responses return as structured JSON.

Preference collection in the loop

Launch pairwise preferences, Likert ratings or step-level rationale tasks directly from your training pipeline between runs. JSONL export feeds into RLHF, DPO and reward-model workflows.

Simulate with real people

Programmatically recruit targeted participants to role-play end users in multi-turn evaluation. Stable cohort hashes mean the same group can be re-recruited for longitudinal comparison.

What customers say

"I want to remove any barrier between my agent and the results."

Emerging Products Director: Fortune 500 software company

Watch the CLI demo

Why Prolific

Built for pipelines

One network for every interface

The same human network and 300+ filters whether you're calling via UI, CLI or REST API. Cohorts defined in the dashboard resolve identically in your pipeline, so there's no drift between how you test and how you ship.

Designed for repeat runs

Built for iteration at scale. Re-run studies on consistent cohorts, export paginated responses in JSONL and keep provenance intact across every record so your training data stays clean and traceable.

Latency that fits a training loop

Studies launch in minutes and return responses in hours. Webhooks on response.submitted and study.completed let a pipeline ingest incrementally, so you're never blocked waiting on a full cohort.

Get started

Do more from where you already operate

Start where it matches your stack.

API reference

Studies, cohorts, responses, webhooks, submissions. OpenAPI schema, auth model, rate limits, idempotency semantics.

docs.prolific.com/api

CLI reference

Launch studies, wait on completion, export responses, manage filters from a shell. Scriptable in any pipeline.

docs.prolific.com/cli

Events & webhooks

Subscribe to response.submitted, study.completed, and submission review events. Signed payloads, retry policy.

docs.prolific.com/webhooks

Fast-moving AI teams using Prolific

Trusted by AI/ML developers, researchers, and leading organizations across industries.

Speak to the team

Prolific's CLI in practice

See how agent can launch tasks to real humans on Prolific - from dataset to results

Watch the demo

Building breakthrough AI faster

Ai2 reduced human data collection from weeks to hours with Prolific, building state-of-the-art multimodal AI models faster without sacrificing quality.

Unpacking human preference for LLMs - The HUMAINE framework

Prolific's human-centered leaderboard ranks frontier AI models by how real users actually experience them - not just technical benchmarks. Featured at ICLR 2026.

Read the paper

Gemini 3 Pro: Frontier safety framework

The frontier safety framework report for Google’s latest model.

Start building automated workflows

Get started for free

Read the API docs

FAQ

Questions from engineers integrating Prolific

Does Prolific have an API?

Yes. Prolific exposes a REST API and a CLI over the same 200,000+ identity-verified participant pool. UI, CLI, and API address the same 300+ filter attributes and the same audit trail. Anonymised participant response data is accessible via any of the three.

Can an LLM agent call Prolific directly?

Yes. Any system that can make authenticated HTTPS requests — agent scaffolding, orchestrators, training pipelines, CI runners — can launch studies, subscribe to webhooks, and export responses. Idempotency keys on study creation make retry safe from automated callers.

Can I wire Prolific into an RLHF or DPO training loop?

Yes. Preference-collection studies launched via API return JSONL responses with stable cohort hashes and filter provenance in every record. The format is designed to feed directly into reward-model, DPO, and RLHF training code.

How fast does a Prolific study return data?

Self-serve studies typically launch in minutes and return first responses in hours. Completion time depends on cohort size, filter strictness, and reward. Webhooks on response.submitted allow incremental ingest rather than blocking on full completion.

Can I filter participants programmatically by profession, credentials, or demographics?

Yes. The API accepts the same 300+ filter attributes available in the UI — demographic, professional, behavioural, and credential. A filter set resolves to a stable cohort hash that can be reused across runs for longitudinal comparison between model versions.

Does Prolific support webhooks?

Yes. Subscribe to response.submitted, study.completed, and submission-review events. Payloads are signed and include filter provenance; delivery retries with exponential backoff on non-2xx responses.

Can I use Prolific as a release gate in CI?

Yes. A CI job can launch a study, wait on completion via the CLI or a webhook, aggregate responses against a threshold, and exit pass or fail. Reproducible filter hashes make the same cohort specification repeatable across releases.

How does Prolific authenticate API requests?

API-key authentication over HTTPS. Keys are scoped to a workspace and can be rotated without downtime. Keys are never embedded in participant-facing URLs or exposed to study respondents.