Safe and Scalable Web Agent Learning
via Recreated Websites

Abstract

Training autonomous web agents is fundamentally limited by the environments they learn from: real-world websites are unsafe to explore, hard to reset, and rarely provide verifiable feedback. We propose VeriEnv, a framework that treats language models as environment creators, automatically cloning real-world websites into fully executable, verifiable synthetic environments. By exposing controlled internal access via a Python SDK, VeriEnv enables agents to self-generate tasks with deterministic, programmatically verifiable rewards — eliminating reliance on heuristic or LLM-based judges. This design decouples agent learning from unsafe real-world interaction while enabling scalable self-evolution through environment expansion. Through experiments on web agent benchmarks, we show that agents trained with VeriEnv generalize to unseen websites, achieve site-specific mastery through self-evolving training, and benefit from scaling the number of training environments.

Why Verifiable Environments?

Existing self-evolving web agents face a fundamental tension: they learn from unsafe, unresettable real websites using unreliable LLM-as-a-judge evaluation. This feedback loop compounds errors and produces agents that overfit to the judge's biases rather than the true task. VeriEnv breaks the loop by giving agents a controlled, reproducible training ground with deterministic rewards.

The VeriEnv Pipeline

VeriEnv automates three stages: (1) a coding agent clones the target website end-to-end (frontend, backend, and database) from screenshots; (2) bug-report passes stabilize the clone; and (3) task instructions are generated alongside executable Python SDK validation programs, so every task ships with a programmatic judge.

A verifiable task

Each generated task is paired with a ground-truth Python SDK call whose return value is the reference answer. At evaluation time, the agent's answer is checked against this reference with deterministic operators (exact_match, must_include, fuzzy_match, etc.).

Site-specific self-evolution

Cloned Websites

VeriEnv clones websites spanning a wide range of domains — travel, commerce, education, healthcare, entertainment, government, and more. The five examples below are highlighted from our full set, with each pair showing the original website that our coding agent was given as input next to the VeriEnv clone it produced.

Sample Generated Tasks

Tasks are generated automatically and filtered by executable validation. The selector below rotates through five representative tasks per site — a mix of information-seeking queries and authenticated multi-step actions.

Benchmark Results

Agents trained on VeriEnv-cloned environments transfer to standard web-agent benchmarks. Tables 4 and 5 show consistent gains over backbone models and prior self-evolution baselines (Synatra, ADP), evaluated on WebArena-Lite (5 real websites) and Mind2Web-Online (held-out tasks across difficulty levels). VeriEnv delivers the largest absolute improvement on both benchmarks for both backbones tested.

Scaling & Reliability

Increasing the number of cloned training environments consistently improves transfer to held-out websites — VeriEnv turns the cost of environment construction into a controllable axis of scale.

Compared to PAE-style LLM-as-a-judge pipelines, VeriEnv produces substantially less ambiguous tasks and more reliable evaluations.

What Fails, and How Long Does It Take?

Citation

If you build on this work, please cite:

@article{chae2026verienv,
  title   = {Safe and Scalable Web Agent Learning via Recreated Websites},
  author  = {Chae, Hyungjoo and Park, Jungsoo and Ritter, Alan},
  journal = {arXiv preprint arXiv:2603.10505},
  year    = {2026}
}