Manuscript · Submitted to NAACL 2027

RELIEFRevision-Based Learning from
Interactive Experience and Feedback

Jiawen Li1, Chenming Ge2, Alice Gao3, Rishab Vempati1, Yi-Hao Peng4, Jason Wu1
1Purdue University2University of Michigan 3University of Washington4MIT
Paper (PDF) ↗ Explore the method Results
Generate a flight-search interface, gather feedback on its date filter, revise the interface, and use reconstructed generation traces to fine-tune the model.
Generate → collect feedback → revise → train.
Figure 1 from the paper.

Learning from interface use

Learn from how people use generated interfaces.

Watching people use a generated interface can reveal confusing controls, missing feedback, and broken task flows.

RELIEF turns those observations into training data. A coding agent revises the interface using feedback. RELIEF then reconstructs a generation trace that would produce the revised interface and uses it to fine-tune the model.

200
generated websites
355
interaction recordings
24
study participants
2
base models evaluated

Four sources, one revision pipeline

Where does the feedback come from?

We compared feedback inferred from real interactions with written user feedback and critiques from automated agents.

All four feedback sources guided the same interface-revision pipeline. Feedback volume differed across sources.

From revisions to supervision

Reconstruct how the improved interface could have been generated.

The revised website provides a target. RELIEF connects its changes back to the original generation steps.

  1. Replay. Record the website source after each file-writing step in the original generation trace.
  2. Attribute and patch. Use code differences and Git-blame-style attribution to identify and revise the steps responsible for changed code.
  3. Verify and train. Replay the reconstructed trace to check that it reproduces the revised website, then use it for supervised fine-tuning.
Training and reconstruction details

We fine-tuned a separate variant of each base model for each feedback source using reconstructed generation traces. The experiments used supervised fine-tuning with LoRA adapters. Reconstruction retained the original reasoning while revising code-writing actions; see Sections 4 and 6 of the paper for the procedure and its limitations.

A date-filter revision is attributed to the original Add filters generation step. That step is patched, and the reconstructed trace is replayed to reproduce the revised flight-search interface.
A revision to the final website becomes a revision to its generation trace. Figure 4 from the paper.

What changed after fine-tuning?

Feedback from human interaction showed the strongest observed results.

Across the two tested model families, human-interaction feedback achieved the highest observed task completion and arena Elo ratings.

Arena Elo results with 95% confidence intervals: human interaction scored 1014 for Qwen and 1023 for Inkling, compared with base model ratings of 1007 and 997. Confidence intervals overlap.
Human preference arena · 400 pairwise comparisons across six annotators.
Figure 5 from the paper. Error bars show 95% confidence intervals.

The confidence intervals overlap. These are observed differences under the tested models and evaluation setup, rather than a definitive ranking of feedback sources.

Could the generated interfaces support their tasks?

Task completion after model fine-tuning · Table 2
Training conditionInkling SmallQwen 3.8 27B
Base model78.8%83.7%
Code critique71.7%83.2%
Computer-use-agent critique82.6%89.7%
Human-written feedback75.5%86.4%
Human interaction84.8%90.8%

Evaluated with a computer-use agent on 184 held-out tasks across 14 websites. Task completion was measured after up to three generation retries; first-attempt eligibility was reported separately in the paper.

Abstract

Code generation models are increasingly used to produce interactive artifacts, such as websites and user interfaces. Many properties that determine whether these interfaces are useful only become apparent through realistic interaction, making them difficult to incorporate into model training. In this paper, we introduce RELIEF (REvision-based Learning from Interactive Experience and Feedback), a method for improving code generation models using feedback on their generated interfaces. RELIEF uses a coding agent to revise these interfaces, then incorporates the improvements into their original generation traces to produce supervision for fine-tuning. To evaluate RELIEF, we collected a dataset of 200 synthetically generated websites paired with 355 interaction recordings and accompanying feedback from 24 users. We used critiques derived from code inspection, automated interaction, explicit human feedback, and human interaction recordings to fine-tune variants of two code generation models. Our results show that feedback effectiveness varies with base model capabilities, with critiques inferred from human interaction recordings leading to the largest usability improvements overall under our tested models.

Citation & resources

@unpublished{li2026relief,
  title={Revision-Based Learning from Interactive Experience and Feedback},
  author={Li, Jiawen and Ge, Chenming and Gao, Alice and Vempati, Rishab and Peng, Yi-Hao and Wu, Jason},
  year={2026},
  note={Manuscript submitted to NAACL 2027},
  url={https://oscar-ge.github.io/files/RELIEF.pdf}
}

Read the manuscript · Download BibTeX · Contact

The paper describes planned public releases of the dataset and fine-tuned model weights.