Skip to content

Normalize observation-only steps without blank agent messages #295

Description

@neubig

Some standardized samples represent environment/tool observations as blank source: agent steps with only an observation field. This is schema-valid, but semantically awkward because the environment, not the agent, authored the observation.

Examples from the audit/heuristics include web/mobile datasets such as WebLINX, Wonderbread, Mind2Web, Go-Browse-WA, TurkingBench, WebArena, AndroidControl, and related browser/mobile traces.

Acceptance criteria:

  • Decide the canonical representation for standalone observations when no preceding tool call is available.
  • Prefer attaching observations to the triggering tool-call step when possible.
  • If standalone observation steps remain necessary, document the convention clearly or add schema support for environment-authored steps.
  • Regenerate affected samples after the representation is settled.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions