For AI agents: the complete documentation index is available at https://a3s-lab.github.io/Test/en/llms.txt, the full documentation bundle is available at https://a3s-lab.github.io/Test/en/llms-full.txt, and this page is available as Markdown at https://a3s-lab.github.io/Test/en/guide/actions-and-evidence.md.

Actions, waits, and evidence

A3S Test does not treat a test step as an unrestricted browser script. Every step is a closed typed action. Static admission and session policy run before dispatch. The selected driver then checks its capability and resolves the target before any platform input. The runner assigns the step result only after the driver returns structured output and evidence.

This boundary has three practical benefits.

  • An agent can propose only protocol-admitted intent, not arbitrary code for page execution.
  • Failures remain attributable to the specification, locator, product assertion, driver, or cleanup.
  • Local exploration, ACL regression suites, and MCP sessions share one action vocabulary.

The current action protocol revision is 15. Treat installed machine output as the authoritative contract.

a3s-test agent schema
a3s-test capabilities --json

How one action completes

A typical Web action follows this path.

ACL or agent proposal
    -> Action JSON Schema
    -> current surface capability
    -> origin, network, and file policy
    -> current observation and target resolution
    -> surface driver and platform input
    -> StepOutput and Evidence
    -> next observation captures any new page revision

An ACL suite completes syntax and attribute admission before opening a surface. A persistent agent session also verifies that the action is in the session allowlist. Actions using @eN, @cN, @gN.M, or @vN are bound to the latest observation. Observe again after any step that can change the DOM, route, focus, tab, frame, or viewport.

Choose a stable target first

Target choice determines whether a test can be repeated across page revisions. Prefer this order.

TargetUse it forStability and boundary
role("button", "Save")A control with a correct accessible namePreferred user-facing identity; traverses open Shadow DOM
label("Email")Form controlsBinds the actual labeling relation
testid("checkout")A product-owned stable test identifierUse when semantics are insufficient
placeholder("Search")An input with a unique, stable placeholderIt should not compensate for a missing accessible label
text("Saved", true)Visible copy that is itself the identityThe second argument requests exact matching
css("[data-row]")Collections, geometry, or exact browser targetsCurrent-document semantics; does not pierce Shadow DOM
ref("@e4")A browser element from the latest observationValid only for the observation that produced it
Page Context @cNAn actionable Test Kit component nodeLatest observation; drift requires an exact delta proving its private node ID unchanged
automation_id("save-button")A GUI accessibility automation IDGUI only
visual_point("@v3", 120, 80)A GUI target absent from accessibility semanticsGUI only; coordinates, image digest, and observation must remain fresh

@uN is Test Kit UI-understanding evidence, not an action target. It can associate style, layout, state, motion, and component facts with a report, but it cannot enter click, fill, drag, or another input action.

ACL locator functions and Action JSON discriminants do not always use the same spelling. Use these machine forms with agent act or MCP test_act.

ACL formAction JSON target
ref("@e4"){"type":"ref","value":"@e4"}
css("#save"){"type":"css","selector":"#save"}
role("button", "Save"){"type":"role","role":"button","name":"Save"}
text("Saved", true){"type":"text","value":"Saved","exact":true}
automation_id("save-button"){"type":"automation_id","value":"save-button"}
visual_point("@v3", 120, 80){"type":"visual_point","snapshot":"@v3","x":120,"y":80}
testid("checkout"){"type":"test_id","value":"checkout"}
label("Email"){"type":"label","value":"Email"}
placeholder("Search"){"type":"placeholder","value":"Search"}

The two easy mistakes are ACL testid() versus JSON type = "test_id", and the CSS target's selector field rather than value. a3s-test agent schema remains authoritative for the installed protocol revision.

When a locator resolves to zero or multiple nodes, the driver returns a missing-target or ambiguity error. A3S Test never chooses the first match and never converts location failure into a product state.

Page, observation, and viewport

ActionWhen to use itMinimal ACLInternal behavior and output
navigateOpen a known HTTP(S) pageurl = "https://example.test"Checks exact origin, navigates, and invalidates old observations
snapshotExplicitly retain an ACL semantic snapshotinteractive = trueProduces a bounded observation and fresh surface references
viewportTest a responsive breakpoint or scalewidth = 390, height = 844Resizes the real viewport and advances the page revision
navigate "checkout" {
    url = "http://127.0.0.1:3000/checkout"
}

snapshot "interactive-controls" {
    interactive = true
}

viewport "mobile" {
    width = 390
    height = 844
    scale = 2
}

An agent session performs the equivalent observation through agent observe. snapshot is an explicit action in the action protocol, while observe is a perception turn in the session loop. Either can advance the observation, so an earlier ref is never a durable selector.

Pointer, keyboard, and form input

ActionPurposeImportant distinction or limit
clickActivate a button, link, or clickable targetDirect Web refs and CSS targets are scrolled into view first
hoverTrigger hover state, a tooltip, or delayed menuMoves the pointer without implying a click
focusEstablish explicit editing or keyboard contextThe current Web protocol requires a ref or CSS target
double_clickDispatch double-click semanticsDo not approximate it with two independent clicks
context_clickTrigger the page contextmenu behaviorWeb does not inspect the browser-native context menu
fillReplace the complete value of an inputBest for deterministic form values
typeAppend text through target-scoped typingResolves and focuses the target
insert_textInsert at an established caret or selectionHas no target and does not reset focus or selection
check, uncheckSet a checkbox or radio stateInapplicable control types fail closed
selectSelect one or more exact option valuesRequires at least one value; Web currently requires ref or CSS
dragDrag one target to anotherResolves and checks source and target independently
pressSend one key or chordUses the current keyboard context
wheelPerform real scroll or modified-wheel inputdelta_y is required; at least one delta must be nonzero

The current Web adapter accepts only a latest ref or explicit CSS for several browser-native commands: focus, double_click, context_click, type, uncheck, select, both ends of drag, upload, download, and a targeted wheel. These actions do not invent a role, label, or text fallback. click, hover, fill, and check are the actions that lower supported semantic targets into an adapter command.

focus "title" {
    target = css("#title")
}

type "append-title" {
    target = css("#title")
    value = " plan"
}

insert_text "insert-at-caret" {
    value = " approved"
}

check "terms" {
    target = label("Accept terms")
}

select "status" {
    target = css("#status")
    values = ["review", "published"]
}

drag "reorder" {
    source = css("[data-item='second']")
    target = css("[data-item='first']")
}

press "submit" {
    key = "Enter"
}

wheel "zoom-canvas" {
    target = css(".document-canvas")
    delta_x = 0
    delta_y = -120
    modifiers = ["control"]
}

fill, type, and insert_text have different input semantics. Use fill when a field must become one exact value. Use type when keystroke-style input behavior matters. Use insert_text only after the editor already has the intended caret or selection.

Tabs, frames, and dialogs

These actions change browser context. Observe again before addressing the next target.

tab "list" {
    operation = "list"
}

tab "open-docs" {
    operation = "new"
    url = "https://example.test/docs"
    label = "docs"
}

tab "switch-docs" {
    operation = "switch"
    tab = "docs"
}

frame "payment" {
    target = css("#payment-frame")
}

frame "return-main" {
    target = main()
}

dialog "accept-prompt" {
    operation = "accept"
    text = "Approved"
}

tab supports list, new, switch, and close. A tab can be addressed by its stable ID or user label. frame accepts the main document, a current ref, or a CSS frame target. dialog supports status, accept, and dismiss; only accept can carry prompt text. A missing pending dialog never becomes an assumed success.

Synchronize on conditions, not fixed sleeps

A Web wait accepts exactly one condition.

wait "document-ready" {
    load = "domcontentloaded"
}

wait "network-settled" {
    load = "networkidle"
}

wait "confirmation-copy" {
    text = "Order confirmed"
}

wait "confirmation-route" {
    url = "http://127.0.0.1:3000/orders/42"
}

wait "dialog-visible" {
    visible = role("dialog", "Checkout")
}

wait "dialog-gone" {
    hidden = role("dialog", "Checkout")
}

domcontentloaded reads current document readiness, so it remains deterministic when navigation finished before the wait began. networkidle uses the driver's bounded idle detector. Runner-owned hidden polls with a read-only visibility probe every 50 ms and completes immediately when the target is already hidden or absent. The scenario deadline, cancellation, and a 1,201-probe ceiling bound it.

A stable locator with no visible match can prove hidden state. An expired ref cannot, because staleness is not disappearance. wait hidden therefore rejects ref() and visual_point(). See Assertions and stability for the complete truth model.

Uploads, downloads, and network stubs

upload "attachments" {
    target = css("input[type=file]")
    paths = ["tests/fixtures/one.txt", "tests/fixtures/two.txt"]
}

download "report" {
    target = css("[data-download-report]")
    path = "downloads/report.pdf"
}

network_route "empty-users" {
    pattern = "**/api/users"
    body = "{\"users\":[]}"
}

network_route "block-analytics" {
    pattern = "**/analytics"
    abort = true
}

network_unroute "users" {
    pattern = "**/api/users"
}

network_unroute "all" {}

The CLI resolves relative upload paths from its current working directory before sending them to the browser. The driver still applies allowed-root, count, and size limits. A download path is always relative to the current scenario or session artifact root.

Each network_route selects exactly one response mode, static body or abort. Routes remain inside browser network policy and cannot reach an unadmitted domain. A patterned network_unroute removes matching rules; an empty block removes all routes created by the current session.

Record only useful evidence

ActionArtifactQuestion it can answer
screenshotPNGWhat did the final UI look like
accessibilityInteractive or complete semantic-tree JSONWhat semantics could a user or agent perceive
consoleBrowser console JSONDid the page emit script warnings or errors
page_errorsUncaught page-error JSONDid page runtime fail
harNetwork HAR for an explicit windowWere requests, responses, and timing correct
traceBrowser trace for an explicit windowWhy did a multi-step interaction fail
videoWebMDoes the state transition need human review
downloadProduct downloadDid a real download produce the expected file
har "start-network" {
    operation = "start"
}

trace "start-trace" {
    operation = "start"
}

video "start-video" {
    operation = "start"
    path = "video/checkout.webm"
}

accessibility "semantic-tree" {
    path = "evidence/accessibility.json"
    interactive = false
}

console "console-log" {
    path = "evidence/console.json"
    clear = false
}

page_errors "runtime-errors" {
    path = "evidence/page-errors.json"
    clear = false
}

screenshot "confirmation" {
    path = "screenshots/confirmation.png"
}

har "stop-network" {
    operation = "stop"
    path = "network/checkout.har"
}

trace "stop-trace" {
    operation = "stop"
    path = "traces/checkout.zip"
}

video "stop-video" {
    operation = "stop"
}

HAR and trace choose the artifact path on stop. Video chooses it on start and attaches the completed file on stop. clear = true clears only the driver's current console or page-error buffer; it does not mutate product state.

Every evidence path must be relative to the artifact root. Traversal, symlinks, Windows reparse points, non-regular files, and resolution outside that root fail closed. A stale file is removed before a fresh capture, so a successful command cannot reuse old evidence. Screenshots also have a 1-byte through 32-MiB admission bound.

TUI-specific actions

TUI is for known deterministic terminal workflows. It does not expose an interactive agent session.

terminal_resize "working-size" {
    columns = 120
    rows = 40
}

terminal_paste "command" {
    text = "open fixtures/report.txt"
}

press "submit" {
    key = "Enter"
}

wait "loaded" {
    regex = "Loaded [0-9]+ lines"
}

terminal_recording "evidence" {
    path = "terminal/editor.vt"
}

terminal_paste honors the application's current bracketed-paste mode. terminal_resize resizes the A3S Test-owned PTY or ConPTY. terminal_recording retains bounded VT evidence. A TUI wait accepts only text or regex; it never guesses how a browser URL, load, or element-visibility condition should map to a terminal.

TUI explicitly rejects browser-targeted type; use terminal_paste for text input. Generic viewport also maps width and height to terminal columns and rows when scale is absent. Prefer terminal_resize in terminal suites because its field meanings are explicit.

Send full Action JSON in an agent session

Common steps have compact commands such as agent click, agent fill, and agent press. Browser-context, network, and evidence steps can use the complete Action JSON form.

a3s-test agent act \
  --session checkout \
  --observation 7 \
  --action-json '{"type":"click","target":{"type":"ref","value":"@e3"}}' \
  --json

An action with no observation-bound ref can omit --observation. If the action contains a current ref, it must include the latest observation ID that created that ref. Strict schema admission rejects unknown fields.

Surface capability boundary

Capability groupWebGUITUI
ObservationInteractive or complete semantics, plus optional Page ContextActionable accessibility semantics; window vision can add a screenshotViewport, scrollback, cursor, and VT state
Common inputFull browser action setClick, double click, context click, fill, type, drag, press, and wheelPress and terminal paste
Browser context, network, Web evidenceSupportedUnsupportedUnsupported
ScreenshotSupportedSupported within bound window and artifact rootUnsupported
Terminal resize and recordingUnsupportedUnsupportedSupported; viewport without scale also resizes the grid

This table describes the current implementation. Runtime availability still depends on the installed inventory, driver version, and GUI certification. The existence of an Action variant does not imply that every surface can execute it.

Use the error code to choose the next step

Error familyWhat it usually meansWhat to inspect
test.spec.*ACL fields, combinations, or values failed admissionAttribute path, single condition, stable target, and bound
test.session.*Session, observation, or revision is no longer validRe-observe, session state, and browser containment
test.driver.web.*Web location, capability, protocol, I/O, or evidence failureTarget uniqueness, driver version, network, artifact path
test.driver.gui.*GUI permission, window, reference, or CUA failureHost certification, PID, window binding, latest screenshot
test.driver.tui.*Terminal action, PTY, or semantic-state failureExecutable, key, regex, output bound, and process lifecycle
test.assert.*Valid evidence was observed but product state differedExpected, actual, and attached evidence
Cleanup error fieldtest.run.cleanup_*, test.session.cleanup_*, test.agent.cleanup_*, or a driver cleanup codePreserve product outcome and inspect the separately reported cleanup proof

An infrastructure error cannot prove a product pass or failure. Preserve its ownership, then use the retryable field to decide whether the same session can retry. See Troubleshooting for the full workflow.

A practical minimum evidence set

Most Web regressions do not need full-session video. A useful, reviewable set usually does the following.

  1. Wait on a typed ready condition.
  2. Prove the outcome with one or more typed expectations.
  3. Save the final screenshot.
  4. Save the interactive accessibility tree.
  5. Save console and page errors.
  6. Enable HAR, trace, or video only when network or timing diagnosis needs it.

This preserves structured verdict evidence without burying the actual failure signal in unrelated binary artifacts.