#LLM, grounding, and design providers

A3S Test defines model requests, responses, provenance, budgets, and authority. It does not bundle model weights or an inference runtime. A deployment may put any model service behind these protocols if its adapter returns the exact generated schema and accepts A3S Test's local revalidation.

Providers add proposal capabilities where machine inference is useful. Browser observation, deterministic assertions, human approval, and workspace authorization remain with their respective authoritative layers.

#The four providers are not interchangeable

Discover each installed protocol first.

a3s-test provider schema llm
a3s-test provider schema contract-generation
a3s-test provider schema visual-grounding
a3s-test provider schema design-audit

Each output contains transport-neutral request and response schemas, standard HTTP envelopes, and safety invariants. Unknown fields are rejected. An incompatible change requires a new protocol ID.

#Shared HTTP and credential boundary

All four CLI adapters use the same deployment principles.

• Endpoints require HTTPS, except explicit loopback addresses may use HTTP.
• An endpoint cannot contain credentials, a query, or a fragment.
• An authorization environment variable must begin with A3S_TEST_PROVIDER_AUTHORIZATION_.
• Its value is the complete Authorization header and never enters ACL, command arguments, session metadata, or reports.
• The adapter does not follow redirects or use environment proxies and bounds request and response bodies.
• HTTP must return 200 with a JSON media type; transport success still undergoes typed response admission.
• The earlier of the configured timeout and wire deadline applies.
• Provider identity, model, request binding, usage, and cost are rechecked locally.

A standard response selects exactly one of success or failure.

{
  "protocol": "a3s.test.visual-grounding-provider/2",
  "status": "failure",
  "error": {
    "code": "capacity_exhausted",
    "message": "queue is full",
    "retryable": true
  }
}

HTTP 200 does not mean the capability succeeded. status = "failure" remains a provider failure, and the caller can use the bounded retryable field only within the original deadline.

#Use an LLM provider for one bounded Web workflow

An external coding agent using agent start -> observe -> act is already the planner and does not need a nested LLM configuration. Use agent run only when A3S Test should own the complete one-shot model loop.

#Configuration

agent_run "checkout" {
    url = "http://127.0.0.1:3000/checkout"
    goal = "Complete checkout with the fixture account"
    success_criteria = ["The order confirmation is visible"]
    allow_origins = ["https://auth.example.test"]
    allow_domains = ["cdn.example.test"]
    allow_actions = ["click", "fill", "wait"]
    max_turns = 8
    max_total_tokens = 20000
    max_cost_microusd = 50000
    max_context_bytes = 524288
    timeout_ms = 120000

    provider {
        name = "deployment"
        model = "planner"
        endpoint = "https://models.example.test/v1/plan"
        authorization_env = "A3S_TEST_PROVIDER_AUTHORIZATION_DEPLOYMENT"
    }

    verification {
        expect "confirmation" {
            text = "Order confirmed"
        }

        screenshot "final" {
            path = "confirmation.png"
        }
    }
}

export A3S_TEST_PROVIDER_AUTHORIZATION_DEPLOYMENT='Bearer ...'
a3s-test agent run tests/checkout.agent.acl --json

Required fields include URL, goal, non-empty success criteria, allowed actions, a cost ceiling, one provider, and one verification block. The complete runtime-budget admission is below.

allow_actions accepts only these unique values.

navigate, snapshot, click, hover, focus, double_click, context_click,
fill, type, check, uncheck, select, drag, press, wheel, viewport,
wait, assert, screenshot, tab, frame, dialog, upload, download,
network_route, network_unroute, har, trace, video, accessibility,
console, page_errors

This CLI workflow opens only a Web surface, so it does not admit terminal_paste, terminal_resize, or terminal_recording. The type allowlist entry governs both targeted type and focus-owned insert_text. Runner-owned verify_contract can never be proposed by the model. The workflow deadline and cleanup deadline remain independent.

#What the model receives on each turn

Protocol a3s.test.llm-provider/1 sends these structured fields.

The provider returns one JSON decision, token and micro-USD usage, and an optional request ID. Even if a model claims structured output, Core deserializes and validates again. An action also passes allowlist, surface capability, origin, observation revision, and target admission.

The internal loop is fixed.

open Web surface
    -> observe
    -> build bounded model context
    -> provider proposes one typed decision
    -> local schema and policy admission
    -> execute one action
    -> observe again
    -> provider proposes finish
    -> run local deterministic verification
    -> close exact owned surface

Model finish is provisional. Verification accepts only snapshot, wait, expect, screenshot, accessibility, console, and page_errors, and requires at least one expect. A final report can pass only when the model finishes, local verification passes completely, and surface cleanup succeeds.

The report protocol is a3s.test.agent-run/1, written by default to .a3s-test/agent-runs/<run-id>/report.json. It retains provider identity and usage, decision digests, observations, action outputs, verification, and a separate cleanup error, with bounded redaction before atomic publication.

#Turn sources into candidates with contract generation

This capability interprets PRD spans and design-image regions as Expected Surface candidates. It does not generate a browser accessibility tree and does not directly publish an executable contract.

contract_generation "checkout" {
    max_cost_microusd = 50000

    context {
        mode = "operate"
        audience = ["customer"]
        primary_outcome = "place_order"
    }

    provider {
        name = "deployment-gateway"
        model = "interface-contract-model"
        endpoint = "https://inference.example.test/v1/contracts"
        authorization_env = "A3S_TEST_PROVIDER_AUTHORIZATION_CONTRACTS"
    }

    source "requirements" {
        kind = "prd"
        path = "./checkout.md"
        uri = "./checkout.md"
    }

    source "desktop-design" {
        kind = "design"
        path = "./checkout.png"
        uri = "./checkout.png"
        media_type = "image/png"
        width = 1440
        height = 900
    }
}

a3s-test contract generate \
  --config tests/contracts/checkout.generate.acl \
  --output tests/contracts/checkout.draft.json

#Source fields and generation limits

context requires mode = "persuade" | "operate" | "read" | "experience", a non-empty audience, and primary_outcome. Every uniquely labeled source requires kind and a relative path contained by the configuration directory; omitted uri inherits path. A design source also supplies the real image media_type and positive width and height. A PRD source rejects those image fields.

Every optional limit is at least 1. max_cost_microusd is required and bounds the cost reported by the provider.

The CLI computes source SHA-256 values and does not accept claimed digests from configuration. A PRD candidate requires an exact source byte span. A design candidate requires an in-bounds pixel or normalized region and consistent visual and semantic parentage. Files are checked before and after the call. Symlinks, directory escape, content drift, cyclic parents, unknown sources, duplicate elements, and budget overrun fail closed.

The model never picks a winner when source fields differ. Generation turns differences into stable conflicts. A human must approve or reject every applicable candidate, resolve every conflict, and provide rationale before contract review publishes canonical Surface Contract ACL. See Turn PRDs and designs into a verifiable interface contract for the complete flow.

#Use visual grounding only for location advice

Visual grounding is intended for canvas, image-only controls, remote desktops, design references, or a real semantic-location miss. Ordinary Web elements should still prefer role, label, test ID, placeholder, text, and CSS.

#Configuration and call

visual_grounding {
    max_cost_microusd = 10000
    timeout_ms = 15000
    max_candidates = 32
    max_query_bytes = 4096
    max_label_bytes = 1024

    provider {
        name = "deployment-gateway"
        model = "ui-grounding-model"
        endpoint = "https://inference.example.test/v1/ground"
        authorization_env = "A3S_TEST_PROVIDER_AUTHORIZATION_GROUNDING"
    }
}

a3s-test agent ground "Checkout button in the canvas" \
  --session checkout \
  --observation 7 \
  --config tests/providers/grounding.acl \
  --reason canvas \
  --json

--reason is one of explicit, canvas, image-only, remote-desktop, design-reference, or no-semantic-match. Natural-language keywords cannot activate the provider implicitly.

#How image and geometry are revalidated

The request binds the latest observation ID, Test Kit surface revision, PNG SHA-256, dimensions, query, trigger, deadline, and cost ceiling. The HTTP envelope carries Base64 image/png; the remote service is never asked to read a client-local path. Decoded PNG is limited to 32 MiB and the JSON envelope to 64 MiB.

A response may use screenshot-pixel or normalized coordinates and return bounded points or positive-sized boxes. A3S Test then performs these checks.

1. Read and hash the image again.
2. Match provider, model, observation, revision, dimensions, and usage.
3. Reject non-finite, out-of-bounds, or malformed geometry.
4. Map box centers to current visual-viewport CSS pixels.
5. Hit-test only visible, non-occluded Page Context nodes with usable semantic targets.
6. A unique hit may return a current semantic suggestion; zero or multiple hits retain image-bound ambiguity.

The result always has authority = advisory. It is not a durable ref, contract evidence, action permission, or Repair Ledger finding. Success dispatches no click. Page revision drift also invalidates the original observation.

#Use design audit for traceable design advice

Design audit combines a screenshot with complete forensic Page Context. It can review hierarchy, composition, spacing, typography, color, consistency, interaction copy, and responsive behavior. It is an explicit persistent Web session operation, not a deterministic expectation.

#Configuration and call

design_audit {
    max_cost_microusd = 20000
    timeout_ms = 30000
    max_findings = 100
    max_summary_bytes = 2048
    max_rationale_bytes = 8192
    max_recommendation_bytes = 8192
    max_page_context_bytes = 8388608

    provider {
        name = "deployment-gateway"
        model = "design-review-model"
        endpoint = "https://inference.example.test/v1/design-audit"
        authorization_env = "A3S_TEST_PROVIDER_AUTHORIZATION_DESIGN_AUDIT"
    }
}

a3s-test agent audit \
  --session checkout \
  --observation 7 \
  --config tests/providers/design-audit.acl \
  --dimension visual-hierarchy,spacing-rhythm \
  --json

Nine dimensions are available.

Omitting --dimension requests all dimensions. A duplicate dimension is rejected before session access.

Each finding requires a unique ID, requested dimension, high, medium, or low priority, summary, rationale, recommendation, confidence from 0 through 100, and exactly one page, current visible node, or in-image region target. A3S Test rejects stale nodes, out-of-bounds regions, duplicate IDs, unrequested dimensions, digest drift, cost overrun, and revision drift.

The admitted report protocol is a3s.test.design-audit-report/1 with advisory authority. A compatible Test Kit can display it in a separate audit layer. Dismissing advice has no product effect, and opening review grants no authority. A finding reaches the existing Repair Ledger only after a human explicitly saves or submits it individually or in a batch.

#Evaluate a model integration by protocol evidence

A model name alone cannot prove integration quality. A deployment review should establish at least these facts.

• The adapter fully implements the current generated schema rather than returning approximate JSON.
• Images are digest-bound and sent as request attachments.
• Context, image, response, candidate, and finding bounds agree at both sides.
• Token and micro-USD usage is truthful and can trigger local budget rejection.
• Timeout, cancellation, and retryable errors retain correct ownership.
• The deployment owns model licensing, weight distribution, inference data retention, and regional compliance.
• Provider unavailability fails closed instead of falling back to keywords or undeclared heuristics.

A vision model may implement visual grounding or design audit, and it may propose design candidates for contract generation. That does not grant browser actions, assertion verdicts, or human repair authority. The protocol boundary is more stable than any specific model family.

#Triage common failures in order

A successful provider response is only intermediate evidence. The result must still pass the corresponding local admission, deterministic verification, or human review.