testing

Testing

Ariadne agents can register a test suite for a project, run it, and report structured results back in chat — without you leaving the conversation to check a CI dashboard. This is the first phase of a larger testing capability (test authoring, change-targeted runs, git watchers and auto-publishing PRs are designed but not yet built); today it covers registering, running and reviewing an existing suite.

Requirements

• A Console or Desktop client connected to the conversation (Code mode), with a working directory pointed at the project — the test project lives on your machine or in your headless work container, not on the Ariadne server. See Code Mode & Remote Development.
• The agent's Testing tool collection enabled (see Agents → Tool Collections).

There is no separate Testing page yet — everything happens through the agent in chat.

Registering a suite

Ask the agent to register your project, e.g. "Set up testing for this project". Behind the scenes this calls create_test_suite with a name and the project's working directory. The agent detects the test framework automatically from project markers (NUnit/xUnit .csproj, pytest.ini/pyproject.toml, package.json with Jest/Vitest) — if more than one is plausible, it asks which one you mean.

A suite optionally records a separate subject path, for cases where the tests live in a different project or repo from the code they exercise.

Listing suites

Ask "what test suites do I have?" — the agent calls list_test_suites and shows each suite's id, project path and detected framework. You'll need a suite id (or its name) to run it or fetch results.

Running tests

Ask the agent to run a suite, optionally with a filter (a test name or category). This calls run_tests, which builds the right command for the detected framework and runs it in your working directory via the connected client, then returns a summary: total / passed / failed / skipped.

A few things worth knowing:

• Runs are capped at 270 seconds. If your suite runs longer than that, the run fails with a clear timeout message rather than hanging silently — split the run with a filter, or ask the agent to run a subset.
• A filtered run is not evidence the whole suite passes. It's recorded as a targeted run, and the agent is expected to say so rather than imply a full green suite.
• Full runner output (the .trx/JSON/JUnit file) is kept in blob storage, not stuffed into the conversation — the agent reads it on demand if it needs more detail than the summary gives.

Reviewing results

Ask "show me the failures from the last run" — the agent calls get_test_results, which pages through a run's results with an optional outcome filter (passed / failed / skipped / error). By default this doesn't include full per-test output; ask explicitly if you want it.

What's coming

The fuller design (see docs/design-automated-testing.md for engineering detail) covers agent-authored tests with provenance tracking and negative controls (so a generated test is proven capable of failing before it's trusted), change-targeted test selection driven by what actually changed in your repo, git watchers that trigger a run on every push, and automatic branch/PR publishing once new tests demonstrably work. None of that is available yet — today's capability is register → run → review.