Skip to content
Amomic-AI

Testing & publishing

Test an agent on the Test tab, record verdicts, read the readiness checks that gate publishing, and know what to do when something goes wrong on a live channel.

Open Agents

Every agent is good at the questions you thought of while building it. Testing is how you find out what happens with the other ones. Every failure you find here is a customer conversation that never goes wrong.

This page covers the Test tab, what to test, the readiness checks on the Overview tab, and what to do when a live agent misbehaves. Publishing itself, version history and rollback are on Versions.

Who can do this: workspace owners and admins.

Test before you save risky changes

Saved agent edits are live straight away on every channel the agent is deployed to. The Test tab runs the agent's saved settings, so the safe rhythm for a live agent is: make one small change, save it, test it at once, and roll back if it is wrong. For a big rewrite, try it in a second agent that is not deployed first. See When your changes reach customers.

The Test tab #

Open the agent and choose Test, or use Test in the studio header or the Test item on the agent card's … menu.

A threaded test. The third answer used no knowledge, which is exactly what you are looking for.

The Test conversation card is a real conversation with the agent. It remembers the thread, so you can ask a question and then follow up on the answer, which is where most agents break.

  1. 1

    Ask a question

    Type in the box and press Enter (Shift+Enter adds a new line), or choose a starter. A message can be up to 1,000 characters.

  2. 2

    Read the answer

    Under each answer you see how long it took, for example 2.4s, and either {n} sources or the badge No knowledge used. Choose 2 sources to see which knowledge passages the answer drew on, with links to their pages where there is one.

  3. 3

    Give a verdict

    Choose Pass or Needs work. The answer is marked Marked pass or Marked needs work, and a test run is recorded. See Verdicts and test runs.

  4. 4

    Start again when you change topic

    New thread clears the conversation and starts a fresh one. A question asked after five turns is not the same test as the same question asked first.

What the Test tab exercises #

The Test tab talks to the agent as web chat:

  • It uses the agent's Web chat persona if you set one under Per-channel persona, and the default persona otherwise.
  • Answers are formatted the way the web widget shows them, with links and lists.
  • It does not exercise the phone or WhatsApp personas, the phone deployment's opening line and voice, or the WhatsApp message formatting. Test those on the real channel.

Test conversations appear in the Inbox with a Preview badge, and the conversation itself reads Preview — not a real visitor. They are not billed, and they are left out of analytics and of a contact's history.

The thread on screen is not kept when you leave the tab. The verdicts you gave are.

The starters #

When the thread is empty, the card offers four starters: Pricing (How much does it cost?), Out of scope (Do you sell refrigerators?), Handoff (I want to talk to a human.) and Contact (What are your support hours and how do I reach you?). The What to try card beside the conversation offers the same four at any time. They are a warm-up, not a test plan.

Verdicts and test runs #

Each Pass or Needs work records a test run against the agent's current configuration. The Recent test runs card lists the last eight, each with the question you judged and the first characters of the configuration fingerprint it ran against.

The fingerprint covers the persona, the knowledge attached (and which content revision of each), and the tool settings. Change any of them and earlier runs no longer count for the readiness check, because a pass on a different setup says nothing about this one. Re-test after any real change.

A passing test is recommended, not required

The empty Test tab says a passing run "unlocks publishing". In fact, a missing or stale passing run is only a Recommended item on the readiness checks. It does not block publishing. Test anyway.

What to actually test #

Do not test the questions you know are covered. You wrote the knowledge base, so of course it answers those. Test the ones your own team gets wrong.

  1. 1

    The five questions your support inbox is full of

    Open your inbox (the real one, with people in it) and take the five most repeated questions from last month. Ask them in the customer's own words, typos included. If the agent cannot handle your top five, nothing else matters.

  2. 2

    The exceptions

    "What if I bought it in a sale?" "What if I'm in Guwahati?" "What if the box was opened?" Published policies describe the rule; customers ask about the edges. An agent that confidently answers the rule when the customer is describing an exception is worse than one that says it does not know.

  3. 3

    The follow-up

    Ask a question, then a second one that only makes sense given the first answer. "Is the drum noise covered under warranty?" then "And if I bought it on a sale site?" This is where thin knowledge shows: the first answer is fine, the second is invented.

  4. 4

    The out-of-scope question

    Ask about something you do not sell. A well-scoped agent declines cleanly. A vague one improvises. If it improvises, the fix is in the purpose, not the knowledge base.

  5. 5

    The hand-off

    Say I want to talk to a human. The agent should open a ticket, gather what it needs without interrogating you, and read back a reference. If it only apologises, check that Create support tickets is enabled on the Tools tab.

  6. 6

    The annoyed customer

    Type the message an annoyed person really sends: short, in capitals, no punctuation. How the agent handles tone decides whether a complaint resolves or escalates.

The two failure modes #

Almost everything you find is one of these two. They need opposite fixes, so telling them apart is the whole skill.

1. It answers from nowhere #

The agent states a policy, price, timing or rule that is not in any of your sources. It is usually fluent and plausible, and occasionally right by luck.

How to spot it: the No knowledge used badge, or sources whose passages have nothing to do with the claim.

What it means: your knowledge is thin here and the model is filling the gap.

The fix is content. Add the source. If the fact is on your website but was not crawled, check that the page is inside the section the crawler was pointed at. If it is written down nowhere, add it as a Text source. See Knowledge bases.

Do not fix this with a rule in the instructions. "Never say the AMC costs ₹2,499" patches one sentence and leaves the gap.

2. It refuses something it should know #

The agent says it does not know, and you can see the answer on your own website.

How to spot it: open the knowledge base, use the Retrieval tester, and ask the same question. It shows which passages would be retrieved, with the model out of the picture.

  • The tester finds nothing relevant: the passage does not exist in a usable form. Usually the answer is inside an image, inside a table that flattened into meaningless text, on a page that needs a click to load its content, or outside the crawl.
  • The tester finds the right passage but the agent still refused: retrieval worked and the model did not use it. That is a persona problem. Check the purpose for a boundary that excludes this topic, and check whether Strictly grounded is too tight for how that passage is written.

Tables and images are the usual culprit. A service-charge matrix or a warranty card that a person reads at a glance often becomes unusable as plain text. Rewrite the rows customers actually ask about as sentences in a Text source.

Readiness checks #

The Overview tab's Readiness card shows whether the agent can be published: Ready to publish or Needs attention. It is worked out from the current agent, its knowledge revisions, its tool settings and the latest test runs.

Four tiles show each check passing, failing or warning. Below them, each open item has its message, a label, and a Resolve link to the tab that fixes it.

Check Message Label Fails when
Agent Add an agent name and purpose. Required before publishing The name or the purpose is empty.
Knowledge Attach at least one ready knowledge base. Required before publishing The knowledge requirement is Require ready business knowledge and no attached knowledge base is ready.
Test Run a successful test against the current configuration. Recommended No Pass has been recorded against the current fingerprint.
Tools — — Always passes. It does not check your tool settings.

Only the two Required before publishing items block publishing. When nothing is open, the card reads Every current check passes.

Next to Readiness:

  • Publish shows Publish first version or Publish new version. The button is disabled until the agent is ready. It opens the publish dialog described in Publish a version.
  • Current state shows how many attached knowledge bases are ready, how many deployments are active, and whether the agent has unpublished changes.

Because the Tools check never fails, readiness does not tell you that WhatsApp sending has no approved template or that booking has no calendar. Check the Tools tab yourself before you publish.

Publish, then deploy #

  1. 1

    Publish

    From Overview or Versions, publish. This records the current setup as a numbered version, with an optional heading. A heading such as Stopped quoting AMC prices in chat is the only human-readable record of why the version exists. See Versions.

  2. 2

    Deploy, once

    An agent must be published before its first deployment. From the Deployments tab, put it on web chat, WhatsApp or a phone number. See Deployments.

    After that, you do not redeploy for agent changes. A deployment always serves the agent's saved settings.

  3. 3

    Try it on the real channel

    The Test tab tests the agent, not the channel. Send yourself a WhatsApp message, open the page with the widget on it, or ring your own number.

    Channel problems do not show in the studio: a domain missing from the widget's allowed list, a WhatsApp template that was never approved, an opening line that reads well but sounds odd out loud. Five minutes on the real channel catches all three.

When something goes wrong on a live channel #

Quickest first:

  1. Take the channel out of service if the harm is ongoing. On the Deployments tab, open the row's menu and choose Disable. The agent stops answering on that channel straight away. See Disable, enable and remove.
  2. Roll the agent back. If a recent change caused it, roll back to the last good version on the Versions tab. It takes effect on every channel at once. See Roll back.
  3. Diagnose. Read the conversation in the Inbox. It shows whether the agent looked something up, called a tool, or answered with no knowledge. Those are three different bugs with three different fixes.
  4. Fix, test, save, publish. Make the fix, test it on the Test tab, save, then publish so the good state is recorded. If you disabled a deployment, Enable it again.

A rollback restores the agent only. A phone number's greeting, voice or recording settings belong to the deployment and are changed in its Voice settings.

Troubleshooting #

What you see Why What to do
Publish first version is greyed out A Required before publishing item is open. Choose Resolve on it.
The test check went back to warning You changed the persona, knowledge or tools, or a knowledge base refreshed. Run the test again and mark a Pass.
No knowledge used on a factual answer The agent did not find a passage and answered anyway. Add the content. See The two failure modes.
The answer is fine on the Test tab but wrong on the phone The phone channel has its own persona or deployment settings. Check Per-channel persona and the deployment's Voice settings, then test by calling.
A test answer shows an error instead of a reply The test request failed or timed out after 60 seconds. Try again. If it keeps failing, start a New thread.
Test conversations in the Inbox Tests are real conversations, marked Preview. Nothing to do. They are not billed or counted in analytics.

Where to go next #

Fix a content problem

Knowledge bases: sources, the retrieval tester, and getting good answers.

Fix a behaviour problem

Agents: the purpose, instructions and grounding policy.

Record and roll back

Versions: publish, compare and roll back.

Check the whole picture

Going live: everything worth confirming before real traffic arrives.