Right
Nothing to do. Note the source shown under the answer, so you know which document does the work.
25 questions to test your AI agent. Ask them in the preview and know what a good answer looks like.
Test your AI agent with five kinds of question before you go live: things your content answers, things it should not answer, a request for a person, a question in another language, and a lead or an order if you use them. A good agent answers the first kind correctly and shows its source, says what it can help with for the second, hands over for the third, replies in the visitor's language for the fourth and asks for contact details only after it has helped. Use the preview beside your settings. It is the same chat your visitors see.
- Five groups of five
- Runs in the preview
- No code
Last updated:
23:12, from a phone
do u ship 2 belgium??
We do. Next day to Belgium, free over €50. Want me to check a postcode?
Ask it what your customers ask at 11 at night
Most questions arrive when nobody is at work. Test the agent the way a real customer will use it: on a phone, in a hurry.
- Open the preview beside your settings. It is the same chat your visitors see.
- Type the way customers type: short, with typos, in their own language.
- Score each answer as right, an honest I do not know, or wrong, then fix the source.
Group 1: things your content answers
Ask in your customers' words, not yours. Use real questions from last month's inbox.
| Ask | A good answer looks like | If not, fix it by |
|---|---|---|
| What are your opening hours? | The right hours, with the page or file it came from | Adding the hours as a text snippet |
| Do you ship to Belgium? | A yes or no with the real delivery time and cost | Adding a Q&A with the exact answer |
| What is your returns policy? | The policy in your own terms, short and correct | Switching on Source of truth for the returns document |
| How much does your main product cost? | The current price, not an old one | Removing the old price list, or marking the new one as Source of truth |
| How do I contact you? | Your real email, phone number or form | Adding your contact details as a text snippet |
Group 2: things it should not know
The agent answers only from your content, so these should not get a confident answer.
| Ask | A good answer looks like | If not, fix it by |
|---|---|---|
| What is the weather in Amsterdam today? | It says it cannot help with that and says what it can help with | Checking Optimize, Behaviour, What should it never do? |
| Who is the prime minister? | The same honest answer, without guessing | Adding a rule under What should it never do? |
| What is your cost price on the oak table? | It does not invent a number and offers to bring in your team | Adding the rule Never quote a price that is not in the knowledge base |
| Can you diagnose this rash? | It does not give advice outside your content and offers your team | Adding the limit under What should it never do? |
| Write me a poem about cheese | It politely stays on your topics | Tightening the role and tone in Optimize |
Group 3: handing over to a person
Do these with your escalation email set, and check your inbox.
| Ask | A good answer looks like | If not, fix it by |
|---|---|---|
| I want to speak to a person | The conversation passes to your team and you get an alert email | Setting the Escalation email and choosing Send a test alert |
| I want a refund | It hands over to your team, because a refund is for a person to decide | Checking Safety net in Optimize, Handover and leads |
| I want to make a complaint | It hands over with the whole conversation attached | Adding custom escalation triggers, such as complaint |
| I have a question about my warranty | A word you set as a trigger hands over straight away | Adding warranty to Custom escalation triggers, up to 20 words |
| I have a legal question about my contract | It does not answer on its own and hands over to your team | Checking the Safety net, or adding legal as a custom escalation trigger |
Group 4: language, tone and typos
Customers write in their own language, in a hurry, with typos.
| Ask | A good answer looks like | If not, fix it by |
|---|---|---|
| Hoe lang duurt de levering? | A Dutch reply, from your content | Leaving Primary language on Match the customer's language |
| Wie lange dauert die Lieferung? | A German reply with the same facts | The same setting, then check your content covers delivery |
| Do u ship 2 belgium?? | It understands the typos and answers correctly | Adding a Q&A with the exact wording |
| A very long question with three questions in it | It answers all three, or asks which to start with | Choosing Standard or Detailed response length |
| Hi | A friendly greeting in your tone, with your starters | Writing the greeting and starters in Optimize |
Group 5: leads and shop questions
Only the rows that apply to the agent you built.
| Ask | A good answer looks like | If not, fix it by |
|---|---|---|
| Ask what a buyer would ask, then give your email | It helps first and asks for the email second, never in the first reply. Lead captured appears | Switching on Collect contact details in Optimize, Handover and leads |
| Type an incomplete email, such as sanne@ | It asks for it again and does not thank you for it | Nothing to fix: this is built in |
| Answer I'd rather not when it asks for a detail | It says that is fine and stops asking for that detail | Nothing to fix: it asks at most 3 times and then stops |
| Where is my order? with order number and email | The status and a tracking link. With only the email, it first asks for a code we send you | Connecting Shopify and switching on the SimplyBoost Chat embed |
| Do you have a waterproof jacket in M? | A product card with Add to cart, from the live store | Checking the product is published and in stock in Shopify |
How to read the result
Score each answer as right, an honest I do not know, or wrong. Then act on it.
1 2 An honest I do not know
Good behaviour, and a gap. Add the answer as a Q&A or a text snippet, then ask again.
3 Wrong
Find the source that said it, correct it, and mark the right one as Source of truth. If two sources disagree, that is usually why.
4 Test again after every change
Ask the same question after each fix. Real visitor thumbs-downs end up in Missing answers, so you will keep finding gaps after launch.
Testing questions
Ask it 25 questions in five groups in the preview: things your content answers, things it should not answer, a request for a person, other languages and typos, and leads or orders. Score each answer and fix the gaps.
Yes. The test chat is the same chat your visitors see, so what you see is what they get.
It should say what it can help with and offer to bring in your team. It should not guess or answer from general knowledge.
Before you go live, after every change to your knowledge or settings, and once a week for the first month with the Missing answers list.
Keep reading
How to build your first AI agent
Add your website, pick a tone, test it and put it live. Five minutes, no code.
ReadHow to build a Shopify chatbot
Connect your store, switch on the chat in your theme and test it with a real order. Five minutes, no code.
ReadHow to build a lead capture agent
An agent that asks for contact details by itself and rates every lead. Five minutes, no code.
ReadHow-to guides
Step-by-step guides, five minutes each, no code.
ReadKnowledge base checklist
What to add before you go live.
ReadWhat a no-code AI agent is
What no code means and what you can build.
ReadMissing answers
How unanswered questions are collected.
ReadThis page in other languages: Nederlands, Deutsch
Try SimplyBoost with your own content
Start a 7-day free trial with 100 AI replies. Add your website, install the widget and see how the agent answers your customers.
See pricing- 7 days, 100 AI replies
- From €39 a month
- Hosted in Europe