A chat question branches to the publisher's own tent review, a separately marked outside source, and a product card that is held back.
Tutorials

How to Test AI Chat on Topics Your Site Never Covered

Readers will ask about things your editors never tested. Use one guide and four questions to check how chat labels outside sources and product links.

4 min read
Publisher AI chat source scopeAI chat uncovered questionsAI chat outside sources

To learn what AI chat tells readers about topics your site never covered, ask it four questions from one real guide: one your articles answer, one next to your coverage, one far outside it, and one where an outside source disagrees with your editors. A passing answer uses your coverage first. It says plainly when your editors have not reviewed something. It labels every outside source, and it never presents an unreviewed product as your pick. Decide section by section, where your setup allows it, whether outside answers and product links can appear at all.

Pick one guide and write down what your editors actually covered

Use a staging copy of a guide with clear edges. In this fictional example, a camping site has reviewed three two-person tents, and its top pick is the Ridgeback 2P. The site has never tested sleeping pads, and it has no coffee coverage at all. Before you open the chat, list the products and claims the guide supports, the related articles that exist, and the topics your site has not reviewed. That list is your answer key.

Agree on the rule before you test. Some editorial teams want chat to stop at their own coverage. Others accept outside answers as long as readers can see where each one came from. Write the rule down so the test has a pass mark.

Ask four questions in fresh sessions

Open the guide in a new browser session for each question so earlier answers do not carry over.

  • Covered. Ask “Which of these tents is lightest?” Expect an answer that matches the guide, with a link or citation to your article.
  • Next to your coverage. Ask “Which sleeping pad should I use with the Ridgeback 2P?” Your site never reviewed pads. A passing answer says so. If it draws on outside sources, it names them. If a product appears, it is not described as tested, recommended or picked by your editors.
  • Outside your coverage. Ask “What is the best espresso machine?” The answer should follow your written rule. Either it says the site does not cover the topic, or it gives a clearly sourced outside answer. An answer in your publication's voice with no source fails.
  • Disagreement. Ask about a tent detail where a manufacturer page or another publication contradicts your review, such as packed weight. Expect your editors' finding first, with the disagreement and the other source named.

For each run, save the question, the full answer, every source link shown, and whether each source is yours. Record any product cards or buying links, and how the answer describes them. A sponsored or affiliate product next to an uncovered question should carry the same disclosure your site uses elsewhere.

Compare source controls in Gist Answers and Brambles.ai

Gist Answers documents three source scopes. A single publication draws answers only from your own site, a whole organization pools the titles you own, and the full Gist network adds more than 1,600 licensed titles. Gist says it always looks at your own site first, and it shows source cards and inline citations with proportional attribution. Its product page does not say what a single-publication setup answers when your site has nothing on the topic, or whether it suggests products. Ask about both in a pilot, then run the sleeping-pad and espresso questions under each scope you are considering.

Brambles.ai's Publisher overview says it indexes your entire site and shows AI answers with inline source links to your articles. The same page describes products from affiliate networks with price and stock checks, and sponsored products in AI results. The page does not describe source-scope settings or say what chat answers when your site has no article on the topic. In a Brambles.ai pilot, run all four questions, and check whether product suggestions appear on the sleeping-pad question, how they are labeled, including any Editor pick tag, and whether you can limit them by section.

Neither page says what its chat answers on topics your site never covered. The four questions show what your chosen setup actually does on your guide.

Decide where outside answers and product links can appear

  • Allow outside answers in a section when the covered question cites your article, uncovered questions are clearly labeled with named sources, and the disagreement case puts your editors' finding first.
  • Keep product links off uncovered categories until every product shown there is clearly marked as not reviewed by your editors and carries your usual disclosure.
  • Turn chat off for a section if any answer presents an unreviewed product as your pick or repeats an outside claim in your voice. Fix the source scope and rerun all four questions.

Rerun the test after you change source scope, add a section or start a new sponsorship. Bring the guide, your coverage list and the four questions when you review Brambles.ai's Publisher plans. If you run several titles, use the sister-publication test to check answers that draw on your other brands, and the editorial corrections test to check what happens after an article changes.

Explore Brambles.ai

Learn more about our AI-powered agentic commerce platform, agentic shopping, and shopping assistance solutions.

Explore More Insights

Discover more articles on AI, automation, and business innovation

View All Articles