Training Quality

The Training Quality dashboard shows how well your AI chatbot answers visitor questions and highlights areas where it needs more training data.

How to get there: Go to Setup → Chatbot in the top menu → click your chatbot → Quality tab (under the Training section in the sidebar).

How It Works

After each chatbot conversation ends, Boei automatically analyzes the exchange and scores each question-answer pair. This helps you see which topics your chatbot handles well and where it struggles.

Conversation Lifecycle

  1. A visitor chats with your AI chatbot
  2. After 20 minutes of inactivity, the conversation is automatically closed
  3. Boei's AI analyzes the conversation and generates quality scores
  4. Results appear in the Training Quality dashboard

If a visitor returns and sends a new message, the conversation reopens automatically. When they go idle again, it closes and gets re-analyzed with the full conversation including the new messages.

Dashboard Overview

Time Filters

View quality data for different periods:

  • Last Hour - Most recent conversations
  • Today - All conversations from today
  • Last 7 Days - Weekly overview (default)
  • Last 30 Days - Monthly overview

Summary Cards

  • Knowledge Coverage - Percentage of questions answered well (score 60+)
  • Average Score - Average quality score across all questions (0-100)
  • Good Answers - Number of well-answered questions
  • Knowledge Gaps - Number of poorly-answered questions

Pending Conversations

A blue banner shows how many conversations are still open and waiting to be assessed. These are active or recently active conversations that haven't been closed yet. They'll be included in the assessment automatically once the visitor goes idle for 20 minutes.

Knowledge Gaps

Questions where your chatbot scored below 60 are listed as knowledge gaps. These are topics where visitors asked questions but the chatbot couldn't provide a good answer.

How to fix knowledge gaps:

  1. Look at the query topics listed
  2. Add or improve training content that covers those topics
  3. Retrain your chatbot
  4. Monitor the dashboard to see if scores improve

Top Knowledge

Questions where your chatbot scored 60 or above. These represent topics your chatbot handles well. The frequency count shows how often each topic comes up.

Quality by Tag

If you have training tags enabled, the dashboard breaks down quality metrics per tag so you can see which content areas perform best.

Tips

  • Check regularly - Review knowledge gaps weekly to continuously improve
  • Focus on frequent gaps - Prioritize fixing gaps that occur most often
  • Add specific content - When you see a gap, add training content that directly addresses that topic
  • Allow processing time - New conversations take a few minutes after closing to appear in the dashboard
Made with ProductLift