Most chatbot dashboards show 40+ metrics. Total conversations. Message volume. Session duration. Response time. Engagement rate.
Most of these are vanity metrics that tell you nothing actionable.
Here are the only 5 numbers that actually matter—with benchmarks and improvement strategies.
The Vanity Metric Trap
Before the useful metrics, here's what to ignore:
Total conversations: High number? Great. But what if 80% are one-message bounces? Volume means nothing without quality.
Average session duration: Longer isn't better. A 30-second resolution beats a 10-minute frustration spiral.
Messages sent: More messages ≠ more engagement. Efficiency beats verbosity.
"Accuracy rate": Too subjective to be actionable. How do you define "accurate"?
Metric 1: Resolution Rate
What it measures: Percentage of conversations where the customer's problem was solved without human help.
Why it matters: This is your chatbot's GPA. Low resolution = expensive human escalations. High resolution = cost savings and happy customers.
How to calculate:
Resolution Rate = (AI-resolved conversations / Total conversations) × 100
What counts as "resolved":
- Customer explicitly says problem is solved
- Customer completes desired action (purchase, signup)
- Conversation ends naturally without escalation
- Customer gives positive rating
Benchmarks:
| Rating | Resolution Rate |
|---|---|
| Poor | Under 50% |
| Acceptable | 50-70% |
| Good | 70-85% |
| Excellent | Over 85% |
How to improve:
- Review unresolved conversations—what questions stumped the bot?
- Add content covering those topics
- Bot learns automatically on next scan
Metric 2: Goal Completion Rate
What it measures: Percentage of conversations where the customer completes YOUR goal—not just their question.
Why it matters: Resolution measures if customers are happy. Goal completion measures if the chatbot drives business results.
Example goals:
- E-commerce: Purchase completed
- SaaS: Trial signup
- Lead gen: Contact info submitted
- Service business: Appointment booked
How to calculate:
Goal Completion = (Conversations with goal action / Total conversations) × 100
Benchmarks:
| Goal Type | Acceptable | Good | Excellent |
|---|---|---|---|
| Trial signup | 3-5% | 5-8% | 8%+ |
| Lead capture | 5-10% | 10-15% | 15%+ |
| Purchase | 2-4% | 4-7% | 7%+ |
How to improve:
- Add clear CTAs in bot responses
- Set up proactive messages on high-intent pages
- Make the path from conversation to goal as short as possible
Metric 3: Deflection Rate
What it measures: Percentage of potential support tickets the bot handled instead.
Why it matters: This is your direct ROI calculation. Every deflected ticket saves money.
How to calculate:
Deflection Rate = (Bot-resolved / (Bot-resolved + Human-escalated)) × 100
Dollar value:
Monthly Savings = Deflected conversations × Cost per human ticket
Example: Cost per ticket $18. Bot deflects 720 tickets/month. Savings = $12,960/month.
Benchmarks:
| Rating | Deflection Rate |
|---|---|
| Poor | Under 50% |
| Acceptable | 50-65% |
| Good | 65-80% |
| Excellent | Over 80% |
Warning: 95%+ deflection is suspicious. May mean bot is refusing to escalate when it should. Check CSAT scores.
Metric 4: Customer Satisfaction (CSAT)
What it measures: How happy customers are after interacting with your chatbot.
Why it matters: High resolution + low satisfaction = something's wrong. Bot may be technically answering but in a frustrating way.
How to collect: After resolution, show simple rating:
"Was this helpful? 👍 👎"
How to calculate:
CSAT = (Positive ratings / Total ratings) × 100
Benchmarks:
| Rating | CSAT |
|---|---|
| Poor | Under 70% |
| Acceptable | 70-80% |
| Good | 80-90% |
| Excellent | Over 90% |
How to improve:
- Read every negative-rated conversation
- Find patterns—what's causing dissatisfaction?
- Usually accuracy issues or tone problems
- Fix and watch CSAT recover
Metric 5: Time to Resolution
What it measures: How long from first message to problem solved.
Why it matters: Speed is a feature. Customers hate waiting. Faster resolution = higher satisfaction = better conversions.
Benchmarks:
| Rating | Time to Resolution |
|---|---|
| Excellent | Under 2 minutes |
| Good | 2-5 minutes |
| Acceptable | 5-10 minutes |
| Poor | Over 10 minutes |
For simple questions (pricing, hours, policies), resolution should be under 1 minute.
How to improve:
- Look at long conversations—what's causing them?
- Usually incomplete information or unclear responses
- Add clearer content on those topics
The Weekly Dashboard
Track these metrics weekly:
| Metric | This Week | Last Week | Trend | Target |
|---|---|---|---|---|
| Resolution Rate | 85% | |||
| Goal Completion | 10% | |||
| Deflection Rate | 80% | |||
| CSAT | 90% | |||
| Avg. Resolution | <2 min |
Red flags to investigate:
- Any metric down week-over-week
- Resolution high but CSAT low (forcing resolutions that don't help)
- Goal completion stalled (bot isn't converting)
The Weekly Improvement Ritual (25 Minutes)
Minutes 0-5: Pull metrics — Open analytics dashboard
Minutes 5-7: Identify biggest gap from target
Minutes 7-17: Review 5 failed conversations in that area
Minutes 17-22: Identify one improvement
Minutes 22-25: Implement (add content, adjust settings)
Compound improvement: 2% better each week = 100% better in a year.
Analytics Location Reference
| Metric | Dashboard Location |
|---|---|
| Resolution Rate | Analytics → Conversations → Resolved by AI |
| Goal Completion | Analytics → Conversions |
| Deflection Rate | Analytics → Performance → Ticket Deflection |
| CSAT | Analytics → Feedback |
| Time to Resolution | Analytics → Performance |
All exportable as CSV for deeper analysis.
Getting Started
Start free with Kya to measure what matters:
After one week of conversations:
- Open Analytics
- Check Resolution Rate
- Click into unresolved conversations
- Read 5 of them
- Identify missing information
- Add it to website or custom training
- Watch resolution rate climb
You can't improve what you don't measure.
But measuring the wrong things is worse than measuring nothing—it gives false confidence while real problems hide.
Five metrics. That's it.
Resolution. Goal completion. Deflection. Satisfaction. Speed.
If those are good, your chatbot is working.
If they're not, now you know exactly what to fix.
