Behind a reliable assistant are three main components: a well-crafted system prompt, a connected knowledge base, and a systematic way to test different language models against real questions before rolling them out to customers.
1. Why businesses deploy AI assistants
In customer-facing channels, an AI assistant handles routine conversational tasks that would otherwise consume staff time:
- Lead capture and qualification: Greeting incoming visitors, asking about their requirements, budget, or timeline, and storing those answers into structured fields for your CRM.
- Customer support: Answering frequent questions about pricing, operating hours, delivery options, and return policies in seconds.
- Product guidance: Helping users navigate a catalog, checking product availability, and guiding buyers to checkout or connecting them with a human specialist.
To do this reliably, the bot needs explicit instructions on how to behave and a dependable source of factual information.
2. Writing clear system instructions
The system prompt defines the assistant's persona, boundaries, and communication style. It tells the model who it is, how to respond, and what information to collect.
┌────────────────────────────────────────────────────────┐
│ System prompt │
│ • Role: Online sales consultant │
│ • Goals: Collect name, phone number, and order item │
│ • Style: Friendly, concise, max 2 sentences per reply │
│ • Limits: Do not guess prices; ask 1 question at a time│
└────────────────────────────────────────────────────────┘
Core principles for effective prompts
- State a specific role: Give the model a focused job description rather than a vague instruction like "be a helpful bot."
- Ask one question at a time: Models often try to ask three or four questions in a single message. Explicitly telling the model to ask one question per turn keeps conversations manageable for users.
- Set clear negative constraints: Specify what the assistant must never do, such as quoting unverified prices, guessing order statuses, or discussing unrelated topics.
- Keep data extraction rules out of prompt copy: If your bot platform handles JSON structure validation, let the platform manage it instead of bloating the prompt text with formatting instructions.
3. Pairing the prompt with a knowledge base (RAG)
A common mistake is pasting entire product lists, price sheets, and company manuals directly into the system prompt.
This creates several practical problems:
- High token usage: Every token in the prompt is processed and billed on every single turn of every conversation.
- Diluted attention: When a prompt stretches to thousands of tokens, models are more likely to miss behavioral rules or invent details.
- Maintenance overhead: Updating a single price requires editing and redeploying the scenario prompt.
Using dynamic retrieval instead
A knowledge base uses Retrieval-Augmented Generation (RAG). Documents (PDFs, DOCX files, text notes) are indexed separately. When a user asks a question, the system searches the knowledge base, extracts relevant excerpts, and passes only those specific paragraphs to the model along with the user's question.
User question: "What is the warranty period on the laptop?"
│
▼
┌─────────────────────────────────────┐
│ Knowledge base search (RAG) │
│ Found excerpt: "Laptops carry a │
│ 2-year limited manufacturer warranty"│
└──────────────────┬──────────────────┘
│
▼
┌─────────────────────────────────────┐
│ AI model │
│ System prompt (behavior rules) │
│ + Retrievable document excerpt │
└──────────────────┬──────────────────┘
│
▼
Answer: "Laptops come with a 2-year manufacturer warranty."
This keeps the system prompt short and focused on conversational rules, while the knowledge base supplies up-to-date factual answers.
4. Why testing prompts and models is difficult
Developing an assistant in a test chat is straightforward, but verifying that it will work consistently across hundreds of customer conversations is harder:
- Multi-turn drift: A prompt that answers a standalone question correctly might fail during a longer dialog when the user changes the topic or gives partial answers.
- Untested live changes: Editing a prompt directly in production without running test cases can introduce subtle regressions.
- Defaulting to expensive models: Teams frequently default to flagship models out of caution, even when compact models could complete the same task with identical accuracy at a fraction of the cost.
- Lack of cost visibility: Standard chat interfaces do not show how many input and output tokens were spent, what the interaction cost in dollars, or how many seconds each response took.
5. Built-in prompt analysis in Botconsole
To make prompt development measurable, Botconsole includes a prompt analysis and testing tool inside the AI Dialog block on the visual canvas.
Clicking the prompt analysis button opens a dedicated audit view:
┌────────────────────────────────────────────────────────┐
│ AI prompt analysis modal │
│ ┌──────────────────────┐ ┌────────────────────────┐ │
│ │ Prompt analysis │ │ Question testing │ │
│ └──────────────────────┘ └────────────────────────┘ │
│ │
│ Quality score: 88% │
│ • Recommendation: Add fallback for out-of-scope queries│
│ [Apply to prompt] │
│ • Recommendation: Limit reply length to 2 sentences │
│ [Apply to prompt] │
└────────────────────────────────────────────────────────┘
The analyzer checks your prompt for logical gaps, missing constraints, and data collection clarity, providing a percentage score and specific recommendations that you can apply with one click.
6. Testing with benchmark question sets
Instead of testing prompts manually one question at a time, you can create a test suite with expected answers:
- Flexible question input:
- Add test questions manually along with expected keywords.
- Generate customer questions from your website URL.
- Import previous customer dialogs.
- Upload question lists from a CSV file (up to 100 KB).
- Automated keyword verification: The runner checks whether the model included required terms, highlights matched and missing words, and calculates an overall test score (for example,
Result: 100% (12/12)). - Knowledge base source badges: When a knowledge base is connected, each answer shows badges with the source document name and the exact text passage used.
- Direct fixes:
- If a model failed to follow an instruction, use Fix -> Edit prompt so the system suggests prompt adjustments.
- If the model lacked specific product facts, use Fix -> Add to knowledge base to save the verified question and answer directly into your bot's knowledge base.
7. Comparing speed, quality, and cost across models
Different tasks require different model tiers. A lightweight model can handle structured lead qualification, while complex advisory flows may benefit from a larger reasoning model.
Botconsole allows you to connect up to 5 models at once (such as OpenAI GPT-4o-mini, Anthropic Claude Sonnet, Google Gemini Flash, Kimi, or Grok) and run them against the same test suite simultaneously.
┌───────────────────────────────────────────────────────────────────────────┐
│ [OpenAI: gpt-4o-mini 100%] [Claude: sonnet 100%] [Gemini: 2.0-flash 100%] │
├───────────────────────────────────────────────────────────────────────────┤
│ Result: 100% (12/12) • 42 sec • 3,120 tokens • $0.0009 │
│ │
│ [User] What are your business hours? │
│ [AI] We are open Monday through Friday from 9:00 AM to 6:00 PM EST. │
│ Source: Company_FAQ.pdf │
│ [ Apply to block ] │
└───────────────────────────────────────────────────────────────────────────┘
Each model tab provides four concrete metrics:
- Accuracy rate (%): How many questions matched the required keywords and validation criteria.
- Total execution time: How long the model took across all test turns.
- Token usage: Total input and output tokens consumed.
- Calculated cost (USD): Estimated cost based on current token pricing from OpenRouter.
Once you find the model that passes all test cases with acceptable latency and the lowest token cost, click Apply to block to set that model in your scenario configuration.
Practical takeaways
To build an efficient AI assistant that keeps operating costs predictable:
- Keep the system prompt focused on conversational behavior and limits.
- Use a knowledge base for product specifications, pricing, and policies.
- Build a test set of common customer questions to evaluate changes systematically.
- Test multiple models on the same questions to find the lowest-cost option that meets your accuracy standards.
