Evaluating AI Responses and Reducing Hallucinations

A hallucination is output that's fluent but unsupported or false: a made-up refund policy, an invented API method, a wrong date. Models can't be made hallucination-free, but systems can make hallucinations rarer, easier to detect, and less harmful.

Reducing hallucinations by design

• Ground answers in retrieved documents or tool results, not model memory. See RAG and Function Calling.
• Constrain: narrow tasks, schemas, low temperature for factual work.
• Permit uncertainty: instruct "if the answer isn't in the context, say you don't know", and reward it in evaluation.
• Verify: validate structured fields, check that citations exist, cross-check numbers against your data.
• Contain: show sources, let users edit, and avoid AI-only decisions in high-stakes domains.

Build a golden evaluation set

Collect 50–200 realistic inputs (with consent and anonymisation) covering common cases, edge cases, other languages, adversarial inputs, and questions that should be refused. For each, note what a good answer must contain or avoid.

@Serializable
data class EvalCase(
    val id: String,
    val input: String,
    val mustContain: List<String> = emptyList(),
    val mustNotContain: List<String> = emptyList(),
    val shouldRefuse: Boolean = false,
    val maxWords: Int = 150,
)

data class EvalOutcome(val id: String, val passed: Boolean, val reasons: List<String>)

fun check(case: EvalCase, output: String, refused: Boolean): EvalOutcome {
    val reasons = buildList {
        if (case.shouldRefuse != refused) add(if (case.shouldRefuse) "should have refused" else "refused wrongly")
        case.mustContain.filterNot { output.contains(it, ignoreCase = true) }.forEach { add("missing: $it") }
        case.mustNotContain.filter { output.contains(it, ignoreCase = true) }.forEach { add("forbidden: $it") }
        if (output.split(Regex("\\s+")).size > case.maxWords) add("too long")
    }
    return EvalOutcome(case.id, reasons.isEmpty(), reasons)
}

Automated checks, from cheap to expensive

Regression testing

1. Run the evaluation set whenever you change the prompt, model, retrieval or temperature.
2. Compare pass rates per category against the previous version.
3. Block releases on significant drops, and review failures manually.
4. Record prompt and model versions with results.

Learning from production

• Add thumbs up/down and "report" on AI outputs, and capture the reason.
• Track acceptance, edit and regeneration rates for suggestions.
• Sample and review conversations (with consent and privacy safeguards), then add failures to the golden set.

Applying it

Frequently asked questions

Conclusion

Quality AI features combine prevention (grounding, constraints, permission to say "I don't know"), verification (validation, citation checks) and evaluation (golden datasets, regression runs, production feedback). Treat quality as an ongoing process, not a launch checklist.

Next step: explore the AI tools that help you build, starting with AI Coding Assistants for Android Developers.