Build an AI-Powered Search Feature for Android

Search is often the highest-intent feature in an app. Users who search know what they want. Semantic search matches by meaning rather than exact words, so it recovers queries that keyword search misses. Done carelessly, it also returns confident nonsense, so build it in layers.

Keyword vs semantic vs hybrid

How semantic search works

Where to run it

• Server-side (most common): embed your catalogue with a cloud embedding model, store vectors in a vector-capable database, and expose /search?q=. The app stays simple, and the API key stays on the server.
• On-device: for private or offline data such as the user's own notes, use a small on-device text embedder (for example MediaPipe Text Embedder with LiteRT) and brute-force cosine similarity for a few thousand items.

On-device semantic search for personal notes

/** Brute-force cosine search: fine for a few thousand notes. Vectors stored in Room as FloatArray blobs. */
class NoteSemanticIndex(private val embedder: TextEmbedderWrapper, private val dao: NoteVectorDao) {

    suspend fun index(note: Note) {
        val vector = embedder.embed("${note.title}\n${note.body}")
        dao.upsert(NoteVector(note.id, vector.normalized()))
    }

    suspend fun search(query: String, limit: Int = 10): List<String> = withContext(Dispatchers.Default) {
        val q = embedder.embed(query).normalized()
        dao.all()
            .map { it.noteId to dot(q, it.vector) }       // cosine similarity of normalised vectors
            .filter { it.second > 0.3f }                 // drop weak matches (tune on your data)
            .sortedByDescending { it.second }
            .take(limit)
            .map { it.first }
    }
}

fun FloatArray.normalized(): FloatArray {
    val norm = kotlin.math.sqrt(sumOf { (it * it).toDouble() }).toFloat().coerceAtLeast(1e-6f)
    return FloatArray(size) { this[it] / norm }
}
fun dot(a: FloatArray, b: FloatArray): Float { var s = 0f; for (i in a.indices) s += a[i] * b[i]; return s }

TextEmbedderWrapper wraps whichever on-device embedder you choose. Check the MediaPipe or LiteRT documentation for current setup.

Hybrid ranking with keyword fallback

suspend fun hybridSearch(query: String): List<String> = coroutineScope {
    val keyword = async { dao.ftsSearch(query).map { it.id } }        // Room FTS, always available offline
    val semantic = async { runCatching { semanticIndex.search(query) }.getOrDefault(emptyList()) }
    // Reciprocal rank fusion: simple, robust way to merge ranked lists
    val scores = mutableMapOf<String, Double>()
    listOf(keyword.await(), semantic.await()).forEach { list ->
        list.forEachIndexed { rank, id -> scores[id] = (scores[id] ?: 0.0) + 1.0 / (60 + rank) }
    }
    scores.entries.sortedByDescending { it.value }.map { it.key }
}

LLMs for query understanding

An LLM can turn "red running shoes under 3000 for women" into structured filters (category=shoes, color=red, maxPrice=3000, gender=women) via structured output. Apply those filters in your own search system, rather than letting the LLM pick products from memory.

Evaluating relevance

• Build a set of real queries with the results you expect.
• Measure whether the right item appears in the top 3 or top 10, before and after changes.
• Track zero-result rate and click-through in analytics.

Use cases

Frequently asked questions

Conclusion

AI-powered search combines embeddings for meaning, keyword search for precision, and optionally LLM query understanding for filters. Run it server-side for shared catalogues, on-device for private data, keep an offline FTS fallback, and measure relevance with real queries.

Next step: protect the AI endpoints behind these features with Integrating Cloud AI APIs Securely.