๐Ÿง  Android AI Projects and Production

Build an AI Photo Caption Generator for Android

Pick a photo, choose a tone, get three caption ideas and accessible alt text. This project teaches multimodal prompting, image handling, structured output and responsible UX. Here's how to build it with Compose and Gemini.

โœ๏ธ Yunok Tech ๐Ÿ“… Updated โฑ 3 min read ๐ŸŽฏ Intermediate
On this page
  1. App flow
  2. 1. Pick and downscale
  3. 2. Ask for structured output
  4. 3. On-device alternative
  5. 4. UI
  6. Responsible design
  7. Troubleshooting
  8. Frequently asked questions
  9. Conclusion

Our AI Image Description guide covers describing images. This project builds a complete app around it: caption styles, structured multi-option output, an on-device path, and sharing.

App flow

Photo caption generator flow The user picks a photo with the photo picker and chooses a tone. The app downscales the image, sends it with a prompt to Gemini requesting structured JSON with three captions and alt text, validates the result, and shows options the user can edit, copy or share. Pick photo+ tone Downscale~1024 px Geminiimage + prompt JSONvalidate Editcopy, share
Users always review and edit before sharing.

1. Pick and downscale

Load a downscaled bitmapKotlin
suspend fun loadScaled(context: Context, uri: Uri, maxSide: Int = 1024): Bitmap =
    withContext(Dispatchers.IO) {
        val source = ImageDecoder.createSource(context.contentResolver, uri)
        ImageDecoder.decodeBitmap(source) { decoder, info, _ ->
            val (w, h) = info.size.width to info.size.height
            val scale = maxSide.toFloat() / maxOf(w, h)
            if (scale < 1f) decoder.setTargetSize((w * scale).toInt(), (h * scale).toInt())
            decoder.allocator = ImageDecoder.ALLOCATOR_SOFTWARE
        }
    }

Downscaling reduces upload time, tokens and cost, with little effect on caption quality. Use the photo picker for selection. See Privacy and Accessibility.

2. Ask for structured output

CaptionGenerator.ktKotlin
@Serializable
data class CaptionResult(val captions: List<String>, val altText: String)

class CaptionGenerator(private val json: Json = Json { ignoreUnknownKeys = true }) {

    private val model = Firebase.ai(backend = GenerativeBackend.googleAI()).generativeModel(
        modelName = "gemini-3.8-flash",                                   // check current models
        generationConfig = generationConfig {
            responseMimeType = "application/json"
            responseSchema = Schema.obj(
                mapOf(
                    "captions" to Schema.array(Schema.string()),
                    "altText" to Schema.string(),
                )
            )
            temperature = 0.8f
        },
    )

    suspend fun generate(image: Bitmap, tone: Tone): CaptionResult {
        val response = model.generateContent(content {
            image(image)
            text(
                "Write 3 short social media captions for this photo in a ${tone.label} tone " +
                "(max 120 characters each, at most 2 hashtags). Also write factual alt text " +
                "(max 150 characters) describing what is visible. Don't guess identities of people."
            )
        })
        val result = json.decodeFromString<CaptionResult>(response.text ?: error("Empty response"))
        return result.copy(
            captions = result.captions.map { it.take(150).trim() }.filter { it.isNotEmpty() }.take(3),
            altText = result.altText.take(200).trim(),
        )
    }
}

enum class Tone(val label: String) { FUN("fun"), PROFESSIONAL("professional"), POETIC("poetic"), MINIMAL("minimal") }

3. On-device alternative

On supported devices, ML Kit GenAI Image Description (beta) produces a short description with Gemini Nano, privately and offline. Use it for alt text and offer cloud captions as an option, or use hybrid inference. See Nano vs Cloud.

4. UI

CaptionOptions.ktKotlin
@Composable
fun CaptionOptions(result: CaptionResult, onShare: (String) -> Unit) {
    var selected by rememberSaveable { mutableIntStateOf(0) }
    var edited by rememberSaveable(selected) { mutableStateOf(result.captions.getOrElse(selected) { "" }) }

    Column(verticalArrangement = Arrangement.spacedBy(8.dp)) {
        Text("AI suggestions. Review before posting.", style = MaterialTheme.typography.labelMedium)
        result.captions.forEachIndexed { i, caption ->
            FilterChip(selected = i == selected, onClick = { selected = i }, label = { Text(caption, maxLines = 2) })
        }
        OutlinedTextField(value = edited, onValueChange = { edited = it }, label = { Text("Caption") })
        Text("Alt text: ${result.altText}", style = MaterialTheme.typography.bodySmall)
        Button(onClick = { onShare(edited) }) { Text("Share") }
    }
}

Responsible design

  • Instruct the model not to identify people or infer sensitive traits (age, ethnicity, health).
  • Let users edit everything, and never auto-post.
  • Disclose that photos are sent to a cloud service, unless processing is on-device.
  • Handle safety blocks gracefully ("Couldn't caption this photo").
  • See Responsible AI UX.

Troubleshooting

  • JSON parse errors: use a response schema, ignoreUnknownKeys, and catch failures with a retry.
  • Slow uploads: downscale, and compress if you send bytes.
  • OOM on large photos: decode with a target size, as shown.

Frequently asked questions

Can it caption multiple photos?

Yes. Send several images in one request for a carousel caption, within the model's limits.

How do I keep costs low?

Downscale images, use a Flash-class model, cap output tokens, and rate-limit per user.

Conclusion

A caption generator combines the photo picker, downscaling, multimodal Gemini with structured JSON output, validation, and an editable UI, plus on-device alt text where available and clear responsible-AI choices.

Next project: an AI Voice-Notes App.

Official references

๐Ÿ“ฑ Apps built by Yunok Tech

Real Android apps on Google Play, free to download.

See all our apps โ†’

Ready to build real Android applications?

Every AI feature still runs inside an Android app that has to be fast, reliable and well structured. Strengthen the Kotlin, Jetpack Compose and architecture skills that AI integrations depend on, then apply these AI concepts with confidence.

Move beyond theory and strengthen your Android development skills with Yunoktech's Android course. Learn core concepts, practical implementation, modern Android development techniques, and the reasoning behind building maintainable applications.

Explore the Android course and take your next step as an Android developer.

Share: LinkedIn X / Twitter WhatsApp