How to Add AI to an Existing Mobile App

You do not need to rebuild your app to add AI. Most AI features are one new endpoint on your backend and one new screen in your app. The real work is choosing the right feature and controlling what it costs to run.

1. Start with one job, not with "AI"

Pick a task your users already do slowly: writing a description, finding an answer in your help content, or typing in data from a photo. A feature that removes one of those is worth more than a general chatbot.

Recipe Lens, an app our founder built for a client, is a good example. The user photographs their fridge, Clarifai identifies the food in the photo, and GPT-4 writes recipes from it. One job, done well.

2. Keep the API key on your server

Never call OpenAI or Gemini directly from the app. Anything shipped inside an app can be extracted, and a leaked key is a bill you pay. The app calls your backend, and your backend calls the model.

Mobile app  ->  your API (sign-in, limits)  ->  model provider
Mobile app  <-  your API (streamed answer)  <-  model provider

3. Stream the response

A model can take several seconds to finish an answer. Streaming shows the words as they are generated, so the user is reading within a second. It feels far faster than a spinner followed by a wall of text.

4. Ground the answers in your own data

A model on its own answers from general knowledge and will guess when it does not know. Retrieval-augmented generation (RAG) fixes this: your content is stored as vector embeddings, the most relevant passages are found for each question, and the model is asked to answer from those passages only.

5. Put limits on cost

• Set a daily limit per user, and a lower one for free accounts.
• Use a smaller, cheaper model for simple tasks and keep the large one for hard ones.
• Cache answers to questions that repeat.
• Log usage per feature, so you can see what each one costs.

6. Plan for wrong answers

• Tell users the content is generated by AI.
• Let them edit the result or ask for another one.
• Have a person review anything with real consequences before it is used.
• Add a way to report a bad answer, and read the reports.

7. Check the privacy side

Read the provider's current data policy before you send it customer data, send only what the feature needs, and say in your privacy policy what is sent and to whom. The App Store and Google Play both ask you to declare the data your app shares, so update those declarations as well.

Prototype before you build

The cheapest way to find out whether an AI feature is good enough is a prototype that runs on your real data. Judge the answers first. Building the feature into the app is the easy part once you know the answers are worth shipping.