Start with a use case, not a mandate
The projects that fail begin with a directive to add AI and then hunt for somewhere to put it. The ones that succeed begin with a specific, painful, measurable task and ask whether AI is the right tool for it.
Good candidates share a shape: they involve unstructured text or judgement, they happen often enough to matter, and a slightly imperfect answer is still useful. Summarising long case notes, classifying incoming requests, drafting a first reply, extracting structured fields from messy documents, or answering questions over a knowledge base are all strong fits. Vague goals like make the app smarter are not.
Before writing code, define what success looks like in numbers: minutes saved per task, percentage of drafts accepted without edits, accuracy on a test set. If you cannot state the metric, you are not ready to build the feature, only to talk about it.
APIs, not training runs
The mental model that trips up newcomers is thinking AI means training a model. For business software it almost never does. You call a hosted large language model over an HTTP API, sending a prompt and receiving a completion, the same way you call any other web service.
This is liberating because it collapses the effort. There is no dataset to assemble, no GPU cluster, no training loop. You shape behaviour through the prompt, through the data you supply at request time, and through a handful of parameters. Fine-tuning exists and occasionally helps for narrow, high-volume, stable tasks, but it is an optimisation you reach for later, not a starting point.
Because you are consuming an API, established engineering practice applies. Keep credentials server-side, never in the browser. Version your prompts like code. And treat the provider as a dependency you can swap, keeping the integration behind a thin internal interface so you are not welded to one vendor's exact request format.
Ground the model in your own data with retrieval
A raw language model knows nothing about your customers, your inventory, or last week's tickets, and if you ask it anyway it will confidently invent an answer. Retrieval-augmented generation is the fix, and it is the single most important pattern for business AI.
The idea is simple: when a request comes in, first fetch the relevant records or documents from your own system, then include that material in the prompt and ask the model to answer using it. The model stops relying on its training memory and starts reasoning over facts you supplied, which slashes hallucination and keeps answers current with your live data.
Retrieval can be as simple as a keyword or database query when the relevant record is obvious, or as sophisticated as semantic search using embeddings, numeric representations of meaning, to find passages similar to the question. Start with the simplest retrieval that works. Many useful features need nothing more than pulling the right record by id and handing it to the model.
- Fetch relevant records first, then ask the model to answer from them.
- Simple lookups beat vector search when the right record is obvious.
- Use embeddings and semantic search only when you must find by meaning.
- Grounding in real data is what turns a demo into a trustworthy feature.
Treat the model as an unreliable dependency
A language model API is a slow, occasionally failing network call with variable latency, and your architecture must respect that. Generation can take seconds, so never run it inside a request that a user is waiting on synchronously if you can avoid it. In a Frappe app, push the work to a background job and update the record when it completes; in any stack, prefer asynchronous handling for anything non-trivial.
Failures are normal, not exceptional. The API can time out, rate-limit, or return an unusable response. Design for it: set timeouts, retry with backoff on transient errors, and always have a graceful fallback so a failed generation degrades the feature rather than breaking the app. A summarise button that occasionally says try again is fine; one that throws an error page is not.
Streaming responses can improve perceived latency for chat-style features by showing text as it arrives, but it adds complexity. Reach for it when the wait is genuinely long and user-facing, not by default.
Design for cost and token discipline
AI features have a running cost that scales with use, driven mainly by tokens, the chunks of text sent to and returned from the model. Ignore this and a feature that is cheap in a demo becomes expensive at production volume. Build cost awareness in from the start.
The biggest lever is context discipline. Do not stuff entire documents into every prompt; retrieve only the passages that matter. Match the model to the task, using a smaller, cheaper model for classification or extraction and reserving a larger one for genuinely hard reasoning. Cache results when the same input recurs, and cap response length so the model cannot ramble at your expense.
Instrument token usage per feature early, while volumes are small and mistakes are cheap. Knowing the cost per operation lets you make honest decisions about which features are worth shipping and which need rethinking before they reach scale.
Keep humans in the loop and evaluate honestly
Language models are fluent, which makes them persuasive even when wrong, and that combination is dangerous for anything consequential. The discipline is to place a human at every point where a mistake has real cost: posting to the ledger, sending a binding customer message, altering records, or producing compliance text.
The productive pattern is AI drafts, human approves. Let the model summarise, classify, extract, or propose, and let a person confirm the action. This captures most of the time savings while keeping accountability with someone who can catch the confident error before it does damage.
Evaluation is what separates a real feature from a gamble. Log inputs and outputs, build a small test set of representative cases with known good answers, and measure accuracy before and after prompt changes. Without evaluation you are shipping on vibes; with it, you can improve deliberately and prove the feature actually earns its place.
How Upeosoft adds AI to systems already in production
Our approach is deliberately unglamorous. We do not bolt a chatbot onto a working system and call it innovation. We find the one task in a client's daily workflow where judgement over text is eating hours, prove that an AI feature moves a real metric, and then integrate it into the infrastructure that already exists.
On Frappe and ERPNext that means exposing a whitelisted method that calls the model, running generation as a background job so it never blocks the desk, and using the client's own doctypes as the retrieval source so answers are grounded in their real data and respect their existing permissions. The AI becomes another well-behaved server capability, not a separate system to babysit.
Done this way, adding AI is an increment, not a rewrite. You keep the software your team already trusts and add a capability that pays for itself on a task you can name and measure. That is the difference between AI that ships value and AI that only demos well.
