Building More Efficient AI Applications with Decision Models

Large language models have become the default choice for almost everything involving AI. Whether you’re classifying support tickets, interpreting search queries, or deciding which workflow to execute, the usual approach is to send a prompt to an LLM and get a response. It works, but are we using generative AI for problems that don’t actually require generation?

Think about it. If an application needs to decide whether a support ticket belongs to billing or technical support, does it really need a model designed to generate paragraphs of text? What if all we need is a structured decision?

That’s where decision models come in. They’re designed to evaluate information and return decisions rather than generate content. I recently explored this approach using Jev, a decision model from TypeSafe, by building a small experimental application called Nutriva. The goal was to better understand how these models work, what they’re capable of, and whether they offer a more efficient approach to certain AI tasks.

What Are Decision Models?

Decision models take an input, evaluate it against a defined set of possibilities, and return a structured result. Instead of asking AI to explain what should happen, you’re asking it to determine what should happen.

For example, imagine a customer sends a message saying, “I’ve been charged twice for my subscription.” A traditional LLM could interpret the message and generate a response explaining that it’s a billing issue. A decision model can simply classify it as billing, allowing the application to route the ticket automatically.

One example is Jev, which TypeSafe describes as a System One Model. It transforms unstructured inputs into typed, probabilistic decisions. Depending on the task, this could mean selecting a category, assigning a score, or estimating the probability of a particular outcome.

The distinction matters because many AI features aren’t fundamentally about generating new information. They’re about interpreting something ambiguous and selecting from outcomes the application already understands. Decision models are built around that narrower problem.

Why Decision Models Are Interesting

The most obvious benefit is cost. Traditional LLM applications sometimes send far more context than necessary and receive generated text that the application then needs to process. Decision models can work with a smaller representation of the problem and return compact results. For applications making thousands or millions of decisions, reducing unnecessary input and output can make a meaningful difference.

There’s also latency. Purpose-built decision models can be optimized for fast inference rather than generating responses token by token. TypeSafe, for example, reports response times in the tens to hundreds of milliseconds for Jev, depending on the workload. While actual performance varies, this makes specialized models particularly interesting for applications where decisions need to happen quickly.

Another advantage is predictability. When the possible outcomes are predefined, the model returns a decision that fits the application’s expected structure. You don’t need to extract a category from a paragraph or handle arbitrary answer formats. Models that return probabilities also give applications a way to identify uncertain decisions and determine whether to proceed automatically or involve a human.

Of course, decision models aren’t automatically cheaper, faster, or more accurate than LLMs. Modern language models also support structured outputs, and straightforward tasks can often be solved with ordinary rules. The real benefit is architectural: using AI specifically where interpretation is needed while leaving deterministic operations to the application.

Where Decision Models Make Sense

Once you start looking at AI applications as a series of decisions rather than generated responses, quite a few opportunities become apparent.

Customer support is an obvious example. Incoming messages can be classified into billing, account access, technical support, or other predefined categories. The application then routes them to the appropriate team without needing AI to generate an explanation.

Search and recommendations are another useful case. Users often describe what they’re looking for without knowing the exact product name or category. A decision model can interpret that intent and select relevant options, while the application retrieves prices, availability, descriptions, and other details from its database.

AI agents and workflow automation can also benefit. An agent frequently needs to decide which tool to use, whether a request requires approval, or when to escalate a task. Similar patterns apply to content moderation, document classification, lead qualification, and quality scoring.

The common thread is that the application already knows the possible outcomes but needs intelligence to choose between them. That’s where decision models become useful, particularly when the input is unstructured or ambiguous.

Experimenting with Decision Models: Nutriva

To better understand how decision models work in practice, I built Nutriva, a small experimental application that uses natural-language queries to discover fruits and vegetables. The idea was simple: could a decision model interpret requests like “What should I eat to support my eye health?” and select relevant items from a predefined dataset without needing access to every nutritional detail?

I used Jev from TypeSafe for the experiment. Nutriva had a local dataset containing produce names, calories, vitamins, minerals, and other nutritional information. One approach would have been to send that entire dataset to an LLM along with the user’s question. Instead, I wanted to explore whether Jev could make a useful selection using only the query and a controlled list of produce names.

You can try Nutriva here.

The implementation was fairly straightforward:

const criteria = Object.fromEntries(
  produce.map((item) => [toChoice(item.name), item.name])
);

const response = await fetch('https://api.typesafe.ai/v1/systemone', {
  method: 'POST',
  headers: {
    Authorization: 'Bearer ' + apiKey,
    'Content-Type': 'application/json'
  },
  body: JSON.stringify({
    model: 'jev-latest',
    state: { userQuery: query.trim() },
    questions: {
      best_match: {
        type: 'choice',
        instructions:
          'Interpret the user query as either a specific produce request or a nutrition goal.',
        criteria
      }
    }
  })
});

Jev evaluated the query against the available choices and returned structured probabilities. The application then used those results to retrieve the corresponding nutritional records locally, including calories, nutrients, and images. The model handled interpretation, while ordinary application code handled the actual data.

The architecture was simple: User query → Decision model → Local data lookup → Results. Adding more nutritional fields to the dataset wouldn’t increase the information sent to Jev, since the decision request only included produce names.

What I found interesting wasn’t the nutrition application itself, but how naturally a decision model could fit into an otherwise conventional application. The experiment helped me understand the separation between probabilistic interpretation and deterministic data handling. It wasn’t a production benchmark, and I didn’t use it to measure precise cost savings, but it provided a practical way to explore the capabilities and limitations of this approach.

Are Decision Models a Replacement for LLMs?

Not really, and I don’t think that’s the right way to look at them. If an application needs to write content, summarize documents, generate code, or answer open-ended questions, a generative model is often the better choice.

Decision models are most useful when the task has a well-defined answer space. They can interpret ambiguous input, but they can’t select something that doesn’t exist among the available choices. Their outputs still require validation, and probabilities shouldn’t automatically be treated as reliable confidence estimates without testing.

It’s also worth remembering that not every decision requires AI. Exact filtering, straightforward validation, and simple routing rules can often be handled more efficiently with ordinary code. The opportunity lies in using decision models where the input requires interpretation but the expected output is relatively simple.

Rather than replacing LLMs, decision models can complement them. An application might use a decision model to determine what action to take, conventional code to enforce business rules, and an LLM only when it actually needs to generate a response.

Final Thoughts

Experimenting with Jev through Nutriva gave me a better understanding of where decision models might fit into modern applications. More importantly, it reinforced a useful architectural principle: before reaching for a generative model, consider whether the problem actually requires generation or simply a structured decision.

As AI becomes part of more everyday software, I think specialized models will become increasingly relevant alongside general-purpose LLMs. They offer another way to balance flexibility, performance, cost, and control, especially for applications built around classification, routing, ranking, and other bounded decisions.

The bigger lesson isn’t that decision models are always better than LLMs. It’s that we should be more deliberate about which parts of an application actually need generative intelligence.

Sometimes the best way to reduce AI costs isn’t to find a cheaper LLM. It’s to recognize that you don’t need one for the task in the first place.