What is a large language model?

How a language model works

An LLM breaks text into small units called tokens and calculates, for each position, which token is most likely to come next. It learned these probabilities during pre-training on large text collections; additional fine-tuning and human feedback teach it to follow instructions and answer in dialogue form. During use – the so-called inference – the model receives an input (the prompt), processes it within its context window and generates the answer token by token.

• Parameters: the learned weights of the model; more parameters usually mean more capability, but also more computing power.
• Context window: the maximum amount of text the model can consider at once – question, instructions, attached documents and chat history.
• Knowledge cut-off: the model only knows what was in its training data; anything later or internal to your company is unknown to it.

Why LLMs matter for companies

LLMs handle language-heavy work that has so far been done manually: drafting e-mails and offers, summarising meeting notes, structuring contracts, answering recurring questions or analysing data. The real value emerges when the model is connected to company knowledge – for example via RAG – and to tools, so that it becomes an AI agent that can carry out tasks.

Limits and risks

• Hallucinations: an LLM produces plausible text, not verified facts. It can invent figures, sources or legal provisions; more on this under hallucination.
• No company knowledge: without access to your documents, the model only answers in general terms.
• Data protection: inputs may contain personal data. Where the model runs, which provider processes the data and whether a data processing agreement exists all matter – see GDPR-compliant AI.
• Costs: usage is typically billed per token; long documents and large models increase costs.
• Model choice: no single model is best at everything; quality, speed, price and hosting region differ.

Checklist for selecting models

1. Clarify use cases: text, code, analysis, images or speech?
2. Check the hosting region and data processing terms of the provider.
3. Compare quality with your own sample tasks rather than relying on benchmarks alone.
4. Make costs per token and usage limits transparent.
5. Avoid lock-in: keep the option to switch models later.

LLMs in AIPE

AIPE is not a language model itself but a platform that makes various LLMs usable in one interface. Model providers can be configured per organisation; published providers in EU regions include Azure OpenAI (Sweden Central), Amazon Bedrock (Frankfurt), Google Vertex (europe-west1) and Mistral (EU). Requests are routed via a model router, and organisations can also store their own provider keys (BYOK). The overview of available models is on the models page; the guide AI in the Mittelstand explains how to get started.