AI literacy for lawyers: the basics under the hood

Many lawyers still do not really know what an AI tool actually does from a technical perspective.

That understanding is important. Not because every lawyer needs to become a developer — contrary to what sometimes seems to be a growing trend — but because anyone working with AI and AI platforms needs to understand what AI can and cannot do, why some answers are better than others, and why the use of AI cannot be unlimited and free.

These insights matter for the way you use AI and for the choice of systems or platforms you decide to work with.

An AI platform ≠ an AI model

First of all: most legal AI platforms do not build their own full foundation model. Behind the scenes, these tools usually work with existing large language models, such as models from OpenAI, Anthropic, Google or other providers. These models are also called LLMs: Large Language Models.

Legal AI platforms build various layers between the lawyer and the model. That layer determines how documents are uploaded, how a case file is analysed, which documents or passages are considered relevant, which instructions are given to the model, how the answer is built, and how the result is shown back to the user.

The quality of a legal AI platform therefore does not only depend on “which AI model” is being used in the background. The real value often lies in everything around it: selecting the right context, giving specific instructions, embedding the legal workflow, using the right set of sources, splitting documents in the right way, checking for hallucinations, building proper legal reasoning, linking one action to the next, processing speed, ease of use and overall user experience.

What is a “call” to an AI model?

When a lawyer asks a question such as “summarise this case file” or “draft an argument”, a lot happens behind the scenes.

First, the user’s question is converted into a clear instruction. That instruction contains not only the question itself, but also guidance on the desired style, structure, role of the AI, language, use of sources and limitations. The platform then needs to determine which information from the case file is relevant. This may include emails, letters, contracts, police reports, expert reports, legal sources or other documents.

Next, a package of information is assembled and sent to the language model. That package may include the instruction to the model, the lawyer’s question, the relevant passages from the case file, any legal sources and sometimes earlier intermediate steps or analyses.

That entire package is sent to the language model through a technical connection, an API. The model processes the input and generates an answer. That answer is returned to the platform, which then displays it to the user, possibly with formatting, source references or additional checks.

That single interaction is often called a “call” to the model. But one action in a legal AI platform may in fact contain many calls. A summary of a large case file, for example, may first be prepared document by document, then merged, then checked, and only then turned into a final text.

AI does not count pages, but tokens

For a lawyer, a case file feels like a bundle of documents. For an AI model, that same information consists of tokens.

A token is a piece of text. It can be a word, part of a word, a punctuation mark or a space. The cost of many AI models is calculated based on the number of tokens you send to the model and the number of tokens the model sends back.

That matters. A short question with a short answer costs very little. But a legal question where the platform first needs to include hundreds of pages of case file content costs much more. Not because the question sounds complex, but because the model has to process much more text.

In legal case files, this quickly adds up. Written submissions, bundles of exhibits, email chains, expert reports and contractual documentation can together represent a very large amount of text. If the AI also needs to take case law, legal doctrine or previous analyses into account, the input becomes even larger.

The context window: how much information can an AI model process?

Every AI model has a so-called context window. This is the maximum amount of information the model can process at once.

You can compare this to a lawyer’s desk. The bigger the desk, the more documents you can lay out at the same time. But even a large desk has limits. And the more documents are lying on it, the harder it becomes to quickly find that one relevant passage.

AI works in a similar way. Some modern models can process larger amounts of text at once. But being able to include a lot of context is not the same as always perfectly understanding what is relevant, which is crucial in legal case files. A case file often contains a lot of noise: duplicate emails, attachments, older versions, irrelevant correspondence, procedural documents that contradict each other, technical documents and documents without a clear structure.

A good AI platform should therefore not simply send as much text as possible to the model. It should be able to send the right and relevant passages.

Why the context window is one of the major challenges in AI

The context window is often presented as a technical limit: how much text can an AI model process at once? But in the legal profession, it is more than that. It is one of the core issues on which good legal AI platforms distinguish themselves.

A model can only answer based on what it “sees” at that moment. It does not automatically understand the full case file in the way a lawyer does after days or weeks of studying it. When a platform sends a question to the model, it must therefore decide which documents, passages, previous analyses or legal sources are included. Everything that is not included cannot, in principle, be used by the model. Everything that is included takes up space within the context window.

That is where the difficulty begins. In a legal case file, not every document is equally important. Not every passage is relevant. And sometimes one short sentence in an email or one detail in an attachment is decisive. An AI platform that simply sends “many documents” to the model will therefore not necessarily perform better. On the contrary: too much context can make the model slower, more expensive and sometimes even less precise.

When drafting written submissions, a lawyer does not include every document as an exhibit either. The question is which documents are relevant to the specific argument, which passages have evidential value, which documents contradict each other and which information is legally decisive at that moment. A good lawyer does not only read a lot; he or she selects, orders and weighs. A good AI platform must do exactly the same.

The limitation of the context window operates on several levels.

First, there is the hard technical limit. Every model has a maximum input and output. The input consists of the question, the instructions, the case file context and any legal sources. The output is the answer the model writes back. If the input becomes too large, it must be cut, summarised or selected.

Second, there is cost. The more context is sent to the model, the more tokens are processed. Sending an entire large case file is therefore not only technically difficult, but also economically inefficient. In a legal environment, where case files often contain hundreds or thousands of pages, this can quickly have a major impact on usage costs.

Third, there is speed. A model that has to process a lot of context needs more time. For a lawyer, this matters. An AI platform that can theoretically process an entire case file but takes minutes to answer every question becomes less useful in daily practice. Speed is not a luxury; it determines whether the tool truly fits into the workflow.

Fourth, there is the quality of the reasoning. A larger context window does not automatically mean that the model uses all information equally well. When a lot of information is provided at once, relevant details may get buried. The model may technically have access to the right passage, but fail to pick it up correctly or give it sufficient weight.

That is why the real challenge is not only “how much context” a platform can process, but above all “which context” the platform selects.

Good legal AI tools are working very hard on this. They do not try to give case files to a model as one large mass of text. They build intermediate layers that first structure the file. Documents are split, recognised, indexed, summarised, deduplicated and linked to metadata such as date, author, document type or relevant topic. When the lawyer then asks a question, the platform first tries to retrieve the right context before the model formulates an answer.

That process is called “retrieval”: the targeted retrieval of relevant information from a larger dataset. But in legal case files, retrieval is particularly difficult. A keyword alone is often not enough. A relevant document may use different wording than the lawyer’s question. An important fact may be hidden in an attachment. A legal argument may depend on a combination of different documents. And sometimes the absence of a document or piece of evidence is just as important as what is present in the file.

That is why good tools need to go beyond simple search technology. They need to understand which information may be legally relevant, which documents relate to each other, which chronology appears from the file and which passages can support the answer. They must also prevent the model from filling in the gaps itself when the right context is missing.

The context window is therefore not a detail. It directly determines the reliability of the legal answer.

A platform that selects the wrong context may produce a beautifully written but legally useless answer. A platform that sends too much context may become slow and expensive. A platform that sends too little context misses nuance. And a platform that does not make context verifiable does not allow the lawyer to efficiently check the answer.

That is why the best legal AI platforms will not necessarily be the ones with the largest context window. They will be the platforms that know best how to deal with that context: what must be retrieved, what must be ignored, what must be summarised, what must be quoted literally and what must be shown to the lawyer as a verifiable source.

For lawyers, this is an important insight. When evaluating an AI tool, it is not enough to ask which AI model is behind it. At least as important are the questions: how does the tool process my case file? How does it search through my documents? How does it select relevant passages? Can I verify what the answer is based on? And how does the tool prevent important context from being lost?

The context window is therefore not only a technical limitation. It is one of the central design questions of legal AI. Whoever solves this well is not building a chatbot, but real legal infrastructure.

What does this mean for lawyers?

Lawyers should therefore be aware of how AI tools deal with these technical aspects. At first glance, many tools may look similar: a chat window, the ability to upload documents, generate summaries and draft texts. But the real differences are often under the hood. Just like with cars, the bodywork may look similar, while the engine makes all the difference. With AI tools, this means the way documents are processed, context is selected, model calls are orchestrated, source references are checked, and cost and speed are managed. These are not details. In legal case files, precisely these choices determine whether a tool merely produces fluent text or truly supports reliable legal work.

AI literacy does not mean that lawyers need to master technical details. But they do need to understand the basics.

When an AI platform gives an answer, that answer does not come out of nowhere. It is the result of a chain: documents are processed, context is selected, a model is instructed, tokens are consumed, answers are generated and the platform tries to make all of that usable within a legal workflow.

That insight helps lawyers ask better questions. It also helps them assess AI results more critically. A lawyer should ask: which documents is this answer based on? What context has the AI seen? Are the source references verifiable? Is the answer complete? Has the platform searched the entire case file or only a selection? Is this a first draft or a final legal product?

AI is not magic. AI is infrastructure. And as with any infrastructure, quality, speed, reliability and cost are the result of choices made behind the scenes.

Those who understand this will use AI better. And those who use AI better will be stronger lawyers.

‍