Understanding Context Windows and Token Limits

Overview

Harvey uses powerful AI models to generate high-quality outputs quickly and accurately. Like all large language models, Harvey’s models operate within a context window: the total amount of information the AI can "see" at once when generating a response.

This article explains how context windows work across Harvey, roughly how much material each part of the product can consider in a single pass, and why more material does not always mean better results. These estimates are flexible. You can usually go beyond them and still get strong output, and the guidance here is most useful as a troubleshooting reference when results feel incomplete.

What Are Tokens and Context Windows?

Harvey processes text in tokens, which are small pieces of language, typically a word or part of a word. For example:

• "contract" = 1 token
• "unbelievable" = 3 tokens
• "Please summarize this contract." = 6 to 8 tokens

Harvey's models also have a context window, which is the total amount of information the AI can consider at once. Within a single response, the context window has to hold several things together, including:

• Your input (your prompt or question)
• Any documents or sources included
• Prior conversation history in the same thread
• Harvey’s internal instructions and tools
• Harvey's response

Think of the context window as the AI's short-term working memory. Everything the model needs for a given answer has to fit inside it at the same time.

Because different models have different context windows, we describe capacity in approximate pages of text rather than exact token counts. This keeps things practical and reflects that the real limit shifts depending on the model you select and the kind of question you ask.

How Much Text Can Harvey Actually Handle?

There is no single number, and that is the most important concept to understand.

It depends on the model. Each model available in Harvey has a different context window. As a rough guide:

• Smaller models can consider on the order of ~100 pages of text in a single pass.
• Mid-range models can consider several hundred pages.
• The largest, newest models can consider roughly a thousand pages or more.

These are approximate, flexible ranges, not hard upload limits. You can safely provide more material than this and still get useful results. By default, Harvey chooses the most appropriate model for your query. You can also pick a specific model using the model selector.

It depends on the kind of question. Two questions over the same 300-page contract can behave very differently:

• Targeted questions ask Harvey to find specific information, for example, "What is the termination provision in this contract?" Harvey only needs to locate and reason over the relevant sections, so these questions scale well even across very large documents.
• Comprehensive questions ask Harvey to reason over everything at once, for example, "Is there anything non-standard anywhere in this contract?" These questions require Harvey to hold and weigh the whole document together, so they are more demanding and reach their practical limit sooner.

If a question needs a complete understanding of a large corpus rather than a specific answer from within it, expect it to be more sensitive to length, and consider breaking it into smaller, more focused asks.

More material can reduce quality even when it fits. Fitting inside the context window is not the same as using it well. As the window fills up, it becomes harder for the model to reliably recall any one specific detail, because that detail is competing with everything else in view. So even within the ranges above, a leaner, more focused request often produces a sharper answer than one packed with extra material.

Tip: When results feel incomplete or inconsistent, the fastest fix is usually to narrow the question or reduce the material, not to add more.

Threads

A thread is a single ongoing conversation, including your prompts, attached documents, and the full back-and-forth history of your earlier questions and Harvey's earlier responses.

Because history accumulates, a long thread uses more of the context window over time, even if no single document in it is large. When a thread grows long enough, Harvey manages this automatically so the conversation can keep going. In practice, that can mean summarizing earlier parts of the conversation to make room, or setting aside the least relevant material. You may notice this as Harvey seeming to "forget" a small detail from much earlier in a long thread.

Agents built through the New Agent Builder (Early Access) also operate within threads and are subject to the same limits and behavior as Assistant queries.

Querying with Sources

You can point Harvey at large amounts of material, including from your vaults, without hitting a wall, as long as your question is focused. This works because Harvey does not simply pour every document into the context window.

Retrieval is agent-driven. When you ask a question over sources, Harvey works out what it actually needs to answer it and pulls in the relevant material, rather than loading everything indiscriminately. Just because something could fit in the context window does not mean Harvey will fill the window by default. Harvey agentically determines the information it needs at each turn in the conversation, and can search multiple times until it finds that information or fills its context window. This is deliberate: keeping only the relevant material in view means the important information does not get drowned out by everything else. We calibrate Harvey's retrieval to keep that signal strong and the noise low.

The clearer and more specific your question, the better this works:

• Higher relevance: "What is the termination clause in Section 9?"
• Lower relevance: "Summarize each of the provisions in the contract."

When Harvey Delegates to Specialists

For larger or more specialized tasks, Harvey does not rely on a single model call. It can hand parts of the work to specialized helpers, each of which works in its own separate context window.

A useful way to picture this: it is like delegating to specialists on a team. You might ask a researcher to dig into one subtopic and a drafter to produce a memo. You give each of them a focused briefing rather than your entire case file, they do the work in their own workspace with their own materials, and they hand back a summary or a finished document.

This matters for two reasons:

• On the input side, because each helper has its own fresh window, Harvey can work across more total material in a single request than any one model window could hold on its own.
• On the output side, large deliverables such as generated documents are produced by these helpers as separate files. That means a request like "draft several memos" is not squeezed into one response. Each document is produced with its own room to work.

Review Tables

Review Tables let you work across large volumes of documents without consuming a single shared context window.

When you build a Review Table, Harvey treats each cell as its own independent task. A cell is the intersection of a column (a question) and a row (a document), and each cell runs as its own separate model call with its own context window. One cell's context has nothing to do with another's.

Because of this, Harvey pulls in only the most relevant parts of each document for each cell, and a table can span thousands of documents without any single call becoming large. The amount of source material considered per cell is intentionally smaller than in a thread, because retrieval has already narrowed things down to what that specific question needs.

If cells consistently return incomplete answers on very long documents, that is the signal to break the document up or ask a more targeted question.

Workflows

Each AI block in a Workflow behaves like its own Assistant query, with the same model-dependent ranges described above.

• When a block references documents or vaults, the agent searches for relevant material from those sources.
• Outputs and variables can be passed from one block to the next, but each block runs independently within its own context window and only the context that has been added to it.

Quick Reference Table

What Happens If You Exceed the Context Window?

Harvey is built to keep working rather than simply fail. When material approaches the limit, Harvey manages it automatically by focusing on the most relevant information, summarizing earlier conversation history in long threads, and setting aside less relevant material.

If a response feels incomplete or inconsistent, try the following:

1. Simplify: Shorten your prompt or break it into smaller asks. Ask a specific question first, then ask follow-ups from the output.
2. Focus: If many files are in context, use the @ mention feature to focus the model’s attention on specific files.
3. Split: Run long tasks across multiple requests or Workflow blocks.
4. Use a Review Table first: Build a Review Table with a column for each data point you care about, then use Ask over Review to run a question over the extracted results. This ensures every document in the table is reviewed.

FAQs