What Are AI Tokens and Context Limits? Why Your AI Forgets What You Told It

What Is a Token? (The Non-Technical Explanation)

Before AI models can process text, they break it down into small pieces called tokens. Think of it like a butcher breaking down a whole animal into individual cuts before working with them. The AI doesn't see "Hello, how are you?" — it sees something more like [Hello][,][ how][ are][ you][?].

Tokens are not exactly words. They're roughly 3–4 characters each in English, which works out to about 0.75 words per token. Here are some real examples:

Why does this matter? Because tokens are the currency of AI. Every word you type, every line of code you paste, and every response the AI generates all cost tokens. And every AI tool has a limit on how many it can handle at once.

What Is a Context Window?

Every AI model can only "see" a limited amount of text at any given moment. That limit is called the context window. Think of it like the AI's working memory — it can hold a certain amount of information in mind at once, and anything outside that window simply doesn't exist from its perspective.

The context window includes everything in the current conversation:

• Your messages (all of them, going back to the start)
• The AI's responses (all of them)
• Any files, code snippets, or documents you've pasted in
• Any system instructions the tool has set up behind the scenes

When you send a message, all of that goes in together. The AI generates its response with full awareness of everything in the window. But as the conversation grows, it creeps toward the limit — and that's when things get weird.

For a deeper look at how context windows are architected, see our dedicated guide: What Are Context Windows? A Plain-English Guide.

Why Your AI "Forgets" Mid-Conversation

Here's the scenario that frustrates everyone eventually:

1. You start a coding session. You explain your project, your tech stack, your naming conventions.
2. The AI follows your rules perfectly for the first dozen exchanges.
3. An hour later, you notice it's using camelCase when you explicitly asked for snake_case. Or it's suggesting a library you said you weren't using. Or it keeps making a mistake you already corrected three times.

What happened? The conversation got long. Your early setup messages — the ones where you established your requirements — have been pushed out of the context window to make room for more recent exchanges. The AI isn't ignoring you. Those messages are gone from its view.

Different tools handle context limits differently. Claude.ai shows a visual warning when you're approaching the limit. ChatGPT silently drops old messages. Cursor manages context automatically but may silently exclude files it deems less relevant. The behavior varies, but the underlying constraint is the same.

Token Limits Across Major AI Coding Tools

Context windows have grown dramatically in the past two years. Here's where the major tools stand as of early 2026:

To understand how context windows work at a deeper level — including how models handle very long inputs — see What Are Context Windows?

Bigger Context Window ≠ Better Results

More room sounds like it should always be better. In practice, there are real trade-offs:

Speed

Processing a long context takes more compute. A conversation with 100,000 tokens of history is noticeably slower than one with 5,000. If you're pasting huge files at the start of every session "just in case," you're paying a speed tax for context you may never use.

Attention Drift

AI models tend to pay more attention to the beginning and the end of their context window — a phenomenon sometimes called "lost in the middle." Critical instructions buried in the middle of a very long conversation may get relatively less weight than the same instruction at the top or bottom. This means a 200k context window doesn't guarantee the AI will remember message #47 as reliably as message #1.

Cost (for API users)

If you're building with the API rather than using a chat interface, token count directly drives cost. Input tokens and output tokens both have a price per thousand. A prompt that uses 50,000 tokens when 5,000 would have worked is ten times more expensive per call.

Practical Strategies to Stay Under Token Limits

You don't need to obsess over token counts to work effectively with AI. These habits will serve you well across every tool:

1. One Task Per Conversation

The single most effective thing you can do. Don't build your entire app in one endless chat session. Use a new conversation for each meaningful task:

• New conversation to design the data model
• New conversation to build the authentication flow
• New conversation to debug the payment integration

This keeps each conversation focused, fast, and within safe context limits. It also makes it easier to re-run a task from scratch if the AI goes sideways.

2. Share Only What's Relevant

Resist the urge to paste your entire codebase. If you're asking about a bug in submitForm(), paste only that function — not the whole file. If the AI needs context about how a variable was defined, you can add that specific section. Most of the time, a focused snippet works better than a wall of code.

Here's my entire app.js file (400 lines). There's a bug
somewhere in the form submission. Can you find it?
[pastes 400 lines]

This function handles form submission in a React app.
When the user clicks submit, it throws:
"Cannot read properties of undefined (reading 'email')"

Here's the function:
[pastes 20-line function]

The form data comes from a controlled component using
useState. What's wrong?

3. Front-Load Critical Context

If you have requirements the AI must follow throughout a session — your tech stack, naming conventions, constraints — put them at the very top of your first message. Don't save them for message five. The start of the conversation is the safest place for important context.

Project context for this session:
- Stack: Next.js 14, TypeScript, Prisma, PostgreSQL
- Styling: Tailwind CSS only (no inline styles)
- Naming: snake_case for DB columns, camelCase for JS
- No external libraries without asking me first
- All components should be server components unless
  client interactivity is required

Task: Build a user profile page that shows their
recent orders. Here's the Prisma schema for orders:
[paste only the orders model, ~15 lines]

4. Summarize Instead of Continuing

When a conversation is getting long and you need to keep working on the same project, don't just keep going. Instead, ask the AI to summarize the decisions and code you've written so far, then start a new conversation and paste that summary at the top. You get a clean context window with all the important information preserved.

Before we wrap this session: please write a concise
summary (bullet points) of:
1. What we built
2. The key decisions we made and why
3. Any constraints or conventions I established
4. What was left to do

I'll use this as context at the start of my next session.

5. Use Structured Prompts, Not Essays

A well-structured prompt uses fewer tokens and gets better results than the same information written as flowing paragraphs. Use bullet points for requirements, code blocks for code, and clear section headers when you have multiple pieces of context to share.

For more on how to write prompts that get better results and use fewer tokens, see the AI Prompting Guide for Coders.

When to Start a New Conversation

Knowing when to cut your losses and start fresh is a skill. Here are the clear signals:

• The AI forgets something you told it more than once. If you've re-stated the same requirement three times and it keeps violating it, the constraint has probably been pushed out of context. Start fresh and lead with that requirement.
• You're switching to a different part of the project. Finished the auth flow? Don't continue in that same conversation to build the dashboard. The old auth context is mostly noise now.
• Response quality noticeably drops. Long contexts sometimes produce more confused, hedging, or lower-quality responses. If the AI suddenly seems less capable, context bloat is a likely culprit.
• You pasted a lot of files early on. Large file dumps fill context fast. If you front-loaded 10 files and the conversation is now 30 exchanges deep, you're probably near the limit.
• The tool warns you. Claude shows explicit warnings. Cursor shows context usage indicators. If a tool is telling you you're running out of context, believe it and start fresh.

How Different Tools Handle Context

Each tool manages context a little differently. Understanding this saves frustration:

Claude (claude.ai)

Claude has the largest context window of the mainstream tools at 200,000 tokens — roughly the length of two novels. It shows you a visual indicator when you're approaching the limit. It handles very long conversations better than most, but quality still degrades near the top. Claude is a strong choice for tasks that genuinely require long context — like reviewing an entire codebase for consistency. For shorter tasks, a new conversation is still better. Learn more in the Claude Code Beginner's Guide.

ChatGPT / GPT-4o

GPT-4o supports 128,000 tokens. ChatGPT silently drops old messages when you hit the limit — it doesn't warn you. You'll notice when the AI starts acting like it forgot things. For coding workflows, this silent behavior is the most frustrating aspect of using ChatGPT for long sessions.

Cursor

Cursor is smart about context management. In Agent mode and Composer, it automatically selects the most relevant files from your codebase rather than dumping everything in. You can also explicitly add files to context using @filename syntax. This means you're less likely to hit raw token limits than in a raw chat interface — but you should still understand what Cursor is including. See the Cursor Beginner's Guide for how to manage context in Cursor specifically.

GitHub Copilot

Copilot's inline suggestions work on a narrow context — the current file, the current function, and sometimes files you have open. Copilot Chat has a larger context but still works best when you're focused on a specific file or feature. The @workspace command tells Copilot to search across your project, but it's more selective than tools with full codebase indexing. Learn more in our guide to what GitHub Copilot is and how it works.

How to Structure Prompts to Use Fewer Tokens

Every word in your prompt is a token being spent. Here are specific techniques that reduce token usage without losing information quality:

Cut the Preamble

Don't warm up your prompts. The AI doesn't need "I hope you can help me with this. I've been working on a project and I've run into a bit of an issue..." Just state the task.

Hi! I was wondering if you could help me. I'm building
a web app and I'm not super experienced with JavaScript
but I've been learning and I have this function that
isn't working the way I expected. Could you take a look
at it and maybe explain what's going wrong?

This JavaScript function isn't working as expected.
Explain the bug and fix it:

Use Code Comments as Your Spec

Instead of a long paragraph describing what you want, use a short comment block directly above your code. This is exactly how inline autocomplete tools like Copilot and Codeium are designed to work — and it uses far fewer tokens than a separate explanation.

Avoid Repeating Yourself

If you've already established context in this session, don't re-state it in every message. A brief reference is enough: "continuing from above" or "using the same User model as before." Full re-statements eat tokens and dilute the AI's attention.

Prefer Bullet Points Over Paragraphs

Bullet points express requirements in fewer words than prose — and they're clearer for the AI to parse. "The form should validate email format, require all fields, and show inline error messages" works better as three bullets than one run-on sentence.

Ask for Shorter Responses When Appropriate

AI output also counts against your context window. If you're in a long session and want to preserve context space, you can ask: "Keep your response brief — just the code, no explanation." That 500-token explanation the AI would have given is 500 tokens not filling up your window.

For a comprehensive approach to writing better AI prompts, read the AI Prompting Guide for Coders — it covers token efficiency in the context of a full prompting strategy.

When Context Problems Look Like AI Incompetence

One of the most common frustrations with AI coding tools: "It keeps making the same mistake even after I correct it." Half the time, this isn't an AI quality issue — it's a context issue. The correction fell out of the window.

Before you conclude an AI tool is broken or bad at something, run through this checklist:

• Is this a long conversation? If you're 30+ messages in, start a new session and see if the issue persists.
• Did you paste a lot of code early? Large early pastes compress the effective "age" at which later context falls out.
• Is your requirement in the current context? In a new session, if you re-state the requirement clearly at the top, does the AI follow it? If yes, you had a context window problem, not an AI problem.
• Is the AI hallucinating or just missing context? These are different failure modes. Hallucination is making up false information confidently. Missing context is giving a correct answer that contradicts something it can no longer see. See our guide on how to debug AI-generated code for how to tell the difference.

What to Learn Next

Now that you understand tokens and context limits, here's where to go depending on what you're working on:

Frequently Asked Questions