Free · runs entirely in your browser

Token Speed Simulator

See how tokens-per-second and output length turn into a real waiting time — before you set response limits on a live product.

Count Tokens Locally

Scope: this is a local animation based on the numbers you enter, not a live call to any model. It's built to make the trade-off between output length and generation speed easy to feel, so you can set sensible response limits before shipping.

Set your speed and length

Pick a preset or set your own numbers, then press start.

How fast the model writes once it starts answering.

How long the full reply is expected to run.

The wait before any text appears — routing plus prompt processing.

5.40 s
Estimated total
0.00 s
Elapsed
0/300
Generated
Progress0%
Time to first token (7% of total)Decode (streaming)
Simulated streamed output
Press start to preview how output appears as tokens stream in.

Compare two speeds

Uses the output length set above (300 tokens) — good for comparing a cheaper, slower model against a faster one before you decide.

6.00s
Speed A · 50 t/s
1.50s
Speed B · 200 t/s

Speed B finishes 75% faster for a 300-token reply.

How output length changes the wait

Your current speed (60 t/s) is highlighted so you can see where it sits against a slow and a fast reference.

Output length50 t/s60 t/s (yours)200 t/s
150 tokens3.0s2.5s0.8s
500 tokens10.0s8.3s2.5s
1,000 tokens20.0s16.7s5.0s
3,000 tokens60.0s50.0s15.0s

What does "tokens per second" actually mean?

Tokens per second describes one specific stage of a model's response — how quickly it writes once it has already started. It's a useful number, but it's easy to over-read it, because a lot of what a user actually experiences as "speed" happens before that stage even begins. Understanding the difference between the wait before text appears and the pace at which text appears once it starts is what makes this simulator actually useful for planning a real product, rather than just a fun animation to watch.

Decode speed

The pace of the answer's body once generation has begun — this is the number the tokens/s slider in the simulator controls directly.

First-token latency

The setup work — routing the request, reading the prompt, checking any retrieved context — that has to finish before a single word can appear.

Perceived latency

What the person actually experiences, which streaming can shorten even when the underlying model hasn't gotten any faster.

How to read the numbers in practice

Short replies feel instant

At 100 tokens/second, a 150-token answer — roughly a two-sentence reply — finishes generating in about 1.5 seconds. Add a typical 300-400ms wait before the first token, and the whole exchange still comfortably fits inside what most people experience as a snappy chat interaction.

Long answers need a different plan

The same 100 tokens/second speed takes about ten full seconds to finish a 1,000-token answer — a length that's common for a detailed explanation, a code review, or a drafted report section. Ten seconds of silence feels broken; ten seconds of visibly streaming text feels like the product is working. The fix usually isn't a faster model, it's showing progress.

The math behind the wait

The simplest useful estimate splits the wait into two additive pieces: the setup time and the writing time.

writing_time = answer_length ÷ speed
total_wait = first_token_delay + writing_time

300 ÷ 60 = 5.00s completion
0.40s TTFT + 5.00s = 5.40 s total for your current settings

What actually changes real-world speed

  • • Model size and architecture set the ceiling on how fast decode can run in the first place.
  • • Prompt length extends prefill time before generation even starts, independent of output speed.
  • • Hardware, batching, and quantization on the provider's side can shift throughput significantly between requests.
  • • Provider load at any given moment means the same model can measurably speed up or slow down hour to hour.
  • • Retrieval, tool calls, and moderation checks add real wall-clock time that sits entirely outside token generation.

Three stages worth telling apart

Time to first token

The wait before any streamed text appears at all — shaped by routing, queueing, and how much context the model has to read first.

Prefill

The model reading and encoding your input before it writes anything back. Longer prompts and retrieved documents both stretch this stage.

Decode

The repeated step that produces the answer one token after another — the stage this simulator's tokens/s number is standing in for.

Worked example: setting a response cap for a support chat widget

Say the underlying model runs at roughly 70 tokens/second with a 350ms time to first token, which is fairly typical for a mid-sized hosted model. An uncapped reply that runs to 900 tokens would take about 350ms + (900 ÷ 70) ≈ 13.2 seconds to fully finish — long enough that a user watching a spinner instead of streamed text is likely to assume something broke. Capping typical replies to around 250 tokens brings the same math down to roughly 3.9 seconds, and pairing that cap with streaming means the first words appear well under half a second in, which is usually enough to keep the interaction feeling responsive even before the answer is complete.

Common mistakes when planning around speed

  • • Treating a published tokens-per-second figure as a fixed guarantee rather than an average under normal load.
  • • Ignoring time to first token entirely and judging "speed" only by the decode rate.
  • • Assuming a non-streaming integration will feel as responsive as a streaming one at the same underlying speed.
  • • Not capping output length, so an otherwise fast model still produces a slow-feeling reply by writing too much.
  • • Benchmarking speed once and not re-checking it, when provider load and model versions both shift over time.

Where speed planning actually matters

Chat products

Use a simulation like this one to pick output limits that keep conversational replies feeling responsive rather than sluggish.

Agent workflows

A long chain of tool calls and intermediate steps can feel slow to a user even when the underlying model throughput is genuinely high.

Document tasks

Summaries, reports, and code reviews often need a visible progress state or partial streaming for outputs that run past a few hundred tokens.

Glossary of speed & latency terms

Tokens per second
The pace of output generation once the model has started writing an answer.
Time to first token (TTFT)
The wait before any streamed text appears, covering routing and prompt processing.
Prefill
The model reading and encoding the full input prompt before generation begins.
Decode
The repeated step that produces each output token in sequence.
Streaming
Sending partial output to the client as it's generated, instead of waiting for the full reply.
Throughput
How much total output a system can produce across all concurrent requests, not just one.
Batching
Processing multiple requests together on the same hardware pass, which can change per-request speed.
Quantization
Running a model at reduced numeric precision, often trading a little quality for more speed.

Provider streaming docs

Check each provider's own documentation for exact streaming behavior and event formats before you build against it.

Embed this simulator

Drop a lightweight version of this tool on your own site — self-contained, dark theme, links back to the live page.

<iframe
  src="https://apextechjournal.com/embed/token-speed-simulator?ref=widget"
  width="100%"
  height="440"
  style="border:1px solid #262e39;border-radius:12px;"
  loading="lazy"
  title="Token Speed Simulator by Apex Tech Journal"
></iframe>

Token Speed Simulator FAQ

It measures how fast the model produces output once it has started answering — the decode stage. It does not include the time spent before generation starts, such as routing the request, reading the prompt, or waiting in a provider's queue.