ttok 0.4 makes supported tokenizers easier to discover

Simon Willison’s ttok counts text tokens and can truncate input to a token limit using the tiktoken library. Version 0.4 adds a command that lists supported model names, alongside maintenance fixes. The tool accepts piped text, making it useful for checking the size of files before assembling them into a model’s input.

Simon Willison · Oct 8, 2026

Why it matters

A small local preprocessing tool can make prompt preparation easier to inspect and script.

More context

Try the released command-line tool through uvx or an isolated Python installation. Use --list-models to inspect the names supported by the installed tokenizer library, and select the intended model explicitly. This is a maintenance release of an existing tool, not a new model.

Keep in mind

Token counts depend on the tokenizer and exact input. They are not a complete invoice estimate for an API request with message framing or tool definitions. The repository’s default-tokenizer documentation also describes later version behavior; check the version you install.

Original read