METHODOLOGY · 0.9

What we measure,
and what we don't.

Being the source people cite is the whole plan, and that only works if the numbers are honest about their own limits. This page says where every figure comes from, including the ones that are still placeholders.

Measured in this build

  • Definitions, senses, synonyms, antonyms, WordNet 3.0 via NLTK.
  • Token splits, each model's own tokenizer. Only tokenizer.json is fetched, never the weights.
  • Embedding geometry, neighbours, opposition ranking, personality axes and the 2D field, from each model's real model.embed_tokens matrix. One shard per model, downloaded and extracted on CPU; no GPU and no inference.
  • Foreign headwords, Open Multilingual WordNet 1.4, joined to English WordNet on the shared synset. See the languages below for what that does and does not carry.

Models

ModelRepositoryLabRegionTiersAccess
Qwen3 8B Qwen/Qwen3-8B Alibaba CN 1, 2, 3 free
DeepSeek V3 deepseek-ai/DeepSeek-V3 DeepSeek CN 1, 2 free
Mistral 7B mistralai/Mistral-7B-Instruct-v0.3 Mistral EU 1, 2 free
Gemma 3 4B google/gemma-3-4b-it (gated) Google US 1, 2, 3 upcoming
Kimi K2 moonshotai/Kimi-K2-Instruct Moonshot CN 1, 2 paid*

Model identity lives in config/models.toml. Access tiers are 1 = embedding matrix and tokenizer, 2 = generation, 3 = real forward passes. Gated models need a HuggingFace token and manual licence acceptance; they are skipped rather than failing the build. Upcoming means a model we intend to add: configured and ready to extract, waiting on that licence click, so nothing on this site is computed from it yet.

* Paid models are built lazily. Their embeddings are not extracted up front: the first paid signup triggers the extraction, which pulls the shard and runs the pipeline, so the data exists shortly after there is someone entitled to see it rather than sitting idle beforehand. Until that first run completes, paid-tier panels for that model have nothing to show.

What each number does not mean

Token splits are the tokenizer, not understanding

Seeing tapestry become tape·stry tells you how the text was chopped before the model saw it. It says nothing about whether the model knows what a tapestry is.

Neighbours are distributional, not synonyms

Two words sit close because they appear in similar sentences. That makes antonyms close too, hot and cold are near neighbours in every model. This is why opposition is seeded from a dictionary and only ranked by geometry, never discovered by it.

Personality axes are projections we defined

Each axis is the difference between two sets of anchor words we chose, such as therefore, moreover, utilize against gonna, stuff, yeah. The model never labelled anything "formal". We deliberately do not show a raw peak dimension as a headline: an index like d902 has no interpretable meaning in a modern model. It stays in the expert view where its limits are obvious.

A word the model splits has no vector of its own

Embeddings are per token, not per word. When a tokenizer splits a word we average its pieces, which is the standard cheap approximation and a visibly rough one: Mistral splits tapestry into tap·est·ry and its nearest neighbour becomes incest, a shared est fragment, not a shared meaning. Every panel built on a split word says so. It also ties Panel 2 to the rest: the models that split a word are the ones that represent it worst.

Personality scores are percentile ranks, not raw cosines

A raw projection onto an axis is tiny, typically ±0.05, because the anchor words span a sliver of a 4,096-dimensional space. Read literally, every word sits "near the middle" of every axis. So each score is converted to its rank across the whole vocabulary: "very formal" means more formal than 90% of the words here. That is the honest claim, and it self-calibrates across models whose embeddings have different scales. Full method below.

How the site is served, and what that means for you

LLMsaurus runs as two separate things. Every word page you can read is a static file on a CDN: there is no server in the request path, so those pages cannot be rate-limited, cannot track you, and will not ask you to log in. That is structural, not a promise we could quietly change.

A small API service sits alongside it for the things that genuinely need a server: the JSON API, accounts, keys and the paid endpoints. Rate limits apply there and nowhere else.

What we store, and for how long

DataFormWhy
Reading a word pagenothing at all static file, no server
Anonymous API callsalted hash of the IP rate limiting; salt rotates daily, raw address never written
Accountemail + scrypt hashlogin
API keySHA-256 hash shown once at issue, never recoverable
Card detailsnever touch this app hosted checkout at the payment provider

IPv6 is truncated to its /64 prefix before hashing, because one subscriber is routinely handed billions of addresses and the individual address is not an identity.

How the personality axes are built

Panel 4 scores a word on eight axes. Each axis is defined once per model, from two lists of anchor words, and the axis is only ever as good as those lists, so here they are in full.

Why these axes and not others

An axis earns its place here by clearing three bars. It has to be interpretable without training, so a reader can predict which way their own word should fall before they look. It has to be anchorable at both ends by ordinary words that are unambiguously at one pole, which rules out most interesting-sounding ideas: there is no clean five-word list for "irony" or "beauty". And it has to be cheap, one dot product against a direction computed once per model, because Panel 4 renders for all 20,660 words in every model.

The four this started with cover the dimensions lexical research keeps rediscovering as the ones that separate words at all: valence (positive / negative), register (formal / casual), abstractness (concrete / abstract), and frequency (common / rare). Between them they explain most of what people mean when they say two words "feel different".

The four added since exist because those four leave real gaps. Arousal (mild / intense) is the second axis of the standard valence–arousal–dominance model of affect, and without it rage and sadness look alike: both negative, wildly different in intensity. Dominance (powerless / powerful) is the third, and separates sovereign from fragile independently of whether either is a good thing. Technicality (everyday / technical) is not the same as formality: enzyme is technical without being formal, and furthermore is formal without being technical. Magnitude (small / big) is the most consistently reported dimension of physical semantics after valence.

Two axes we tried and dropped, for the record. Modern / archaic failed the anchor bar: the vocabulary is frequency-ranked, so it contains thou and alas but not thine, hath or whence, and a two-word pole is not an axis. Body / mind failed the independence bar: it scored almost identically to concrete / abstract, so it added a slider without adding information.

AxisPositive poleNegative pole
common / rare the, and, make, time, people esoteric, quixotic, abstruse, recondite, perfidious
abstract / concrete justice, idea, freedom, concept, theory rock, table, dog, chair, hand
formal / casual therefore, moreover, utilize, furthermore, consequently gonna, stuff, yeah, kinda, okay
technical / everyday algorithm, molecular, enzyme, polynomial, voltage thing, food, house, water, walk
positive / negative wonderful, joy, love, excellent, delight awful, hate, misery, terrible, grief
intense / mild furious, rage, terror, frenzy, agony calm, gentle, mild, quiet, soothing
powerful / powerless powerful, mighty, dominant, commanding, sovereign helpless, fragile, weak, subordinate, inferior
big / small enormous, vast, massive, gigantic, immense tiny, petite, slight, miniature, slender

Listed in the order Panel 4 shows them, grouped lexical, register, affect, physical. The four older axes are anchored partly in deliberately rare words (quixotic, gonna) that extraction has to add on top of the vocabulary. The last four are anchored only in words the vocabulary already contains, which is why they could be added without re-extracting 14.2 GB of embedding shards.

The arithmetic

Take the mean vector of the positive anchors, subtract the mean vector of the negative anchors, normalise the result: that is the axis, a direction in the model's own embedding space. A word's raw score is the dot product of its unit vector with that direction, which lands roughly in −1..+1 and in practice within about ±0.05. The score you see is that value's percentile rank across the whole vocabulary, which is why the wording is "more abstract than 90% of the words here" and never "90% abstract".

What this is not

These are not labels the model assigns, and not dimensions the model has. They are directions we defined by picking ten words per axis, and a different ten would move every score. Nothing about the model says formality lives on one axis; the claim is only that words the anchors treat as formal end up on the same side. The anchors themselves sit at the extremes of their own axis by construction, so seeing enzyme top the technical axis is not evidence of anything; the interesting cases are the words that were never anchors.

The failure mode worth knowing

Anchors are deliberately extreme, which is exactly why a frequency-ranked vocabulary does not contain them: quixotic and gonna are not common words. Extraction therefore adds the anchor words explicitly, on top of the vocabulary. When it does not, an axis whose anchors are missing is dropped without complaint, which is how two of the four axes silently disappeared on the first run against real embeddings. Code: pipeline/personality_axes.py.

The four languages, and what a synset pivot can carry

Spanish, French, Japanese and Chinese come from Open Multilingual WordNet 1.4, which maps lemmas in 25 languages onto English WordNet synset ids. The join is the whole mechanism: a foreign headword is matched to the English sense it shares and answered with the geometry already computed for that sense. Nothing is re-extracted per language, and no multilingual model is involved.

What that means for a number on a foreign page. The neighbours, the personality axes and the usage lift were all measured on the English pivot, and the page names that word rather than implying otherwise. The token split is the exception and is computed on the headword itself, because a tokenizer is language-agnostic. Definitions are English WordNet's: OMW carries lemmas, not glosses.

Coverage, against the 20,660-word vocabulary: Japanese 19,010 words (92.0%), French 18,962 (91.8%), Spanish 15,480 (74.9%), Chinese 14,748 (71.4%), for 123,338 headwords in total. Where a language has no headword for a sense, the page shows that language as a gap rather than guessing.

Two artefacts of the source, stated because they are visible. Within one sense, OMW's lemma order is the file's rather than a frequency order, so a leading headword is occasionally not the one a speaker would pick first. And a headword's pivot is chosen by how central the shared sense is to the English word, then by frequency; before that rule existed chien led with click, because both reach the mechanical-catch sense and click is the commoner English word. Code: pipeline/omw_lemmas.py.

What the language cost table measures, and what it does not

The table on the use-cases page gives a mean token count per language per model. It is real tokenizer output — no estimate anywhere in it — but the number it reports is the product of an aggregation choice, and a different defensible choice gives a different number. Both are stated here so the table can be read for what it is.

The sample. Every concept in the build that has a headword in every language carried: 3,983 English synsets. Foreign headwords come from Open Multilingual WordNet, joined on synset id, so the languages really are naming the same concepts rather than being compared across independent word lists. Each headword is tokenized with a leading space, exactly as it would appear mid-sentence.

The mean is unweighted. Each concept counts once no matter how common the word is. The vocabulary is frequency-ranked and that rank is loaded and indexed, but this stage does not read it, so aardvark and able carry equal weight. Proper nouns are the clearest cost of that: the Chinese headword for the sense pivoted on aaron is fifteen Mistral tokens, and it counts as much as a function word. A frequency-weighted mean would be a different and also defensible measurement; it is not the one on that page.

The mean is over headwords, not concepts, and the two sides are not symmetric. English contributes one headword per concept — the pivot — while every other language contributes all the headwords OMW lists for that synset. Measured over the shared set that is 1.0 headwords per concept for English, 2.0 Spanish, 2.2 French, 3.3 Chinese and 7.5 Japanese. The foreign columns therefore include orthographic variants and archaic forms with no English counterpart in the average (Japanese offers four spellings of one verb for abandoned alone), and concepts with many synonyms pull harder than concepts with few.

How much this moves the answer. Averaging within each concept before averaging across concepts shifts the published range from 1.8x–3.5x to about 1.8x–3.6x. Taking the cheapest headword per concept instead — the cost a writer who picks the shortest word would actually pay — narrows it to roughly 1.4x–2.7x. English comes out cheapest under all three. The within-language model comparison is far more stable than the between-language one, because two tokenizers reading the same string differ only in their vocabularies: the Chinese DeepSeek-against-Mistral gap stays at 1.9x under every aggregation above. Code: pipeline/language_cost.py.

Vocabulary

This build ships 20,660 words: 18,000 everyday plus every scientific term WordNet's topic domains can supply. That is the size at which lookups essentially always land, without carrying hundreds of thousands of entries nobody searches for. Everyday words are the most frequent ones that have a dictionary entry. Method and measured output: docs/vocabulary.md.