"""tokens — the ONE char↔token estimate, and the only place the ratio is written.

Alexandria measures "how much context is this?" in four places that had drifted
into three private copies of the same constant:

  - :mod:`vault_core.query.load` — the context pack's budget shares;
  - :mod:`vault_core.query.vaultgraph` — bfs' cumulative-token accounting;
  - :mod:`models.loop` — the conversation-size guard (provider ``input_tokens``
    → the char scale it evicts on);
  - :mod:`mcp.tools` — paged tool results, which until now budgeted by ITEM
    COUNT and so could hand a seat 50 rows of 40 chars or 50 rows of 4,000.

They all meant the same thing and all said 4, but nothing made that agreement
checkable, and a paged result set was measured on a different scale from the
pack it would be read alongside. One module, one constant: a budget quoted by
any of them now means the same thing to the others.

**It is an estimate and stays one.** ~4 chars/token is the English-prose ratio,
confirmed against a provider report in the context incident behind
``models.loop`` (4.2M chars ≈ 1.06M tokens). Dense JSON, CJK, and base64 all run
richer, so a budget spent here is a floor on the real spend, not a meter —
size *guards* off it, never bill off it.
"""
import json

# Rough English prose ratio. Float so callers can divide without surprise; the
# helpers below return ints because a fractional token is not a useful answer.
CHARS_PER_TOKEN = 4.0


def est_tokens(text) -> int:
    """Estimated tokens in a string. ``None``/empty → 0."""
    return int(len(text) / CHARS_PER_TOKEN) if text else 0


def est_tokens_json(obj) -> int:
    """Estimated tokens of a value **as it will go over the wire** — i.e. of its
    JSON encoding, so quoting, key names, and escapes are counted too.

    This is the honest measure for a tool result: a row of 40 visible chars is
    not 10 tokens once it is ``{"path": …, "line": …, "text": …}``. Anything
    unserialisable falls back to its ``repr`` rather than raising — a size
    estimate must never be the thing that fails a call.
    """
    try:
        return est_tokens(json.dumps(obj, ensure_ascii=False))
    except (TypeError, ValueError):
        return est_tokens(repr(obj))
