Skip to content

The method

This page describes how the report is produced and why it is built the way it is. It deliberately does not include the extraction prompts, the marker taxonomy, or the model bindings — those are the parts we sell. Everything that governs whether you should trust the output is here.

The short version: the pipeline has ten stages, only four of which call a model, and the discipline is concentrated in the seven that don't.

The two rules everything else follows from

1. The observable / interpretive wall

The pipeline extracts things that are measurable in the text: how often a topic returns, whether a stated plan is revisited, how a message changes after the assistant disagrees, how the length and hour of messages drift over months.

It does not extract internal states. There is no stage that concludes "the user is anxious," because that is not a property of a text file. Meaning is left to you.

This sounds like a small distinction. It is the difference between a tool that reports "this topic returned in eleven sessions across four months" and one that reports "you seem preoccupied." The first is checkable. The second is a stranger's opinion dressed as a measurement.

2. The reasoning gate — the "Ctrl+F test"

A pattern that could be found by searching for a word is not a finding, it is a keyword match. If the only thing separating a marker from grep is a synonym list, we do not want it: it tests recall, not judgment, and it is exactly the kind of thing that flags a novelist for writing a villain.

So every marker type must force an assessment of a relationship rather than the presence of a word:

  • a claim against the evidence offered for it
  • this session against what earlier sessions established
  • a stated belief against the behaviour in the same conversation
  • what is said against what happens next

Any candidate marker type that survives a plain-text search is cut. This is the single most expensive rule in the project and it removes most of what a naive version of this tool would report.

The exclusion rule

A model may not hold a judgment role in analysing a corpus it generated.

Your ChatGPT export is roughly half ChatGPT's own output. A large share of what is worth looking at lives in the assistant turns — did it push back, and what did you do when it did? How much agreement and validation was in the room?

Using GPT to judge those turns is self-assessment, and it is biased in precisely the dimension being measured: a model's sense of "how much warmth and agreement is normal from an assistant" was set by the training that made it that assistant. This is a calibration offset, not a question of honesty.

So for a ChatGPT export, no OpenAI model extracts, synthesises, or reviews. The rule is written into the configuration rather than the documentation, and it is relative to the corpus — if we analysed a Claude export, Anthropic models would be excluded instead.

Four different model families do four different jobs, from four different vendors. That is not about cost. It is so that no single vendor's blind spot becomes the report's blind spot.

The reviewer is adversarial by construction

The review stage does not ask "are these findings good?" A reviewer asked that question agrees, because agreeing is what these systems do.

Instead, for every candidate finding, an independent model from an unrelated vendor is told to construct the strongest case that the finding is wrong — that the benign reading is the true one, that the evidence does not support the claim, or that the pattern is an artifact of how much text there is. It is instructed to default to refuted unless the evidence forces otherwise.

It also gets one question nothing else in the pipeline can answer: what would a different framework look for that this one misses? Every other stage operates inside our taxonomy, so this is the only step that can catch a hole in it.

Reviewer verdicts do not silently delete anything. They attach to the finding, and the report shows contested findings with the counter-argument visible. A finding the reviewer refuted and the synthesiser could not defend is precisely the thing you should see, not the thing we should hide.

On the first real run, the reviewer could not dismiss 4 of 7 candidate patterns. The other three are in the report too, marked as refuted.

Evidence honesty

Every finding cites specific messages — timestamp, session, message index. A finding with zero citations is not printed; that is an automated gate, not an editorial habit.

Every finding carries a benign interpretation, generated as part of the finding rather than bolted on. Also an automated gate: a finding without one does not render.

Findings are tiered by how much support they actually have. A pattern seen twice in a year is not presented like one seen in forty sessions.

Every finding in every report we currently produce is Bootstrap tier — theoretically motivated, not validated against chat data. The report says so, on the finding, every time. When that changes it will change because of measurement, not because we got more confident.

What the pipeline refuses to do

  • It never names a condition. A panel of four independent judges checks the finished report for diagnosis language, clinical register, and unsupported claims about isolation before it is released to you. If any judge raises a concern, the report does not ship unreviewed.
  • It never analyses a corpus without consent from the person who wrote it.
  • It never presents an absence as a finding without disclosing what the extractor is known to miss — because "this person never does X" and "our extractor cannot see X" produce identical-looking output, and only one of them is about you.

That last one is the most important sentence on this page, and it has its own page: what this cannot do.

Why sessions marked "roleplay" are treated differently

Fiction, roleplay, and prompt-testing produce text that looks alarming and means nothing. A person writing a villain, or stress-testing a model's guardrails, generates exactly the surface features a naive tool would flag.

Sessions that look like fiction or testing are detected and down-weighted rather than fed to the extractor at full strength. The detector is deliberately crude and over-inclusive: over-flagging costs a down-weight, under-flagging costs a false statement about a person.

What we are not telling you

For completeness, the things this page withholds:

  • the marker taxonomy — which types are extracted and the dimensions tagged on each
  • the extraction system prompt
  • which specific models hold which role, and the probe cases used to qualify them
  • the question sets used to measure reproducibility and safety
  • the clustering rules

We think the durable advantage here is the measurement discipline rather than the prompts, which are cheap to copy. But they are still ours.