Methodology

This site is an evidence graph with an owner. I set what it is for, what counts as a person, how strong a claim may be, and what may be published. Models help me open records and file the statements those records support. They do not own the standard those statements have to meet.

That split is the heart of the method, but it didn’t arrive fully formed. I wrote the specification, AI helped add more bones to it, and then we tried it on actual historical records. The oldest Waltons we could find, modern people with machine-readable identities, my own family, and plenty of people who may or may not be related to any of them. We watched what broke, fixed it, and tried again.

This is still happening. The methodology isn’t a declaration that we’ve solved genealogy. It’s an account of the decisions we’ve made, why we’ve made them, and how those decisions can be inspected, challenged, and improved.

Someone has to answer

I have been writing about a gap in generative systems: a person can keep control of an AI-shaped decision and still fail to author it. The missing act is a judgment over the frame the system is using — what it is for, what it treats as good, and what it refuses. If that frame is nobody’s, there is no one to ask. The papers are on lukefwalton.com:

The Decision No One Authored — control is not the same as authorship. The gap is a missing judgment over the evaluative frame.
Building Answerable AI — hold the junctures where a frame enters force, and automate what only executes between them.
The Captured Oracle — a synthesis can have an author. It is the party who set the retrieval policy, the ranking, and the refusal floor, and who comes forward for that frame.
Answer Engine — a small implementation of the same pattern: evidence rules live in schema and tests, outside the model.

Those are preprints and a reference implementation, not a finished theory of genealogy. They are the reason this archive is shaped the way it is. A family tree built by asking a model to “fill in the Waltons” would be an unauthored composite. A family tree built by a written standard, with every claim attributed and every override on the record, can be asked why.

The method grew out of mistakes

The first version was much more rigid than what you see now. In some places it was too permissive. In others, it was so conservative that we could barely build a family tree at all.

There were straightforward technical problems. Wikidata records weren’t being resolved into people correctly because the system wasn’t creating the necessary join keys and defensible fuzzy matches. We fixed that, while retaining the distinction between a possible identity and an established one.

There were visual problems. Siblings were being arranged in ways that made them look like spouses. People connected by a speculative parentage could appear to belong to an established family. We changed the layout so that the relationships were legible without making them appear more certain than they were.

And there were problems of judgment. The early system demanded too much evidence before it would permit a relationship to be represented at all. That sounds rigorous, but genealogy doesn’t work entirely through definitive receipts. Sometimes you have a respected genealogist who reports a relationship but whose underlying document you cannot locate. Sometimes you have several records pointing in the same direction without any one of them settling the question.

I don’t want the AI to invent the missing evidence. But I also don’t want it to pretend that nothing can be said.

That’s where the scalars, hypotheses, and explicit human overrides became important. Rather than forcing a binary choice between related and unrelated, we can record the relationship, explain why we think it might exist, and represent how strongly the evidence supports it.

The research and the user experience have grown together. A relationship that can be expressed in the database but not understood by a visitor is not good enough. Neither is a beautiful family tree that hides the uncertainty behind its connections.

The system is still imperfect. That’s precisely why I want the method to be visible.

Five judgments that stay mine

The constructive pattern in the answerability writing above is a short list of junctures. Each one is a place an evaluative frame enters force. Between them, a model may work. At them, someone is named.

Ends. The job is the richest historical graph the sources support: identifiable Waltons, and the people needed to understand them, every connection tied to a source. The graph is not a single clean pedigree, and it is not a dashboard of everyone who has ever been called Walton.
Standards. A specification states what may anchor a person, how a scalar works, which collections stay out, and how a private fact is held. A model does not get to relax that because a page was interesting.
Verification. Imports, the public boundary, and representative rich profiles are checked by tests. A biography that a model summarized without a source fails those checks. So does a public file that leaks a held passage.
Acceptance. An ordinary sourced claim can be approved under a written policy, and the assessment is attributed to the actual model that made it. My explicit overrides win, including when the same record is imported again. Four people on my own line, and three parent relationships among them, are mine at full strength. So is Bill Walton as the father of Luke Theodore Walton, the basketball player. That Luke is not me. Nothing else becomes Established because a catalog or a famous name said so.
Release. What you see is rebuilt from committed import files and an append-only decision journal. A correction that exists only in a local database disappears on the next build. Publication is a replay, not a mood.

What you are looking at

Time runs downward. A family column is kinship the sources support at Probable or better. Weaker kinship still seats a parent above a child and a spouse beside them. The line is dashed or dotted, and a ground sits behind those cards, so the uncertainty stays visible. Identity and association — “these two records may be the same person,” “these two people were associated” — sit in a side column. They are not kinship.

Everyone else is in a pile I call StubCity. A public person stays there until the public graph connects them to another public person. A birth, a death, a place, an occupation, or a list of books does not extract them. One parent, spouse, sibling, identity claim, or person-to-person association does, at any strength, including a thin one. Leaving the pile does not make the edge stronger, and it does not merge two people who share a name. Drop the last connecting edge and the next build puts them back.

That is triage. Green is a family the sources support. Yellow is a connection still open. StubCity is the work left. People who share a county are not, by that fact, one family.

Every relationship you can click carries a scalar from 0 to 1, or no scalar at all. 1 is the strongest affirmative reading. 0 is the strongest negative. A null scalar means the claim was not assessed. It is not a rejection. 0.85 is the line for Established. It is a cutoff, not a ceiling. A birth or death that a Library of Congress heading and another retrieved identity record both state is filed at 0.9. A date from one unchecked record stays at 0.55. A published sourced genealogy, such as Kerns 2005, is the same kind of scaffold until a cited primary is opened. The compiler’s reputation does not promote the claim. An anonymous tree cannot anchor a person.

Two people who share a name stay two people until an identity claim supports a merge. A card on the atlas is not that merge. A path of weak associations is not a cousinship. The scalar, the source, and the split between kinship, identity, and association are what make a path readable.

What a hypothesis is allowed to say

My own family is a good example of why I don’t want to eliminate human judgment.

I know who my father is. I know who his father was, and who his father was. I assign those personally confirmed relationships a scalar of 1. That is my judgment, and I’m the person who has to stand behind it.

Then there’s Minjum Walton.

Trying to find Minjum — sometimes written Minjim — is what got me into this mess in the first place. I have spent a great deal of time trying to establish whether he was the father of John T. Walton, the ancestor at the uncertain end of my documented line.

There are reasons to suspect that relationship. There are also missing records, uncertain identities, and assumptions that must be true for the proposed connection to work.

I currently assign that parentage a scalar of 0.40, or Possible. It appears in the graph because I think it is a legitimate research hypothesis. But the graph must not imply that I have established it.

That distinction is fundamental.

A model can read the records, identify contradictions, compare dates and places, find possible connections, and challenge my interpretation. It can contribute enormously to the investigation. But the decision to present a particular historical hypothesis, at a particular strength, is ultimately an editorial judgment.

I’m willing to make bolder judgments than an automated system should ordinarily make on its own, provided I make them legible and stand behind them.

That doesn’t mean anything goes. A genealogist’s published assertion is still a compiled assertion until its underlying evidence is examined. A possible identity is not a merge. A path through several weak associations doesn’t make two people cousins.

The worst failure state is to perpetuate a lie: to declare that someone is definitely related to someone else without evidence that warrants the claim.

But the other failure state is friction and inaction. A system so reluctant to entertain hypotheses that it produces thousands of isolated people and no useful historical understanding.

The point of the scalar isn’t to turn historical truth into a mathematically precise probability. It’s to make the strength of an assessment explicit, revisable, and inspectable.

I want a visitor to be able to ask not only “How are these people connected?” but “Why do you think they’re connected?” And I want the answer to be available right there on the claim.

How a source gets in

An anchor decides that there is a person to collect: a deed, a retrieved Wikidata, ORCID, GND, ISNI, or Library of Congress record with a usable name, or an inspected page of a published sourced genealogy. Then the record is read. Birth, death, places, occupation, authorship, affiliation, and kinship are statements. Each one is imported as a claim. The record is named in the argument. The scalar is set. The provenance stays.

A source can be wrong without being useless. For Wikidata, what the item asserts is kept distinct from what its references support. Qualifiers, references, and the retrieval date stay on the claim when the record has them. A thin or conflicting statement is still imported, in the band the evidence supports. A source that offers only a name and an identifier is still a useful stub. A fuller biography does not, by itself, move the person out of StubCity.

On a slave schedule or an estate list, a line that states a name, age, sex, color term, or value is a person. The label uses only those words and the source location. They are not given the holder’s surname. A bare count stays one enumeration.

Some files stay out on purpose. IPUMS census extracts prohibit genealogical use and prohibit linking a row to its enumeration image. Census Bureau surname counts are national totals, not people. The National Archives catalog can find a federal record; the description is not the record.

Four research fronts

This is a genealogy of the Walton surname more than a genealogy of one bloodline.

Some Waltons share ancestors. Others acquired the name through marriage, adoption, or choice. Others belong to entirely unrelated families. The name itself has a history, and that history is larger than any one family’s inheritance chart.

I started with my own Virginia-to-Alabama-to-California line because that’s the history I knew something about. I’m also an American, and the United States is where I’ve put most of my research energy. But I want this archive to extend wherever the name has been used.

For the initial research, I’ve chosen four fronts that expose the system to quite different kinds of evidence.

Early Waltons. Go as far back as the records permit. Medieval England gives us land grants, charters, court rolls, and some of the earliest identifiable people using forms of the name. These records establish an early frame, not a single ancestral family from which every modern Walton must descend.

Immigrant Waltons. Look for the people who brought or established the name in colonial America. Passenger lists, headrights, wills, deeds, parish registers, and early households help us understand how the surname entered and spread through different American communities. These are potential connections between early European history and later American families, but only where the evidence supports them.

Black Walton histories. The United States cannot be understood without its history of slavery, emancipation, and the people whose families were affected by both. This includes people who were enslaved, free people of color, freed people establishing their lives after emancipation, and the generations that followed. The surname may have been inherited, adopted, chosen, or acquired in circumstances we cannot yet establish. These histories are worth recording in their own right, not merely where they intersect with white Walton families. An enslaver’s surname is never automatically assigned to someone the records do not name.

Modern Waltons. Start with people who already have independently discoverable, machine-readable identities. Wikidata, ORCID, library authorities, institutional biographies, obituaries, and other public records give us people we can reasonably distinguish and research backward from.

That last front has an obvious bias. A scholar, an athlete, a politician, or a billionaire is often much easier to identify than someone who lived an equally interesting but less publicly documented life. That bias is inherent in much of the historical record, and it’s not particularly fair.

But it is a reasonable early research strategy. These people are legible. They have identifiers, biographies, institutions, dates, and sometimes named relatives. From those anchors, we can begin reconstructing households and families that extend beyond the famous person.

The aim isn’t to privilege those lives permanently. It’s to use accessible records to build a substantial, varied population graph and then investigate the gaps.

We know from the United States Census that tens of thousands of people bear the Walton surname. We don’t yet know how their many families connect, or how many are entirely unrelated. I want enough documented people and sufficiently rich connections that we can begin asking meaningful questions across that population, without pretending our early sample is statistically representative.

These four fronts aren’t four different family trees. They are four ways of entering the same historical graph.

Some will eventually meet. Some won’t. Both outcomes tell us something.

How the repository is cut

The repository is the frame made inspectable. I expect to open-source it once a few remaining kinks are worked out, so someone else can run the same kind of archive on another surname or another question. This page is the public account of that shape. The specification in the repository is the full text.

Specification: docs/Walton-Historical-Graph-Specification-v1.md is the standard: evidence, publication, privacy, identity, and the public data contract. When an instruction conflicts with it, the specification wins.
Operating notes: AGENTS.md tells a model how to work inside that standard: what to import when a page is open, when a path is exhausted, and which assessments are already locked. NEXT_STEPS.md is the current queue. docs/research-todos.md is a lead list, not the queue.
Imports: data/imports/ holds the committed batches. A batch is evidence, not a vibe. Reimporting it replays the same claims. It does not get to lower a locked override.
Decision journal: data/decisions/journal.jsonl is append-only. A hold, a withdrawal, an override, or a retraction made outside an import file is appended there and replayed on every build. Each entry names the decision, the entity, the authority (agent_policy or luke_explicit), the actor, the policy version, and a short public-safe reason. Sensitive detail stays in a private ledger and is referenced, not copied, into the journal.
Authoring database: data/authoring.sqlite is local. It is rebuilt from the imports and the journal. It is not committed, because a fact that lives only there is not a fact the next build will know.
Public snapshot: The site reads generated files and nothing else. The pipeline is import, replay the journal, validate, snapshot, then lay out the atlas. The browser never sees the authoring database. A withheld representation or record removes its passage, its links, and its argument text from every artifact. The hold is the smallest unit: it does not unpublish a person who is merely anchored to that record.

The atlas is a view. Coordinates are not evidence. Nothing on a person stores “this one is in StubCity” or “this one is family.” The layout recomputes both from the current public edges, so a retraction shows up as a move on the next build without a second editorial opinion.

What a model is for, here

A model is good at reading a long record and tedious about filing every statement on it. That is the work I want from it. Birth, death, place, occupation, authorship, affiliation, and kinship, each as a claim, each named to the record. Stopping after the identifiers, or stopping after a handful of easy people while the same source still yields a useful statement, is an unfinished pass.

A model is bad at the judgments above. It will merge namesakes because the string matches. It will treat a well-known biography as settled. It will smooth a conflict into a single year. The schema makes those moves expensive: two person rows until an identity claim says otherwise, a band on the edge, a scalar that is not allowed to rise because the person is famous, and tests that fail if the public files invent a field the contract does not have.

The model’s name stays on the assessment. When you open a relationship, the page says who assessed it and when. An agent assessment and one of mine are not the same kind of sentence, even when the scalar is high.

Built to be corrected

One idea I keep returning to in my work on answerability is that a responsible automated system has to expect mistakes.

This is familiar thinking in manufacturing and the Toyota Production System. Regardless of how much of a process you automate, something will go wrong. The question is whether the system allows that problem to be seen, understood, corrected, and prevented from recurring.

Genealogical research is no different. Neither is software engineering. And neither is AI.

I will make mistakes. The models will make mistakes. Historical records contain mistakes. Respected genealogists have made mistakes that later researchers repeated for generations.

The question is what happens when we find one.

I want a correction to become part of the history of the research, not erased. I want the original source, the assessment, the person responsible, and the subsequent change to remain inspectable wherever publication and privacy permit. The decision journal and the relationship page are where that inspection lives.

That includes my own judgments. Giving the human final authority doesn’t make the human infallible. It makes someone answerable.

The same goes for the specification. Publishing the methodology is an invitation to examine it, disagree with it, and improve it. I expect my future self to find mistakes in it. I expect other genealogists to find mistakes in it. And I expect future automated systems to become better at finding mistakes than the ones helping me build it today.

That is also why I intend to open the repository, on the timing already stated above: so other people can inspect the machinery and perhaps use it to investigate other surnames or historical questions.

Doing good research means being open about your evidence, your assumptions, your limitations, and your biases. It doesn’t require making private or sensitive information public. It requires making the judgments behind what you do publish answerable.

There is a balance to strike here.

One extreme is reckless automation: millions of apparent facts, invented relationships, and confident conclusions that no one can defend.

The other is a process so bottlenecked by caution and approval that nothing ambitious ever gets done.

I’m looking for something closer to the Aristotelian mean between those failures. Use the machines for what they can do. Allow the human genealogist to make judgments, including uncertain ones. Require those judgments to be legible. Make corrections possible without dismantling the whole archive.

The goal isn’t perfection. We’ll never get there.

The goal is to get better.

And the only way I know to do that is to be open, honest, answerable, and iterative.

What this page is not

It is not a claim that the graph is finished, or that a model’s reading of a deed is as good as the deed. It is not a family tree of every Walton. Living people without a public record of their own stay off the snapshot. Case-file prose from the earlier Minjum investigation stays in the authoring database; the public pages get the citation, the people, and the enumerations.

The short version of why I started is on About. The longer personal account, including Find Minjim, stays on lukefwalton.com. This page is the method those stories sit on.