How We Designed Around Responsible AI in Genealogy

A couple of posts back I wrote about testing AI on census handwriting — where it broke, what changed, and what’s honestly true about it today. That post was about one question: can it read the document correctly? This one is about the question underneath it, which matters more and gets discussed less: when a tool uses AI on your family’s records, what happens to your data, and how do you know which parts of your tree came from a machine?

Those aren’t paranoid questions. Genealogy is one of the few fields where a mistake propagates — a wrong parent gets copied into a dozen other trees and outlives everyone who could have corrected it. Add a confident AI to that, and a plausible guess can become “a fact everybody knows” in about two years.

There’s an actual standard for this now

The Coalition for Responsible AI in Genealogy (craigen.org) formed in 2024 out of an ethics panel at RootsTech. It’s backed by the National Genealogical Society and a group of people whose names you’d recognize from the conference circuit, and it publishes five plain-language principles: accuracy, disclosure, privacy, education, and compliance.

Worth being precise here, because vendors get sloppy about this: CRAIGEN is not a certification body. Nobody is “CRAIGEN-certified,” including us. We’re not a member organization. What the principles are is a good, short description of what responsible looks like — which makes them a useful thing to design against and to be measured against publicly.

So here’s the measuring.

Accuracy: nothing touches your tree without you

The single most important design decision in Gather the Family is that AI output is never applied automatically. Every extraction from a document lands in a review inbox first. You see what the AI read, what it wants to change, and a confidence score — and low-confidence reads get flagged rather than quietly folded in with everything else.

This is a real tradeoff, not a free win. It means more clicks than a tool that just fills your tree in for you. I think that’s the correct trade in this specific domain: a tool that silently populates a family tree is optimizing for a demo, not for research you’ll still trust in ten years.

Disclosure: you should always know what came from a machine

AI-written content is labeled as AI-written, not slipped in as though you wrote it. AI answers in search carry a badge saying exactly that. Document review keeps AI-extracted values visually separate from what you typed yourself, so months later you can still tell which is which.

The narrative generation follows the same rule at the sentence level — it’s instructed to hedge where the underlying data is thin (“around 1847,” “possibly”) rather than smoothing a gap in your research into a confident sentence. An AI-written paragraph that reads like a finished conclusion is worse than no paragraph, because it’s harder to un-believe.

Privacy: your family’s data doesn’t go to ChatGPT

This one is concrete and easy to check. All AI processing runs through AWS Bedrock inside our own infrastructure. Your documents and family data are not sent to a third-party consumer LLM, and they’re not used to train anybody’s model. Living people get additional privacy protections by default.

CRAIGEN’s own guidance warns researchers against pasting sensitive family information into public AI tools, and that’s good advice — but it’s advice most people can’t follow if the software they use is quietly doing it on their behalf. Worth asking your other tools where their inference actually runs.

Photo integrity: the original is untouchable

This is the principle I’d argue is most quietly violated across the industry. AI photo restoration and colorization can fabricate facial features — invent a jawline, guess an eye color, sharpen a face into someone who never existed. In a family archive that isn’t a cosmetic issue. Alter the only photo of your great-grandmother and you’ve corrupted evidence about identity, era, and place, permanently.

So: we never modify your original. Enhanced or restored images are saved as separate, clearly labeled versions in a version history, with the original preserved as version zero — and the system won’t let you delete it. Enhance a photo five different ways and you still have the untouched scan sitting underneath all five.

Education: saying what it can’t do

This is the principle most companies skip, because the honest version is bad marketing. The census-handwriting post is what it looks like in practice — publishing the failure mode (dense grids, not old handwriting), and naming the variable that actually predicts a good result (scan quality, not era).

I’d rather you have a correct mental model of where the tool is weak than an impressive one that fails you on a hard document.

Compliance: provenance on external data

When record data comes in from an external source, source attribution is mandatory and licensing is tracked alongside it. Genealogy runs on other people’s archives, and “where did this come from and what am I allowed to do with it” is a question that should have an answer stored in the database, not one reconstructed from memory later.

How we talk to you

All of the above collapses into a language rule that shows up hundreds of times across the app.

What we don’t say: “Your ancestor was born in Cork in 1847.”

What we say instead: “This document suggests a birthplace of Cork, circa 1847 — worth verifying against the original record.”

Same underlying data. Completely different claim. The first one ends your research; the second one tells you where to point it next. AI suggestions are starting points, and the grammar should say so.

Where we’re not done

Honest gap, since a post about disclosure shouldn’t have a disclosure problem: we don’t yet have in-app education about AI limitations — no “About AI in GTF” page, no tooltips on AI features explaining what they’re weak at. Right now that context lives in posts like this one, which means you only get it if you read the blog. That’s backwards, and it’s on the list.

If you want the full breakdown of each principle and what backs it, it’s on the Responsible AI page — and if a claim there doesn’t match what the app actually does, I want to hear about it.