Governance
Data lineage
Definition
Data lineage is the recorded path from a source system to a decision — what data was used, where it came from, when, and under what permission.
Lineage is unglamorous and decisive. It determines whether you can answer an examiner, honour a deletion request, explain a decision, or safely retire a source without breaking something you did not know depended on it. AI raises the stakes because retrieval systems pull from many places at once and blur provenance unless it is deliberately preserved, in a way that a single-source manual process rarely had to worry about.
Permission travels with the data
A document that one team may read does not become universally readable because it was indexed into a shared retrieval system; the original access restriction still applies and has to be enforced at the point of use. Enforcing entitlements at retrieval time is the only durable approach, because relying on the interface layer to hide restricted results is fragile and tends to break the first time a new access path to the same index is built without the same restriction logic applied to it.
Lineage makes deprecation possible
Without it, no one can say which decisions depended on a bad source, so nothing is ever confidently corrected or switched off, because turning off a source of unknown importance feels riskier than leaving a known-flawed one running. With lineage, a team that discovers a document was outdated or wrong can immediately identify every decision that used it, which turns a vague worry into a specific, resolvable remediation task with a bounded scope.
Lineage supports both audits and deletion requests
A regulator's examination request and a customer's data deletion request are, structurally, the same query: what data was used where, and by what process. A system with real lineage answers both quickly; a system without it treats each as a bespoke, manual investigation that consumes days of analyst time and produces an answer nobody is fully confident in, which is a poor position to be in under the deadline pressure either kind of request typically carries.
In practice
- Record source, version, and timestamp for every retrieved passage
- Enforce entitlements in the retrieval layer, not the interface
- Be able to list every decision that used a given source
- Treat audit requests and deletion requests as the same underlying lineage query
