Methodology
Quality assurance and revisions
PFD Monitor uses AI to analyse a large collection of reports and responses. Quality assurance helps keep that analysis connected to the source evidence and stops incomplete or inconsistent records from being published.
These checks reduce the risk of error. They do not make every AI judgement infallible.
Checks during processing
PFD Monitor checks its work at several points rather than waiting until the end. Depending on the type of record, checks may confirm that:
- the original document and supporting passage are present;
- required information has been extracted;
- names, dates, and links use valid formats;
- a decision uses one of the permitted outcomes;
- connected records refer to the same report, response, or organisation; and
- uncertainty has not been presented as a confirmed finding.
If a check fails, the affected record or release can be stopped rather than quietly published.
Quality check
Passing a structural check means that a record is complete and internally consistent. It does not prove that every judgement about the meaning of a complex passage is correct. That is why PFD Monitor also preserves the source words and accepts corrections.
Testing the AI
Before a method is used across the collection, it is tested on real reports and responses. The results are compared with careful human reading to find common errors and unclear decision boundaries.
Testing may examine whether the AI misses concerns, combines separate actions, creates unsupported descriptions, confuses an action with a respondent's position, or links concerns that belong separately.
Methods can be revised when testing reveals a material weakness. New versions receive another check before publication.
When people review the work
Human review is targeted rather than universal. It may be used for:
- unclear or incomplete source material;
- possible new recurring issues;
- warnings raised by automated checks;
- significant changes to an existing issue;
- sampling and quality evaluation; and
- corrections submitted by readers.
A person does not routinely approve every AI-produced record. The site describes a particular record as human reviewed only when that review has taken place and has been recorded.
Publication checks
PFD Monitor publishes records in fixed releases. Before a release goes live, checks confirm that its evidence, public definitions, method versions, and required approvals fit together.
An incomplete processing run does not automatically become a public release. Publication is a separate decision, and an earlier release can remain live if a new one is not ready.
Technical note
Each published snapshot records the versions of the source collection, processing method, issue list, and public vocabularies used to create it. This makes a release traceable and allows it to be replaced or rolled back without silently mixing versions.
Corrections and revisions
An error can affect the source collection, extracted information, an AI description, a classification, or a link between records. A correction changes only the affected layer and leaves the original evidence intact.
Important changes appear in the methodology change log. Each entry states what changed, why it changed, which records were affected, and whether earlier material was processed again.
Interpret with care
A revised finding does not mean the source document changed. It may mean that PFD Monitor improved or corrected its interpretation of the same source.