A document-assistance pilot needs a narrower purpose than making government work faster. Start with an administrative task whose output staff can inspect, and explicitly exclude decisions affecting eligibility, enforcement, benefits, priority of service or public rights.
What the pilot may—and may not—do
Specify the document class, task, sample size, duration and accountable official. A possible task is suggesting internal folder labels for already-public meeting packets. Exclude sensitive submissions and any ranking that could change access to a service. State that suggestions cannot trigger external notices or decisions.
Human review can miss errors, and a small sample cannot establish fairness or reliability across all document types. Records obligations, procurement rules and permitted data uses differ by agency. Expansion requires a new scope decision, not an assumption that a successful labeling test authorizes public decision automation.
Authorize data handling before testing
Ask records, privacy, security and accessibility staff what may enter the system, who may see outputs and how inputs, prompts, revisions and logs must be retained or disposed of. Review vendor data use and subprocessors. Use approved test material until authorization is documented; public availability alone does not settle every reuse question.
Keep routing decisions inspectable
Display the source alongside the suggestion. Give reviewers an accept, correct or reject path and preserve their choice. Send uncertain cases to an exception queue with a named owner. Do not let an unreviewed label hide a submission or move it behind a service deadline.
Hypothetical example: A fictional clerk office tests 30 public meeting packets for folder suggestions. Staff find three packets mislabeled and one with an unreadable attachment. They retain all four in the manual queue and log the source problem separately from the model error. No citizen request is prioritized, approved or denied by the tool. These figures illustrate a test design, not observed pilot results.
Expansion requires evidence, not enthusiasm
Before starting, define acceptable and unacceptable errors. Count mislabels, omitted pages, sensitive-data exposure and reviewer time, including corrections. Pause on unauthorized disclosure or any output used for an excluded decision. Compare with a manually processed sample before considering a broader trial.
- Named accountable official and written exclusions.
- Approved data, access and retention plan.
- Manual route remains available.
- Errors and corrections logged against source documents.
- Pause criteria and expansion decision documented.