Compliance first: We never take HIPAA/PHI data or personal customer data. Everything is de-identified before it leaves your control.
Home / Guidelines / Open data policy guideline 17
Open data policy guideline 17

Archival Material: Open Data Policy Guideline 17

Archival Material: Open Data Policy Guideline 17

Guideline 17 asks an open data policy to reach backward as well as forward: paper records, old budgets, minutes, maps and photographs should be digitized and published on a prioritized, realistic schedule.

Most open data policies are written about the data a government produces today. This guideline is a reminder that much of a community's institutional memory still sits in filing cabinets, microfiche and scanned images that no one can search.

What the guideline means

An open data policy that only covers born-digital records leaves a gap: decades of council minutes, capital budgets, zoning maps, inspection logs and historical photographs remain effectively invisible. Guideline 17 asks the policy to name archival material as in scope and to commit to converting it into digital, reusable form.

The guideline is not a demand to scan everything at once. It treats digitization as a pipeline that should be ordered by the same prioritization logic used for current data (see Guideline 5), shaped by public input and by the departments that hold the records.

Model policy language patterns

Policies that address archives tend to use a few recurring constructions, which can be paraphrased like this:

  • A commitment to "provide or support access" to historical copies of data the city has already released, so that each update does not overwrite the past.
  • A direction that the inventory of holdings (see Guideline 4) include non-digital records, flagged for later conversion.
  • A requirement that digitized records be published as structured data where possible, not just as page images.
  • A link to records-retention schedules, so archival publication builds on existing law rather than competing with it (Guideline 2).

Where it appears in adopted policies

Archival language is one of the less common provisions in municipal open data policies. In this index it appears in:

  • Milwaukee, WI (2016), whose policy commits the city to provide or support free access to historical archives of data it has released.
  • Naperville, IL (2016), which lists archival practices among the commitments of its open data program.
  • Tulsa, OK (2015), whose executive order addresses historical data in its definitions and scope.

Later policies in Modesto and Salinas, California, used similar language.

Common pitfalls

  • Scanning without structure. A PDF image of a 1987 budget is better than nothing, but it is not data. Budget tables, permit logs and ledgers should be transcribed into rows and columns.
  • No prioritization. Without criteria, digitization follows whichever office has spare staff time rather than public value.
  • Losing history on update. Portals that replace a dataset in place quietly destroy the time series that researchers need. Versioning is part of archival access (see Guideline 23).

A practical checklist for archival provisions

  • Name archival and non-digital records explicitly in the scope section.
  • Require the inventory to flag records that exist only on paper, film or legacy systems.
  • Rank digitization candidates using published criteria, such as records-request volume and research value.
  • Prefer structured transcription for tabular records like budgets, permits and ledgers; use scans plus indexes for maps, photographs and correspondence.
  • Publish digitized material with the same metadata, license and permanence commitments as current data.
  • Report progress in the annual open data report so the public can see how much history has been opened.

Why this matters for your business data

Archives are where the long time series live, and long time series are what model builders pay for. A business that has kept years of work orders, invoices, scheduling logs or sensor readings holds something similar to a city's archive: a record of how operations actually behaved across seasons, price changes and market cycles.

The same lesson applies as in government. History locked in scanned paperwork or a retired system has little value until it is structured, documented and consistent. If you are weighing what your back catalog is worth, start with how to value my data and the valuation calculator, then read the bridge guide from open data policy to data value.

We never take HIPAA/PHI data or personal customer data. Everything is de-identified before it leaves your control.

Our compliance promise: We never take HIPAA/PHI data or personal customer data. Everything is de-identified before it leaves your control.

Frequently asked questions

Does an open data policy have to cover paper records?

Not necessarily, but policies that ignore paper records leave most of a government's history inaccessible. Guideline 17 recommends at least naming archival material as in scope and setting a prioritized plan for digitizing it, even if the work happens over several years and depends on available funding and staff.

Is a scanned PDF enough to satisfy this guideline?

A scan improves access but falls short of open data. The intent is machine-readable records that can be searched, sorted and analyzed. Where full transcription is impractical, policies often start with scans plus structured indexes, then convert high-demand series such as budgets or permits into tabular data.

How should a city decide which archives to digitize first?

Use the same criteria applied to current data: public demand, accountability value, cost, and relevance to active policy questions. Records requests are a strong signal. Public input helps, but it should be balanced with input from departments that understand which records are complete and reliable.

Policies in the index that reflect guideline 17

PlaceAdoptedInstrumentGuidelines
Tulsa, OKDec 21, 2015Executive order123458910
Milwaukee, WIJul 14, 2016Council resolution12356789
Naperville, ILNov 15, 2016Council resolution45789101112
Syracuse, NYJul 26, 2017Executive order1456781113

Sources

  1. Sunlight Foundation Open Data Policy Guidelines (historical reference)
  2. Sunlight Foundation open data policy collection (public repository)

Updated December 31, 2024.