Compliance first: We never take HIPAA/PHI data or personal customer data. Everything is de-identified before it leaves your control.
Home / Guidelines / Open data policy guideline 22
Open data policy guideline 22

Real-Time Updates: Open Data Policy Guideline 22

Real-Time Updates: Open Data Policy Guideline 22

Guideline 22 asks open data policies to require ongoing publication and regular updates, as close to collection time as practical, because a dataset released once begins going stale the next day.

A one-time data dump makes a good press release. It does not make a useful public resource. This guideline is about turning publication into a routine.

What the guideline means

The ideal is publication close to the moment data is collected. Guideline 22 recognizes that releasing a dataset once is not enough: the moment new records are created and not published, the public copy becomes incomplete. Policies should therefore require ongoing publication and specify how frequently datasets are refreshed.

Real-time publication is demanding, but the guideline notes that it becomes much easier when combined with structured collection (Guideline 21), a central portal (Guideline 18) and APIs (Guideline 20). Automation, rather than staff effort, is what makes frequent updates sustainable.

Model policy language patterns

Update provisions commonly take these forms (paraphrased):

  • Published datasets shall be updated on a regular schedule appropriate to how often the underlying data changes.
  • Each department shall set and publish an update frequency for every dataset it releases.
  • Datasets shall be kept current and, where feasible, refreshed automatically from source systems.
  • The public shall be notified when datasets are updated.

Where it appears in adopted policies

Ongoing-update language is common among municipal open data policies. Examples in this index include:

Portland, OR (2009) addressed ongoing publication through agreements with regional partners to publish and maintain datasets.

Common pitfalls

  • Launch-day spike, then silence. Many portals show a burst of uploads around adoption and few refreshes afterward.
  • No stated frequency. Without a published schedule, users cannot tell a current dataset from an abandoned one.
  • Manual pipelines. Updates that depend on one employee exporting a spreadsheet stop when that person changes jobs.
  • Silent schema changes. Frequent updates are only useful if columns and definitions stay stable or changes are documented.

A practical checklist for update provisions

  • Require a stated update frequency for every published dataset, recorded in its metadata.
  • Show the date of last update prominently.
  • Automate refreshes from source systems wherever possible.
  • Notify users of updates through feeds, change logs or portal alerts.
  • Document schema changes before they take effect.
  • Track and report datasets that miss their stated schedule, so stale data is visible to oversight bodies and the public.

Why this matters for your business data

Freshness is one of the clearest drivers of data value. For AI and machine-learning buyers, a historical extract is useful, but a historical extract plus a reliable recurring refresh is worth considerably more, because models trained on current conditions outperform those trained on stale ones.

Businesses already produce data continuously through their operating systems. The question is whether a consistent, de-identified extract can be produced on a schedule without heroic effort. Get a bid to see how buyers price one-time versus recurring delivery, or use the valuation calculator for a first estimate.

We never take HIPAA/PHI data or personal customer data. Everything is de-identified before it leaves your control.

Our compliance promise: We never take HIPAA/PHI data or personal customer data. Everything is de-identified before it leaves your control.

Frequently asked questions

Does real-time mean every dataset must update continuously?

No. The goal is publication as close as practical to collection, matched to how often the data changes. Transit positions may update by the minute, permits daily, budgets annually. The key is a stated, reliable frequency rather than one-time release.

How can a small government keep datasets current?

Automation is the main answer. Connecting the portal directly to source systems, collecting data electronically, and scheduling exports remove the need for manual uploads. Starting with a handful of automated, high-demand datasets is more sustainable than many manual ones.

Should policies require update schedules to be published?

It is good practice. A published frequency tells users whether a dataset is current and gives oversight bodies a simple metric for compliance. Many policies place the update frequency in each dataset's metadata.

Policies in the index that reflect guideline 22

PlaceAdoptedInstrumentGuidelines
Washington, DCJun 12, 2006Executive memorandum1131822
Portland, ORSep 30, 2009Council resolution37810162230
New York, NYMar 7, 2012Local law (Local Law 11 of 2012)14578101118
Philadelphia, PAApr 26, 2012Executive order (No. 1-12)34578182224
Madison, WIAug 15, 2012Ordinance457810111318
Chicago, ILDec 10, 2012Executive order34678101113
Montgomery County, MDDec 17, 2012County legislation (Bill 23-12, Open Data Act)457811131822
Austin, TXAug 26, 2013Administrative memo123457810
Honolulu, HINov 27, 2013Ordinance237810111820
Los Angeles, CADec 18, 2013Executive directive (No. 3)37818222630
Hartford, CTMar 26, 2014Executive order34789182224
Nashville, TNMay 12, 2014Executive order (No. 043)2345781011
Cincinnati, OHMay 29, 2014Administrative regulation8101122242629
Washington, DCJul 21, 2014Executive directive (Mayor's Order 2014-170)12356789
Minneapolis, MNAug 1, 2014Council-adopted policy34568131819
Houston, TXSep 18, 2014Administrative policy (AP 8-7)234567810
Denton, TXFeb 9, 2015Administrative directive357811121618
Suffolk County, NYJul 31, 2015Administrative policy78131820222426
Cambridge, MASep 21, 2015Ordinance12345678
Asheville, NCOct 13, 2015Council resolution13489101118
Tulsa, OKDec 21, 2015Executive order123458910
San Jose, CAApr 5, 2016Council policy1681113182230
Las Vegas, NVApr 12, 2016Administrative policy (City Manager policy CM510)12345678
Saint Paul, MNApr 27, 2016Council resolution12578101113
Naperville, ILNov 15, 2016Council resolution45789101112
Fort Collins, COFeb 7, 2017Council resolution12345678

Sources

  1. Sunlight Foundation Open Data Policy Guidelines (historical reference)
  2. Sunlight Foundation open data policy collection (public repository)

Updated December 31, 2024.