# Crime and heat: upstream source freshness audit

Checked 25 September 2026, 21:45–21:46 UTC; San Diego Police aggregate replay checked 21:52 UTC. This is a source-version audit. It does not refit models, alter frozen study inputs, extend the arrest series, or revise statistical conclusions.

## Findings

| Source | Freshness result | Consequence |
|---|---|---|
| California Department of Justice (DOJ) | All five frozen CSV, contextual PDF, and jurisdiction-workbook files are byte-identical. Live catalog relationships still list the July 2026 release covering data through 2025. | Existing release and reporting-gap qualifications remain applicable. |
| San Diego Association of Governments (SANDAG) | All 14 aggregate/dictionary responses are byte-identical. Four raw metadata responses have different hashes, but every retained scientific metadata field, column definition, and update timestamp is unchanged. | No detected change to the acquired SANDAG counts or schema. |
| San Diego Police Department (SDPD) | The 2020–2025 annual CSVs and dictionary are byte-identical. The partial 2026 CSV increased from 56,793 to 57,037 source records, a net addition of 244. | Refreshable contextual aggregate data; not a replacement for the Sheriff arrest outcome. |
| National Oceanic and Atmospheric Administration (NOAA) | Of 92 files, 76 are byte-identical and 16 daily station files differ. All 29 daily stations retain identical 1991–2025 and 2021–2024 records. All 60 hourly archive files are byte-identical, including decompressed records. All three station inventory/history files are byte-identical. | No detected change to the checked historical station exposure records. |
| U.S. Census Bureau | The fixed 2020 incorporated-place population table is byte-identical. | Contextual population values remain tied to 2020; this check does not convert them into newer estimates. |
| San Diego County Sheriff | Direct requests for the activity-report listing and agency information page returned HTTP 403. Search-indexed official listing text identifies August, May, April, March, February, and January 2026 reports. | The previous three-PDF/six-point benchmark is not an exhaustive current monthly series. Latest August report contents and bytes remain unverified. |
| Open-Meteo | Current historical-weather documentation retrieved. The documentation continues to distinguish model products and recommends explicit ERA5 or ERA5-Land selection for comparisons across decades. | Documentation verification only. This audit did not redownload the study's complete reanalysis responses, resolve quota-limited locations, or certify unchanged reanalysis values. |

The request log contains 133 HTTP requests: 131 successful responses, two access failures. Among 124 responses with an expected original hash, 103 match and 21 differ. Seven successful documentation/catalog responses have no prior byte comparison. These counts describe HTTP resources, not independent datasets or complete agency coverage. Source servers can return cached responses; retrieval time is retained separately from server timestamps.

## SDPD revision and safe refreshed data

The [official SDPD portal](https://data.sandiego.gov/datasets/police-nibrs/) links the [2026 source CSV](https://seshat.datasd.org/police_nibrs/pd_nibrs_2026_datasd.csv). Its retrieved response contains 57,037 rows covering occurrence dates 1 January–24 September 2026, with no missing occurrence dates and 57,037 distinct offense identifiers. The response SHA-256 is `465589d9a9f809924014601759a6b9a29c413fcc76978358554c93db93898230`; size 22,046,787 bytes; HTTP Last-Modified 25 September 2026, 14:07:44 GMT. A second retrieval for aggregate production matched that same hash.

There are 27 revised daily totals. The new 24 September date contributes 95 records; revisions to 26 earlier dates contribute a net 149, including 90 additional records dated 23 September. The earliest changed date is 1 January. Counts for 4 and 13 September each decreased by one. The revision therefore includes delayed additions and corrections, not merely a new day's append.

`sdpd_nibrs_2026_daily.csv` and its JSON equivalent preserve the prior aggregate schema: date, reporting group, crime-against classification, source-row count, distinct offense count, and distinct case count within that cell. `sdpd_nibrs_2026_monthly_offense.csv` and JSON preserve the prior month/group/offense-category counts. There are 1,335 daily cells and 416 monthly offense cells; each aggregation reconciles to 57,037 source rows. Distinct case counts are cell-specific and must not be summed as though cases could occur in only one cell.

No record-level CSV, case identifier, address, coordinate, or person-level detail is retained in these outputs. Raw CSV bytes were processed in memory. SDPD offense records are not arrests, victims, or domestic-violence cases. This source covers a different agency and carries no explicit domestic-violence flag in the documented schema. Partial 2026 remains subject to reporting delays and revisions.

## Source-specific evidence

- **DOJ:** The [arrest file relationship](https://data-openjustice.doj.ca.gov/jsonapi/node/dataset/041eca85-2c60-4f38-a995-805231a75e3c/field_source) lists `OnlineArrestData1980-2025.csv` and its context PDF. The [domestic-violence file relationship](https://data-openjustice.doj.ca.gov/jsonapi/node/dataset/25b28e8b-42d2-42c2-ab28-ebf15669910f/field_source) lists `DVRCA_2001-2025.csv` and its context PDF. Both report July 2026 creation dates. The [context PDF](https://data-openjustice.doj.ca.gov/sites/default/files/dataset/2026-07/Domestic%20Violence%20Related%20Calls%20for%20Service%20Context_06222026.pdf) remains byte-identical to the previously reviewed qualification concerning Sheriff reporting gaps. Annual calls for assistance are a different outcome from arrest records.
- **SANDAG:** Retained metadata from [Group A](https://opendata.sandag.org/api/views/pr74-d3tr.json), [Group B](https://opendata.sandag.org/api/views/huzf-mi2z.json), the [offense dictionary](https://opendata.sandag.org/api/views/knre-fqwi.json), and [historical domestic violence](https://opendata.sandag.org/api/views/qbrv-e75t.json) matches the prior normalized metadata exactly. The raw metadata hash differences alone do not imply data revisions. Aggregate requests remain at 791,760 Group A rows and 112,170 Group B rows. Latest recorded timestamps in the agency-coverage responses are 23 September 2026, 00:42 and 00:55 respectively. The historical annual domestic-violence series still has 37 observations through 2022. Exact aggregate query URLs and hashes appear in `freshness_results.json`.
- **NOAA:** [Daily station documentation](https://www.ncei.noaa.gov/pub/data/ghcn/daily/readme.txt) describes the fixed-width daily format used for record comparisons. The [Integrated Surface Database](https://www.ncei.noaa.gov/products/land-based-station/integrated-surface-database) is the hourly archive source. All downloaded response hashes are compared against `expansion_sources/weather/raw_manifest.json`. Daily historical comparisons retain entire source records, including source and quality flags. The hourly primary-window comparison uses UTC instants from 1 January 2021, 08:00 inclusive through 1 January 2025, 08:00 exclusive, the America/Los_Angeles boundaries for the study years. The audit does not interpret changed 2026 daily records as changed historical study exposure.
- **Census:** The retrieved [2020 California incorporated-place table](https://tigerweb.geo.census.gov/tigerwebmain/Files/acs26/tigerweb_acs26_incplace_2020_tab20_ca.html) is the same source used in the frozen geographic context. The `acs26` URL path does not make the table's 2020 population counts into 2026 estimates.
- **Sheriff:** The [official monthly-activity listing](https://www.sdsheriff.gov/resources/open-data/law-enforcement-monthly-activity) was discoverable through its search-indexed text, despite direct access failure. An [April 2026 report](https://www.sdsheriff.gov/home/showpublisheddocument/10213) was readable through indexed PDF text, but no new Sheriff values were incorporated into the released benchmark. No current Sheriff PDF hash or complete latest-month series is claimed.
- **Open-Meteo:** The [historical-weather documentation](https://open-meteo.com/en/docs/historical-weather-api) distinguishes hourly variables, model selection, and daily output. It does not, by itself, validate the frozen request parameters or the study's daylight-saving-time transformation. Those remain matters for the separate exposure-construction audit.

## Reproduction and publication boundary

Run `audit_freshness.py` with the adjacent frozen `expansion_sources` directory available to repeat the upstream comparison. Because upstream responses are mutable, a later run is a new audit, not a guaranteed replay of this date. `refresh_sdpd_2026.py` refuses to generate refreshed aggregates unless the source response matches the audited hash. It retains only aggregate outputs.

`verify_freshness.py` verifies the saved audit's reconciliations and the frozen NOAA/DOJ inputs without network requests. `PUBLIC_FILES.json` is an explicit publication allowlist, with byte sizes and SHA-256 hashes. Do not recursively publish this directory: excluded working-response HTML includes a portal's public sample records, and full documentation snapshots are unnecessary for the public audit. Source bodies transformed into sanitized metadata have separate file hashes in the allowlist; their request-log SHA-256 values describe original HTTP response bytes.