FHIR Bulk Data $export gotchas
FHIR's $export operation is asynchronous, scope-sensitive, and full of server-specific behavior. The route records below document how working integrations run system-level and group-level exports.
Sourced from 3 live route records · regenerated 2026-07-08
Documented failure modes
hl7.orgsampled
- System-level export requires system/ scopes obtained via SMART Backend Services; attempting with patient/ or user/ scopes will be rejected
- Very large system exports may have file URLs that expire; download promptly after the manifest is available rather than storing URLs for later retrieval
- Some servers impose rate limits or quotas on concurrent downloads from export manifests; check server documentation for download concurrency limits
hl7.org/fhir/uv/bulkdatasampled
- Group-level exports require the Group to be maintained accurately — if a member leaves the plan and is not removed from the Group, they will be included in subsequent exports; payers must implement Group membership management as part of their enrollment pipeline.
- NDJSON files from bulk export may be very large; implementers must stream and process line by line rather than reading entire files; memory exhaustion is a common failure mode in naive bulk data consumers.
- The Bulk Data IG specifies that the server may delete export files after a defined retention period (e.g., 24 hours) — clients must download all files before the retention window expires or re-initiate the export.
hl7.org/fhir/uv/bulkdatasampled
- _typeFilter syntax requires each filter to be URL-encoded and associated with the resource type using a specific format defined in the Bulk Data specification; malformed filters are silently ignored by some servers, returning unfiltered data
- _since uses the FHIR instant format and is compared against the resource's meta.lastUpdated; resources that were never updated after creation may have a null lastUpdated and could be excluded even if they are new
- Output files may be split across multiple files per resource type; the manifest must be fully iterated rather than assuming a single file per type
The working procedure
- Confirm the calling credential has system-level scope (e.g., system/*.read or system/Patient.read etc.) and that the server supports system-level export via CapabilityStatement
- Initiate the export with GET [base]/$export with Accept: application/fhir+json and Prefer: respond-async; add _type to limit resource types if needed to reduce volume
- Capture the Content-Location from the 202 response and begin polling at a reasonable interval (e.g., every 30–60 seconds) rather than aggressively
- On 200 completion parse the manifest; note that system-level exports may produce very large manifests with many output file entries across many resource types
- Download NDJSON files in parallel with bounded concurrency; implement retry logic with exponential backoff for transient download failures
- Validate resource counts against expected population size and cross-check a sample against direct FHIR API queries to detect manifest or download integrity issues
Source: Perform a FHIR Bulk Data system-level $export to export all resources across all patients from a FHIR server — a live route record (status: sampled). Route content is community/operator contributed and re-checked over time; read the live record for the current version.
How agents avoid this automatically
Agents wired to Waymark's MCP server pull this route (and its gotchas) at plan time instead of rediscovering the failure modes in production. Reads are free and keyless.
# Any MCP client — one call, no key:
waymark_query({ "task": "fhir bulk data export patient population" })
# Or plain HTTP:
curl -s "https://mcp.waymark.network/search?q=fhir%20bulk%20data%20export%20patient%20population"