Analyzing FHIR Data in a Tabular Format With Python
min. reading time
Last updated: May 20, 2025
Roles:Informaticist
Learning objectives
Understand the high-level approaches for converting FHIR-formatted data into tabular for analysis in Python.
Learn how the FHIR-PYrate library facilitates requesting data from a FHIR server, and creating tidy tabular data tables.
Data analysis approaches in Python often use Pandas DataFrames to store tabular data. There are two primary approaches to loading FHIR-formatted data into Pandas DataFrames:
Writing Python code to manually convert FHIR instances in JSON format into DataFrames.
This does not require any special skills beyond data manipulation in Python, but in practice can be laborious (especially with large number of data elements) and prone to bugs.
Using a purpose-built library like FHIR-PYrate to automatically convert FHIR instances into DataFrames.
It is recommended to try this approach first, and only fall back to (1) if needed.
To use FHIR-PYrate, you will need a Python 3 runtime with FHIR-PYrate and Pandas installed.
However, any FHIR server loaded with testing data can be used. See Standing up a FHIR Testing Server for instructions to set up your own test server.
The code blocks in the following section show sample output immediately after. This is similar to the code cells and results in a Jupyter notebook.
2 Retrieving FHIR data
Once your environment is set up, you can run the following Python code to retrieve instances of the Patient resource from a test server:
# Load dependenciesfrom fhir_pyrate import Pirateimport pandas as pd# Instantiate a Pirate object using the FHIR-PYrate library to query a test FHIR serversearch = Pirate( auth=None, base_url="https://hapi.fhir.org/baseR4", print_request_url=True,)# Use the whimsically named `steal_bundles()` method to instantiate a search interaction## For more information, see https://github.com/UMEssen/FHIR-PYrate/#piratebundles = search.steal_bundles( resource_type="Patient", request_params={"_count": 10, # Get 10 instances per page"identifier": "https://github.com/synthetichealth/synthea|", }, num_pages=1, # Get 1 page (so a total of 10 instances))# Execute the search and convert to a Pandas DataFramedf = search.bundles_to_dataframe(bundles)df.head(5)
It is easier to see the contents of this DataFrame by printing out its first row vertically:
# Print the first row of the DataFrame vertically for easier reading.pd.set_option("display.max_rows", 100) # Show all rowsdf.head(1).T
0
resourceType
Patient
id
ed6a17f3-deb2-4e7a-9548-5d15d2d47c05
meta_versionId
1
meta_lastUpdated
2026-09-29T01:27:02.704-04:00
meta_source
#nOZOegRefKiCm14q
meta_tag_0_system
https://smarthealthit.org/tags
meta_tag_0_code
synthea-5-2019
text_status
generated
text_div
<div xmlns="http://www.w3.org/1999/xhtml">Gene...
extension_0_url
http://hl7.org/fhir/us/core/StructureDefinitio...
extension_0_extension_0_url
ombCategory
extension_0_extension_0_valueCoding_system
urn:oid:2.16.840.1.113883.6.238
extension_0_extension_0_valueCoding_code
2054-5
extension_0_extension_0_valueCoding_display
Black or African American
extension_0_extension_1_url
text
extension_0_extension_1_valueString
Black or African American
extension_1_url
http://hl7.org/fhir/us/core/StructureDefinitio...
extension_1_extension_0_url
ombCategory
extension_1_extension_0_valueCoding_system
urn:oid:2.16.840.1.113883.6.238
extension_1_extension_0_valueCoding_code
2186-5
extension_1_extension_0_valueCoding_display
Not Hispanic or Latino
extension_1_extension_1_url
text
extension_1_extension_1_valueString
Not Hispanic or Latino
extension_2_url
http://hl7.org/fhir/StructureDefinition/patien...
extension_2_valueString
Alana Blick
extension_3_url
http://hl7.org/fhir/us/core/StructureDefinitio...
extension_3_valueCode
F
extension_4_url
http://hl7.org/fhir/StructureDefinition/patien...
extension_4_valueAddress_city
Bourne
extension_4_valueAddress_state
Massachusetts
extension_4_valueAddress_country
US
extension_5_url
http://synthetichealth.github.io/synthea/disab...
extension_5_valueDecimal
0.145932
extension_6_url
http://synthetichealth.github.io/synthea/quali...
extension_6_valueDecimal
35.854068
identifier_0_system
https://github.com/synthetichealth/synthea
identifier_0_value
f06c0800-7e56-45e7-85eb-0047c2c5b4b1
identifier_1_type_coding_0_system
http://terminology.hl7.org/CodeSystem/v2-0203
identifier_1_type_coding_0_code
MR
identifier_1_type_coding_0_display
Medical Record Number
identifier_1_type_text
Medical Record Number
identifier_1_system
http://hospital.smarthealthit.org
identifier_1_value
f06c0800-7e56-45e7-85eb-0047c2c5b4b1
identifier_2_type_coding_0_system
http://terminology.hl7.org/CodeSystem/v2-0203
identifier_2_type_coding_0_code
SS
identifier_2_type_coding_0_display
Social Security Number
identifier_2_type_text
Social Security Number
identifier_2_system
http://hl7.org/fhir/sid/us-ssn
identifier_2_value
999-17-9471
identifier_3_type_coding_0_system
http://terminology.hl7.org/CodeSystem/v2-0203
identifier_3_type_coding_0_code
DL
identifier_3_type_coding_0_display
Driver's License
identifier_3_type_text
Driver's License
identifier_3_system
urn:oid:2.16.840.1.113883.4.3.25
identifier_3_value
S99913919
identifier_4_type_coding_0_system
http://terminology.hl7.org/CodeSystem/v2-0203
identifier_4_type_coding_0_code
PPN
identifier_4_type_coding_0_display
Passport Number
identifier_4_type_text
Passport Number
identifier_4_system
http://standardhealthrecord.org/fhir/Structure...
identifier_4_value
X55105329X
name_0_use
official
name_0_family
Johnston
name_0_given_0
Paola
name_0_prefix_0
Mrs.
name_1_use
maiden
name_1_family
Schinner
name_1_given_0
Paola
name_1_prefix_0
Mrs.
telecom_0_system
phone
telecom_0_value
555-573-4647
telecom_0_use
home
gender
female
birthDate
1924-12-30
deceasedDateTime
1962-02-22T02:28:09+00:00
address_0_extension_0_url
http://hl7.org/fhir/StructureDefinition/geoloc...
address_0_extension_0_extension_0_url
latitude
address_0_extension_0_extension_0_valueDecimal
42.33196
address_0_extension_0_extension_1_url
longitude
address_0_extension_0_extension_1_valueDecimal
-71.020173
address_0_line_0
742 Howe Spur Apt 91
address_0_city
Boston
address_0_state
Massachusetts
address_0_postalCode
02108
address_0_country
US
maritalStatus_coding_0_system
http://terminology.hl7.org/CodeSystem/v3-Marit...
maritalStatus_coding_0_code
M
maritalStatus_coding_0_display
M
maritalStatus_text
Custom InteropX mapping rule — not present in ...
multipleBirthBoolean
False
communication_0_language_coding_0_system
urn:ietf:bcp:47
communication_0_language_coding_0_code
en-US
communication_0_language_coding_0_display
English
communication_0_language_text
English
If you look at the output above, you can see FHIR-PYrate collapsed the hierarchical FHIR data structure into DataFrame columns. FHIR-PYrate does this by taking an element from the FHIR-formatted data like Patient.identifier[0].value and converting to an underscore-delimited column name like identifier_0_value. (Note that Patient.identifier has multiple values in the FHIR data, so there are multiple identifier_N_... columns in the DataFrame.)
3 Selecting specific columns
Usually not every single value from a FHIR instance is needed for analysis. There are two ways to get a more concise DataFrame:
Use the approach above to load all elements into a DataFrame, remove the unneeded columns, and rename the remaining columns as needed. The process_function capability in FHIR-PYrate allows you to integrate this approach into the bundles_to_dataframe() method call.
Use FHIRPath to select specific elements and map them onto column names.
The second approach is typically more concise. For example, to generate a DataFrame like this…
id
gender
date_of_birth
marital_status
…
…
…
…
…you could use the following code:
# Instantiate and perform the FHIR search interaction in a single function calldf = search.steal_bundles_to_dataframe( resource_type="Patient", request_params={"_count": 10, # Get 10 instances per page"identifier": "https://github.com/synthetichealth/synthea|", }, num_pages=1, # Get 1 page (so a total of 10 instances) fhir_paths=[ ("id", "identifier[0].value"), ("gender", "gender"), ("date_of_birth", "birthDate"), ("marital_status", "maritalStatus.coding[0].code"), ],)df
While FHIRPath can be quite complex, its use in FHIR-PYrate is often straight forward. Nested elements are separated with ., and elements with multiple sub-values are identified by [N] where N is an integer starting at 0. The element paths can typically be constructed by loading all elements into a DataFrame and then manually deriving the FHIRPaths from the column names, or by looking at the hierarchy resource pages in the FHIR specification (see FHIR Resources in Depth for more information on reading the FHIR specification).
4 Elements with multiple sub-values
There are multiple identifier[N].value values for each instance of Patient in this dataset.
# Instantiate and perform the FHIR search interaction in a single function calldf = search.steal_bundles_to_dataframe( resource_type="Patient", request_params={"_count": 10, # Get 10 instances per page"identifier": "https://github.com/synthetichealth/synthea|", }, num_pages=1, # Get 1 page (so a total of 10 instances) fhir_paths=[("id", "identifier[0].value"), ("identifiers", "identifier.value")],)df
This will give you separate identifier_0, identifier_1, … columns for each Patient.identifier[N] value.
5 Retrieving related data
To retrieve instances of related resources, additional request_params can be added. See Using the FHIR API to Access Data for more information on constructing the parameters for FHIR search interactions.
In the example below, instances of Patient and instances of related Observation resources are requested:
# Instantiate and perform the FHIR search interaction in a single function calldfs = search.steal_bundles_to_dataframe( resource_type="Patient", request_params={# Get instances of Observation where `Observation.patient` refers to a fetched Patient instance"_revinclude": "Observation:patient","identifier": "https://github.com/synthetichealth/synthea|","_count": 10, # Get 10 instances per page }, num_pages=1, # Get 1 page (so a total of 10 instances))# `dfs` is a dictionary where the key is the FHIR resource type, and the value is the DataFrame## Split these into separate variables for easy access:df_patients = dfs["Patient"]df_observations = dfs["Observation"]# Look at the first row of the Observations DataFramedf_observations.head(1).T
To work around this, you can also iterate over all the rows in a DataFrame and request related resources using trade_rows_for_bundles():
df_observations2 = search.trade_rows_for_dataframe( df_patients, resource_type="Observation", request_params={"_count": "10", # Get 10 instances per page }, num_pages=1,# Load Observations where `Observation.subject` references the instance of Patient# identified by `id` in the `df_patients` DataFrame df_constraints={"subject": "id"}, fhir_paths=[ ("observation_id", "id"), ("patient", "subject.reference"), ("status", "status"), ("code", "code.coding[0].code"), ("code_display", "code.coding[0].display"), ("value", "valueQuantity.value"), ("value_units", "valueQuantity.unit"), ("datetime", "effectiveDateTime"), ],)# Look at the first row of the Observations DataFramedf_observations2.head(15)
Note that this will only display value for instances of Observation that record a value in Observation.valueQuantity. Typically, you would filter by Observation.code and then choose the appropriate data type for Observation.value[x] to import. For example, http://loinc.org|72166-2 is the LOINC for smoking status. To get smoking status records for all patients in df_patients:
# Directly search for smoking status observationsdf_observations2 = search.steal_bundles_to_dataframe( resource_type="Observation", request_params={"code": "http://loinc.org|72166-2", # LOINC code for smoking status"_count": 20, # Get more observations since we're not limiting by patient }, num_pages=1, fhir_paths=[ ("observation_id", "id"), ("patient", "subject.reference"), ("status", "status"), ("code", "code.coding[0].code"), ("code_display", "code.coding[0].display"), ("value", "valueCodeableConcept.coding[0].code"), ("value_display", "valueCodeableConcept.coding[0].display"), ("datetime", "effectiveDateTime"), ],)# Look at the first row of the Observations DataFramedf_observations2.head(15)