Skip to main content

Welcome to InterSystems OMOP

InterSystems® OMOP™ is a cloud-based, software-as-a-service solution that transforms HL7® FHIR® data into the Observational Medical Outcomes Partnership (OMOP) Common Data Model (CDM). InterSystems OMOP looks at FHIR data stored in an cloud storage bucket and automatically transforms and sends the data to the cloud-hosted repository in the OMOP CDMOpens in a new tab format. You can then use external Observational Health Data Sciences and Informatics (OHDSI) tools, such as ATLAS or HADES, in conjunction with a database driverOpens in a new tab, such as JDBC, to perform analytical tasks on your data.

InterSystems OMOP Deployment

As part of the onboarding process for your service, the InterSystems Managed Services team will work with your organization to stand up and configure your OMOP deployment. During that process they will require information about your cloud service provider, storage bucket details and how access to your OMOP service will be secured.

Once your OMOP deployment is set up, the InterSystems Managed Services team will provide your organization with the necessary access information.

Try Out Your Deployment

Once your deployment is available, you can try it out. Log into the service using the information provided by the InterSystems Managed Services team. From the Overview page, you can see the details and status of your deployment, the status of each service, and connection information for connecting external applications to your data.

You can try out the following actions:

  1. Upload your FHIR files to your cloud storage bucket to trigger the transformation. For more details on how these files should be formatted and how the transformation works, see Transform FHIR Data.

  2. Once you have uploaded a file, navigate to the Metrics page in your service. If multiple files are listed in the left pane, ensure the file you just uploaded is selected. On the Metrics tab, the Run Information section shows that the Bulk FHIR file you uploaded is listed with the time the ingestion began. Once ingestion is complete, the number of patients will also be listed in the table, indicating that the transformation from the FHIR format to the OMOP CDM has begun. When the transformation is complete, the time that the transformation finished will be shown.

    Once the file has been completely processed, the Metrics tab provides an overview of the data, including the number of resources loaded along with error and warning counts.

  3. You can view a summary of any data errors or warnings on the Error Summary tab. It includes sections for a summary list of unrecognized coding systems, a detail list of unrecognized codes under each unrecognized coding system, and an error/warning list of missing data.

    The Ingestion Errors tab lists FHIR validation errors, where values do not confirm to the FHIR standard.

    The Transformations tab lists all the transformations that occurred, with their status. You can also use the search window to search for a specific value across the entire batch file. To understand how the transformation maps FHIR fields to OMOP CDM fields, see InterSystems OMOP Transformation Details. For information on the errors and warnings that may be reported, see the Understand Process Errors and Warnings section.

  4. To view the data itself you can connect an external application, such as DBeaver, to view the schema and run queries on the data. If access is configured to an external OMOP repository, you can also access the data through that external source.

Transform FHIR Data

Your OMOP service accesses your cloud storage bucket, processes files added to the cloud storage bucket, and stores the transformed records in a repository. The service periodically looks in the cloud storage bucket to see if new files have been added and performs the data transformation on these files automatically. Simply add bulk FHIR files to your cloud storage bucket to start the transformation of your data to the OMOP CDM. FHIR data added to your cloud storage bucket can be stored in an NDJSON file with a standard FHIR bundle structure, a folder containing multiple layers of such NDJSON files, or contained in a zip file in the Bulk FHIR format. A zipped file should be structured as shown in the picture below:

generated description: intro zipped file format

For more information about the NDJSON format used in Bulk FHIR, see the official HL7 documentationOpens in a new tab.

All transformed data is stored in the OMOP repository in your deployment. If new FHIR data is for an existing patient then, to ensure data integrity, the service deletes all the existing data for the patient in the OMOP repository and replaces it with the new FHIR data. Therefore, the new FHIR data should include all the historical patient data previously sent to the OMOP platform. Any data the references a patient that is not included in the new FHIR data is considered “orphaned” and is reported as an ingestion error.

Important:

When updating a patient, ensure the new FHIR data contains the historical patient data as the OMOP platform deletes all existing data for the patient.

The deployment automatically maps nonstandard terminologies in FHIR data to the standard terminology in OMOP ontology. Below is an example of mapping ICD10 to SCT:

generated description: intro mapping example

For more information about mapping non-standard terminologies in FHIR data to standardized terminologies in OMOP, see Terminology Mapping.

Access OMOP Data

You can access the data in the deployment from a Python, Java, .NET, or C++ application through the use of a database driver, in just a few steps:

Once you have accessed the data from your application, use OHDSI toolsOpens in a new tab to assess and analyze the data that has been standardized in the OMOP CDM. See OMOP Odyssey - CelebrationOpens in a new tab, an article in the InterSystems Developer CommunityOpens in a new tab, for more information on using the iris dialect with the DatabaseConnectorOpens in a new tab in HADES.

Understand Process Errors and Warnings

The tabs in the pane on the right provide information for a selected file. The Error Summary tab reports the summary of any issues that arose during processing. The Ingestion Errors tab reports issues that occurred when processing particular resources in the file.

Issues fall into two distinct categories:

  • Warnings: The deployment raises a warning when there is a failure to map data in a field from a FHIR resource to a non-required field in an OMOP CDM record; this OMOP CDM record is created, but contains one (or more) empty fields. If a warning is triggered when processing a coded field with a value not found in the OMOP CDM ontology, it can be addressed by adding a custom terminology mapping to include the data.

  • Errors: The deployment raises an error when there is an issue processing the data contained in a particular field. In this case, the deployment fails to create an OMOP CDM record and reports what FHIR resource the error originated from. The issue can arise from any of the following cases:

    • Missing a required field: The deployment raises an error when the transformation would create an OMOP CDM record that lacks one of the required fields of the OMOP CDM. This typically occurs when the FHIR resource does not have data in the field that maps to the required field.

    • Unrecognized terminology: When the deployment cannot recognize the FHIR terminology, it cannot map to standard OMOP concepts.

    • FHIR validation error: The deployment validates all FHIR resources before attempting to transform them and any resource that fails validation is not transformed into an OMOP CDM record. Any FHIR resource that does not conform with the standard fails validation. For example, if a property has wrong datatype or if the FHIR resource does not contain a required property.

    • Storage access error: The deployment raises an error when it cannot properly connect with the configured cloud storage bucket.

    • External data source access error: The deployment raises an error when there is an issue with the connection between the deployment and the external data source in which it attempts to store an OMOP CDM record.

To fix most of these issues, you must modify the input FHIR data. Error messages reported on the Transformations tab indicate which resources the OMOP service was unable to transform, so you can find the origin of the error, modify the source, and re-upload the data that failed the transformation.

To fix a storage access error, make sure that the input properties specifying how to access your bucket are correct. If they are not, contact your InterSystems Managed Services team.

Transformation Errors and Warnings

The Transformations tab lists all the transformations that occurred, together with their status. To see more information about an error or warning, you can expand an entry by clicking the relevant plus (+) sign. These are examples of the errors and warnings you might see:

No source data found

The transformation cannot find a value in the source FHIR resource. For example, a missing gender value is an error and a missing race/ethnicity value is a warning.

Unrecognized or missing coding system

The coding system element is missing or contains a value that cannot convert to OMOP’s vocabulary ID. To remediate, add an entry to the system lookup table.

Unrecognized or missing code

The code element is missing or contains a value that cannot convert to the concept ID of a standard OMOP concept. To remediate, add an entry to the primary lookup table.

Terminology Mapping

If a Terminology Prefix was defined when creating your deployment, you can extend the default vocabulary of the OMOP Platform by uploading a .csv file into your cloud storage bucket at the defined location. Each file is processed individually when it is added to the cloud storage bucket.

There are two valid formats of .csv files that can be uploaded to your bucket. One format adds values to the OMOP vocabularyOpens in a new tab file and the other adds values to the OMOP conceptOpens in a new tab file, enabling you to define extensions on the default vocabulary sets. By adding various vocabularies and concepts, you can ensure the proper transformation of data from the OMOP CDM into the FHIR format.

OMOP Vocabulary

Add a .csv file that has exactly two columns: system and vocabulary (in that order). In the system column, denote the FHIR code system’s canonical URL. In the vocabulary column, specify the VOCABULARY_ID of the code system from the OMOP CDM. These two columns create a mapping between a specific FHIR code system and a supported OMOP vocabulary, ensuring that data is transformed correctly.

To add a new mapping or update an older one, a value in both columns is necessary; this will add or update the name of the vocabulary associated with the specified coding system’s canonical URL. To delete a mapping, leave the vocabulary field blank, but specify a coding system URL; this will remove the coding system URL from the system’s transformation process.

The following example demonstrates what this .csv file might look like. It contains three rows: the first row adds the LOINC vocabulary, the second row adds the ICD9CM vocabulary, and the third row removes the vocabulary (RxNorm) associated with the urn:oid:2.16.840.1.113883.6.88 coding system.

system,vocabulary
http://loinc.org,LOINC
urn:oid:2.16.840.1.113883.6.2,ICD9CM
urn:oid:2.16.840.1.113883.6.88,
OMOP Concept

Add a .csv file that has seven columns:

  1. domainId: The DOMAIN_IDOpens in a new tab of the OMOP concept.

  2. system: The FHIR code system’s canonical URLOpens in a new tab.

  3. code: The code in FHIR to be translated to a standard OMOP concept. This value typically goes with system and display.

  4. conceptId: The CONCEPT_IDOpens in a new tab of a standard OMOP concept.

  5. relationshipId: The mapping relationshipOpens in a new tab, either “Maps to” or “Maps to value.”

  6. display: An optional column that represents the meaning of the code within the system.

  7. sourceConceptId: An optional column that specifies the unique CONCEPT_ID of a non-standard concept that corresponds to the code extracted from FHIR.

All rows must have unique values for domainId, system, and code. display, and sourceConceptId are optional for all rows.

To add or update a row, specify values in the domainId, system, code, conceptId and relationshipId columns. To delete a concept mapping from the table, simply specify values for domainId, system, and code, while omitting the conceptId.

The following example demonstrates what this .csv file might look like. It contains three rows: the first two rows add concept mappings, and the third row removes one.

domainId,system,code,conceptId,relationshipId,display,sourceConceptId
Drug,urn:oid:1.2.36.1.2001.1005.17,AGRPL,40213153,Maps to,Agrippal,
Unit,http://unitsofmeasure.org,m[IU]/L,9040,,Maps to,m[IU]/L,
Unit,http://unitsofmeasure.org,{score},,,,

The following diagram (also seen above in Transform FHIR Data) demonstrates many of these fields in action. A FHIR record is represented as the source data, which is ingested into the OMOP CDM as a non-standard concept. Using the concept relationship defined by the mapping file, the non-standard concept is mapped to a standard concept.

  1. domainId: Found under the standard or non-standard Concept > domain_id, with the value Observation (related standard and non-standard concepts typically have the same domain_id).

  2. system: Found under Source Data > system, with the value http://hl7.org/fhir/sid/icd-10-cm.

  3. code: Found under Source Data > code, with value W61.51.

  4. conceptId: Found under the standard Concept > concept_id, with value 4314097.

  5. relationshipId: Found under Concept_Relationship > relationship_id, with value Maps to.

  6. display: Found under non-standard Concept > concept_name.

  7. sourceConceptId: Found under the non-standard Concept > concept_id with value 21222, which corresponds with the ICD10CM code W61.51, representing the non-standard “Bitten by goose” concept_name.

generated description: intro mapping example

Default Vocabulary

By default, InterSystems OMOP already supports a subset of the OHDSI Standardized Vocabularies. Transformations that require these vocabularies work out of the box and do not require you to manually add any additional vocabularies. These supported vocabularies are:

Learning Resources

To learn more about InterSystems OMOP, explore the following resources:

To learn more about working with OHDSI tools with InterSystems OMOP and other InterSystems products, explore the following articles in the InterSystems Developer Community:

InterSystems OMOP Transformation DetailsNext section
FeedbackOpens in a new tab