Modularity and Packaging

Overview

Currently, every CIMPL project must fully contain all other CIMPL definitions it depends on. For example, CIMPL projects must have local files with definitions for Quantity (and its elements) for the toolchain to work. This is annoying for users and raises the barrier to entry. In addition, this can lead to many problems, including unnecessary duplication of data, stale or conflicting definitions, and more.

CIMPL needs a packaging mechanism to allow dependencies to be declared in a simple configuration file and automatically downloaded and used during the import and export process. This should work in much the same way as NPM does for Node.js, Gradle/Maven does for Java, Bundler does for Ruby, etc.

NOTE: Also see Versioning

Desired Features:

• Simple declaration of dependencies in a text file (that can be version controlled)
  • to enable importing specific namespaces or even specific elements without having to import all of SHR
  • Potentially remove the requirement for the Uses statement
• Support for resolutions of dependency versions using semantic versioning
• Some level of version deconfliction for transitive dependencies (e.g., A depends on X v1.1 and B depends on X v1.2).
• Simple to use with minimal-to-no extra software installation needed
• Leverage existing software as much as possible (i.e., we probably don't need to invent something)
• Leverage existing repositories as much as possible (i.e., we probably don't need to maintain a repository server)
• Just-in-time compilation of dependencies (nice-to-have)

Proposed Approach

Adopting the NPM packaging mechanism best addresses the requirements:

• NPM dependencies are declared in a simple package.json file
• NPM has robust support for semantic versioning and using wildcards to identify versions
• NPM/Yarn have capabilities for resolving conflicting transitive dependencies in smart ways
• Since authors already need NPM/Yarn to run the tools, no additional software is needed
• The NPM repository can be used for publishing/retrieving

In addition to fulfilling requirements, there are additional advantages:

• The shr-cli can be easily setup and run from the package.json, potentially avoiding the need to separately install shr-cli
• FHIR has also moved to NPM packaging for IGs (doc), so there is consistency (but note we can't/shouldn't share the same package name between IG and CIMPL)

Potential Project Structure

- cimpl-oncology
  - src/
    - oncology.txt
    - oncology-cp.txt
    - oncology-map-r2.txt
    - oncology-map-r3.txt
    - oncology-map-r4.txt
    - oncology-vs.txt
  - doc/
    - index.html
  - examples/
    - breast-cancer.json
    - staging.json
  - config-r2.json
  - config-r3.json
  - config-r4.json
  - package.json
  - package-list-r2.json
  - package-list-r3.json
  - package-list-r4.json
  - README.md

Potential Dependency Declaration in CIMPL

Grammar:      DataElement 6.1
Namespace:    oncology
Description:  "SHR implementation of oncology data elements."
Dependencies: cimpl-obf // NOTE: cimpl-obf-datatype is always implied

Note that Dependencies replaces what used to be Uses.

Potential NPM package.json File

{
  "name": "cimpl-oncology",
  "version": "1.0.0",
  "description": "CIMPL Module for Oncology Definitions",
  "dependencies": {
    "cimpl-obf-datatype": "^1.0.0",
    "cimpl-obf": "^1.0.0",
  },
  "devDependencies": {
    "shr-cli": "^6.9.0"
  },
  "scripts": {
    "build-r2": "shr-cli -c config-r2.json ./src",
    "build-r3": "shr-cli -c config-r3.json ./src",
    "build-r4": "shr-cli -c config-r4.json ./src"
  }
}

How It Works (High Level)

• Author runs npm/yarn to download dependencies
• Author runs yarn build-r4 to start the build process (formerly shr-cli)
• CLI enumerates dependencies and imports each one as SHR models
  • resulting models are stored in a map w/ dependency name as the key
  • CLI should suppress all messages from dependencies except errors
• CLI builds current project passing in built dependency map

This means that the importers/exporters need to be updated to know how to find and use dependency definitions -- and potentially to treat them differently (e.g., don't necessarily export the dependencies, use the correct canonicals when referencing profiles from a dependency, etc.).

NOTE: This is likely the simplest approach but the least efficient. We should consider another approach that would not compile dependency elements until they are actually used.

Open Questions

• Should packages support/allow multiple namespaces?
• How do exporters work re: dependencies (e.g., how/when should they export them?)
  • For example, ES6 classes are no good if they don't also have the exported ES6 classes they depend on
  • But you don't want the Oncology IG to contain all the profiles from its dependencies
• How should we handle potential conflicts/mismatches between transient dependencies w/ different versions?
• At what granularity should we split packages? For example, do all "base classes" (like Condition, Observation, etc) go in one base classes package? Or do we have packages for cimpl-conditions, cimpl-observations, etc?

---

Further Discussion (extracted from https://github.com/standardhealth/shr-cli/issues/137)

Mark Kramer:

No matter which implementation guide we generate from SHR sources, or what kind of filtering we do, or what we put in the config file, profiles and extensions must always have the same fully qualified names. Right now, if I run the oncocore IG (aka MCODE), the Patient profile is named: http://hl7.org/fhir/us/oncocore/StructureDefinition/shr-entity-Patient.

If I run the same source but generate the occupational health IG, the same profile is named: http://hl7.org/fhir/us/odh/StructureDefinition/shr-entity-Patient.

The name can't change. It's the same profile.

cmoesel commented on Dec 10, 2018

To support this feature, we need either the planned modular feature (CIMPL packages) or we need a way to associate namespaces/elements to URL prefixes. Otherwise there is no way that the FHIR exporter can know that you already exported the same patient definition in a different IG with a different IG URL prefix.

markkramerus commented on Dec 10, 2018

I did some investigation, and it seems the pattern for FQN is [fhirURL]/StructureDefinition/[namespace]-[ElementName]. If we always use the same fhirURL in every config file, e.g., http://hl7.org/fhir/us/shr, then the FQN should be stable. The downside is that the namespace is packed into the element name (such as shr-entity-Patient), not earlier in the URL. Is that a workaround? Regardless the documentation on the profile page says "Reference(oncocore Patient Profile)" when it should say Reference(shr-entity-Patient).

However, value sets march to a different drum. For example:

Name: oncocore PresentAbsentVS ValueSet Defining URL: [projectURL]/shr/base/vs/PresentAbsentVS

To be consistent with above, this would be [fhirURL]/vs/shr-base-PresentAbsentVS, e.g., http://hl7.org/fhir/us/shr/vs/shr-base-PresentAbsentVS.

cmoesel commented on Dec 10, 2018

You got the basic algorithm right. If I recall, the VS URLs are different because we wanted them to be independent of FHIR (e.g., a value set definition is applicable across many representations whereas a FHIR profile is by-definition FHIR only). I'm not sure that distinction really holds up, but I think that's the deal. If I recall though, when the projectURL == fhirURL, some additional magic happens and the algorithm for the VS URL changes to match the FHIR scheme. I think.

As for the redundancy w/ shr in the url prefix and in the namespace, that redundancy is created because shr exists in the URL and in the namespace. In other words, if that redundancy should be removed, I think it's up to the author to either modify the namespace or modify the url. That said, I supposed we could create some other convention that allows for "default" namespaces (or "default" namespace roots) that get suppressed.

That said, if we fix this so URLs for things like shr-entity-Patient don't change, then we have a different problem: multiple IGs all publishing the same profiles (but potentially with differences in content). I think the real solution will have to be a more modular approach such that when you use shr-entity-Patient, you're clearly importing it from another package/IG (along with its original URL) and not re-publishing it in your own IG. We need to get there.

Regardless the documentation on the profile page says "Reference(oncocore Patient Profile)" when it should say Reference(shr-entity-Patient).

As for this, the FHIR IG Publisher creates the text of the reference using the name of the profile it references. We currently export the name as ${config.projectShorthand} ${element.name} Profile. That can be changed.

markkramerus commented on Feb 21

I noticed the version value on the profile is the version of the IG (see code snippet). This is not correct since the profile version and IG versions are independent of each other. You should be able to rev the IG without revising the profiles.

<identifier> <system value="http://hl7.org/fhir/us/projectURL"/> <value value="shr.encounter.Encounter"/> </identifier> **<version value="0.6.0"/>** // this is the IG version <name value="Encounter"/> <display value="shr-encounter-Encounter"/> <status value="draft"/>

From another open bug:

To further modularity, we need the ability to have the fully qualified path reflect the publisher or owner. For example, the profile of AnatomicalLocation (may-2018-ballot branch) is:

http://hl7.org/fhir/us/breastcancer/StructureDefinition/entity-AnatomicalLocation

I'd like to be able to change this to:

http://hl7.org/fhir/us/**cimi**/StructureDefinition/entity-AnatomicalLocation

to reflect the fact that this particular profile is part of core cimi.

--------------- another conversation, perhaps related (through naming issues) ----------------

markkramerus commented on Nov 13, 2018 •

Our naming convention for extensions is namespace-element name-extension, for example, shr-base-TopicCode-extension. This seems out of step with the way other extensions (defined by FHIR and other work groups) are named; they seem to just use a readable name. Could we simplify our naming to "TopicCode" for the purposes of the IG? We don't have to change the unique URL. Note that the full URL for the extension is included in the instance data, so in practice, this shouldn't cause a name collision.

cmoesel commented on Nov 13, 2018

I'll take a look at this to see what we can do. It seems to be a reasonable request, but I need to see where the IG documentation generator is pulling the name from -- and if that is required to be unique. One potential issue is that we have some things that have both a profile representation and an extension representation -- and their identifiers need to be unique.

Unfortunately, when an extension is used in a profile, the snapshot view in the IG displays the extension's id rather than name or title. In the example above, the id is shr-base-TopicCode-extension.

The problem with simplifying the id is two-fold:

• the id must be unique in the IG
• the id must match the last part of the identifying uri

If we simplify something like my-namespace-MyElement-extension to just MyElement, there are two potential ways we could end up w/ duplicate ids:

• If we need to represent MyElement as both a profile and an extension, then they'd both have the same simplified id. You could disambiguate by adding -extension, which is what I did.
• If there was a MyElement in two different namespaces, then they'd both have the same simplified id. You could disambiguate by adding the namespace, which is what I did.

We could dynamically determine if either of these name clashes happen and only disambiguate if necessary, but then there is a possibility that if we add elements in a future version that cause an ambiguity, we'd change the already published id -- which is not a good thing.