Release notes for NLP API 2.21

Contents

• Summary
• New Qualifier Relation Types
• Clinical/CCD subcategory
• Python SDK updates

Summary

Added

• Two new qualifier relation types: “compliance” and “stage/grade”.
• Added class NUM qualifiers to the stage/grade qualifier relation type.
• Annotation updates for status type qualifiers for fractures.
• New Clinical document subcategory called CCD.
• Annotation updates for referrals/consults and biomarkers.
• Added new measurement units.
• Added section based entity disambiguation.
• Multiple annotation updates for SNOMED, RxNorm, HGNC.

Fixed

• Improvements to ejection fraction and FEV1 measurement relations.
• Improved measurement unit handling.
• Improved qualifier relations.
• Improved measurement relations.
• Remove unitless measurements not in measurement relations.
• Entity finding bug fixes for trailing + or - and hyphenated terms and dangling single quotes.
• Fixed false positive unitless measurement relations.
• Fixed a bug that raised ValueError exception in followup relation finding.
• Improved negation handling.

Changed

• Remove punctuation insertion for faster processing.
• Bare/unitless measurements entities that are not in a relation are pruned from the engine output.
• Tokenizer improvements for better word splitting.
• Improve plain/Clinical/generic section handling.
• Added qualifier relations detection in Clinical documents.
• Improved plain/Clinical/generic section handling to avoid false positives on “alcohol” and other sub-section headings.
• HGNC ontology limited to entitylink matches.

Compatibility

• API version 2 Release 21
• Python SDK and client v4.3.0 to v5.2.0
• Java SDK and client v3.0.2 to v3.1.0
• JSON result output schema v2.14 for emtellipro-json-2

The latest version of the SDK is required for the new features in this release. The JSON result output schemas are backward compatible for all v2.x versions of emtellipro-json-2.

New Qualifier Relation Types

Existing qualifier relation types supported by the NLP API are:

• size/severity
• status
• duration/time

Two new types of qualifier relations were added:

• stage/grade
• compliance

For the following example sentences:

Stage IV gastric adenocarcinoma.
The patient refused a LP.

The output from the engine using output json result format v2.14.0 is shown below. The output is truncated to the relevant parts (the ... indicate the ellipsis).

1
...
2
"entities": {
3
    "found": [
4
        {
5
            "label": "E0",
6
            "spans": [
7
                {
8
                    "start": 0,
9
                    "end": 8
10
                }
11
            ],
12
            ...
13
            "entity_type": {
14
                "snomed": "qualifier value"
15
            },
16
            "text": [
17
                "Stage IV"
18
            ]
19
        },
20
        {
21
            "label": "E1",
22
            "spans": [
23
                {
24
                    "start": 9,
25
                    "end": 31
26
                }
27
            ],
28
            ...
29
            "entity_type": {
30
                "snomed": "disorder",
31
            },
32
            "text": [
33
                "gastric adenocarcinoma"
34
            ]
35
        },
36
        {
37
            "label": "E6",
38
            "spans": [
39
                {
40
                    "start": 37,
41
                    "end": 44
42
                }
43
            ],
44
            ...
45
            "entity_type": {
46
                "snomed": "person",
47
            },
48
            "text": [
49
                "patient"
50
            ]
51
        },
52
        {
53
            "label": "E7",
54
            "spans": [
55
                {
56
                    "start": 45,
57
                    "end": 52
58
                }
59
            ],
60
            ...
61
            "entity_type": {
62
                "snomed": "qualifier value"
63
            },
64
            "text": [
65
                "refused"
66
            ]
67
        },
68
        {
69
            "label": "E8",
70
            "spans": [
71
                {
72
                    "start": 55,
73
                    "end": 57
74
                }
75
            ],
76
            ...
77
            "entity_type": {
78
                "snomed": "procedure"
79
            },
80
            "text": [
81
                "LP"
82
            ]
83
        },
84
        ...
85
    ],
86
    ...
87
    "qualifiers": [
88
        {
89
            "label": "RQ0",
90
            "attributes": {
91
                "qualifier_type": "stage/grade"
92
            },
93
            "args": {
94
                "qualifier": {
95
                    "ref": "E0",
96
                    "text": [
97
                        "Stage IV"
98
                    ]
99
                },
100
                "qualifies": {
101
                    "ref": "E1",
102
                    "text": [
103
                        "gastric adenocarcinoma"
104
                    ]
105
                }
106
            }
107
        },
108
        {
109
            "label": "RQ1",
110
            "attributes": {
111
                "qualifier_type": "compliance"
112
            },
113
            "args": {
114
                "qualifier": {
115
                    "ref": "E7",
116
                    "text": [
117
                        "refused"
118
                    ]
119
                },
120
                "qualifies": {
121
                    "ref": "E8",
122
                    "text": [
123
                        "LP"
124
                    ]
125
                }
126
            }
127
        }
128
    ],
129
},
130
...

The processing feature to ask the engine to produce qualifer relations is qualifier-relations.

Clinical CCD

Both emtellipro and the Python SDK now support a new document subcategory called CCD under the Clinical document category.

A new CCD option has been added to the list of subcategories for the subcategory part header in the NLP API.

Python SDK updates

You will need NLP API Python SDK 5.2.0 for the new features listed below . If you are using Output Database client, please use the migrate command to upgrade DB to the new version of engine and SDK.

Plugins for ingesting novel document types

The Python database client now supports plugins for different input file types. Currently XML files in the CCD format are supported via a new plugin emtellipro-sdk-python-ccd-plugin which is available from the Downloads page.

To use a plugin you should pip install the wheel that you downloaded. The Python database client will automatically use the plugin for file types (specified by the suffix, e.g. .ccd or .xml) that the plugin registers with the client. Any such file types will be processed by the plugin and it will communicate the parsed information to the database client which will send it to the emtellipro API for processing.

The text spans are reported on the output of the plugin. For instance, the CCD plugin will return a plain text version of the structured XML input file. The spans produced by emtellipro will be over this plain text version (not the original XML document).

New qualifier relation types

New qualifier relation types called stage/grade and compliance are now supported by emtellipro_db (the Python database client).

The new qualifier relation type appears in the qualifierrelation table in the Output Database:

 id | qualifier_type | qualifier_id | qualifies_id
----+----------------+--------------+--------------

The qualifier_type field can now contain stage/grade and compliance in the Output Database.