Clinical Extractions

Social History

Goal of Extraction

The Core Clinical Social History Library is a social history-focused clinical data extraction tool designed to identify, extract, and ontologically map social determinants and lifestyle factors from unstructured clinical narratives and patient health record documentation. It extracts social and behavioral information documented across social history sections, clinical notes, and patient interviews, standardizing representation through SNOMED CT concept mapping.

Extraction Methodology

The extraction process operates at the sentence level, capturing each social history element with precise character-level locations (start and end positions) that enable direct traceability back to the “document”. Each extracted social history record includes substance use patterns (alcohol, nicotine, drugs), consumption and cessation information, marital status and length, and employment history. Status fields (alcohol_status, nicotine_status, substance_status, marriage_status, employment_status) are standardized to controlled vocabulary. A social history record will only be extracted if at least one of the following is present: [alcohol_status, nicotine_status, substance_status, marital_status, employment_status]. When a patient’s social history element appears multiple times within a single document—whether from repeated documentation, updates across different note sections, or serial assessments—each instance is extracted and recorded separately to preserve the complete social history documentation. The library performs extraction on a document-by-document basis; there is no automatic cross-document reconciliation or cross-patient data merging.

Note: A document is determined by our intelligent document splitting feature. Medical PDFs can contain a single document or multiple documents. A document’s type can provide clinical context important for the data extraction. This feature separates and categorizes an input PDF into individual documents by document type to prepare them for data extraction. An input PDF can remain one document or can be split into multiple documents through this process. A “document” in the context of this data extraction refers to the documents identified after the intelligent document splitting process.

Interpretation & Validation

Status fields (alcohol_status, nicotine_status, substance_status, marriage_status, employment_status) are standardized to controlled vocabulary to ensure consistent representation. These standardized classifications guide appropriate reliance on extracted data and support informed clinical interpretation.

Clinical Considerations

Social history context is critical for clinical decision-making. Understanding substance use patterns, employment status, and marital circumstances directly impacts patient management, risk assessment, and care planning. The distinction between current and former substance use patterns has significant implications for clinical decision-making and patient counseling. Duration and intensity of substance use provide essential context for health risk stratification. Clinical judgment should be applied when interpreting extracted results, particularly for sensitive social history elements.

Target Concepts

Coding System: SNOMED CT

All extracted social history elements are mapped to SNOMED CT (Systematized Nomenclature of Medicine Clinical Terms), with each element assigned a valid, active SNOMED concept identifier (6–18 digit numeric codes), along with its associated full SNOMED concept description. This structure provides both the standardized data needed for system integration and the contextual information needed for informed clinical interpretation.

Clinical Concepts Extracted

Social history elements are clinically documented information about patient substance use, employment, and marital circumstances. This includes substance use patterns (alcohol, nicotine, drugs), consumption and cessation information, marital status and length, and employment history.

We define social history status fields with SNOMED concepts. Status fields (alcohol_status, nicotine_status, substance_status, marriage_status, employment_status) are standardized to controlled vocabulary.

The extracted concept must be a specific, named element. Vague qualifier terms, attributes, and organism concepts are excluded — we are looking for identifiable, valid social history concepts.

Extracted Fields

Field NameData TypeDescription
filenamestringThe source PDF name with a suffix appended. The suffix refers to the subdocument after document splitting
subdoc_idstringUnique subdocument identifier
manifest_indexintegerThe row index of the file in the manifest submitted with the processing request. Used to retrieve the corresponding OCR output from the document metadata JSONL.
measureconst stringRow-label indicating extraction type
chartdatestringClinical note or chart date; standardized to YYYY-MM-DD format
subject_idstringPatient identifier as assigned in the manifest.csv (required)
processing_datestringDate/time record was processed; system-generated
alcohol_statusstringCurrent alcohol use status; controlled vocabulary: “current_drinker”, “ex_drinker”, “non_drinker”, “never_drinker”; if not mentioned, return “no_mentions”
alcohol_drinker_typestringType of alcohol use pattern; controlled vocabulary: “heavy”, “moderate”, “light”, “occasional”, “social”, “rare”, “alcoholic”; if not a drinker or no mention of drinking, return “n/a”
alcohol_consumptionstringQuantity of alcohol use per week, if a drinker; if not a drinker or no mention of drinking, return “n/a”
alcohol_quittingstringA date or temporal terms associated with drinking cessation, if applicable; if no history of drinking, is a current drinker, or there is no mention of drinking, return “n/a”
alcohol_sentencestringMost relevant sentence or paragraph about alcohol use; if no mention of alcohol use, return “no mentions”
alcohol_concept_idstringSNOMED concept for alcohol use status
alcohol_descriptionstringThe SNOMED fully specified name (FSN); derived from alcohol_concept_id
alcohol_mention_startintegerCharacter start position in sentence related to alcohol use mention
alcohol_mention_endintegerCharacter end position in sentence related to alcohol use mention
nicotine_statusstringCurrent nicotine use status; controlled vocabulary: “smoker”, “ex_smoker”, “non_smoker”, “never_smoker” (for smoked product users); “user”, “ex_user”, “non_user”, “never_user” (for non-smoked product users); if not mentioned, return “no_mentions”
nicotine_productsstringType of nicotine product(s) used; if not applicable, not a user, or if data is not present, return “n/a”
nicotine_consumptionstringThe quantity of nicotine use per day (e.g. 1 pack/day) and/or per year (e.g. 10 pack-year history), if present; if not applicable, not a user, or if data is not present, return “n/a”
nicotine_quittingstringA date or temporal terms related to nicotine cessation, if applicable; if no history of use or is a current user, return “n/a”
nicotine_sentencestringMost relevant sentence or paragraph about nicotine use; if no mention of use, returns “no mentions”
nicotine_concept_idstringSNOMED concept for nicotine use status
nicotine_descriptionstringThe SNOMED fully specified name (FSN); derived from nicotine_concept_id
nicotine_mention_startintegerCharacter start offset of the sentence for this mention relative to the start of the document text
nicotine_mention_endintegerCharacter end offset of the sentence for this mention relative to the start of the document text
substance_statusstringCurrent illicit or recreational substance use other than alcohol, nicotine, and caffeine-containing products; controlled vocabulary: “current_user”, “ex_user”, “non_user”, “never_user”; if not mentioned, return “no_mentions”
substance_productsstringType of substances used (excludes alcohol, nicotine, and caffeine-containing products); if not a user, then return “n/a”
substance_consumptionstringThe quantity of substance use per day or per year; if not applicable or no history of use, return “n/a”
substance_quittingstringA date or temporal terms related to substance use cessation, if applicable; if no history of use or is a current user, return “n/a”
substance_sentencestringMost relevant sentence or paragraph about substance use/abuse; if no mention of use, return “no_mentions”
substance_concept_idstringSNOMED concept for substance use status
substance_descriptionstringThe SNOMED fully specified name (FSN); derived from substance_concept_id
substance_mention_startintegerCharacter start offset of the sentence for this mention relative to the start of the document text
substance_mention_endintegerCharacter end offset of the sentence for this mention relative to the start of the document text
marriage_statusstringCurrent marital/partnership status; controlled vocabulary: “married”, “unmarried”, “common_law”, “separated”, “divorced”, “widowed”, “engaged”; if not mentioned, return “no_mentions”
marriage_lengthstringDuration of current marriage, if married; if not married, returns “n/a”
marriage_sentencestringMost relevant sentence or paragraph about marital/partnership status; if no mention of status, return “no mentions”
marriage_concept_idstringSNOMED concept for marital status
marriage_descriptionstringThe SNOMED fully specified name (FSN); derived from marriage_concept_id
marriage_mention_startintegerCharacter start offset of the sentence for this mention relative to the start of the document text
marriage_mention_endintegerCharacter end offset of the sentence for this mention relative to the start of the document text
employment_statusstringCurrent employment status; controlled vocabulary: “employed_full_time”, “employed_part_time”, “employed_seasonal”, “employed_intermittent”, “unemployed”, “retired”, “student”; if not mentioned, return “no_mentions”
employment_typestringType of work or job title; if not applicable, return “n/a”
employment_lengthstringDuration of employment, unemployment, student, etc; if not specified, return “n/a”
employment_statusstringMost relevant sentence or paragraph about employment status; if no mention of status, return “no mentions”
employment_concept_idstringSNOMED concept for employment status
employment_descriptionstringThe SNOMED fully specified name (FSN); derived from employment_concept_id
employment_mention_startintegerCharacter start offset of the sentence for this mention relative to the start of the document text
employment_mention_endintegerCharacter end offset of the sentence for this mention relative to the start of the document text

Social History Extraction Fields

Example of Extracted Output

  • Note: This example demonstrates the content of the output, not the structure
nicotine_sentencenicotine_statusnicotine_consumptionnicotine_concept_idnicotine_description
Patient currently smokes 1 pack/daysmoker1 pack/day77176002Smoker (finding)
Social hx: non-smoker, drinks occasionallynon_smokern/a8392000Non-smoker (finding)

Social History Illustrative Output Example

  • Example JSONL object showing a typical extraction result
1{
2  "filename": "Case_1_APS_HSHS_Redacted2.pdf",
3  "subdoc_id": "Case_1_APS_HSHS_Redacted2.pdf-0009",
4  "manifest_index": 0,
5  "measure": "social_history",
6  "chartdate": "2021-05-19",
7  "subject_id": "10000001",
8  "processing_date": "2024-03-28T00:00:00",
9  "data": [
10    {
11      "employment_status": "retired",
12      "employment_type": "n/a",
13      "employment_length": "n/a",
14      "employment_sentence": "no mentions",
15      "employment_concept_id": "105493001",
16      "employment_description": "Retired, life event (finding)",
17      "employment_mention_start": null,
18      "employment_mention_end": null,
19      "marriage_status": "no_mentions",
20      "marriage_length": "n/a",
21      "marriage_sentence": "no mentions",
22      "marriage_concept_id": "n/a",
23      "marriage_description": "n/a",
24      "marriage_mention_start": null,
25      "marriage_mention_end": null,
26      "substance_status": "never_user",
27      "substance_products": "n/a",
28      "substance_consumption": "n/a",
29      "substance_quitting": "n/a",
30      "substance_sentence": "Drug use:  | Never",
31      "substance_concept_id": "228368007",
32      "substance_description": "Has never misused drugs (situation)",
33      "substance_mention_start": 6759,
34      "substance_mention_end": 6774,
35      "nicotine_status": "ex_smoker",
36      "nicotine_products": "cigars",
37      "nicotine_consumption": "n/a",
38      "nicotine_quitting": "1991",
39      "nicotine_sentence": "Tobacco Use |  | Smoking status:\tFormer Smoker | Types:\tCigars | Quit date:\t1991 | Years since quitting:\t30.4 | Smokeless tobacco:\tNever Used",
40      "nicotine_concept_id": "8517006",
41      "nicotine_description": "Ex-smoker (finding)",
42      "nicotine_mention_start": 6373,
43      "nicotine_mention_end": 6501,
44      "alcohol_status": "current_drinker",
45      "alcohol_drinker_type": "social",
46      "alcohol_consumption": "n/a",
47      "alcohol_quitting": "n/a",
48      "alcohol_sentence": "Alcohol use:  | Comment: social  | Yes",
49      "alcohol_concept_id": "219006",
50      "alcohol_description": "Current drinker of alcohol (finding)",
51      "alcohol_mention_start": 6723,
52      "alcohol_mention_end": 6756
53    }
54  ]
55}