Automated de-identification of free-text medical records

Question

I am looking for a program that can perform automated de-identification of free-text medical records. The de-identification should abide by Health Insurance Portability and Accountability Act (HIPAA) rules (US regulations), i.e. the following information should be removed from the text:

(i) Names of patients and family members
(ii) Addresses and their components
(iii) Dates (month and day parts, unless the inclusion of the year part identities an individual to be older than 90 years old)
(iv) Explicit mention of ages over 89 years old
(v) Telephone and fax numbers
(vi) Social Security numbers
(vii) Medical record numbers
(viii) Health plan beneficiary numbers
(ix) Account numbers
(x) Certificate or license numbers
(xi) Vehicle identifiers and serial numbers
(xii) Device identifers and serial numbers
(xiii) Electronic mail addresses
(xiv) Web universal resource locators (URLs)
(xv) Internet protocol (IP) addresses
(xvi) Biometric identifiers
(xvii) Full face photographic images
(xviii) Employers
(xix) Any other unique identifying number, characteristic or code

Any license, price, and OS is fine. I would be interested to know what techniques the program use to perform the de-identification (heuristics, LR/SVM/RF, CRF, ANN, etc.) as well as some metrics to assess the quality of the de-identification.

Answer 1

You can use MITRE Identification Scrubber Toolkit (MIST):

• free and open source (BSD license)
• CRF-based

The MITRE Identification Scrubber Toolkit (MIST) is a suite of tools for identifying and redacting personally identifiable information (PII) in free-text medical records. MIST helps you replace these PII either with obscuring fillers, such as [NAME], or with artificial, synthesized, but realistic English fillers.

For example, MIST can help you convert this document:

Patient ID: P89474

Mary Phillips is a 45-year-old woman with a history of diabetes. She arrived at New Hope Medical Center on August 5 complaining of abdominal pain. Dr. Gertrude Philippoussis diagnosed her with appendicitis and admitted her at 10 PM.

into this:

Patient ID: [ID]

[NAME] is a [AGE]-year-old woman with a history of diabetes. She arrived at [HOSPITAL] on [DATE] complaining of abdominal pain. Dr. [PHYSICIAN] diagnosed her with appendicitis and admitted her at 10 PM.

or this:

Patient ID: ID586

Sandy Parkinson is a 34-year-old woman with a history of diabetes. She arrived at Mercy Hospital on July 10 complaining of abdominal pain. Dr. Myron Prendergast diagnosed her with appendicitis and admitted her at 10 PM.

To use MIST, see How can I use the MIST library to de-identify a text?