Splunk, Big Data and Healthcare Analytics in the Federal Government – Part 3 DHMSM

Welcome to part three of my three-part blog on the ascending role of big data for healthcare analytics in the federal government. In this final part of the series we look at DHMSM, a very large project to find efficiency and insight in near real time. Part one and part two can be found here and here.

DHMSM and the problem to be addressed

Department of Defense Healthcare Management System Modernization (DHMSM) Program is administering an RFP for a potential $11B effort which calls for the modernization the Department of Defense healthcare system by uniting multiple legacy healthcare systems and data stores, developed over decades. I’ve reviewed most of the RFP consisting over 20 attachments which also calls for the implementation of an EHR (electronic health record) and HIE (health information exchange). This RFP was originally part of a joint DoD and Veterans Administration RFP that was “considered to be too costly.”

DoD has recognized the rising costs of healthcare as a percentage of the total DoD budget. Between 2006 and 2015, DoD health case related costs have moved from 8% of the overall budget to 12% by 2015. This is due in large part to:

  • “Medical forces are not sufficiently net-centric and interoperable to enable a fully integrated future health system and accelerate its ability to know, decide and act in real time.
  • Current medical information systems do not fully facilitate data sharing between government agencies, DoD and industry partners.
  • Current medical training strategies and platforms cannot adequately prepare medical forces to operate in a future operational environment in which joint forces deliberately create dynamic situations that change at great speed and intensity.
  • Current joint medical logistics processes are not sufficiently integrated and networked to respond efficiently and effectively to medical material requirements of future medical forces.
  • Current medical systems, equipment, and forces lack capabilities to operate in all types of environments, including multinational operations; security, transition, and reconstruction operations; operations with NGOs and IGOs; medical capacity building; and public health services.
    The growing population of persons to be treated will increasingly challenge the ability of the MHS to provide the expected level of care.”

The EHR System is expected to unify and increase accessibility of integrated, evidenced-based healthcare delivery and decision-making. The EHR System will support the, “availability of longitudinal medical records for 9.6 million DoD beneficiaries and approximately 153,000+ MHS personnel globally. MHS personnel include activated members of the National Guard and Reserve, estimated at 50,000 additional personnel.”

The system will need to support:

  • Access to historical information with role-based access control,
  • Synchronization/integration with legacy systems,
  • Migration required to support transitions for legacy information portals,
  • Data, Identity Management and Access Management Architecture and Design Inputs,
  • Health Data Sharing Recommendations, while;

Establishing “appropriate administrative, technical, and physical safeguards to protect all Government data, to ensure the confidentiality, integrity, and availability of Government data.”

The system is created to support, “…first responders, advanced trauma/emergency, Ancillary support services (e.g., Laboratory, Pharmacy, Radiology), in-theater hospitalization including robust care for resuscitation, surgery, and postoperative care, Critical Care services from Critical Care Air Transport Teams (CCATT).

In other words, the DoD wants to gather a high volume (I’m guessing north of 100 terabytes) of EHR data from a wide variety of legacy systems that likely produce a wide variety of structured and unstructured data from across the DoD health systems at high velocity. All while looking for better cost efficiency and better patient outcomes across a wide variety of use cases.

The DoD has finally recognized that it must, “…fully exploit information technology, infrastructure, training and research to support the four basic categories of military activity.”

Data types

While no specific data types are mentioned in the DHMSM RFP, we have some hints about the types of data that must be collected for the project to succeed.

  • Access to historical information – Building baselines and trending for insight requires this kind of access.
  • Synchronization with legacy systems – Tons of mainframes and legacy applications likely no longer supported putting out a variety of different structured and unstructured data.
  • Data, Identity Management and Access Management Architecture and Design Inputs – Data access and identity management data are critical for HIPAA compliance and mitigating data breach risks.
  • Health Data Sharing Recommendations – It is desirable to have data flow out of the big data system in ways that allow it to be used in other systems or as part of a mash-up in a web portal.


The emphasis on security can be seen in several parts of DHMSM. Specifically, the contractor shall, “…provide cybersecurity that conforms to the DoD cybersecurity and the DoD Risk Management Framework requirements as outlined in the Defense Information Systems Agency (DISA) Information Assurance Support Environment (IASE).

The system shall also:

  • “Ensure the confidentiality, integrity, and availability of Government data in compliance with all applicable laws and regulations, including data breach reporting and response requirements, in accordance with Defense Federal Regulations Subpart 224.1 (Protection of Individual Privacy), which incorporates by reference current versions DoDD 5400.11, “DoD Privacy Program,” and DoD 5400.11-R, “DoD Privacy Program.”
  • Comply with federal laws relating to freedom of information and records management.
  • Analyze any breach of PII/PHI for which it is responsible under the terms of this Contract under both the Privacy Act and Health Insurance Portability and Accountability Act (HIPAA), if applicable, to determine the appropriate course of action under each requirement, if any.”

RMF for ITFigure 1 — RMF for IS and PIT systems (page 28) – DoD Risk Management Framework

It should be noted that this approach closely mirrors the new CDM framework published by DHS.

“The system shall maintain situational awareness/continuous monitoring of any vulnerability, flaw or weakness in the network defense that could be exploited to gain unauthorized access to, damage or otherwise affect the network.” This is in accordance with the approved DoD Information Assurance Certification and Accreditation Process (DIACAP) baseline in accordance with the DoDI 8510.01. Also, [the system operator shall] mitigate all newly discovered vulnerabilities within the specified timelines based on Category level of the findings in accordance with DoD (DoDI 8510.01 section E2.56), DHA policies, and/or PEO DHMS.


Ideally, to support the use cases listed, the system must be able to be supported through full data redundancy, disaster recovery, and emergency failover and have 100% up-time. First responders (a use case listed in the RFP) can’t count on a system with only ‘five-nines’ of up time. The performance standards summary (appendix A) in the RFP calls for the EHR system reliability to be “The EHR System shall meet > 98% Systems Operational (network) Availability. Operational availability assesses the total time the system is capable of being used to perform clinical functions during a given interval.” Established baselines and benchmarks for application performance, error rates, and system downtime can all be tracked for accountability and alerts routed to the right person for


In addition to all applicable HIPAA compliance standards, the DoD has it’s own Department of Defense Privacy Program. The section of collecting personal information discusses how SSNs are used, how private data shall be accessed (For example, an individual requiring access, “…may be required to provide reasonable proof his or her identity.”).

Splunk has the ability to collect all authentication records and correlate them with context data about the user to understand their rights to view the data based on who they are, where they are, what system they are authenticating from, the authentication channel or method (i.e. VPN), and when the data is being accessed. The user can assign a risk score to these factors and use the information to determine whether or not the user should be allowed access.

Splunk’s big data solution provides role based access controls and auditing capabilities needed to restrict, monitor and alert on user access to specific patient data. This meets DoD privacy objectives. Alerts and access dashboards can be created that call out any combination of abnormal access based on a lack of relationship between the patient and caregiver, time or date of access, or access by a caregiver that may be inappropriate based on a current or past family relationship.

Use Cases and Splunk’s Role

Of all the documents that are a part of the RFP, the most interesting documents in the RFP package are Attachments 8 and 9 both titled Healthcare Service Delivery Concept of Operations – CONOPS and Attachment 2 Government Requirements Traceability Matrix. These document outline some of the use cases that the DoD has thought of and seen to include in the RFP package. In the table below we’ll some of the capabilities in the tracibility matrix and outline Splunk’s possible role.



Splunk Capabilities

Risk Management

The ability of a hospital or other healthcare facility to direct the identification, evaluation, and correction of potential risks that could lead to injury to patients, staff members or visitors and may or may not result in financial loss to the Government.

Information can be gathered from a wide variety of systems and used to identify risk by watching for treatment incongruences based on diagnostic codes and comparing them with data about the patient. Data can be queried to ensure that all pre-visit questionnaires have been filled out and received in their electronic form and that all
appointments have been kept for any preparatory visit. Accurately tracking
this information can help reduce risk.

Patient Safety

The ability to maintain freedom from accidental injury due to medical care or medical errors.

Splunk is a proven solution for medical device monitoring and supply chain monitoring. Battery life, fault rates and RFID information can be linked back to specific manufacturing lots leading to better
performance and better patient outcomes.

Quality Improvement

The ability to maintain a formal approach to the analysis of performance and the systematic efforts to improve it. QI is embedded in the culture of every aspect of HSD.

QI starts with data being able to be extracted from the systems and presented in ways that are meaningful for those trying to make QI. This requires statistical analysis and visualization to understand log
term trends in patient medical history. Fast ad-hoc groupings of  patient types based on condition, location, sex, age and other data allows for QI to occur.


The ability to detect disease in asymptomatic populations. DoD beneficiaries will receive health screening that has been demonstrated to be effective (reduces mortality, reduces morbidity and/or enhances quality of life).

Population modeling and disease detections in asymptomatic populations requires correlation of a variety of different
data types including in and outpatient visit data location of service, Periodic Health Assessment (PHA), mental / behavioral health, inpatient, extensive outpatient, telehealth, residential, sensitive records, etc. and other variables. The wide variety of data types calls for a data indexing system
that allows any type of data to be collected and expose correlations in the data.     

Health Counseling

The ability to provide patient education on preventive measures that have been demonstrated to be effective by reducing mortality, morbidity and/or enhance quality of life.

The system shall be able to push information to patients based on medication counseling, missing documents, checkups and missed appointments. The system can provide data to drive information to a patient healthcare portal.   

Community Health Education

The ability to provide any combination of learning experiences provided to DoD beneficiaries with the end goal of
attempting to bring about behavioral changes that improve or sustain an
optimal state of health. Community health education programs begin with a
needs assessment to identify population requirements and to determine whether a particular health education program is warranted and/or will be successful.

Periodic data collection based on electronic patient survey can be presented back to the patient indicating “health grades” for
the patient and compared to a peer group or groups. 


The ability to protect susceptible patients from communicable diseases by administration of a living modified agent, a suspension of killed organisms, a protein expressed in a heterologous organism, or an inactivated toxin in order to induce antibody formation. Military members will receive all DoD- mandated routine immunizations (currently Hepatitis A/B, tetanus-diphtheria, inactive polio virus, MMR, and seasonal influenza) and all required contingency and travel-related
immunizations (e.g. smallpox, anthrax, yellow fever, etc.).

Additionally, military members and other DoD healthcare beneficiaries will be offered all immunizations recommended (beyond those required by DoD) by the Advisory Committee of Immunization Practices (ACIP). These immunizations and any adverse events will be tracked and monitored. Future emerging/novel infectious disease threats to DoD forces may require rapid vaccine development and production capabilities beyond that which can be generated short-notice in the civilian sector. The DoD should have inherent capabilities that can be activated to meet this national security need.

Depending on immunization data that is not present in the system, the system can prompt a user to review data in a secure portal to validate current records. The data can be compared to the DoD Immunization
schedule with any modifications for country of deployment. Vaccination updates can be analyzed across different populations to target immunization messaging to specific populations by age, sex, race, or location down to a specific military installation or location.


Occupational Health Services

The ability to protect the safety, health, and welfare of the warfighter, civilian employees, and contractors in the
workplace. Occupational Health Services includes occupational medicine, occupational (or industrial) hygiene, public health, safety engineering, chemistry, health physics, ergonomics, toxicology, epidemiology, environmental health, industrial relations, public policy, sociology, and occupational health psychology.

On the job exposure to specific environmental factors can happen under any variety of situations. The key here is to know if the person that indicates they may have been exposed to a specific hazard was actually assigned as special duty personnel. Personnel that have not been given this designation and are exposed to hazards may require the DoD to report to OSHA. Splunk can correlate the patient’s visit, specific hazard exposure and
information about the patient to create an alert that the exposure needs to be communicated with OSHA.  

Public Health Lab Services

The ability to provide services to test and monitor the environment for specific health threats; assess the population’s health status; detect and track communicable diseases; and, support medical officers, preventive medicine staff and deployed Preventive Medicine (PM) units/forces as they investigate and control disease outbreaks. Public Health (PH) Laboratory Services also provide the ability to assist military preventive medicine, veterinarian specialists and public health officials in assuring the safety of food and water through provision of laboratory testing and analytical services.

Splunk can ingest data from a wide variety of sources and geo-locate outbreak clusters. Look-ups can be created to a wide variety of other data sources and can be included in reports. Examples are personnel rosters. Equipment location based on RFID data, inventories of vaccines and other drugs.  

Diagnostic Services

The ability to use various radiological techniques, mostly noninvasive, to
diagnose an array of medical conditions using x-rays, computed tomography (CT) scans, magnetic resonance imaging (MRI) scans, and ultrasound. This includes the ability to provide oral and maxillofacial imaging techniques (e.g., bitewing, peri-apical, and occlusal radiographs; ultrasound, cone beam CT, MRI) and special tests (e.g., sialograph) to help diagnose oral or maxillofacial conditions or disease.

Metadata can be collected identifying the who, when and where of radiology documents and patient groups analyzed for similar injuries/issues. 

Medical Transport

The ability to effectively coordinate and transport stabilized patients who
require special medical attention from one location to another.

Vehicle GPS data can be collected and monitored for real-time geographical views of en-route emergency vehicles.



The ability to provide the initial evaluation, diagnosis, treatment, and
disposition of any patient requiring expeditious medical, surgical, or
psychiatric care. Emergency services may be provided in a hospital-based or
freestanding emergency department (ED), in an urgent care clinic, in an
emergency medical response vehicle, or at a disaster site.


Assess to patient history data in Splunk on mobile devices can allow emergency service teams to review patient history looking for any possible reactions to drugs that may need to be administered as well as provide better information hand-offs between emergency services and admitting
hospitals with first responders and hospital admitting personnel having access to the same data sets with restrictions based on what which teams are allowed to see.

The table above represents a small sample of use cases for data driven healthcare decisions.

Splunk provides two very compelling solutions that can be applied to healthcare data in support of health information exchanges and electronic health record systems:

Hunk (Splunk Analytics on Hadoop) – provides the benefits of Splunk’s statistical analysis capabilities across extremely large datasets in Hadoop. Use cases for this data would be related to historical data analysis and the monitoring of ongoing trends. This cost effective solution supports non real-time use cases.

Splunk Enterprise – provides support for extremely large data sets where answers to questions in specific use cases are needed in near real-time. Emergency workers, first responders, and military theaters of operations should be using Splunk Enterprise for real-time insight.

Big data has a huge role to play in bringing together desperate healthcare data sources and types without the cost of upfront normalization, monitoring for proper usage of data, finding efficiencies that reduce cost and finding data relationships that can lead to better quality of care.


SavedURI :Show URLShow URLShow URLShow URLShow URLShow URLShow URLSavedURI :SavedURI :SavedURI :SavedURI :SavedURI :SavedURI :
SavedURI :Hide URLHide URLHide URLHide URLHide URLHide URLHide URLSavedURI :SavedURI :SavedURI :SavedURI :SavedURI :SavedURI :





Posted by


Show All Tags
Show Less Tags