Using Hadoop to Change the Game in Healthcare
At its current growth rate, the healthcare system in the U.S. will soon comprise zettabytes of data this is Big Data on a massive and highly complex scale. Traditional data management tools are simply incapable of managing this data, and that’s where Hadoop comes in. For us, and for the industry as a whole, the Hadoop ecosystem has been transformative.
"Hadoop's ecosystem of technologies has given Inovalon a distinct advantage in bringing healthcare into the age of Big Data"
Across the healthcare sector, data is captured from many different sources. Much of the data associated with population health remains in disparate silos in various formats and it is interchanged and processed without automation. And, in cases where investments have been made to digitize health-related data, the result has been ‘‘walled gardens’’ where the data is static and is not easily shared or analyzed. We have developed our Big Data technology platform to address these challenges.
While scaled data integration and processing have traditionally represented barriers for many organizations in the healthcare industry, Inovalon has relied on Hadoop, Spark and a rapidly evolving ecosystem of technology and tools to capture seamless integration and high velocity computation as true differentiators. Our platform enables advanced intelligent data integration collecting, transforming, cleansing, managing and analyzing data of any type, from any source, on any platform, in any format at extremely high speeds.
Our previous data integration methods required significant time for loading, pre-processing, analyzing, and post-processing data. Client specific SQL scripts and pre-defined, static schemas dictated that only a subset of the data would be usable for analytics and non-mapped data would have to be discarded. In addition, data topography and tracking, when feasible, was extremely time consuming, expensive and cumbersome. Finally, conventional relational database management system (RDBMS) architecture is not linearly scalable and adversely affects mass data storage and the costs of required hardware and licenses were becoming prohibitive.
Understanding these limitations, Inovalon became an early adopter of Hadoop, which offers tremendous value for our data integration and distribution framework. It enables dynamic schema definition directly against raw data, thereby eliminating product and client-specific ETL scripts, making data available more quickly and more efficiently. The dynamic schemas ensure that all available data can be stored and consumed without having to throwaway unmapped data. Hadoop’s integrated data tracking functionality provides the lineage of every field, significantly increasing and improving the quality, depth and accuracy of data.
In addition, Hadoop’s distributed storage and processing capabilities translated into significant speed improvements. Inovalon’s pilot implementation of Hadoop accelerated data integration,resulting in a 94 percent improvement.
Adaption of Hadoop technologies is also what enabled Inovalon to partner with Quest Diagnostics to create our flagship product, Data Diagnostics™, an industry breakthrough that draws on two of the world’s largest datasets of clinical, claims, quality and laboratory data to deliver on-demand analytics and intervention strategies at the point of care. The solution brings the promise of Big Data analytics to reality empowering healthcare providers to make informed decisions and offer higher quality care by providing up-to-the-minute information and insights on an individual patient level.
Today you can buy a coffee using your credit card and it’s immediately debited to your account. The transaction is nearly instantaneous. We should have the same expectation for healthcare. Thanks to Hadoop and its ecosystem of tools, which includes Spark (another technology our platform leverages), we’re approaching this level of immediacy. When a clinician orders Data Diagnostics™ at the point of care, there’s confidence that the data is current and that the analytics are therefore actionable.
If healthcare reform is going to work with its focus on value-based care to replace fee-based care providers, payers and the entire system must rely on data more than ever. Hadoop’s ecosystem of technologies has given Inovalon a distinct advantage in bringing healthcare into the age of Big Data. As the ecosystem matures, we’ll continue to leverage it in new ways so that our customers across the healthcare landscape can deliver better patient outcomes for diminishing costs to us all.