Hadoop and Hanna Live in the Cloud with Sparkle and Hue
Nope, that’s not gibberish or the start of a children’s story. It’s the world we live in today. Many of you may be familiar with these terms, but I’d bet some of you aren’t. And, I don’t blame you. There is a tremendous amount of change going on in the information technology field everywhere and my field of healthcare is no exception. Two of the biggest changes happening globally are the use of “Big Data” and the move to the “Cloud.” It’s not coincidental the two are happening at the same time. While they don’t quite follow the adage of fitting “hand-in-glove,” they’re pretty close…more like “arm-in-sleeve.”
Big Data requires a couple of things. First, Big Data needs a place to live—or storage hardware—and lots of it. Storage has become pretty affordable over the last several years, enough so that there’s really not much thought given to purchasing more storage when you’re getting close to running out of it. However, Big Data requires so much storage that even in today’s world, it makes you think twice about purchasing enough to meet those requirements. Then, you have to give that storage a room to reside, power to run, and cooling so it doesn’t melt itself down. It’s because of the latter you find many newer data centers springing up where very reasonably priced power options exist: Quincy, Wash. and San Antonio, Texas, for example.
The second thing Big Data needs is “compute.” A lot of applications require compute, but Big Data requires exponentially more compute because of the sheer volume and velocity of the data. To get to that amount of compute, companies have to purchase lots and lots of—you guessed it—computers. Or, in this case, servers that have multiple computers inside them and are strung together to make one gigantic server capable of attacking Big Data at the volume and velocity it’s arriving. Those kind of servers aren’t cheap, and you have the exact same space, power, and heating and cooling issues you had with storage.
So, what’s a person to do about these two challenges? Move to the Cloud, move to the Cloud! Oh, by the way, “The Cloud” simply is a bunch of these newer data centers in affordably priced power places connected together through the either private or public (internet) networks. Dang, took the mystique out of that one, right?
Instead of trying to buy enough storage to hold your big data, simply pipe that data to a “Cloud Provider,” and let them hold it for you. The great thing about these Cloud providers is that they usually have some super compute along with storage. That means you can smash the big data you stored in the Cloud against some of the super compute already living in the Cloud and collect some great output that will help in your various endeavors. Of course, it’s critical to ensure the Cloud can also keep data secure and protect privacy – an essential element for healthcare, as well as for many other industries.
“Two of the biggest changes happening globally are the use of “Big Data” and the move to the ‘Cloud’”
There are some idiosyncrasies about the Cloud that you should know. The way you pay for “Cloud services” is unique. First, putting data, big or small, into the Cloud is very financially friendly. It doesn’t cost Cloud providers that much to keep data storage, but they charge considerably more when you move data out of the Cloud. But, and here’s the cool thing, you only have to pay for compute by the minute in the Cloud. So you simply conduct the analysis of your data that’s in Cloud while it’s still there and only bring back the results of the analysis. All around it’s pretty great deal. Now you can see how a couple of folks in a garage can build something like Uber.
The products/ways used for storage, and more importantly retrieving, the big data you sent to the Cloud and smashing that big data against compute are Hadoop and Hana. One of them, Hadoop, is super-duper fast, which it has to be for big data; the other, Hana, is extra super, super-duper fast – for those really inpatient folks. Most of the time, Hadoop stores on a spinning disk, while Hana uses in-memory storage. The tools used to help with the analysis and visualization of the data are Hue and Sparkle.
I hope the Cloud has become much less mysterious to you and that now you can see why “Hadoop and Hana live in the Cloud with Sparkle and Hue” isn’t a fairy tale – it’s a 21st Century mainstay for business.