In an earlier blog post titled, “Use the Four V’s to Better Understand the Big Data Ecosystem,” I discussed the concepts of volume, velocity, variety and variability that represent the measurable dimensions of big data. I then reviewed some research on how the various tools that make up the big data “ecosystem” address these dimensions. Further vetting of these ideas has helped to fuel discussions about the role of enterprise search in addressing big data with customers, partners, analysts and a number of big data practitioners I met at the recent Strata conference in Santa Clara, California. One of the key takeaways of this research is the real-time element that search can add to a big data deployment—more on that later. As promised in my initial post, I developed the topic of search and the value it can bring to big data into a Vivisimo White Paper titled, Optimizing Big Data.

(more…)