Tag Archives: data set

Big Data done……. but what’s next ?

Today big data, is considered to be yesterday’s news as most successful companies have already moved on with it. But what’s next is more important and for these global or progressive organizations.

Few of the major advancements are data integration(automated machine learning or AI),  big little data (basically the tremendous proliferation of small data sets across the web) & video data. The emergence of these are a result of the limitations that big data has as it has been successful in finding trends & correlations across large dimension like population, GDP, timelines, etc. But big data itself is formed with small fractions of data and each fraction has a significant impact on the overall output. 

Data integration is also currently happening with the help of technology and combined with big little data, it will throw insights that may change the way we see at information and decision-making. For e.g. models based on not just industry data but sentiment data, inflation, demographics or any other predictive models which may seem unrelated.

But the transition to big little data isn’t going to be easy, as there is tons of information stored in HTML tables across the web that may look ambiguous but will have contextual relevance. Leading this phase are companies like Google & University of Michigan who are looking at these smaller sets and how to make sense of them, so that it can be made user-friendly.

Data integration has been spoken about for sometime but currently its limited due to human intervention. Its needed since massive data sets need a human mind to figure common aspects between otherwise 2 unrelated pieces of data. Currently the algorithms can relate 2 sets of known data sets which are linked but cleaning the data, resolving issues with data and then finding a pattern by its own is still a challenge which can be tackled only by a beautiful mind now. Although there are noticeable breakthroughs achieved this by crowd-sourced data repositories but there is still some way to go. Yet one of the biggest sources of big little data will  be the smart phone that houses tons of data which isn’t available out in the market once that comes out data integration will reach another level.

 Now the most interesting one – Video data. This actually means extracting streams of data from video. Today world is moving towards video and everywhere videos have taken precedence over pictorial data and this trends has just started and growing. This make big data seem like big little data tomorrow.

All 3 are game changing innovations that are currently worked upon and highly anticipated, just cant wait to see how they change the way we see the world.