The Good, the Ugly and the Bad Situation Awareness in the Big Data: A Cognitive Architecture for Social Forecasting

1. Introduction

The Good, the Ugly and the Bad¹ is the Italian Spaghetti Western film directed by Sergio Leone in 1966, starred by Clint Eastwood (The Good), Lee Van Cleef (the Bad), and Eli Wallach (the Ugly). The plot rolls around three gunslingers that, while participating in many battles along the way, race each other to search luck in a cache of gold. The film’s title has entered the English language as an idiomatic expression². The respective terms refer to upsides (The Good), down- sides (the Bad) and the parts that could, or should have been done better, but were not (the Ugly).

This paper tells a similar story, which respects the same plot, but with three different protagonists. The Good represents the situation awareness (hereafter SA), a challenging approach that aims to realize artifacts being aware of what happens in the environment around them. Just like upside, SA has a view, from above, seeing all directions: collecting sensing data, analyzing them, and forecasting the near future using these data. The Ugly counterpart of this ideal situation, which the artifact wants be aware of, are big data that usually include datasets whose magnitude³ is far away of being processed by traditional techniques and that, as such, asks for methodologies able to integrate evidence from datasets of different nature, in order to automatically formulate hypotheses/models/theories that are able to predict and/or, at least, forecast, the near future. At this point of the story, it appears on the scene the Bad guy, our cognitive inspired framework. The Bad is that, since he even knowing about the ideal situation, represented by The Good, he realizes the presence of the Ugly and, well, faces a decision, even knowing that it will not be the best, but it is still important in order to mine nuggets from the buried cache of gold. As said above, he could have accomplished better his task but he didn’t. Anyway, in the light of the “The Good” inspiration, he is always available to question his believes, and as soon as the Ugly guy offers him new evidence, through new datasets, the Bad updates his attitude. In our case, the Bad will infer patterns from the data who have no claim to be the best (i.e. they are not predictive models), but they would only work as guideposts to follow at the next junctions; in other words, decisions, based upon these emerging pattern/regularities, aim only at forecasting the near future.

Section 2 introduces, more closely, the three protagonists of our story. It provides only one point of view, that is our, and, as such, the treatment is part of and not at all exhaustive. Section 2 reports on a strict and relevant related work. Otherwise, the literature on big data, and the one on techniques do deal with them, would be practically exterminated. Case studies reported are the most recent and similar to ours because of their datasets sources (i.e. Twitter), and the similar approach they employ to deal with them. Section 3 describes the overall architecture of our proposal. The goal was to have a framework, that can be easily adapted as soon as he became necessary to formulate and test new hypotheses in order to explore new domains. Section 4 reports on the framework when it is action. The modularity of the architecture allows to integrate evidence from datasets of different nature and formulate hypotheses on different domains. In this Section we show some simulations coming from USA primary elections. We are interested to know social opinions about candidates. The starting point is represented by Google Trends: then, based on them, we mine opinions on Twitter. The framework is general purpose oriented. We tuned it in order to mine opinions also for other tasks (e.g. the launch of new products, and so forth). Our objective, with these experiments, is to explicit under opinion form the information of the query that have an increasing trend in a given period of time. Otherwise, thanks to Google Trends, they would be available only searches’ peaks, which, although important to detect people interests, they say nothing regarding users’ sentiment about these trends. The opinion so extracted is then translated in a visual form in order to speed up the decision process.