Sunday, May 29, 2011

More on detection of human sentiments


There are several works that attempt to use social networks to detect human sentiments.  In a recent article, John Bates (Progress Software CTO and one of the founding fathers of the event processing discipline and community) reacts to some claims that using Twitter one can predicts trends in the stock market with high accuracy.   While the article deals with stock market applications, I'll give some comments  within a broader context of detecting human sentiments in any domain.  I'll  quote John's claims in italics and different color and mine in this font and color.


John states three problems with that assumption:

  1. Social networks are not secure, thus the results may not be accurate, and may also be malicious or just wrong.  His example is that somebody twits about the fact that USA started a war on France. If somebody uses it to effect the stock investment, or even travel plans, the results may not be that good. I agree that this is a major issue.
  2. Social networks are unlikely to contribute anything that is not being disclosed in mainstream news. I am not sure that I totally agree with this one,  sometimes individuals on twitter or facebook reveal news before the mainstream media detect it, we saw some examples recently all over the world.
  3. The time it takes to process the sentiments may make the results obsolete, since the reality may move faster. I guess that it might be true in some cases, but with real-time analytics, it may be good enough in other cases. 


Overall, the first problem stated is the most serious one.  Perhaps when there will be some validation mechanism to social networks postings, then it can be more reliable.  One may claim that if we can assume that most posts on social networks are correct, then there can be noise removal filter.  But still social networks may not be reliable sources (BTW - mainstream media news may not be accurate either!).


It should be noted that I've learned about John's article since he twitted about it on Twitter.  So maybe the fact he wrote this article by itself is not accurate?     More on this -later.

No comments: