[p2p-research] building a knowledge commons, the p2p way --> feedback from Natalie

[Dan Brickley]

On Sun, Sep 5, 2010 at 8:15 PM, Kevin Carson
<free.market.anticapitalist at gmail.com> wrote:
> On 9/3/10, Natalie Pang (Asst Prof) <NLSPANG at ntu.edu.sg> wrote:
>
>> I share your sentiments about Twitter – to the point that I raise my eyebrows whenever I come across any research project aspiring (or claiming) to mine knowledge from Twitter.
>
> IMO the value of Twitter can be pretty well summed up by the fact that
> every single news item about Sarah Palin comes from her Tweeting or
> updating her Facebook page.

Re twitter mining, I'd urge folks to reconsider. Even the connectivity
patterns (who you follow, who follows you) on twitter can be
incredibly revealing, since they are correlated with other aspects
(taste, preferences, political opinion, ...).

The company Hunch.com are building a business around this stuff, and
have some interesting services and statistics already. Hunch has its
own growing database of opinions, but large enough to cross-reference
with Twitter data.

See http://twitter.com/hunch

eg
Profile of @hunch users who like The Coen Brothers film directors:
http://hun.ch/8Y3g51

This is a hunch-centric demo, not twitter, but to excerpt from
http://hunch.com/explore/result-correlations/1760663/

"People who liked Coen brothers were:
Much more likely to answer…
"Do you like Andy Warhol's art?"   with "Yes" (chart)
"Do you tend to support liberal or conservative politicians?" with
"Liberal" (chart)
"Does one of these people represent your general voice of reason?"
with  "Jon Stewart" (chart)
"Do you support the death penalty?" with "No" (chart)
"What did you think of the movie Napoleon Dynamite?"   with "Loved it" (chart)"

To this kind of data they add Twitter links, hence

http://hunch.com/goodies/

"Twitter Follower Stats
What does who you follow on Twitter say about you?
Learn Interesting Stats about people who follow famous Twitter users."

Here's the analysis of followers of Sarah Palin,
http://hunch.com/twitter-followers/sarahpalinusa/

(the Q/A is from Hunch multiple choice questions)

"When Hunch asks this / @sarahpalinusa followers are more likely to answer:

Q: Do you tend to support liberal or conservative politicians?
A: Conservative

Q: How do you feel about Walmart?
A: Walmart is a great way to save money and I'd welcome one within
driving distance of my home

Q: Do you think standards should change from generation to generation,
or stay the same?
A: They should stay. What worked for me will work for today's children.


By contrast, see statistically salient quirks of followers of eg.
Stephen Fry, http://hunch.com/twitter-followers/stephenfry/

There is also a "Twitter predictor" game which will estimate answers
for such questions for you, given a twitter account ID. Apparently it
only uses who-follows-who information, and ignores text, links, tags
etc.

http://hunch.com/games/twitter-predictor/

This is built on a decent sized dataset now, http://blog.hunch.com/?p=20404
"Hunch’s ‘taste graph’ now exceeds 10 billion connections ...  There
are currently more than 60 million distinct “items” in the taste
graph.  An item can be a person, a place, an opinion, a movie, a pair
of shoes…or just about anything that you might either agree with,
disagree with, like or dislike. ... There are more than 10 BILLION
connections (proper math terminology would denote them “edges”) in
Hunch’s taste graph. An example of a connection/edge is the fact that
a particular person likes the Honda Civic, or whether someone happens
to be a vegetarian. It’s these edges that give Hunch the ability to
make smart predictions even when it only knows a tiny bit about
someone."

So Twitter itself might be shallow, facile or whatever. But mining the
connectivity graph, especially when bridged to other datasets (like
Hunch, or Facebook, or other link-sharing networks) is well worth some
attention...

cheers,

Dan