Is there any insight as to how this works? I got an Openness rating of 97% and a Harmony rating of 100%, both of which I know are not true. (I also received a Love rating of 1% under my Needs, although that's pretty accurate.)
I, too, got a "Love" rating in the low single digits. That's probably not too far from the truth in a global sense, but remember that in this case, the API is working with a local context (HN comments). I'm probably unlikely to have discussed the subject of love on HN, or anything tangentially related to it, or anything that would somehow give an indication of my need for love. Now, if you were to run my Facebook history through the same process, I imagine you'd find a slightly different analysis.
To some extent, our personalities are our personalities. We behave, on some level, the same in every context and in every community. But the extent to which that's the case is up for debate. We probably use different approaches, or if you prefer, we show different aspects of our personalities, in different contexts and in front of different groups. That's why it's extremely difficult to take one context (HN, for instance), and extrapolate universal characteristics from it.
Even within the set of HN history, I got some oddball results. For instance, Watson considers me very "Fiery" (51%) here. There are plenty of areas in my life in which the word "Fiery" makes a bit of sense. HN isn't one of them.
A problem I've observed with these kind of blackbox systems is that the process from input to output really is a mystery.
When the results are right, they're just "right" so you should accept them, when they're wrong they're actually also right by whatever magical hamster wheel is operating inside of the thing and you just don't "get it".
The problem is that humans like to have some clue as to how the results were derived, something easy to explain that gets the gist across. Something like "Watson counted all the words you use and compared them to different reference lexicons to arrive at the score". This provides a little bit of context so we understand the semantics of the result and how to consider them and reason with them.
But for all we know the results we're seeing are from some arbitrary stochastic method:
openness=rand(90,99)
harmony=rand(90,100)
etc.
For things like this to be accepted by the users (humans) there needs to be a quick explanation for how this works otherwise we get head scratchers.
We plan to add more information to our docs soon about the service, including a description of each of the traits, and possibly reference some of the many data sources used.
Meanwhile, you can do a search on "IBM System U" (the project's not-so-internal code name.) This particular slideshare.net prez has some great info on the methodology, validation tests and references: http://slidesha.re/1ri0vPV
I also get rather high Openess and harmony scores, though I'm particularly amused that Hedonism is scored. How can I be 72% sympathetic and 47% coopoerative but not agreeable?
The IBM documentation doesn't really say anything about how these numbers are calculated
