This is amazing. I think the quantified self movement is going to be huge. With more personal tracking gadgets (fitbit, jawbone, nike fuel), measurements are going to become more and more seamless, passive, and complete. Not only will you get historical insights into your blind habits, but you can finally have an objective feedback loop on your behavior, and make necessary adjustments.
But one of the biggest challenges is going to be privacy...
Privacy will be one challenge, but I think seamlessness (as you mentioned) is currently the bigger challenge.
I'm a runner, so diet is important. For 9 months I tracked everything I ate. Weighed portions, counted calories, logged everything. The data was insightful. I learned that I need to eat 3,300 calories a day to maintain my weight, with a range of +/- 300 calories.
After 9 months I stopped tracking things because it became time-consuming. Partly was because I learned what it "felt" like to be full, but mostly because tracking it all was really annoying. If all of that could be automated somehow, I'd love to have a spreadsheet of data which I could graph and chart. "At 8:16am, you drank 7.8oz of Orange Juice, totalling 108 calories." Cool, my O.J. container and glass measured that automatically, and uploaded it to my database!
But right now doing it is extremely manual. All of Wolfram's amazing data comes from some sort of automated system he setup once and never worried about. A pedometer. A script chewing through 20 years of archived emails. That form of automation doesn't exist for many types of data people would like to collect, such as food intake, or blood sugar testing for diabetics.
Inventing these non-invasive automated sensors is key for society-wide personal data collecting. Otherwise most data gathering will be sentenced to the dedicated few, rather than the busy many.
My grandparents are diabetic, and I've long imagined an implant that could measure blood sugar levels and report it back when they sit down at the computer or pick up a device. NFC is used in subdermal implants today, it could be done. My grandfather has a pacemaker, perhaps it could be a combined function. The problem would be getting blood samples without actually causing damage to vessels, but I am no medical researcher so I don't know all the hurdles.
I agree with you that measuring is the most intensive task of tracking personal statistics. I can enter how many servings I had of what in any fitness tracker website, but what if I had 1.3 servings? What if I had .2 servings (just a taste)? How do you estimate that without measuring it first? And if you're measuring everything on a scale, where has the fun gone in life? What's the difference between medium intensity running vs high or low? If I biked at 15mph it might be medium, but what if I kept that 15mph steady up steep hills? Or what if I kept 15mph down a hill without pedaling? We're built for rough estimates and are amazingly adept at convincing ourselves we're right.
I have enough difficulty with time tracking at work either breaking my productivity or cutting into my limited "me" time, and that's just a petty little thing. I usually end up estimating at the end of the week then fixing discrepancies that always come up.
woah, "NFC is used in subdermal implants", is that already being done? I've googled and seen it as something on the agenda, but have no idea if people are already doing it?
Well, if you define RFID as a near-field communication. This could be a matter of debate, I guess. Obviously it's well-known for having your pets "microchipped", but there's also this:
At 8:16am, you drank 7.8oz of Orange Juice, totalling 108 calories.
It just occurred to me that the new breed of payment services (Square, Google Wallet etc) should combine with food vendors to offer this service.
In Australia at least fast food restaurants publish the calorie count of each food option. It wouldn't be impossible for the payment company to do the correlation.
Also sports (and weigh loss) food vendors should embed NFC chips in their packaging so you can scan them before you consume them. There are plenty of cross-promotion possibilities there...
Vendors are already tracking all my itemized purchases and sharing with every advertising company in existence, it would be nice if they would share the data with me, too.
> That form of automation doesn't exist for many types of data people would like to collect, such as food intake, or blood sugar testing for diabetics.
When I was looking into it, I noticed that a lot of the blood sugar testers were electronic and could record the series of readings for you. You still had to prick yourself, but it looked pretty easy.
(Sadly, the one non-invasive blood sugar recording device I found was apparently discontinued for causing electrical burns.)
It probably wouldn't be too tough to make an arm-band device which periodically pricks you, possibly beeping a few seconds before it happens. I'm not sure when diabetics need to calculate glucose levels, if it's either after each meal, or periodically throughout the day.
I would say that this 9 month period was important because it probably trained you to intuitively know how much to eat. I went through a similar process 5 years ago: I had to monitor my vitamin K intake so I grabbed the USDA nutritional database, wrote up a simple little web app for my own use (made public at cookingspace.com) and after using my little app for about 6 months, I had trained myself to intuitively know how much vitamin K I was consuming every day.
It is pretty amazing, but in a way, Wolfram is also demonstrating how not to do Quantified Self: the point is not stamp-collecting, but data-collecting + experimentation + analysis. Right now, Wolfram has maybe #1 and #3, but he doesn't have any of #2, and he doesn't seem to really appreciate this:
> And as I think about it all, I suppose my greatest regret is that I did not start collecting more data earlier.
Hopefully he can change that. For example, his observation about the 7% use of the backspace key is an obvious one. If he eliminated backspace entirely, he could speed up typing by as much as that much - nontrivial given how much time he apparently spends on email! An experiment would be using some typing tutor program for a few months and seeing whether the keylogging notes a decrease in backspace.
Good point. When I saw his email volume, I wondered how I would feel looking about at an entire decade and seeing how much time I spent on emails. I'd probably feel a lot of regret. But like you're saying the point of QS shouldn't be regret (or novelty), but identifying a 'problem' and fixing it asap.
Do they actually work for coder-style usage? Would I actually be faster in Emacs if I was constantly looking to see what my keyboard had been redefined to this time?
Intellisense/autocomplete in Eclipse and Vim is the one are of desktop computing where I do get assistance, and it works great. It's not autocorrect, but it's two characters and then tab of crtl-space or whatever and tab or arrow key to the word of my choice.
I've been doing it in bits for a few years, mostly because I'm a sucker for interesting-looking graphs of data, but imo it's a mixed blessing. I've definitely found interesting trends, but measuring is also something I'm actively conscious of, and, as is often the case, it's hard to measure things without the very fact of measurement changing them.
I find myself struggling to avoid falling into the trap of doing or avoiding-to-do things specifically so that they'll show up in the analytics in a certain way that looks nice, which feels like a weird "teaching to the test" but applied to my actual life. Some things I've since stopped tracking because I really disliked the meta-ness of it all.
Motivation is also another big challenge. Very few people care about their long-term health, and thus very few will care about tracking their health stats.
But one of the biggest challenges is going to be privacy...