The title is misleading ... this is what the real finding is
"But instead, the performance was largely similar, except when it came to the timing of events in the story. "The Kindle readers performed significantly worse on the plot reconstruction measure, ie, when they were asked to place 14 events in the correct order."
I wouldn't have expected this result necessarily but it is interesting how with eReaders, you completely lose that spatial memory that you get from reading traditional paper bound books. I bet you could approximate where a passage you remember is without knowing the page # after reading a traditional book. Of course, eReaders have the advantage of being able to search for text so finding passages isn't really a problem.
I would love it if eBooks supported custom, programmable side-bars, one where the layout designer could add a "map" of the book that was more unique and memorable than the percentage tracker of, e.g. the eInk Kindle.
A significant strength of Kindle is its distraction-free nature. While a progress-bar might be good, I don't think anything more complex than simple line would be suitable. Also its natural place would imho be the bottom edge.
I thought I agreed (and even disabled said progress bar a couple weeks ago). However, the research suggests that there's a counter-productive degree of "distraction free" and that some eReader interfaces pass that threshold.
As a side note: I know that my expectations for a book change depending on my current position. I notice this every time I think I have a large amount of the story left, only to find myself unpleasantly surprised when it ends "abruptly" and that the next half is actually another story bundled with the first in a single file.
The Kindle had one. One of the things I miss from upgrading from a Kindle Keyboard to a 2013 Paperwhite, where I can't figure out any way to get the bottom progressbar back.
Possibly for later novels. The one I have in mind (Planet Oz) keeps the focal characters in close proximity, at least compared to the scale of their travels, so a geographic map could suffice.
"In most respects, there was no significant difference between the Kindle readers and the paper readers: the emotional measures were roughly the same, and both groups of readers responded almost equally to questions dealing with the setting of the story, the characters and other plot details. But, the Kindle readers scored significantly lower on questions about when events in the story occurred. They also performed almost twice as poorly when asked to arrange 14 plot points in the correct sequence."
So, using a battery of test questions divided into five subjects, INCLUDING TIME AND EVENTS, there was no significant difference between the two groups. But on an entirely different test methodology, the Kindle readers scored significantly worse than the paper readers on plot orders. So they go, "Well, we have a battery of results that shows no difference, and one result that shows a difference. Let's backfit an explanation to the one interesting result!" It's that XKCD strip about jelly beans.
> So, using a battery of test questions divided into five subjects, INCLUDING TIME AND EVENTS, there was no significant difference between the two groups
It says right in the part you quoted that the Kindle readers scored significantly lower on the event time questions.
They also did poorly on the separate plot ordering test.
I fail to see any problem with this, since I would expect the to be good correlation between how well people do on questions about event timing in the story and how well they can order a timeline of the plot.
I think parent is pointing out that there could be a problem of "mutliple testing" in the statistics used. If you do enough tests for statistical significance, and do not correct for multiple testing, then some fraction (your cutoff, typically .05) will be expected to test positive even when nothing is happening (null hypothesis holds).
One study with a grand total of 50 subjects and we're already saying that readers absorb less on Kindles? While interesting, the headline is click-bait and is not something we can say without further study.
(Note: They don't link to the study in the article so naturally I cannot comment on the soundness of the study in question).
(EDIT After writing this comment the link was changed from the Guardian to the NYT, which provides more information, though the study has still not been published.)
Studies (especially psychological ones) with human subjects are very difficult and expensive to conduct, which is why sample sizes are often small. 50 is by no means an unusual size for an experiment of this sort.
An alternative is to use observational data, but it's very hard to differentiate between useful observational data and garbage observational data. Not only do you introduce a whole host of problems, but it's more difficult to parametrize the problems that are introduced. So you could easily create a study that boasts a large sample size, with a respectable p-value[0], but have no way of knowing which confounding variables were introduced during the data collection process.
A third option is self-reported data, which comes with an even bigger asterisk after it. For something like this, I'd much rather trust a controlled study of 50 than a self-reported survey of 300 (at that point you might as well post it as an 'Ask HN' and judge based on the comments!).
By contrast, while controlled experiments on human subjects are by no means unbiased or immune to confounding of variables during data collection, it's almost always easier either to limit these in a controlled setting or at least to parametrize them after they happen.
So in the end, this usually ends up being the best feasible option (not the best (theoretically) possible option), short of massively increasing funding to such studies.
[0] Which is usually the wrong way to look at studies anyway, but that's a separate topic of discussion
Something did ping my radar, although it's hard to say because it's not published yet. What the news article says is:
But instead, the performance was largely similar, except when it came to the timing of events in the story. "The Kindle readers performed significantly worse on the plot reconstruction measure, ie, when they were asked to place 14 events in the correct order."
What I would like to know is: how many other performance measures did they test? How "significant" is "significantly worse"? If, say, they tested for 100 performance measures (unlikely, but I'm using a large number on purpose), then random chance means that there are likely to be some measures that are "significantly worse." If, on the other hand, they only tested 3 performance measures, then it's less likely to be random chance.
Basically, if you run an experiment and you test for a large number of things, you can't say much about the outliers. With large enough numbers, there are bound to be outliers. However, after you run such experiments, and you see those outliers, you can run more experiments to test if that was random chance, or if there really is some correlation there.
While the XKCD comic has a lot of truth to it, it's mainly about many different individual experiments (as well as some poorly done ones.) When running large sets of correlations, standard operating procedure is to use one of several techniques to counteract this effect.
The Guardian article linked actually doesn't present the statistics of the study, which hasn't been published yet. Absent further information, critiquing the sample size sounds pretty reasonable to me.
Withholding evidence isn't a defense against criticism. If you won't TELL ME your effect size, but you do tell me the sample size, I can certainly say, "I am skeptical of your conclusion, because of your sample size."
I don't buy it, right off. Seems just as likely as people that read more recall less. Especially at the levels that people read more when they get an ebook.
More likely, actually, is people that are used to reading paper books don't read as well in a new format. Also consider, many more people are reading "mystery" novels than did in the past. I know I would likely have never picked up a mystery myself, yet I have picked up a few on digital sales.
While that may be a true factor, it's still undoubted that the tactile feedback is much more vibrant, and engrossing than digital.
Physical books are cognitively superior in that they allow a reader to firmly establish the progress of narrative through physical feedback. The very act of turning a page, and accumulating each page as physical entities allows a user to see the accumulative progress of a story continuously. In that you can physically see the beginning, and the end, the only obstacle is familiarizing one's self with the contents of the book.
Digital reading however is totally abstract. Chronology, and pagination, beginning, and ending, don't really exist, but are completely an abstract object established by some magical forming of pixels. Every time you "turn a page" there's nothing actually happening, save for the movement of pixels, there's no visual, or physical feedback, no interactive feedback, at least not as in depth of a feedback as physical books provide. And as encoding only enhances with increased sensory input, I would say that digital does comparatively have a decreased sensory input, and therefore decreased ability to encode.
So perhaps the missing link is the vast array of cognitive feedback that one receives when physically turning pages, that is sacrificed in digital reading.
Another possibility is that a reader is more distracted when reading digital books, due to the multiplicity of tasks one can actively partake using computers. Since there are many tasks, a reader could actively seek out, or remember other tasks that are more enticing, or urgent. In comparison there's only one thing you can do with a book. That does not take into to account the physical surrounding of a reader, as that also may have a profound effect. This is only conjecture.
I think you raise good points, but what if we look at it from a perspective where paper books are not the norm:
> it's still undoubted that the tactile feedback is much more vibrant, and engrossing than digital.
If we look for book-like feedback in digital, we'll be disappointed. However, consider that many digital works, such as web pages, games, etc. provide much more feedback than books, and digital feedback can be fine-tuned for the story being told much more than paper books' interface.
> Physical books are cognitively superior in that they allow a reader to firmly establish the progress of narrative through physical feedback. ... Digital reading however is totally abstract
Again if books are the norm, then of course they meet that standard better than digital. Perhaps a more abstract experience is better, or better in some cases and worse in others. Maybe authors unconsciously have written for books and will write differently for digital. If digital had come first and was the norm, I think we would say that books have too much feedback, and they distract from the elements in the text that tell the story.
See, this is where I think you have over thought it. I think it is just as likely that people read their ebooks in more places and more "on the run" than they do physical books. In large part because you can.
Now, I fully agree with the "more distracted" portion. Especially if we are considering such reading devices as iPads or phones. Reading from my paperwhite, though, I would be surprised if this is much different than reading a paperback. Other than that I am now reading more than I used to. (Seriously, I got how many books from the humble bundle?)
Also... just because I "don't buy it" doesn't mean I'm right. :) More research will show more data that should help actually answer these questions. I'm just saying I don't expect more research to necessarily confirm this one.
Yup! Not saying you were right or wrong, just throwing ideas into the fray, and seeing what other people think of it. Aren't discussions amazing? :) Anecdotal consensus while not always accurate is often a cheaper, and quicker method of determining the accuracy of certain ideas.
I can agree with people being on the run more often due to the portability of digital devices, but still don't see why you disagree with physical books simply having a greater array of sensory input.
I disagreed with the first point you made, so I think I just had the knee jerk reaction to the whole post.
My disagreement is in most of the sensory input being irrelevant. I can see the appeal at one level, but I also know I find reading from paper bound books borderline annoying if they don't have lay flat bindings. Most of my sensory input is how to get the previous page out of the way without damaging it. A distraction to what I'm reading.
Sadly, I don't have hard numbers here. Just going with sources such as this[1]. Basically, the fiction market is growing by about 40% currently. To my (admittedly layman) eyes, that is pretty impressive growth.
Right. I am not saying that these folks did bad research. I just don't expect the findings to stand a long test of time. And, just because I don't think so, doesn't mean I'm right. :)
I real incessantly with Kindle on an iPhone 5S ... so this is really relevant for me. Thoughts:
1. The worse temporal understanding could be a problem for history and bio books especially. I find myself making a quick diversion to Wikipedia quite often for additional context and information. For non-fiction you can quite easily pull up timelines etc. if you're having trouble remembering. For some reason on the iPhone I can do a quick web search without getting distracted, which is a problem for me if I try to read on my laptop.
2. Interesting to know the "reading history" of the participants. I've been reading paper books since I was quite young, I couldn't be any more comfortable with the medium. I only got comfortable with eBooks after powering through the entire ASOIAF (i.e. Game of Thrones) series on a phone screen. I find that after being acclimatized it is much easier to fully focus on the material.
3. Using the dedicated hardware (Kindle DXes) was probably quite helpful to the performance of that group. It is important that page turns etc. are fast and there is no resistance of any kind to movement through the book. I might quite naturally take short breaks during reading but having them imposed on me or any other kind of focus break would be intolerable.
These are some really interesting results. As someone who has almost completely converted to digital books, I would be really interested to see these results used to inspire better UI design on the part of people like Kindle. I mostly use the Kindle Android app, and it's getting better and better, but there's still a lot of room for improvement.
There was another article posted here about this the other day, with more detailed information than this one (not much discussion though): https://news.ycombinator.com/item?id=8186517
I made the other submission and it's actually the exact same article, there is just a small difference in the URL. Interesting to see how this took off when the other one did not.
> "the performance [of paper vs e-reader] was largely similar, except when it came to the timing of events in the story."
Without seeing the paper, we don't know what measures were investigated and on how many points the e-reader experience was as effective as the paper experience. This article seems to be picking out one metric where there was a difference, but we don't know how large the absolute value of the difference is.
Sure would be nice to be able to read the paper and comment on the actual work.
[EDIT: this comment was in response to the original link to the Guardian article, since changed to link to a better NYT article.]
I'd like to also see a comparison with reading vs audiobook, (and maybe vs being read to by a live person). That would also help address their hypothesis about the feedback of the book pages.
Agreed -- my own personal experience with this sort of phenomenon is via chess.
I am a much stronger player over the board than I am online and I think the visual representation/spatial relationship is a large part of it. I just engage in a different way.
While different visual representations may seem equivalent on some level, the way our brains process them may be very, very different.
I would imagine many of us also experience a similar phenomenon via writing by hand vs. typing.
I am suddenly inspired to write our head UX guy a letter.
It would be interesting to see a study that compared reading performance on larger books.
I just finished the Idiot, by Dostoyevsky. It was challenging; I frequently flipped back to earlier passages to clarify details.
It was easy to find passages. When I read paperback I have some kind of tactile memory for where details are. I don't necessarily remember words, so I couldn't search. But I can find any passage in 2-10 seconds. I can't do that electronically.
While I don't care about this time ordering stuff (never noticed anything is missing from my enjoyment, thank you very much), who says ebooks couldn't help with that? It would be easy to include all sorts of fancy progress trackers.
And not only progression of time. I sometimes have trouble following all the characters in complex novels. ebooks could provide handy help pages visualizing the entanglement of the actors as the story progresses.
And who knows what more. I suppose current readers are not really equipped for it, but the next generation might be. I see no technical reason not to have that kind of stuff.
Whether this is true or not, I find my own comprehension/recall much much higher when longform text is read to me than when I read it. Consequently, I currently get all my books either in audiobook form, or if not possible I use my smart phone to read the ebook to me. (With the exception of highly technical books with lots of formulas.)
There is still a bit of a taboo in our society I think to not read books manually, hopefully this will pass- It's much more important what you read, not how you read it.
The first thing I used to do with a new book was smell the paper. I find having that constant smell near me as I read a book makes the experience somehow more visceral.
To be honest, I still remember how the paper in my primary school maths books smelled, and I remember how the pages felt (shiny).
Thankfully I also still remember some of the contents.
I can imagine the act of holding the book and thumbing through the pages as you go along might help embed the timing of things into your subconscious, as it's quite obvious exactly where you are in a book due to how you are holding it, where as on tablet its more of just a continuous stream on a screen.
Definitely. The subtle but ever present tactile reminder of how much book is left to be read also helps build anticipation as the end approaches with certain works.
It's like the metadata is being delivered kinesthetically, without distracting from the simultaneous visual delivery of the message itself. Page numbers and percentage read indicators just can't compete.
> I can imagine the act of holding the book and thumbing through the pages as you go along might help embed the timing of things into your subconscious
I wonder what someone who spent their life reading only on tablets would say when first reading a book?
(I agree with you, but like you and everyone here I spent most of my life reading books.)
Even if the participants had "equivalent reading habits and experience with tablets", just imagine if you were presented with a device as unusual as a Kindle DX. I would be distracted for many minutes. "Ooh, shiny! What does this button do?"
Maybe because content on Kindles consumed on the run / when commuting, during short coffee breaks.. while paper book somehow attracts the reader when he knows he's able to dedicate more time for getting deeper in those pages..
Here is my N=1 armchair hypothesis: If there really is a difference, I could imagine that the limited tactile experience is the key factor. Often times when I go for a walk with a podcast on I remember it much better because I associate parts of it with things I encountered along the way. Our finger tips are like magnifying glasses and produce a very rich (though semi-semiconscious) experience we associate parts of the book’s content with (especially the thinkness of both sides). With ebooks it’s reduced to visual experience, the tactile experience is pretty limited in variation and overall unchanging.
"When you read on paper you can sense with your fingers a pile of pages on the left growing, and shrinking on the right," said Mangen. "You have the tactile sense of progress, in addition to the visual ... [The differences for Kindle readers] might have something to do with the fact that the fixity of a text on paper, and this very gradual unfolding of paper as you progress through a story, is some kind of sensory offload, supporting the visual sense of progress when you're reading. Perhaps this somehow aids the reader, providing more fixity and solidity to the reader's sense of unfolding and progress of the text, and hence the story."
You could have at least glanced at the link before commenting.
This comment really confused me as I couldn't find the quote in the NYT until I read the other comments that the link had been changed from The Guardian. Nevertheless, this seems plausible.
I'd really like to know if they had that hypothesis going in and were testing it, or if they ran the test and then put in a plausible-sounding explanation for the results they found.
Well, I know(but don't have sources as of now) it works like that for memorizing stuff, as an example you probably remember where hundreds (or even thousands) of objects are placed within your home, or your office without thinking twice about it, you just "draw" a path to that you were looking for.
But is the absorbing in the article about remembering or understanding what's being read?
I totally agree with this. This is a problem I've faced and recognized it myself. I also feel very similar when it comes to writing. Writing on a computer just never works for me. Plain old paper and pen always get me going. I feel it's just easy to focus our vivid imaginative powers when we don't have some thing we use day to day to do things other than read. I mean would you be able to read if a story was written in an accounting book? Between those lines?
"But instead, the performance was largely similar, except when it came to the timing of events in the story. "The Kindle readers performed significantly worse on the plot reconstruction measure, ie, when they were asked to place 14 events in the correct order."
The NY Times article about it is much better and includes actual number (http://mobile.nytimes.com/2014/08/14/arts/reading-literature...)