I'm going to geek out here for a minute at the fact that Facebook made a screenshot diffing tool for testing. I've been wanting to build that very thing for a long time, but every developer I've talked to about it looks at me like I'm crazy.
So, am I still crazy for thinking screenshot diffing might be an effective way to verify builds?
It's a great idea. Even ability to render different views and inspect them quickly by eye is really valuable when you have huge configuration space (charts for example). Adding automatic diff testing on top of that is a good idea especially when the UI style stabilizes.
It can be very effective. For example, WebKit has used it extensively. On the other hand, it can slow you down if your software involves rapid iteration of the UI, in which case semantic UI testing (using, eg, accessibility) may be a better solution.
HTML5 is not effective at the moment, but as phones become more powerful. I think it will take over. The same way the web app took over from desktop apps when personal PCs and network bandwidth improved.
Perhaps, but the same has been said for what the last two or three years and in that time frame we've gone from single core to quad core and 800 MHz to like 1.9 GHz and HTML5 apps still are not up to snuff.
I wanted HTML5 apps to be awesome because the developer experience was easier, but it makes for a sucky product, so I started just digging into native and making better software.
I've learned my lesson. At some point, we all have to realize that you do the hard, messy work to build the best experience possible for the end user. If getting to that was cheap and easy, we woudln't be worth what they pay us to build software.
With the benefit of hindsight, do you still think the developer experience is easier with HTML5, or was it just that you were already familiar with HTM5, but weren't familiar with the native tools?
I think it will just become easier to hash out ideas with html5 apps, but I don't think there will ever be a point where we can do 100% html5. I mean, obviously if you're doing game dev, or require other system intensive tasks, you're probably gonna want to stick with native. Really though, this reminds me a lot of ruby on rails.. meaning, easily solves most problems, but doesn't solve all problems.
Seems like no one's really struggling to scale mobile I suspect because 1) usage, although large, is less intense and 2) much less goes over the wire (usually just data, not pages).
I think 'scaling' in this sense has less to do with data efficiency but how to develop large-scale projects for mobile, as in efficient release cycles and how to catch bugs before they go out into the wild permanently.
I think mobile app development has lots of similarity with 'boxed' products. Microsoft invests a huge amount of money in its test infrastructure for its boxed products, because once it's shipped it's very difficult to fix. I think the mobile development community incorrectly tends to treat development more like the web than they do a boxed product. For reference, more than 15% of Facebook users are using a release from 2 years ago (I will try to find the source for this info).
So, am I still crazy for thinking screenshot diffing might be an effective way to verify builds?