They should also ban reviews from jailbroken devices, where users may have installed custom fonts or other "features" for which you can't possibly test and account.
And yes -- any test Apple does for this could be defeated, but I expect most jailbreak developers would side with their app writing comrades and let it stand.
The reason that Cydia doesn't have reviews (or even ratings) is that we haven't found a way to make a working review system either, so we don't even try. I don't think that any of us would argue or work against something like that, but I also seriously doubt that Apple will do it.
(I'm a contributor to the Cydia open source project, but I don't speak for saurik or Cydia, and that's probably not the official stance.)
So, I /am/ saurik, and I probably /would/ argue with and work around something like that, as I think I should be allowed to review things in the App Store.
Really, the "official stance" I have is that developers who believe that having their application fail horribly when the platform is even slightly different is not their fault are deluded: Apple is going to change the platform over time, and when your app fails for Apple they tend to get pissed and just block your app from the store and may even de-prioritize your updates.
Seriously: if your app fails because the font is different, there is something epically wrong with your coding methodology; and yes: Apple actually did change the font slightly for the iPhone 4 (on retina screens they switched to Helvetica Neue from Helvetica).
Meanwhile, in desktop-land, everyone from large companies to small independent developers deal with this supposedly horrible problem of random fonts, different operating system versions, upgraded libraries, client-side firewalls, or any other number of things that would make developers like kogir freak out, and somehow the world doesn't end, and almost all of the software I download works without issue (with the exception of Chrome on Tablet editions of Windows, and that's squarely Google's fault). (Hell: even almost all of the software on the App Store works under these conditions.
What should /really/ happen is that Apple should internally have a different "test version" of iOS that developers are not allowed to see; one that has a newer version of SQLite (which I state in specific as developers make all kinds of incorrect assumptions about the SQLite ABI) and has every core library compiled to the opposite instruction set (Thumb if normally ARM, and vice versa), that has every font replaced with Comic Sans, and uses a different pixel density, on a screen with a slightly different aspect ratio, using a CPU that runs slightly faster than the shipping devices.
The reason this needs to be done is that an app that fails to run correctly in that (hidden) configuration is pretty much guaranteed to be an app that is going to stop working when any of the following occur: iOS 6, iPhone Nano, iPad+Retina, or another "game changing" device (such as the iPad when it came out). Rather than let these apps in the store just to fail later, they should be weeded out before they even get in.
(And yes: Cydia itself strives not to make any assumptions about any of the things listed, and in fact worked almost entirely correctly when loaded for the first time on an iPad with the flag flipped for "don't show me in a 1x/2x box"; certainly well enough that the reviewer in this scenario would have given it a pass were they testing it in that environment. The one place it tends to get burnt is with what it does to WebKit; and the current implementation, which works fine on everything from 2.0 through 5.0, is likely stable enough to "stand the test of time".)
(FWIW: I do believe that people using beta versions of iOS should not be able to review applications. These people are using a version of the firmware that is under NDA... they should not be tempted to be providing feedback that might violate that agreement to applications that fail to operate correctly, possibly due to no fault of the app. I have a good example of this related to Cydia on 5.x, but I can't tell you what it is, for this very reason ;P.)
One of the main reasons beta versions of an OS are made available is to find bugs in the OS. Apps can crash because the OS isn't doing what it's supposed to. Which is why we're up to iOS Beta 5. [edit: didn't see your FWIW - was that an edit?]
I'm afraid this reads like a hacker rant who is angry that lesser mortals get to write code too. I'm a bit disappointed this comes from someone as influential as yourself.
[edit] It is a great luxury to have a standardised platform to write for and it cuts development costs. Any Android devs want to comment on their experience of writing code for diverse hardware and setups vs iOS? Or indeed Android users wanting to comment on the quality of software being written?
A) Yes, my FWIW was an edit.
B) I do not see how anything you have said here responded to any of my arguments. (edit: in specific, I don't even know what your one comment about finding bugs in the OS has anything to do with this conversation)
C) My friends who are developers on Android claim that supporting multiple devices is not actually a serious issue.
[edit] D) Further, I believe that a response like mine is the only serious response that can be given to someone (kogir) who actually just suggested, "with a straight face", that a full 10% of the people using the App Store should be /banned from leaving reviews/ because he doesn't want to deal with /fonts being changed/. Are you telling me here that you actually agree with him?
"Further, I believe that a response like mine is the only serious response that can be given to someone (kogir) who actually just suggested, "with a straight face", that a full 10% of the people using the App Store should be /banned from leaving reviews/ because he doesn't want to deal with /fonts being changed/. Are you telling me here that you actually agree with him?"
Considering that that "full 10%" of the people you speak of have broken their terms of service and are running a non-sanctioned OS, why should Apple be obligated to publish their reviews? I understand the anger at Apple bricking jail-broken phones, but I really don't see how they should be obligated to accommodate them.
If Apple brings out an new device with different physical characteristics (Apple TV, iPad 3 w/ retina display), you'll have to port your apps to that device anyway. Using iPhone apps on an iPad (or non-retina apps on a retina display) sucks.
While you shouldn't ever make assumptions about CPU or memory specifics (which is why apple doesn't release that information, even to developers) you absolutely can make assumptions about screen size and aspect ratio. Sure, Apple changing the system font shouldn't cripple your app. But having to code for every possible hardware configuration? What is this, Android?
...and yet, people do this sort of thing constantly on every other development environment I have ever used, whether it be Windows, Unix/X, or Mac OS X, and this has not been a serious problem. If your application can't suddenly tolerate the screen being 20 pixels taller or shorter than it used to be, you are probably doing something wrong. (Yet, in fact, a lot of developers screw this up, and their applications don't even work on iOS when there is a double-height status bar.)
Well, the in-call double-height status bar is documented behavior. Of course you have to account for that possibility. But a different aspect ratio altogether?
On other platforms different screen sizes aren't a problem since that's been the norm from the very beginning. But why shouldn't we be able to make assumptions on iOS? It's an entirely different environment, one where, among other things, screen size and aspect ratio are fixed.
And yes, I don't think you should hardcode every last pixel in your iOS app either, but it's perfectly acceptable for your graphics to rely on the documented sizes. How should e.g. GarageBand account for a suddenly different aspect ratio?
To be clear, the double-height status bar changes the "aspect ratio" of the device. On WebOS, they have a really great notifications mechanism that involves slowly making the screen smaller (up to a point) as more messages need to be rendered. This is the kind of feature that Apple should be allowed to implement without causing havoc on existing applications, and would cause even more drastic disturbances in aspect ratio.
Also, on other platforms, there was at some point "the one monitor" that came with the system: the fact that so far we've only had two different aspect ratios on iOS devices (3:2 and 4:3) is no reason to believe that the next device won't be something more like 16:9. There is no reason that most applications couldn't be designed to be ready for such a seemingly minor change.
People do it constantly on desktop operating systems because they have no choice. It is still resources spent and if the platform was more uniform, they could have made some assumptions and spent time making better features instead.
If you do this from the beginning, it is not a big deal: and in fact, it really helps save you time later when things change. It simply is not difficult to compute a layout rather than drawing one in MS Paint... I mean, even Xcode's Interface Builder solves this problem quite well... I frankly am surprised so many people are disagreeing with this "aspect ratio" bullet point, given that if you take a random non-game app from the App Store and change its aspect ratio it is almost always going to do this correctly: Apple makes it hard to screw this one up. ;P
If you have to design for arbitrary resolutions and aspects, you are way more limited in what you can do with your design. On a touch device this is even more severe because buttons that are too narrow will be hard to reach etc.
Also, if you are not using standard controls and you have a designer make a couple of images for you, you will now have to split those up into parts to render your UI. It seems that you would argue that this is the only way to go but surely we can agree that it is more expensive for the developer?
So how are iOS app developers supposed to test for these undocumented variations? Are you planning on writing a Jailbroken-phones-crazy-ass-hacks test suite for us?
In one of our games, certain jailbroken phones broke the shadows under a character. How do we even begin to debug that? Especially when the only bug reports are 1-star reviews with no way to contact the user?
One of the beauties of the iOS platform is that there is very little variation, and any changes are well-documented and you have plenty of time to test for them in advance. It's one reason it's so popular to develop for and the apps are so high-quality.
The reality of the situation is that 10% of users jailbreak their device. It is not reasonable to ban all of these people from leaving reviews in the App Store.
That said, I am very curious what this bug was that caused "broken shadows". There are an insane number of heavily varied modifications available in Cydia, and I don't even claim to know what even most of them are, but I'm seriously dying to know what kind of change would have caused behavior like that.
In my experience (yes: I sometimes reverse engineer the developer's app, find the bug in their code, and send them a "pseudo-patch" when they get uppity enough ;P), almost every single case of a developer making claims about bugs specific to jailbroken devices actually had a "root cause" of "memory allocation bug in original application triggered by slightly different malloc order".
To further explain what this kind of bug is, this is where the application ends up having a corrupted or obsolete pointer to some other object due to some very serious issue.
A very simple example is just "released an object, but did not clear the pointer to it, and ended up continuing to use the object later". In this situation, the pointer usually ends up not just pointing to garbage: a later object allocation reuses that memory, and due to alignment often ends up at the exact same address.
Of course, the cause can be more indirect: once in this scenario, you might accidentally run the release logic again, only now the release message (or call to free) ends up affecting the /new/ object, causing pointers to /that/ object to become stale, which in turn causes them to point to a further new object. ;P
In this scenario, testing the application may simply "work": the objects may not be used all that often, and the ramifications may be something mostly benign (new also mostly useless objects are always created anew within enough time to make the pointers keep sort of working).
Yet, there is still a bug here, and it may be triggered at any time: if the order of memory allocation and deallocation in other parts of the codebase change slightly, then these stale pointers may end up pointing at something "less benign": maybe you end up freeing the entire shadow of the character, for example, rather than just the word balloons that are constantly being reallocated at different positions anyway.
This kind of thing makes you feel safe if you only test on a single system: you tell yourself that the user's device is going to work the same as yours, and that there is no way you are going to have plenty of time to fix these errors in production if they ever occur.
...of course, unless those bastard jailbreakers do something stupid to your process, breaking your app.
In reality, however, you don't, for a very important reason: deep library code is also using the same memory heap, and you do not get ahead warning of these changes, as Apple (rightfully) does not believe you need to know about them.
When new devices come out, which to date have been heavily correlated with new versions of iOS, they are not running "the gold master" build of the OS: instead they have random changes strewn throughout, and usually have a different build number (despite having the same firmware version).
Sometimes the difference is quite dramatic, though, like with the Verizon iPhone, which came out with a bang: complete access to the App Store with a massively different version of AppSupport (or whatever the moral equivalent is these days), a key library used by UIKit.
You can get really subtle changes like this, however: when 4.3.5 came out, the SSL routines for verifying certificates had to become more complex, and this was released with no notice to developers and only a week of testing even internally at Apple. There is no reason why a change like this couldn't cause changes in dynamic memory ordering.
Now, you might ask "how do we debug stuff like this"? You could wait for your app to crash in the field, but preferably you are debugging this as you write the code. Clang is trying to help, as are specific new features of Xcode 4 built on it, but static analysis can only catch so much.
At the Objective-C level, there is a feature called NSZombie you can use to help debug stale pointer references (no memory is deallocated: it is simply replaced with an NSZombie; a class that is not allowed to receive any messages without terminating the program and logging a stack trace), but at the C level you are on your own.
However, one great way to test stuff like this, is to get a jailbroken device, install a bunch of sketchy hacks on it (preferably the ones that I personally claim are "dangerous and should be used by no one", bonus points if you can find one where "the developer should be ashamed of himself" ;P), and then see if your app keeps working.
If you aren't doing this, you are going to get burned one day by a normal upgrade: you are taking a risk, and to save something that probably will just take an hour of your time, and will improve your codebase because of it.
I recall Cydia being pretty broken in iOS 4 beta days, and using undocumented APIs for no reason (it was obviously written before the public SDK, but there was no reason NOT to rewrite it once the SDK was out).. So that seems pretty hypocritical of you to want Apple to 'weed out' apps using a special environment that app can't even be tested against by the dev. Also, average iOS app lifetime isn't long enough to even remotely justify such draconic testing measures - most will be out of the app store charts long before there's a breaking change in iOS.
There is always a reason to not rewrite something once it is released and operational: there are almost certainly other things you can be doing that would provide higher value.
(Also, I am reasonably certain that the issues you are eluding to with Cydia not working correctly on iOS 4.0b* were, as usual, WebKit issues, which I specifically admitted in my post: and, in fact, during 4.x is when I came up with a much simpler way to handle that specific implementation detail. That said, looking through my git repository, the only changes I seem to have made for 4.0 seem to be to things that fundamentally are not public APIs: restarting SpringBoard being the major one.)
Yeah, there was _UISwitchSlider class or something being used. I ripped almost all of that stuff out for "Cydia 1.1"; except for saurik's WebKit code to replace the crippled UIWebView, it's almost entirely public APIs now (UIViewController, UISwitch, etc).
So, to clarify, the "feature" I was referring to was replacing the system font (Helvetica). Even when we requested Helvetica by name, we'd get the random replacement. Are you suggesting we license and distribute Helvetica with our application? It's just not fast enough (at run-time or code effort wise) if we have to to calculate font metrics for every single static label in the application to make sure it fits in the space allotted.
Some assumptions are valid, not lazy or bad. It's simply not possible to account for every possible scenario when all bets are off.
[Edit]
Note: In this particular case the label flowed over a button on a non-scrollable pop-over dialog. Not being able to click that button in the registration flow kept people from using the app at all. This was ages ago so I can't point you to the reviews, but there were enough of them that it mattered. It was also in the days before we added in-app reminders to leave good reviews, so basically the only reviews were from font-breakers.
So, I do not consider slightly incorrectly placed labels to be a serious issue, and doubt users who change fonts do either. I therefore would like you to point me to the negative App Store reviews you are receiving due to these font replacements that are strong enough and in great enough number that you are seriously suggesting banning 10% of users (a subset that includes me, so I'm rightfully taking this personally) from leaving reviews to stop the behavior.
Apple should remove the reviews section from the App Store for people running pre-release versions of iOS and replace it with a form to send a bug report to the app developer (and possibly even a place for a list of known issues that the app has on iOS 5).
Potentially, but if so, it should be something which you'd have to opt into or open only for fellow developers.
The last thing I want is to get spammed with "bug reports" from lay users with trivial or non-existent issues.
If they implemented this but made it take even one extra step like flipping some switch in the settings, it would filter out most lay users and garner some more useful feedback.
I strongly agree that the PyPI ratings were mostly useless and that removing the feature was a good decision.
iPhone app ratings can be much more useful for popular apps, but for the long tail of apps, they suffer the same problems as PyPI's rating and review system did.
Amazon and Yelp ratings, on the other hand, are quite useful. I think it's partly because users have profiles and are better able to express themselves.
I think the app ratings for the popular apps are all useless (they're well know apps, other people rave about them, so no need to read all the reviews). I agree on the long tail being useless as well.
Instead, I think it's most useful for the "middle tail": those apps that do get a fair bit of usage, but are not considered popular: the developer may not have build up a reputation yet, but the reviews help confirming the application does what it is supposed to do (or not of course).
An alternative fix would be to have the iOS version used when submitting a review. It doesn't limit "free speech", but it gives context on what the problem actually was.
I think this is a great idea. Apple could allow filtering to see _IF_ I really want to upgrade to iOS 5 right away or wait until iOS 5 reviewers are happy.
I pointed this out the last time this came up, but an additional measure would be to also crack down on sites that sell access to iOS betas, if they aren't doing that already. That would (likely) drastically reduce the number of non-developers running beta OSes since isn't it $100/year for a developer account, but you can install betas on many devices? There are sites selling access for less than $10. If it's not against the TOS of the developer accounts, it definitely should be.
Apparently Apple has noticed this issue and they have taken down a few sites. (http://appadvice.com/appnn/2011/08/apple-takes-steps-against...)
In fact Apple can try to increase the technical barrier to install iOS beta. (Must upgrade in Xcode or must own at least one app in App Store to be eligible for beta access.) This way can block the vast majority of non-developers while not affecting real iOS developers.
> must own at least one app in App Store to be eligible for beta access
That doesn't work for many reasons. App Review, as if they didn't have enough of a hard time with a bad reputation, would just be flooded by trash from first time developers or contract developers/companies that just created an account. Apple could make it mandatory to install using Xcode, but in the past, Xcode upgrades have bricked devices where iTunes doesn't. So I have a feeling that's not really a priority and more of a burden. Plus, it's braindead simple to install and use Xcode to install betas, and anyone desperate enough to pay $10 but not $100 should easily be able to get around that.
I really think all Apple needs to do is to block reviews from iOS 5 devices. Developers running betas won't be reviewing apps in the store (that's incredibly hypocritical and cruel if so) so nobody's going to be missing that. This doesn't harm beta testers and devs that want to install iOS betas.
Your proposals take things a bit too far without any real return.
Blocking access to devs who don't have an app in the store is counterproductive, because that prevents the developers from testing against the latest iOS and finding bugs -- which is the entire point of access to the iOS beta! Not to mention some devs might be working for other clients and not yet shipped something themselves (as in my case).
Requiring installs via Xcode would only be a temporary slowdown before someone figures out the secret handshake and hacks around it.
Blocking reviews from devices with the latest iOS beta installed should be sufficient to eliminate most of these unfair reviews.
Maybe Apple doesn't block beta reviews because it incentivizes developers to fix their apps before the release? Most of those problems should be trivial to at least work around with version-dependent logic to minimizes regressions on versions < 5.x.
Bad reviews don't exactly hurt Apple, and it's not like iOS developers are scarce these days, so it might just be Apple having their priorities set straight (users > devs) and not a coding blunder..or maybe it's just me overthinking it.
Except that developers are not even given the chance to fix issues before Johnny Public is allowed to start rating apps. The whole point of the beta is to ALLOW the developer to test their apps and find the bugs, as well as to iron out the bugs in the core OS. What if it is an issue with a core API that is now returning something that it shouldn't and it wasn't caught by Apple in testing, then the developer is getting blamed for something they can't even fix.
You have to be a part of the developer program ($99) to get pre-release software, and as part of that you go with the understanding that it isn't meant for everyday use.
Johnny Public shouldn't be running the beta in the first place. If by the time iOS 5 is out the developer still hasn't fixed the bugs in their app, THEN it is perfectly fine to start leaving bad reviews as the developer has had plenty of time to fix their app, but during the beta period, no.
It is additionally time consuming building work arounds for issues that might be altered by the time the OS makes it to gold master. Beta is beta, it might mean the OS does not work with your app not your app does not work in the OS.
I guess the grandparent was suggesting that, yes, it’s unfair but Apple has no incentive for banning reviews. They are ok with users heckling developers for updates since developers aren’t exactly scarce.
Really Apple should't be making changes that break apps between one major release and another. There should be at least one round of deprecation first. Microsoft used to be really good about this kind of thing. Admittedly Apple has always been much more willing to break working code.
Yeah, but non-developers who manage to install iOS 5 probably haven't actually signed the NDA.
The funny thing is as a developer you're not even allowed to mention in your description that something in your app is broken on iOS 5, since that would be breaking the NDA.
Apple also doesn't like it when you fix an issue, release an update and mention that you fixed a bug which affected iOS 5 compatibility. I needed to change it say "compatibility fixes for future iOS versions".
There are reasons to block the iOS5 users comment or review the applications.
1. iOS5 is beta , so chances are there that inclusion of new APIs , pre release iOS version can adversely affect the already approved application. But as the problem is not with the Application but with the Operating System , the review can be biased or completely irrelevant.
2. As a Developer I will welcome the step by Apple that, this will allow users to comment and review apps from the users who run most stable Operating system, so any bug on my application is because of me, not because of the iOS5.
All the effort developers are putting into complaining about people on iOS 5 Beta leaving negative reviews could better be spent testing and fixing their app for iOS 5.
Are you going to complain when iOS 5 goes live and your app still isn't ready because you stayed up until 2am writing that blog post rant instead of ironing out that last bug?
Even if Apple blocks iOS 5 devices from leaving reviews, you can still leave a review through iTunes.
And yes -- any test Apple does for this could be defeated, but I expect most jailbreak developers would side with their app writing comrades and let it stand.