Overall, pretty interesting. A lot of those are definitely usability bugs. I wish someone would do a comparison between mac os x, windows, ubuntu with the same tasks to see how they stacked up against each other.
However:
5/11 participants (P2, P3, P5, P9, P10, P11) crashed Unity during their hour of testing. And towards the end of her test, P11 opened a zombie quicklist that stayed on top of everything and didn't respond to clicks.
This ubuntu release is shaping up to be pulseaudio 2.0. I know everyone will just say "use LTS", but somehow debian testing manages to be both more up to date and more stable than normal ubuntu releases. I'm not sure why canonical can't at least match that.
Are you running 11.04 beta2? I've been running it for months now and I LOVE Unity. I installed on my wife's machine, and while she doesn't love it (she really doesn't care one way or another about computers etc etc) she appreciates it in many ways. Though, the best way I can describe my wife's interaction with Unity is she gets less annoyed with it than she does/did with our iMac. True story.
Unity quickly became part of my workflow and improved it greatly. Super-W, Super-# are awesome, Dash is good (not great) and I like that something is built-in to the system that allows me to do basic searching etc (I hope this improves, but initial pass is really quite good). My workflow feels more fluid and solid w/ Unity, and I feel more comfortable, and hence, more productive than when I was in OSX just a short while ago.
And stability is something that is hard for me to gauge as I've been running since January on my test machine and beta1 on my main machine without too much problem. I get some quirks here and there, but nothing drastic. I'm running it on three machines: a mac mini, a lenovo t61p and a lenovo t60p. The t60p is the least responsive (naturally) and that is mostly b/c the ATI drivers are not as good as Nvidia's. The mini and t61p are nvidia and after installing those drivers everything works great.
And to compare, I used a Gnome3 live CD for several days trying to decide if I wanted to install something with Gnome3 (most likely Ubuntu w/ Gnome3 if that was the case, tbh...Ubuntu packaging makes it worth it alone) and while I think Gnome3 is a good release, it can't really compare to Unity. I like where Gnome3 is going, but I don't see it as something that directly competes with Unity at the moment.
I think Unity has a ways to go, but if you compare Unity in 11.04 to, say, OSX when it first came out...by god man, Unity is lightyears ahead in usability an consistency. People tend to forget how rough OSX was for a couple of releases. Apple has their act together now, but it took some time. And I LOVE something like Unity existing on Linux and being FOSS...pushing boundaries...
I've been running Unity for 24 hours now, and I can honestly say it's been the best desktop experience I've had in all my years. Notice I said best "desktop experience", and not "best Linux desktop experience". I have been running Windows since 3.1, Linux since Red Hat 5.2, and OS X since, well, Leopard. Unity is by far the best and most cohesive experience I've had so far. It tops OS X by a wide margin...it has many of the same sort of niceties with window management that actually makes sense (yes, I think OS X window management is horrid). Add to this the fact that it bakes some of my favorite compiz features (grid plugin anyone?) in and I couldn't be happier.
I know there will be some naysayers...many, in all likelihood, but in this daily Linux desktop user's eyes, Unity is a VERY big win for Ubuntu.
Can you please give a couple of examples of why is so? The usability test this thread refers to is really just oriented to new users (mainly: how easy it is to find this and that). I wonder what Unity does for everyday users.
I've been using Unity for about a month (or more?) now. It is indeed an awesome experience. Definitely much better than Windows 7. I can't compare it to OS X because I haven't used it, but your testimony makes me really excited :D
If you haven't tried it yet, give original Gnome 3 / Gnome shell a go. You might like it even more than Unity. I get a feeling that by jumping straight to Unity, they will cause many people to be excited about it, not knowing they've lost something at least as good.
Unity is not bad by any stretch, but it feels unfinished to me when I use it. I kept hitting the wrong thing and breaking the flow of whatever task I was doing. I'm sure with some more experience with it I'll quit doing that, but it kept violating my expectations and stealing my attention from more important things.
Gnome 3 was a big change, but felt much tighter and polished. I didn't seem to have to think too much about how to do stuff.
Based on that experience, I think I'd like to use Gnome 3 on Ubuntu long-term.
Actually, the usability test pretty much replicated my experience with Unity. The dock was fine, but the way it folded was awkward. I wasn't sure what the Ubuntu menu did either and I eventually got sick of never being able to find system settings and the really unhelpful search and switched to Xubuntu. I was reading this usability test and going "...And they went ahead with it!!??"
I'm not sure what you were disagreeing with, but I have to disagree with your disagreeing because if 5/11 people crashed the thing in an hour, it's completely unusable for many people, me included.
When someone says "pulseaudio 2.0", they mean it in a derogatory sense ;)
Also, the statement about Ubuntu being less stable than debian testing. I don't know the GPs specific instance, but Ubuntu has basically made debian a usable desktop.
When ubuntu first switched to pulseaudio it was apparently a big mess, and the "fix" was basically to uninstall it. So, calling Unity "pulseaudio 2.0" implies that the first thing everyone will want to do is switch from unity to "classic" GNOME because Unity is too buggy (right now).
As I understand it anyway, I wasn't using Ubuntu Desktop by the time the pulseaudio thing happened.
Some of these are humbling. They reinforce how disconnected I-- a programmer-- am from the average person.
> P1 recovered amazingly well after trying to save "Letter to Mr Smith 08/04/11", and getting the vile response "Error stating file '/home/ubuntu/Documents/Letter to Mr Smith 08/04': No such file or directory"
That's something I'd never consider, what with various directory separators baked into my subconscious.
Windows (certainly 7) handles this perfectly well - try and type a character that isn't allowed in a file name and it pops up a tool tip saying it isn't allowed, along with a quick list of other characters that are disallowed. It also throws that key press away, so it doesn't have to be deleted and can't cause another kind of issue.
This is the difference between products which have undergone long rounds of user testing and ones that haven't. For me this is one of the biggest issues facing Linux based operating systems today, because this is generally not an itch that people working on the project feel like scratching (see: design, documentation, etc.) With commercially backed projects like Ubuntu, it can get there over time, but I would look at a user testing result summary like this and say "we have a LONG way to go" if it was a product that I was leading.
Because we need to identity directory markings. Its as simple as that, we could use *, but then you couldn't put that in a Linux file name. Whatever character is used to separate Directories, is going to have to be disallowed in file names, understanding that basic concept and why it can never be fixed, will take you a few hows of research in bassic operating system file manipulation.
Yes, I thought the same thing! Really shows the value of running tests with people with other backgrounds.
Just for fun I tried saving a document in Microsoft Office 2007 here with a filename containing slashes. I got a slightly better response: "Ogiltigt filnamn" (Illegal filename). Not as confusing, but still no help at all on what was wrong with it or how to correct it.
I just tried the same on the Mac (in TextEdit). If you start with a slash, you get a 'go to directory' dialog. If you enter a slash later in the filename, it saves with a slash in the filename. The 'magic' is that it actually uses colons, which are shown as slashes in the UI. If you try to enter a colon in the filename, it replaces it with a dash. Renaming a file in finder to use a colon, gives a message similar to your 'Illegal filename'.
My father once asked for my help because his computer (Windows, either 2000 or XP) told him he was doing something "illegal" and he was worried he was breaking the law.
Yes, it really shows the disconnect between programmers and users. But to me this write-up also shows that there can't be a one-size-fits-all user interface. If the interface would be changed to conform better with the mental model of these users, surely it will conflict in some ways with the model of more experienced users. That's why the choice of window managers is great IMO.
>If the interface would be changed to conform better with the mental model of these users, surely it will conflict in some ways with the model of more experienced users.
I think it's tough to do, but certainly doable. What was outlined in the OP is just basic confusion amongst novices. Something that can easily be improved upon without hurting the experience for 'senior' users. I imagine this is how Mac does it. They throw their software up against the shores of a million and one newbies and iterate until they have a product that they are happy with. Testing is an integral component of UI design & creation, and from the looks of it, Ubuntu is slacking.
I work for a small non-profit radio station where we've got a couple of Ubuntu boxes (running 10.10, or whatever the latest version with standard GNOME 2 is). I've only done it because we recently expanded our volunteer base and didn't have XP licences to cover computers for every desk!
I've actually been really surprised at how well the volunteers have taken to using the machines, and how some even seem to prefer them to the tried-and-tested XP installation we've got on other computers in the building.
It helped that we were already using Chrome, Thunderbird and OpenOffice as our standard applications throughout the station - so when I installed the machines, I just stuck big, bold icons for each one on the desktop to make it obvious what to do. Audacity was a new one (we're using Adobe Audition on other machines) but people took to it reasonably well.
Comments I've received have been along the lines of "There's a lot less shit on the desktop than on those other [XP] computers," "It's easier to find what you're doing, and it seems a bit faster too," and so on. For some reason, our XP machines accumulate junk on the desktop, whereas the Ubuntu boxes stay relatively clean!
It works for a lot of our older, less experienced members who find it difficult to find, say, the Thunderbird icon among a pile of old documents and folders on the Windows desktop, or get confused when Start -> Email opens up an empty install of Outlook Express for some reason. It often feels like I spend more time helping people out when something's disappeared, they can't print, they can't find something on a Windows box than actually doing my job, so the Ubuntu machines have been a boost - they seem to just keep trooping on.
Of course, our users are simply booting the machine and opening up some standard applications that also run on Windows, and not going in and changing settings, or anything - I suspect that's where it would fall down in usability.
However, I've tried out recent pre-releases of Unity and GNOME 3 and found them pretty confusing, and I expect a lot of the users who've made positive comments to me about Ubuntu at work would too. It seems like a step backwards to me, with too many unclear mysterious icons, and bits of the UI whizzing on and off the screen while I try to work - and I'm sticking with the LTS at home, and not updating the work machines, either.
In all seriousness, what's the problem calling it "LibreOffice Spreadsheets"? Along with "LibreOffice Presentations", "LibreOffce Documents", "LibreOffice Databases"; are we so pressed for space we can't just say exactly what it is? The "name" is libreoffice, I don't see why the individual components have to be unique or catchy.
For windows users in particular, the bundled calculator is called calc.exe. That alone might create confusion. Besides, "Calc" does have a "Calculator" feel when you read it. A comparable thing would be to have a text editor named "Term", or "Conso". Any unix user might expect those to fire of a CLI, not an editor. This is comparable, IMO.
I don't see how "Numbers" is any more intuitive than "Calc", not that Calc is really a good name for a spreadsheet application, but at least it makes sense. Spreadsheets do calculate things, it's just the name collision with the Windows app calc and the fact most people associate calculating with, well a calculator. Calc may be a poor name for a spreadsheet app but Numbers is definitely worse.
> I don't see how "Numbers" is any more intuitive than "Calc"
In an office, ask a bookkeeper, sales manager, whomever, "Hey, show me the numbers!", and you're going to see a spreadsheet. If you ask, "Hey, show me the calc!", you'll get a blank look.
Evidence suggests "Numbers" is a perfectly cromulent word.
Where I've worked, you'd be more likely to see a spreadsheet poorly pasted into a PowerPoint presentation with a poorly scaled bitmap of a logo in the corner that is a 6 MiB attachment to an email...
That's one thing that always bugged me about Linux desktops. I've been using Linux as my desktop OS since Debian 2.0 and the naming convention tendencies of desktop environments are counterintuitive to non-Linux users. The uninitiated just don't know what GNU is or the differences between KDE and GNOME, other than how they look and feel. If Linux desktops are aiming for usability, naming everything Gnusomething, Ksomesomthing or Gsomething is just a poor choice for application names.
Funny enough, spreadsheet applications are called "table calculators" in my language, so naming it "Calc" makes perfect sense to me :). I can see how it would make much less sense to a native english speaker, though. Anyway, perhaps "LibreOffice Sheets" would be an improvement on the name of this app.
This was my favorite -- DESIGNERS, THIS IS WHAT AD BLINDNESS LOOKS LIKE IN AN APP: Don't design stuff like this in, people don't notice it anymore.
" Only 2/6 noticed an XChat Gnome notification, despite (1) a
notification bubble appearing, (2) the Ubuntu button going blue,
(3) the messaging menu envelope going blue, and (4) an emblem
appearing on XChat Gnome's launcher. "
I run with a laptop (display on, but rarely used) connected to an external monitor. Linux assumes the laptop is the default, and changing the external to primary is next to impossible.
Unity puts everything important on the laptop display in this configuration. That alone is a deal breaker for me.
Note that Ubuntu Classic works perfectly in this setup. My panels go to the external monitor when it's plugged in, and move back to the laptop immediately when it's unplugged.
Unity (as it's currently designed) would pose some serious issues in a dual-display setup even if I could move it to the external monitor. As the launcher is stuck on the left, I'd be constantly overshooting it (laptop display is to the left).
I have it running on dual displays with no problems whatsoever. That said, I have two 22-inch monitors and not a laptop screen. Unity is early days now...I'm sure your issue will be fixed soon enough.
Ubuntu on my dell latitude, running 10.10, doesn't handle unplugging/switching primary monitors. So I am far less confident that the parent post's issue will be fixed anytime soon...
I remember about 10 years ago, with the rise of XP and the advent of OS X, expecting Linux desktop to fall ever further behind on a usable & full-featured desktop.
Here we are discussing the usability tests of big UI innovations.
I'd like to thank everyone who got us here! (cough including Redmond cough)
I wonder: 11 people seems like a rather small sample size; does Canonical typically do any testing with larger samples?
I'm surprised that only 5/10 people tried to open another Firefox window by clicking the launcher icon. Personally, I think clicking the launcher icon not opening a new window is silly, but that's because I'm used to Windows 7 and not OSX.
Jakob Nielsen does suggest [1] that only 5 users will get you about 80% of the problems in your software, and 15 will get you damn-near-everything. So 11 is pretty good.
Also, if you do have the resources to do, say, 100 user tests, you actually actively don't want to do them all at the same time or it will be a waste because they will all say the same thing.
If anything, this was overtested. There's a lot of redundancy in those results, just as Nielsen would predict. This is not a scientific experiment to determine the effect of a drug, and you do not always need or even necessarily want a 95% confidence interval, as the costs would exceed the benefits here.
I am a dev (though not on frontend/UI stuff). But that's exactly the point. Many users are for the most part pretty stupid, and that is reality. Unusable software is unusable.
This article made me want to try out the new Ubuntu in a vm. I am looking for a download as I type.
I absolutely HATE the fact that when I click an running application's icon in the taskbar in Windows 7 it only sometimes makes that application active. If the app has multiple windows (sometimes tabs) open, it shows a list for me to choose from. That sucks as it requires more clicks, just show the last used window on click of the taskbar icon,on hover show that list of open windows/tabs. OS X got this right (even though they make all of the windows/tabs for the app come to the front).
YUP! Unity has this right. I opened multiple Firefox windows then opened a home folder. When the homefolder has focus, I single click the firefox icon and it brings all of the ff windows to the front. If ff has focus and I click the icon, i get an expose-like view of all of the ff windows. Windows 7 window management isnt as good. It slows me down A LOT.
I could see myself using this version of Ubuntu. The very first thing that I noticed is that they managed to get rid of the fat-tipped cursor pointer. The os over all looks good and smooth, a lot like a website, I'd just like to see a more dock-like experience with the unity bar (move around, resize etc)
ehhh, I hate the hidden options on the title bar (file, edit, etc) not a big deal, but could be confusing to someone
Agreed. It's great to know that there's some formal usability testing going on (and very interesting to see the results), but without comparing the usability of the existing desktop it's hard to know if we're going forwards or backwards.
I agree, you'd think they'd conduct usability tests BEFORE making the choice to switch to Unity. It seems that Ubuntu makes UI changes for the sake of being different, not better. Like moving all the window controls to the left side of the title bar in the current release of Ubuntu. Why?
Agreed, and the same thing happened, I believe, in Gnome 3. They did usability testing, but only AFTER the features were essentially frozen. Instead of using users as a guide for what to develop, they used them as a confirmation that what was already written in stone was good or bad. That's just not good practice.
If almost 50% of the users managed to crash the GUI with routine tasks in the span of a single hour, that's a sure sign that it isn't ready for prime-time yet, regardless of what anybody's opinions on features or functionality are.
I personally think this is almost definitely going to end up as another Pulseaudo-style debacle that'll jade even more Ubuntu users. This kind of stuff is exactly why I never recommend Linux to friends, even though I personally use it on my day-to-day machine: because it's just not (crashwise) stable enough for Grandma, and Shuttleworth has a very bad habit of making his end-users his beta-testers. Grandma isn't going to loyally log in to Launchpad, report a bug and reboot; she's going to complain to me, and then I'm going to reinstall Windows 7, which for all its faults at least doesn't crash once every two hours.
I know people hate this answer, but this is how I look at that: LTS.
Seriously, I install LTS for several things: servers and non-techies. Why? b/c I view non-LTS releases as awesome for me, you, people on this site but not for my grandma.
What I've seen Ubuntu do is they have given people who want to be on the bleeding edge a way to be on the bleeding edge every 6 months, and that is freakin' awesome. But if you don't want to be there, and frankly, most people don't, they give you an LTS and 10.04 is a pretty freakin' good OS for people who check their webmail and need a writer/spreadsheet with minimal fuss.
Maybe true, but as a web developer I can't in good conscience install an OS for a user that won't upgrade to FF4 for more than 6 months. That's the PPA/milestone release system's blessing and curse: easy to install software, hard to get the latest version without terminal witchcraft or upgrading the entire distro and eating whatever other changes (Unity) that upgrade brings.
Seems perfectly reasonable, and more importantly, prudent given the LTS nature.
To summarize:
* FF4 is available in a PPA right now - ppa:mozillateam/firefox-stable
* 10.04 will continue use FF3.6 in main as long as Mozilla supports it (probably about 6 more months)
* Once mozilla stops support for 3.6, 10.04 will switch to the latest stable (if that is FF5, that is what gets installed).
Seems very reasonable. Imagine if you are a huge corporation and you standardized on Ubuntu 10.04. Would you want every package to get updated to the latest and greatest each time it comes out or would you want security updates to those packages? If you want latest and greatest, jump on the 6 month cycle. If you want to get security updates for a known system, stick to the LTS. Even so, for those that WANT to get the latest of select packages, there are ways.
So, if you feel that having the latest and greatest for your users is important, stick to the non-LTS cycle and pick it and go. You can also install 10.10.
Have a look at the EOL dates...you can use 10.10 with full support until April 2012, getting you through 2 more Ubuntu releases and to the next LTS.
This is what annoys me about the package management/release system of nearly every Linux distro. There's no easy way to cherry-pick just one package (and its dependencies, obviously) from the bleeding-edge branch. It's all or nothing.
Ultimately, flexible package management is what keeps me using Gentoo and dabbling with Exherbo, in spite of the fact that I don't really want to tinker with my OS anymore.
I'm not sure what your Grandma manages to do to the machine, but I've never, ever had Ubuntu crash on me while running the long-term-support versions (currently 10.04). At the same time, if I could get a dollar for each Win7 crash...
On the other hand this testing doesn't include any other ubuntu user. Of course you'll have some learnability gap. I see also a lot of windows people having trouble to figure out in their first few hours in OS X.
While Unity definitely needs some polish, it already makes it more efficient to manage my desktop experience on my netbook running Ubuntu. The guys at Cannonical are trying to do major innovation and provide some consistency with the old UI.
I installed 11.04 and when I was first setting everything up (because the video drivers weren't installed yet) I played around with Gnome and actually ended up with a setup that looked a lot like Unity. The difference was I had a lot more control over the panels which I really liked.
Then, once the drivers were in place, I switched to Unity and was initially really wowed by the way it looked. I really like the idea behind the side menu but right now it is a bit finicky. Sometimes it will partially open when I move my mouse to the edge and other times it works just fine. I absolutely hate the launcher menu (might not be the right term - app drawer or whatever else might be more fitting) simply because it makes it a chore to find applications.
I've also found Unity relatively unstable. I've had it simply lock up and I haven't found a way to get around it other than resetting my computer.
Even stuff like the way moving a window to the edge of the screen and it taking up a portion works seems inconsistent. I can sometimes get 1/3, sometimes 1/2 and sometimes 2/3 but I honestly haven't spent enough time trying to figure out why it works in different ways. Nor do I feel I really should have to... UI should be relatively intuitive.
Anyway, I think Unity is promising but it is really rough right now... really rough.
Where is P6? Was there one and their comments were so bad that they didn't add it to the list? Numbers missing in a list like that really bug me. It also causes me to wonder why it never comes up. I haven't read the thread since a lot earlier and being as I know a few people around the Ubuntu camp I asked them and no one seemed to have any idea why P6 was not on the list.
I work with Ubuntu for work ( we make ARM devices ) so every so often I have to run Natty to test if it is usable yet. If you have a "decent" OpenGL graphics card, then depending on when you install it may or may not work. I currently have an issue with working with the dock or launcher or whatever name they are calling it. Right clicks are being passed through to the desktop so I can't remove or add anything to it.
If you don't have an OpenGL based video card ( no ARM machine does, they all use a subset of OpenGL called OpenGL ES ) then upon logging in, you get a long dialog explaining that you need to logout and choose classic desktop. And it has an exit button. When you click said exit button, after a bit it loads the classic desktop, however the xsettings manager doesn't get started (or it might, just depends on if it feels like running it seems) and gnome becomes very ugly.
There is no mention of Unity-2D which is a version that is QT based. And it suffers from the same issues, xsettings may or may not launch and then you have a ton of dialogs about apps crashing in the background ( this is with either Gnome "classic" or with Unity-2D ).
Ubuntu made great strides in making the Linux desktop accessible for everyone, but this latest release shows just how much further they need to go.
Keep in mind that at the time the decision to write and use Unity was made, the Gnome 3 desktop was in an absolute mess. A lot of work has happened since then and it would be even better if Mark Shuttleworth could swallow his pride and just scrap the Unity project and work with upstream like before.
I mention the ARM bit at the beginning of my statement if only out of frustration due to the fact that our company provided Ubuntu developers with over 50 machines that are in either a desktop configuration or netbook yet it still doesn't run anywhere close to where it should on them.
Great post! I started using Ubuntu Unity on my netbook, however, I had first installed Ubuntu Desktop and added Unity later so I can switch between them at loging time.
I think that Unity is great! Really encapsulates the spirit of a netbook - a small, versatile and fun communications device. I have an asus 1005pe - which I can also highly recommend.
I am really surprised one thing didn't come up. It's about the super+(?) shortcut.
If you press them very quickly, say 'super+D', then it will take to desktop and open dash as well. To be safe, you have press 'super', wait till numbers appear on the icons in launcher and then press 'D'
I am really surprised one thing didn't come up. It's about the super+(?) shortcut.
Really?? You are surprised that the test group didn't somehow magically discover an edge case in a feature they didn't know existed and has no discovery mechanism?
I use a Logitech G15, and it has a little switch on it to switch between "computer" and "gaming" mode. Since I rarely use(d) the super/windows key, I tend to just leave it in "gaming" mode. Without running into a post like this, I would have found it extremely tedious to click the icon type the application name, and then click it's icon.
While I'm sure I'm in a minority, it is always something to consider when writing shortcuts to consider that people may not have a key available.
I haven't tried recently but last attempt I believe alt+f2 did not work as a "run" dialog
It's not really "invented" so much as having been called the "super" key for a long time by Unix convention, and having been adopted by Microsoft to become the Windows key where it actually showed up on consumer keyboards. If you look at a classic Solaris keyboard for example, you'll find four classic modifier keys: control, alt, meta, and super.
You also can't just demand people who have been using naming conventions for twenty plus years to write off that history just because Microsoft decided to use something different.
I'm pretty sure Ubuntu adoption rate isn't affected in the least regardless of whether you call it the Super key or Windows key, or Logo key, or any of the other names it's gotten over the years. "Normal people" don't ever use that key anyways.
2. It's not about adoption. It's about usability. Everyone will know what you mean if you call it 'the Windows key'. The same is not true if you call it 'the Super key'.
I don't have my Macbook at hand, but IIRC correctly the Mac error message is more like "The file can't be saved; try a shorter name or removing special characters".
This isn't unity specific, the only reason they did it was to see if the user could then find the document. The failing in this specific case is more on the side of Libre/OpenOffice.
However:
5/11 participants (P2, P3, P5, P9, P10, P11) crashed Unity during their hour of testing. And towards the end of her test, P11 opened a zombie quicklist that stayed on top of everything and didn't respond to clicks.
This ubuntu release is shaping up to be pulseaudio 2.0. I know everyone will just say "use LTS", but somehow debian testing manages to be both more up to date and more stable than normal ubuntu releases. I'm not sure why canonical can't at least match that.