People produce less text with more errors in more time on mobile. I actively avoid having to type anything on mobile. If I have my laptop open, I'll use whatsapp or signal on that. I often wait replying on either of those until I get my laptop. Just way less frustrating to not have to correct >25% of my key presses. If I have to use the mobile keyboard, I'll often just send a message with typos, no capitals, and skip some of the more redundant words. It seems lots of people do that. Phones just suck for text input. Longform text entering on a phone is by and large not a thing for most people. It's mostly a read only device for passively consuming news and media. And taking photos.
I find it telling that one of the more popular addons for ipads are covers with a builtin keyboard. It's a way bigger device than an iphone. But yet the keyboard sucks enough that Apple sells covers with a keyboard. Of course, all the touchscreen keyboard problems that the ipad has are magnified on their iphone. Yet, they don't have a solution for that. And they also sell a stylus for the ipad. Because fingers lack precision. It's the same OS but there seem to be no such options for the iphone. Does the stylus even work with an iphone? Is that deliberate? It's not like people are going to be magically more precise on an iphone relative to a huge ipad. Conclusion, Apple just accepts that that's the way things are. And besides, Steve Jobs would turn in his grave if they dared to ship an iphone with a stylus.
Hardware keyboards on phones used to be a thing. I worked at Nokia back in the day. Really nice keyboards. Blackberries were popular too. People wrote lots of stuff on those things. I wouldn't mind a little pocket laptop. It's not like my pixel 6 is small or subtle in my pocket. It would be more useful with a slide out keyboard.
I find that typing on a phone (iPhone or Android) these days is substantially easier than typing on an iPad because of swipe. It's still nowhere as efficient as a keyboard in the hands of a skilled typist, but I can often produce text fast enough that it isn't worth getting out my laptop. On an iPad it's far worse, because I actually do have to use the bad QWERTY touchscreen key-by-key.
What I keep hoping for as far as input methods go is a swipe keyboard layout that is optimized for swipe, because QWERTY has a few groupings that make for ambiguities. Someone calculated one a few years ago that puts the vowels as far apart as possible [0].
The only reason swipe works so well is because of how well people know the QWERTY layout, though. You'd have to have a pretty unique situation for it to be worth your time to thoroughly learn a new keyboard layout just to speed up typing on your phone.
The iPad keyboard has a floating option that takes swipe input. There used to be a split keyboard option, which I actually used all the time, but it was removed from newer models for unfathomable reasons.
Wait, wait.... The new iPads don't support the split keyboard SOFTWARE FEATURE that is still present in the latest OS release and still working on "supported" hardware? I thought the removal of the orientation lock switch was user-hostile, but damn; this is another level.
I would think it does support it... In my iPad Air (latest model) I somehow ended up stuck with a split keyboard for a bit until I managed to restore it.
Works well for me. You just type with one hand without ever lifting your thumb. You do need to set the correct language, and if you mix languages, your phone's OS has to be able to deal with that, but at least on iOS that is the case and the prediction is very good. If the first choice isn't correct, usually the first or second displayed alternative is what I wanted.
There are two caveats, though. Firstly, I type differently when using swipe. As it favors words from the dictionary, my texts sound less spoken and more written. And secondly, having backspace delete the whole word is a critical feature to avoiding annoyance. If none of the predictions are correct, I just tap backspace and then type the word like I usually would. This, combined with the rarity of those hiccups makes swipe typing quicker than regular typing to me.
It also helps with relaxing my wrists as I can also hold the phone with one hand and swipe with the middle finger of my other in a gesture like holding a pen, but without the pen.
Maybe there's actually a third aspect; screen and hand size. I haven't owned any big phones, my iphone 13 mini is the biggest phone I ever owned, but I do have large hands, so swipe typing with a single thumb is not very difficult. Your experience might be different depending on your anatomy and device.
There is another caveat with swipe in languages that write some words capitalized. The prediction fails to capitalize correctly in the middle of a sentence. The overhead to edit the word to the correct capitalization is so large, that swipe doesn't feel useful at all. A menu entry or gesture to capitalize the selected word would be really handy.
I've been using swipe for years now on my Android phones (currently a pixel 5). I hold the phone in one hand and use the thumb of the same hand to swipe. Feels much more effortless than "actual" typing. Especially when comparing it to typing with both thumbs.
I've been using swipe since the beta version of Swype. However, Google seems to be enshittifying Gboard. It continues to decline in accuracy, so I can't swipe as fast as I used to. ...sigh...
It's indispensable for me. Works nearly flawlessly, and in English and Spanish at the same time, without having to tell the keyboard to change languages. I'm magnitudes faster with it, and rarely have to go back and correct anything.
This whole comment was written with swipe and didn’t require any corrections, I do much better with swipe than trying to tap all the letters. It’s pretty fast too.
Try downloading Gboard. Been using it for years. Dramatically better than iOS built in, which makes horrible decisions about when and how text is corrected and has worse gesture recognition.
After a relatively short time, I stopped having to look at the keys or think about letters. Touch typing from keyboard transfers over well.
Swipe is good on keyboards other than the stock iOS keyboard. That one is worse than every other option imaginable and it sucks to see because iOS used to have the best mobile keyboard bar none back before they tried to get smart with it.
I don't get along with it either -- it's the worst possible option for me. But I know a lot of people who get along very well with it. Different people are different.
What I don't understand is why speech-to-text is so bad (on the iPhone at least), and it's related to this editing issue. Text to speech will never be a perfect replacement for other forms of text entry, because you're limited in when you can use it. But it's so close to being good.
The main issues are so easy to solve, they're just silly UI issues around editing. You can enter text verbally, it works surprisingly well at understanding what you're saying. But... you can't delete the words you just said if it was wrong. It's literally the first thing you run into - you're transcribing, it gets one word wrong, boom, you're stuck in normal tap-editing-land again. Trying to enter messages while driving? Good luck, unless you can transcribe the message perfectly on the first go, you have to use your hands.
FWIW, I read the corrected comment and still read it as "text-to-speech". Then I saw the replies and scrolled up, and realized that my brain had read it wrong!
Oh man, I long for the days of phones with T9 input and hardware keys you could physically feel. It was so easy to walk down the street and tap out a text message with results much more predictable than "predictive" text, only occasionally having to glance down at the phone to check what you wrote. You could do it all with one hand, it was very unlikely you would drop the phone in the process and even if you did, it would just bounce instead of smashing the whole precious touch screen.
Yeah, these days I only type on the touch screen if it's absolutely unavoidable - if I can't get to my laptop, but can still sit down, I much prefer a little flip out Bluetooth keyboard with the phone on a stand.
I see mention of T9 in "On phones that used the 9-key numeric layout, T9 predictive text was used. Other phones used the full-hand layout with the familiar QWERTY layout, with other proprietary predictive methods." I'm unable to verify this app has T9 as an input method. Is there a custom layout for this?
T9 was pretty sweet as you could type without needing to look at the device.
It was much better when you could get phones with physical keyboards. Only recently has swipe on android gotten to the accuracy of typing on a physical phone keyboard, but not the accuracy.
But, that was lost because of the iphonification of the whole smart phone sector.
There was a near-perfect solution on Android for years: Swype.
Why the people that bought it buried it, I can never understand. It was so much better - even ten years ago - than modern swiping keyboards. It even had a layout that was just for editing.
swype was so damn good and nothing comes even close. too bad it got killed off.
windows phone keyboard was good too. the key "hitbox" would grow or shrink based on predictive text and typing speed, so if you're writing something from the dictionary you could just slap the rough area of the button and it would register the most likely one. You'd feel it though when typing in something from outside the dictionary, suddenly it felt like you forgot how to type. Once you slowed your typing down it returned to behaving like a standard keyboard.
Also, it had a little trackpoint-type nub on the virtual keyboard which you could drag for moving the cursor. It was significantly more accurate than anything else in any mobile OS.
Apple lets you do this, too, but it's really not better than Swype's method. The Gboard/Apple method resembles the description in Hitchhiker's Guide to the Galaxy about radios that could be tuned by pointing - "[it] meant that you had to sit infuriatingly still if you wanted to keep listening to the same programme." Take your finger off the screen too roughly and you get a last little bit of motion.
That's not necessarily an everyone problem, but for all the people who complain about how big phones are and how they would like a smaller one, I have big hands and find an iPhone Pro Max to be a comfortable one-handed fit (I also found the OG Xbox "Duke" controller quite comfortable).
On Swype, you could click over two spaces to the right, or left, or up, and konw that it wouldn't move.
Did I mention the Swype key? Swype-A, Z, X, C, V worked exactly like you expected from computers. Swype-space did something, maybe bring up the edit board? Capitalize a word by going up above the keyboard before finishing the word, and double a letter (as vs ass, for example) by squiggling around on it before moving on to the next letter.
It had one fault for most of its life: the word "me", which is very common especially in text messaging, was frequently interpreted as "nee", which given that I'm not a wedding planner doesn't come up often. And yet even that was better than the horrendous Apple swipe misread that gives me "Abbas" when I mean "and". The only Abbas I know is the Palestinian politician, and he's not in my contact list. And I don't ever write about him except to complain that Apple just keeps doing this substitution.
> It was so much better - even ten years ago - than modern swiping keyboards.
Unless it learned too much about you. I found that regularly clearing all data from it improved swiping accuracy… It’s just like with all automated personalization, it doesn’t work.
This take is a little ironic. In the original iPhone keynote address [1], Steve Jobs specifically called out physical keyboards as a negative user experience. The advantage of the iPhone is that the entire device could adapt to whatever application you were currently using.
That being said I still think you are right. Typing on iOS could be improved.
To be honest, the problems an adaptive keyboard fixed back then - entering contacts, entering email addresses and passwords have long since been fixed by other solutions.
Bluetooth, vCards, contact sharing, QR codes, one-tap logins, password managers..
To say nothing of NOT displaying the keyboard at all. If you look at an old phone, you notice how tiny their screen is, because half of the front side is reserved for a sucky 12 key keyboard.
> The advantage of the iPhone is that the entire device could adapt to whatever application you were currently using.
This is the same argument as with Tesla dashboard: it's true, but incomplete. Virtual keyboard require your visual attention, while physical keyboard can be easily navigated using only touch.
I would say this vision could be realized if the iphone were able to grow a physical keyboard, and until then the entire device of the blackberry is more adapted to the applications I am using
I don’t think stylus is the answer on phone for text input. Maybe on a phablet. I haven’t experienced it being better and only use pencil on my iPad when in a meeting where typing or using a laptop would be obnoxious.
The Apple Pencil doesn’t work on the phones. But even if it did, it would be ridiculous and un-Apple given that the pencil is disproportionately huge compared to the phone itself. I personally wish they would do something for the rare customer who does need it (e.g. for precise drawing on the go). But they’d need a separate Apple Pencil Mini product.
I think Apple’s answer is that you should use speech-to-text or just serve voice messages, both seem quite popular with the crowd that isn’t on HN.
My problem on the Android Samsung keyboard is the positioning of the backspace key. When typing a word containing an "m", I'll often hit backspace instead, causing a lot of editing, for example "example" becomes "exple".
I agree it’s awful, but I very much disagree that people are doing it “less” — think of the chat apps. I think more text passes between more humans on iOS/Android than ever before in the history of humanity.
It’s probably a pain, but a familiar one, and you mostly tune it out.
"People produce less text with more errors in more time on mobile."
I absolutely agree, typing or editing text on a smartphone is just horrible. There are many times I've started to write an email or post to say HN and I've been so frustrated I've aborted the process and restarted it on my laptop. It's down outright infuriating I have to do this.
[I just hope someone of influence at Google reads these posts and acts accordingly. If people at Google don't understand then it's just another instance—like Microsoft the king of offenders—of where Big Tech won't fix the damn obvious (I wish someone would write a book about it and fully expose the problem).]
1. You mention the keyboard matter. I went to the trouble of buying a small Bluetooth keyboard to use with the phone to help with the problem and it was hopeless, for starters, it had no separate keypad and I couldn't use any of the multiple alternative ways of keying ALT-# etc.—to key in, say, a Unicode character. None of the known 'fixes' actually worked for some reason.
Question (to anyone): we have any amount of hardware available but no decent small Bluetooth keyboards—why on earth not!? There's precious few Bluetooth keyboards anyway let alone decent small ones for portable use or that are convenient to use with smartphones. Manufacturers where the hell are you?
2. The finger problem and selecting text. First, there's the selecting text problem. Either one's fingers are too big or selecting text is difficult because the resolution and or sensitivity of the screen is wrong and it cannot be adjusted. Yes, some phones have a 'glove' mode but it's nigh on useless. Why don't phones have a sensitivity control that makes selecting text with a finger much, much easier?
3. Along the same lines, selecting or editing, say, a URL from a browser address line is painful. When the URL text exceeds the screen width then trying to get to the end of the text is a damn awkward. More often than not either one can't, or the highlight comes on and soon as one tries to get the cursor back the URL disappears altogether. Getting the cursor into the correct position between the text and the GUI element is usually difficult, why doesn't Google allow some blank spaces here so it's easy to edit? Surely, it could be made so that tapping in this vicinity and sliding right would place the cursor at the end of the text without losing it? Editing here really does need fine tuning.
4. UNDO, UBDO! I'm typing this into my browser's HN edit box! If I accidentally refresh the page before posting (which is damn easy to do) then I will have lost everything and have to start over again. Why on earth hasn't the 'undo' problem been solved with browsers, especially so on smartphones? It was solved
over 30 years ago on wordprocessors so why not with browsers?
If I know I'm going to type some long text on my phone then I'll first do so in a text editor and save continually as I go then copy the finished text into the browser. I should NOT have to go to this amount of trouble, it's stupid and ridiculous that phone ergonomics aren't easier. (They say smartphone sales are slowing, well I'd suggest they'd quickly pick up again if Google fixed these important ergonomic issues.)
5. That raises the problem of where are 'undo' plugins for browsers? If browser manufacturers won't do 'undo' then why doesn't someone provide one (if ever a browser plugin was needed for a smartphone then this is it). BTW, there used to be an excellent plugin that did this for desktop Firefox called Lazarus but I haven't seen it for years. Lazarus not only did undos but also kept track of multiple edits—one could go back to any edit point even days or weks later and use or reuse it. Why are we now deprived of such a useful tool nowadays? There's no doubt computers are getting harder to use these days—not easier.
Right, I'm damn annoyed with the primitive smartphone environment I'm forced to use. It may have been OK 10 or more years ago but it's not OK nowadays. It's notable that with every new version of Android that Google adds junk I mostly don't need (and often useful features are removed that I do need and done on the pretext it's for security or such). It really is high time Google started to address these essential usability issues that everyone needs and wants.
This is such an issue one has to question what's going on at Google. Are people at Google incapable of entering more than one line of text into their phones at a single sitting? Can't they comprehend any concept longer than a single Twitter/X entry at a time? Given the neglect, one has to wonder.
Speaking of selecting text, at least on Android there is a trend in both Google and third-party messaging apps of only letting you copy the entire message, not a substring thereof.
I wish I could take a peek into the brain of whoever came up with that idea -- ideally, after removing it with a large, sharp rock.
More than once I've copied an entire message, pasted it into Google Keep, selected the bit that I want and pasted that elsewhere, and then deleted the note in Keep. Frustrating.
"...there is a trend in both Google and third-party messaging apps of only letting you copy the entire message, not a substring thereof."
Yeah, right. But it's not just confined to Android apps, for years selecting text with the mouse and copying from within Windows Firefox has been flaky, it can work but often it doesn't. If I want such copy to work for certain then I highlight the text and use Ctrl-c/v to finish the job.
Similarly, selecting some text such as part of a URL is nigh on impossible in Firefox (the URL becomes a single entity). Also, when selecting such text it's better approached starting after the text rather than before and working backwards—and even then it's best to start from a few words further on and discard the extra after the paste. Sometimes, with 'awkward' pages it's best to do a 'Select all' on Android or Ctrl-a elsewhere then paste into a text editor and fine tune one's selection from there.
It's ridiculous. I cannot understand why this hasn't been fixed, surely I'm not the only one who experiences these problems.
I can't imagine why anyone would ever want to share an entire speech bubble, without any context (as in a screenshot).
On the other hand, it's obvious that given a message like "My address is 123 Xyzzy Rd, come anytime," the user would only want to select a subset, not the entire message.
Right, it saves typing when using some of the words elsewhere, and or when there's something in the bubble like a URL or awkward Unicode character. Copying the text is much quicker than regenerating it from scratch—or it ought to be!
Also, with an address and or phone number it saves making copying errors (I regularly copy text this way).
> 3. Along the same lines, selecting or editing, say, a URL from a browser address line is painful. When the URL text exceeds the screen width then trying to get to the end of the text is a damn awkward.
Because editing text on mobile is so tedious for all the reasons already outlined, this is basically the extent the of "editing" text that I do on my phone and it still drives me crazy! Stripping the UTM parameters from a url I've pasted into the address bar, pressing down, and watching the cursor sloooowwwwly move all the way to the end is excruciating. Likewise, the cursor often disappears on me as well.
> 4. UNDO, UBDO! I'm typing this into my browser's HN edit box! If I accidentally refresh the page before posting
Haha, I obsessively copy all if I'm writing > a couple sentences on HN on mobile; accidentally refreshing has wiped my text far too many times
Logitech makes a few good Bluetooth keyboards that are relatively compact. The K480 even has a lip to use like a stand. These are often multi-device so you can pair with a desktop and a phone and quickly change between them. I use the MX Mechanical Mini as my main keyboard these days and switch between my laptop, my desktop, and my phone. Were you thinking of something smaller than these options? Or just lamenting it's mostly just Logitech making decent products in this space?
At the time I bought my Bluetooth Armaggeddon keyboard (see link) a year or more ago I looked at the Logitech ones and decided on the former specifically because it was more compact and almost an ideal size to use with a smartphone although I wouldn't make that decision now for reasons stated.
Moreover, despite being a mechanical keyboard, my 'A' key has deteriorated to the point where I have to push hard (often I only notice after the speller picks up the missing 'A'). This is another reason not to recommended it. And to make matters worse the key is soldered to the board in an awkward way (I was planning on swapping it with a key that wasn't used as much but it was too much trouble). There are other reasons too including bad (buggy) software that drives key LEDs. All up, it was a bad choice.
This stuff is always changing so I'll do another survey, also I've had good results from Logitech in the past so its KBs will be high on the list.
BTW, at the time I bought the Mk-17 Am. I'd first done a bit of a survey and I noted several comments/reviews that lamented the lack of Bluetooth keyboards due to the fact that there was little demand for them. I suppose that makes sense with WiFi availability but it also indicates that few bother to use an external KB with their phones or tablets. Thus, despite editing text on a mobile being such a pain it seems few do anything about it.
Incidentally, the mouse I use with my phone is the Logitech dual wireless unifying/Bluetooth model M590. It works well.
For me, the biggest problem is lack of meaningful feedback. Even a small keyboard lets you feel the edge & centre of each key, and feel when it's acted (the click).
And a physical keyboard does not require you to be looking at it, just to use it.
> People produce less text with more errors in more time on mobile.
I mean yes, of course, but how could it be otherwise? A desktop keyboard is 10x the size of an entire mobile phone. The idea that it should somehow be possible to attain the typing performance provided by such a large dedicated input device on a phone the size of a keyboard's space bar is pretty ridiculous.
And I'm unconvinced that the "Eloquent" design presented in the article is a step forward. The "T-Menu" looks terrible and confusing, and I find the drag animations quite jarring.
The only obvious, intuitive mobile interface for text entry is speech-to-text. I know it isn't quite there yet, but that doesn't mean fiddling with details of how cursor movement works on a phone screen has any realistic chance of ever solving the problem.
This is absolutely a problem, but I'm not convinced the author has found a solution. I'd have to try it to know for sure, but from the description, it still sounds finicky.
I believe touch screens are fundamentally a bad interface for productivity. Consider the range of actions provided by a mouse: You can hover without clicking, you can left click, or you can right click, all with nearly pixel-level precision. Add in a keyboard and your options expand even further.
A smartphone is like a computer with a one-button mouse and an abnormally large, irregularly shaped cursor, where can never be sure which part of the cursor indicates your actual click target. Software on this computer is not aware of the cursor's location until after the mouse has been clicked, and portions of the screen are blacked out when you move the mouse to certain positions. Your keyboard only works when you bring up an on-screen overlay which takes up ~35% of your screen real-estate, on a monitor which is abnormally small to begin with.
Could any amount of well-designed software make text entry efficient on this machine?
This is a hardware problem, not a software problem.
Why not something akin to "fixed-offset cursor"? Actual cursor/pointer is always a centimeter above the finger position (thus always visible [bottom of the screen to be handled as a special case]), and finger movements manipulate position, while hold duration accesses alternate modes.
For a somewhat similar implementation (touchscreen is essentially handled as touchpad) look at how Teamviewer handles remote sessions to Windows desktop from smartphones.
I would really like any/all of those as a toggle-on mode for use on Android itself... selecting text with arthritis fingers is a pain – literally and figuratively.
I have trouble imagining that more than 2-3 fingers can be realistically used for input at once. For one, you normally have to use one hand to hold the phone, which uses up 3-4 fingers. And to enable frequent multi-touch gestures you'd have to hover your hand awkwardly over the phone, which is fine for the occasional pinch-to-zoom but not if you have to do it all the time.
But with 2-3 fingers you already have lots of combinations. 2 thumbs are enough.
I have been toying with some ideas for a game and thought about the potential for normal use cases.
Intuitive (like swiping and pinching) is none of it though and it can only be useful once muscle memory kicks in.
So I guess this is the reason, we do not have a global standard for anything advanced multitouch.
Given 2 finger pinch-spread is already taken for zoomout-zoomin, what are the remaining useful combinations for 3 fingers? It can't detect which finger you're using. It should support one-handed operation, so the 3 finger gesture should be possible with one hand. You'll likely have to use middle and ring fingers, which are mechanically connected, so relative motion would be difficult.
I think you're wrong. There are very few combinations available for things like text editing.
"You'll likely have to use middle and ring fingers"
Well, people who played the violin or alike can do this, but yeah, it was hard work to learn and cannot be expected of the general user.
So yes, it cannot be anything complicated.
One of my ideas is to use the position of the fingers and the timing they touch. No matter which finger, what matters is the position they touch down relative to each other. So 2 (or 3) touches, like double click, but with 2 fingers and you evaluate the positions.
Simple variant: "pointing finger", "middle finger" -> one action (one touching point, then another touching point shortly afterwards left of the first touching point)
"middle finger", "pointing finger" -> another action (first touching point, then the other touching point to the left of the first one)
Or you can use the 2 thumbs, or thumb and pointing finger. And use up and down as well. So you have "middle" "up"; "middle" "down"; ...
And the actions could be anything, like "middle" "left" could select the next word to the left or go to the beginning of the line.
But my main thinking evolved around game actions, but it is the same principle.
And then you could also advance it with "middle" "left" "middle", the only limit are the coordination skills of your average user.
And ... the hardware limits. I have never really implemented those ideas, because I got frustrated very quickly(some years ago) with how the native touch handling of the OS got in the way, and some unprecise sensor input. But if you keep it simple, it should be doable. I guess I am now motivated to give this experimentation another shot real soon ..
It's a good point, but I'm not sure how to use those gestures for text entry. TFA didn't go down the multi-touch route either in their proposal. But it's possible I'm not being creative enough!
If you forced me to implement something without doing any research, I’d suggest:
First, (iOS perspective but it’s not that different than Android) to convert the suggestions row to a row of function keys anytime Shift was held down. No annoying delays or extra taps waiting for the menu pop-up. So now you can use two fingers to cut copy paste undo instead of using several seconds and a popup that can appear anywhere onscreen. Next, arrow keys. Heck, use the same Shift hack and, when Shifted, replace the space bar and the dead useless area below it with a big inverted T.
Our phones have gotten much bigger and yet especially on iOS the keyboard hasn’t grown a pixel since what, iOS 5? Case in point: no number row still for Apple, even as an option. Wtf.
iOS already has three-finger pinch to copy/paste, and three-finger swipe to undo/redo. But I find that these gestures are too unreliable for regular use, the gestures almost always activate some other click target plus my other fingers are busy holding the phone.
2 fingers is already too much. Maybe I could've done it with 2010 smartphones, but new ones are so large that I switched to double-tap then drag ages ago. It's annoying having to involve a second hand just to zoom something, and trying to do it with a single hand is just a recipe for destroying your phone.
The problem was they made deep press and long press do different things, so it chose the wrong one regularly. I don't know why they ever thought that was a good idea. It was obvious that deep press should have just been "faster long press".
But seriously, I suppose I just never use enough pressure when I'm "just" regular-tapping to inadvertently activate force touch under any circumstances, so I never had any ambiguous inputs.
I LOVED it. I could move the text cursor with precision AND do text selection with the same precision and FAST. In some situations faster and with more precision than I would have been able to do with a mouse even.
cursor move is still available with a long press of the spacebar, but I dearly miss the selection. that was the killer application of 3d touch I think.
The first gen or so of Android phones had an optical 'trackpad' below the screen, some were terrible but some were really good, allowing far more precise cursor movement than a touchscreen. I wish this feature had survived, it was awesome for text editing.
It did in a fashion, on my Fold 5, for instance, I can half fold it and get a trackpad on the bottom half of the screen, which is great! I wish that was just a general toggle between "direct touch" and "screen as trackpad with a cursor" that'd be just grand.
> This is a hardware problem, not a software problem.
This is a UI validation problem.
Apple proved in 2007 that you can port tons of applications over to the smartphone. They had to invent their own language for interactions though ("pinch-to-zoom" etc) and it took them two weeks of focus with all their software development staff involved to fix keyboards on capacitive touch.
It may not be possible to reach the same kind of flexibility on a mobile device when it comes to rich text editing, but it's certainly possible to port over a lot of functionality from the desktop.
Do you really think Apple invented pinch-to-zoom? CMU Sensor Lab had it back in ’85. Steve Jobs coincidentally visited soon after and later claimed to have patented the technology for the iPhone. That was shown not to be true in the big Apple vs. Samsung patent case.
No, I don't think they invented multitouch, I was referring to the name of the gesture. "Pinch to zoom" is afaik their invention, as opposed to "two or more input points applied to the touch-sensitive display that are interpreted as the gesture operation", as the article mentions.
The issue is about naming and communicating UI in an intuitive manner. Which should be solved by acceptance testing.
As far as clicking and basic functions, is a two button mouse really that different from tapping + short holding to open a menu wheel? There's a lot of pain points to me when comparing a touch screen to a PC setup but the mouse isn't one of them.
I agree, styluses are great! Placing the cursor and highlighting text on e.g. a Nintendo DS is a lot easier than on an iPhone. (Everything else about typing on the DS sucked of course, but it's not like Nintendo put substantial effort into that experience.)
I do find that e.g. the Apple Pencil doesn't have a small enough tip for text selection to work well, it's really made for drawing.
I sometimes wonder if modal editing (like vim uses) might be a good approach for navigating/editing longform text on touch devices.
It does seem like a missed opportunity to have taken the keyboard/mouse approach and then transferred it to touch devices. Even the keyboard layout has no real advantage for two thumb typing on a screen.
Approaches that adapt the interface whilst leaning heavily on letter based inference could be interesting for one handed / single digit entry of letters. Something like dasher:
https://www.inference.org.uk/dasher/dashersummary.html
On an iPhone if you hold the space bar down the whole keyboard becomes a kind of trackpad which you can use to move the cursor.
Since all the keys disappear, it doesn’t seem like a stretch to add something that works like mouse buttons so you could select text or paste in a specific spot in that “mode”.
Right now it’s so frustrating to do any kind of selecting.
You can use that for selection as well, tap somewhere else on the keyboard. Used to be even better with 3D Touch, you did not have to wait for the long press to register.
I frequently use the selection, but I don't like the implementation of it. Rather than the selection being between the starting "anchor" point and your moving cursor's position, the cursor you're moving around just creates an ever-growing selection that can't be subtracted from. So if you're e.g. selecting across multiple lines and you bring the cursor up to the line above, it'll select everything from the starting point to where the cursor is now, but if you bring it back down it doesn't unselect that text. So if you accidentally select something you didn't want to, the selection is essentially ruined and you need to start over. I have no idea why they implemented it like this.
Selection was introduced at the same time as “space bar cursor”, which is with the introduction of 3D Touch in the iPhone 6s.
Actually, it wasn’t even a “space bar cursor”, as the way of triggering it was with force pressing and not by holding the space bar. It works much better with 3D Touch, as you can move the cursor around and press harder without lifting to start selecting, then release a bit to stop. My iPhone 8 can still do it, and I’m really going to miss it when I upgrade.
Unfortunately this affordance isn’t available when trying to make a selection in noneditable text.
Since often on mobile devices I am responding to comment threads (in slack or gitlab or Jira, or indeed right now on HN, for example) the challenges I have in copy-pasting are often in grabbing text from a prior comment to quote, rather than in selecting my own text.
To be fair you can't drop a caret (the edit cursor) into non-editable text in most desktop applications as well. And that's a missed opportunity there too. I think Chrome got less agreesive in changing your selection to match word bounaries, but it was very annyone for a while. I think you could void it by holding Shift, but that had some other side-effects in some pages.
I think internet explorer did this selection thing at one point years ago. I don’t recall chrome doing it. Firefox used to have a ‘caret browsing’ feature but people didn’t use it so it was eventually removed. So I don’t really think it’s a missed opportunity so much as a feature people didn’t sufficiently love.
Thanks, that comes in very useful from time to time. I thought that got disabled.
Chrome did double checked that I know what I’m doing with a popup, I hope this is not a sign this will get deprecated.
Thanks. I don’t know how much RSI I’d be saved from if I knew this years ago. Apple, if you’re listening, you should advertise this better. Ok, maybe not for the new iPhone user, but maybe for a user in his 2nd year or one which does a lot of text input.
> You can use that for selection as well, tap somewhere else on the keyboard.
Doesn't appear to work.
What does work is pushing a little harder (i.e. "force touch") with the same finger/thumb that initiated the move-cursor-via-spacebar gesture.
Only problem there is that sometimes the forcetouch doesn't register (no matter how hard you pinch the screen), or is too trigger happy and starts selecting text when you only wanted to move the cursor.
What do you mean "tap somewhere else on the Keyboard?" Once I hold down the space bar and then let go, the keyboard disappears. And I just tried some combination of this and got into a frozen state where the keyboard was missing and I had to kill Safari.
I've been using nvim in termux on foldable phone since I've bought phone in that form factor. It works great, I'm using "unexpected keyboard" as input method for faster special symbols access. It works pretty well. Good enough for me to program on the go.
Thanks for letting me know about the Unexpected Keyboard app. It'll take some getting used to, but having arrow keys and punctuation available without changing modes is pretty awesome.
I can even hit Ctrl-A to select all, Ctrl-C to copy, etc. This alone will help text editing on my phone.
Seems to be quite an improvement. Thank you for the tip!
But it would be even better if it had "stylus mode" because, with a precision stylus, it's easier to tap directly alternate characters than to drag from the center of the button toward the needed variant.
In gboard we've already got modal stuff for numerical keypad, emojis, special characters (2 different modal pages), etc.
Adding one more modal keyboard page for cursor-editing (arrow keys, ctrl-arrow-keys, home/end, pgup/godown, select-toggle-button, delete, rclick menu) would just make sense. Would just be getting the rest of the desktop keyboard into the phone keyboard, nothing groundbreaking.
Whoa, I had no idea (obviously). Note, that article is out of date, instead of the G icon, now it's a four-square icon. I don't know why they're not just using a standard hamburger icon, but whatever.
They already have an option to move your visit by dragging across spacebar. Why not just replace half of the oversized spacebar key with something useful?
You can install blink shell, panic prompt, or whatever the popular iPhone ssh client of the day is and ssh over to your desktop to get a preview of how this would work.
Paired with a Bluetooth keyboard, it is fine IMO. The screen is a little bit small. Sometimes if I’m going to SSH from my phone, I’ll put the phone on a little stand in front of the keyboard, so it can sit more like a foot from my face, or whatever (normal cellphone usage distance). Or, I’ll put it in portrait mode farther away and think of it as “half a screen.”
Either way works fine for short stints. Nebulous concerns about eyeball heath for long sessions, although I have no real evidence to back that up, and we’re all screwed on that front anyway, right?
I had a Logitech Keys-to-go I was quite happy with but recently got myself this 60% mechanical keyboard, mostly because you can get it with a case that doubles as a phone & tablet stand
There's tons of professional writers though, not to mention business users, lawyers, and other professionals that do lots of writing. It's pretty obvious that desktops are going the way of the dodo, and the aforementioned users need to write lots of text on mobile devices. I don't think modal text entry is any more onerous to learn than a graphic artist learning Blender or Photoshop, or an engineer learning Solidworks.
There may be fewer desktops, but laptops work the same way and they don't seem to be going away any time soon. I'm not aware of "tons of professional writers, lawyers, and other professionals that do lots of writing" who spend most of their writing time on the type of mobile devices described in this article.
I really doubt people are every going to give up big stand-alone monitors.
Putting cellphones on big standalone monitors might work. But in that case, we’ll probably also need to attach a pointing device and some way of entering text, so from a UI point of view, we’ve got a desktop.
Cellphones definitely have sufficient processing power for lots of typical office workloads nowadays… but using them in this way doesn’t seem to have caught on.
I dunno, eventually this sort of discussion ends up at “why didn’t DeX take over the world,” a question for which I have no good answers, since, like everyone else, I never tried it out.
I occasionally use DeX. It is fine, if you are needing to read email, browse the web, or do simple word processing, or need a touch interface for drawing. If I'm going to sit down and work for multiple hours, I'd rather be using my desktop applications.
No, the aforementioned users do not need to write lots of text on mobile devices - a professional writer will choose or adapt their devices and environment to best suit the needs of their writing work, not adapt their writing to better suit the limitations of the devices; instead it's all the "casual" users who need to make do with whatever device they have on hand even if that device was optimized for entirely different needs.
Sadly, no. There certainly are lots of novel keyboards out there that can be googled up, but perhaps the kinds of features that would make a vim/modal-type keyboard be interesting and useful on mobile require further API hooks into the standard text-input controls (to be able to do things like find word and line boundaries, perform complex selections and replacements, etc).
I wonder if ultimately, the advantages of a "stay on the home row" philosophy like vim has just don't really manifest when there's no longer a physical keyboard as the underlying HMI. But the core idea of doing something modal and separating movement/selection from input does still feel valuable.
The problem is far from invisible (to me at least). A few months ago on HN I used text editing as an example of how iOS wasn't ready for "business" use. I can't find the comment now, but my memory is that I listed several of the issues listed in this post and said something like "Text creation/editing is core to the 'business', non-content-consumer use case. Apple needs to either acknowledge these problems and work to solve them, or admit they cannot solve them and stop pushing this narrative." It's especially telling that the author references Apple's 3D Touch as being an enabling technology here, when Apple shipped it without thinking of a valid use case for it, and then discontinued it after several years of still not thinking of a valid use case for it.
It's interesting to me that Google had Tablet Tuesdays. One of the things I've said many times is that it's obvious when a company actually does the thing they're pressing users to do. Google obviously uses gmail, and just as clearly never used any of the social products they released. But I think Tablet Tuesdays doesn't accomplish the thing they hoped it would: if you can use a regular computer 4 days out of 5 -- or maybe 5 out of 6 :-( then you can limp along on that one day and not have enough incentive to actually solve the problems. "Tablet Tuesdays" should have been "Tablet Teams" -- whole groups of people forced to use nothing but a tablet, with no way to hide from the problems that caused.
Eloquent seems like an excellent existence proof that better is possible. Personally, I would try multi-touch gestures to solve some of the problems. It might be (okay, likely is) too complex a solution, but Apple (at least) can detect up to ten(?) separate touchpoints. That would be absurd, but I'd be curious to try copy/paste with multi-touch shortcuts. And it seems that selection might benefit from multi-touch as opposed to the (admittedly clever) pressure hack Eloquent is using.
Front page of HN (or any story page), search box at bottom of page. Enter “gcanyon iOS”. Click on “Stories” and change it to “Comments”. Click on “Popularity” and change it to “Date”.
(Actually I first entered “gcanyon iOS business”, but did not find it.)
> Google obviously uses gmail, and just as clearly never used any of the social products they released
Employees use for example Gmail and Google Docs, and it's critical to their productivity.
It can be hard to get the same feedback loop for other products. Some Google employees used Google+, but not that many, and they didn't need the same features as external users. And even if they complained internally about some things, it was not critical anyway.
My biggest problem in editing on mobile is changing a text selection (for example a long URL or long text paragraphs) that don't fit on screen at once.
Once I have to scroll (vertically or, even worse, horizontally) in addition to moving the selection handles the UX is just terrible.
I don't know about IPhone but on Android its simply impossible to Paste text between two words. You can't paste text into cursor position you can only replace selected text with Pasted text.
Thats my main biggest issue.
Also, why the fuck there no Copy/Paste buttons on mobile keyboard??
You actually can paste between words but it's hidden (like most things with mobile text editing) You need to place the text cursor between the words carefully, tap the 'teardrop' on the text handle, and that will bring up the menu to paste. (not saying that's good!)
Just tried this. I find tapping to bring up the teardrop sometimes changes the cursor position.
It's quite unfortunate the paste button is not always shown when editing. At least the teardrop (if not an edit bar) should be shown when using long press spacebar to move the cursor.
I really like the additional controls AnySoftKeyboard[1] brings: I like to configure it so that it always displays a left/right arrow above the keyboard to move the cursor. And if you "lift" (gesture up) the space key, there is an additional copy/paste (with clipboard manager)/arrows/selection/undo/redo/etc menu, which is quite useful[2].
Unfortunately I stopped using it in favor of OpenBoard[3] due to subpar autocorrect, especially when typing in French[4].
I'll have to try OpenBoard. Right now I'm using ASK and I really want to like it, but it's just slightly too janky and rigid. The customizability is nowhere near good enough. There's too many behaviors that seem hardcoded and impossible to change. Like when my battery is below 15%, autocorrect and haptics are turned off. Who thought that was a good idea?! Now my phone is almost useless for typing when the battery is low.
I eventually learned to place the cursor, squiggle a little random swipe (and a blank) that I can then select and replace. It's certainly an unintuitive nuisance, but my personal biggest issue is something else:
No advanced keyboard seems to have a setting to not force an automatic blank after each and every swiped word, no advanced keyboard beyond the original Swype that was abandoned a long time ago. Some have elaborate workarounds for fixing blanks before punctuation, but none seem to allow to leave space bar operation to me. I use a lot of composite words (hello from Germany) and the forced blanks are just infuriating. So I use anancient keyboard that was abandoned at some point between flappybird (composite word!) and 2048 or earlier and can only hope that Android keeps doing acceptable backward compatibility...
have you tried florisboard? i would probably use that as my main keyboard but the way it doesn't add a space after words when you swipe something kind of bugs me, especially after getting so used to it with other keyboards
There are copy/paste buttons on Gboard, but they're kind of hidden. Press the 4 squares in the top left of the keyboard and select Text Editing. You get arrow keys, a button for toggling select, and cut/copy/paste. In a way it's like switching out of insert mode in vim.
What if I don't want the keyboard monitoring my clipboard? Nothing should be monitoring my clipboard. If I choose to paste from the clicked, at that point you can look at it, not before.
I agree, URL editing is infuriating especially with long URLs with massive tracking query strings (e.g. anything opened from Facebook). But not only. Sometimes I want to change subdomain of a long URL (e.g. from reddit to old.reddit) and it requires a lot of scrolling to the left.
> We saw this in our user testing when users tried to place the text cursor accurately: they would miss by a few characters...
One small thing: I don't know how it works on Android, but I used to have an N9 and then the Jolla phone and you could tap anywhere in text to place the cursor there.
iOS doesn't let you, except confusingly on the very first tap that activates the cursor. For subsequent taps, you can tap exactly where you want in the middle of a word and it always snaps the cursor to the start or end of that word.
I'm pretty good at aiming at the right character to edit even with big fingers on a small screen. Let me do it!
I know about it, but that's a second action (moving it after tapping), or a more difficult one (moving it all the way there via the spacebar touchpad). Most of the time if a tap would put the cursor where I tapped, it'd already be there.
Edit: I just discovered now that if you long-press on the text to place the cursor, you get a little magnified view and it does let you place the cursor in the middle of a word. So that's probably the most efficient method currently available.
> Edit: I just discovered now that if you long-press on the text to place the cursor, you get a little magnified view and it does let you place the cursor in the middle of a word. So that's probably the most efficient method currently available.
I chuckled a little at this as this is one of the oldest features of iOS, probably even from back in the pre-iPhone 4 days.
Holding down the space bar on my (Google) keyboard brings up a popup to change the keyboard. Sliding my finger along the space bar shifts the cursor.
I miss my very first Android phone (the original HTC Desire) which had a tiny hole they called an "optical trackball" that worked incredibly well for selecting text.
Ah, that makes me think of the PlayStation Vita and its back-side touchpad. This could be a nice addition to smartphones. The only (substantial!) challenge is to make it usable while holding it.
On Android it often stutters, the cursor jumps across the characters at random speeds, sometimes it teleports sometimes it moves one character at a time smoothly. When you lift your thumb it often registers as an extra motion in an unexpected direction. And this is on a flagship Google phone. And it only allows left-right motion, not up-down.
So even if you know about it. It's more an exercise in frustration than anything.
It's not "on Android", it's whichever keyboard came with your phone. On iOS extremely small number of people use non-standard keyboard(it was later allowed), but on Android the keyboard was always an app and there are many.
The Google keyboard on a Google phone for some locales can be very different. Notably, the keyboard for Japanese doesn't do the spacebar thing, on the other hand, it contains actual keys to move left and right.
That's not been my experience on Android, I can go left right up down just fine. I've tried this on SwiftKey keyboard however but I'm sure it's the same in GBoard.
It's really bad in GBoard. I just tap the text itself and if I get close enough to what I want to edit I just backspace and retype. I regularly remove 5-10 characters just to fix a single mistyped one.
> I'm surprised more people don't know about this feature.
I have an iPad without that feature, and it's maybe 5 years old? I think it's a newer feature.
Also, in the past you could tap and hold on text and it would magnify the view around where you were tapping, but that feature was dropped at some point.
> Also, in the past you could tap and hold on text and it would magnify the view around where you were tapping, but that feature was dropped at some point.
I got my iPad, tried several different apps, and I now see that the feature is present in all of the ones I tried.
I think what I did differently this time was hold the spacebar down long enough for the feature to kick in. On my android phone there isn't that long of a delay, so I think that's what threw me off.
I know about this in Swiftkey, but since I'm using more than one language, this action is used to change languages instead. It's unfortunate because moving the cursor is very useful.
You know what was great about text selection on the N9? When you dragged the cursor, you got that haptic tick-tick-tick for each character the cursor passed over. I'll admit I don't entirely understand why this was so beneficial, but somehow it made it substantially easier to get the cursor to the right spot.
Yeah man, everything about the N9 merged hardware and software well. Like the curved glass and smooth edges going along with the edge swipe interaction.
There are 3 ways to position the cursor on iOS in normal input fields:
1. long tap 2. long tap on spacebar 3. move 2 fingers simultaneously on the iPad virtual keyboard (iPad only).
For all the frustration i had with the iOS keyboard, it‘s actually quite good.
Real problems are often created by 3d-Party JavaScript tools, a old version of codemirror for example.
I typed 55wpm on my blackberry without looking. The fact that you have to constantly look at the screen keyboard and correct it is a huge attention suck and kills my input speed
On blackberry, a mistake was one wrong character. On screen keyboards with swipe and autocorrect, a mistake can be inserting 1 or 2 random words
Screen keyboard doesn't work in the rain
while we're griping:
- on an older android device the built-in keyboard is such a pig that it sometimes requires you to slow down to like 1 character per second. Note that this worked fine on a nexus 5 with aosp a million years ago, so it's not like it's not a solved problem
- Swipe keyboard is in theory good, but the keyboard can't switch from swipe <-> tap smoothly enough and usually causes an error
- droid has the ability to drag inside the spacebar to move the cursor, but the first time you do this, it inserts a word instead because it's confused about what mode it's in
Typing is a separate problem. And if you want a physical keyboard on a mobile device you can actually have one. But that still won’t solve editing.
Editing on your blackberry was even clunkier than the touch affordances that this post is about - just cursor keys, right? Now, you might argue that if you can type fluently you won’t need to edit as much - but the point here is about enabling mobile devices to be much more than just message input devices, but actually to do things like revise documents. You’re not using text manipulation affordances just to correct typos but to make significant changes to existing bodies of text.
Though in fairness to awinter-py, given that text entry is a simpler task than text editing, if typing support is already insufficient, editing won't be any better.
I'm typing this at a hardware keyboard on a desktop computer. I've had to make multiple short edits, mostly backspace/retype, as I enter this short comment. The fact that I can look at the screen rather than have my attention focused on the keyboard is itself a huge benefit to writing.
I find I almost always prefer using a stylus to a finger. That's the Onyx's pencil (previously as Staedler, unfortunately since broken). I can see what I'm tapping on with the stylus, positioning is far more precise, and it doesn't smudge the screen.
It's still hell for text editing, however.
Unless I'm doing actual handwritten note-taking, which might be another option.
no the blackberry had a physical trackball with toothy tactile feedback for motion and a mechanical click action
you could scroll vertically and horizontally
I don't remember if you could highlight, but there was a shift key so I'm assuming yes, and I vaguely recall double-tapping backspace for whole-world deletion
I was the same way. I only had a Blackberry for a short while -- a BlackBerry Bold 9000 (got it in 2009). It was my stop-gap between plain cell phones and my first Android phone (a Droid 4, got it in 2013). Then I moved to touch screens with the Samsung Galaxy S4. I'm currently a Pixel 7 user.
Nothing ever matched the typing speed of the Blackberry Bold 9000. I was slowed down a bit by the slide-out on my Droid 4, and slowed down a TON by switching to touch. I could actually "do email" on my Blackberry. At essentially the same speed as my laptop. But, even today, with many years of practice, including swipe, text prediction, and all the rest -- writing a mid-size email on my phone is excruciatingly slow vs my typing/thinking speed. Even more frustrating is that when people observe me typing on my touchscreen, they'll say, "my goodness, you type SO FAST." And I'll just think, this is half-speed for me.
As a simple example, sometimes I try to transcribe text from a podcast I'm listening to on my phone. It's basically impossible -- I have to go back frequently, slow it down to 0.7x audio speed, etc. But, if I transcribe text on my laptop, I can do it so fast that I type ahead of the speaker, even a fast talker. It makes a difference. I could have transcribed a live speaker on my Blackberry Bold. I could even type at thinking speed, which was great. Alas. Two steps forward, one step back.
I think autocorrect is overrated and actually makes the typing process much more annoying. Also I’m not sure if 55wpm is supposed to be fast or not? I just did a typing test (monkeytype.com) on my iPhone and got 80 wpm with 0 mistakes, and I don’t use autocorrect.
Just tried and got 72wpm with 97% accuracy without looking… though honestly I think not looking is a bit pointless, since the keyboard is so close to the text anyway. It’s not nearly as jarring as looking at your keyboard when typing on a desktop.
Yeah your laptop/desktop speed is faster than mine, I’m somewhere between 100-120wpm there. I wonder if my faster phone speed is due to my weird typing style where I use my left thumb and my right index finger? heh
Earlier this year I spent some time with a Unihertz Titan Pocket, which sports a physical keyboard. It gave a vastly better typing experience than any other smartphone I've interacted with.
You can disable autocorrect and get your old behavior. I don’t understand this complaint. The only thing missing is feeling the keys, but this is completely fixable:
The vast difference in editing things on a MicroPC with its touchpad and physical keyboard versus a phone is immense. I'm pretty sure I can legitimately type faster and usually navigate UIs faster on a capacitive touch phone, but the frustration of typing and targeting the cursor is unbelievable even after having used smartphones for over a decade now. It's just bad.
It is amusing that it's hard to convince people this is a problem, but I sort-of understand. Over time people have learned to just, not edit text on mobile. There's relatively powerful versions of office suites on modern mobile OSes, certainly more powerful than Windows CE devices that had full keyboards would ever ship with, and yet most people don't even really consider doing much on mobile other than sending messages and taking notes, two things that rarely require dragging the cursor. When editing things you quickly type out, gestures like dragging the spacebar to move the cursor around is usually "good enough" for making small edits to fix typos or change the wording, which makes it feel like a non-issue.
On Pinephone with Squeekboard, I greatly miss the ability to drag on the spacebar to move text, and even slightly miss the ability to swipe across keys to type. And yet, the weird thing is, even though text editing on Phosh is significantly less refined than either Android or iOS... I ultimately don't have much harder of a time doing it. And I think that speaks volumes on its own.
Every time I pick up my pinephone I feel immense disappointment in what forms of interaction most devices are stuck with today, despite how easily alternatives can be implemented in an open platform as the pinephone has shown. But for just editing text, I find vim and emacs surprisingly usable with a touch keyboard, as long as numbers and needed symbols and modifiers are on the base layer.
Precise pointing / fat fingering is solvable for text editing and in general by using the touchscreen for relative input - like a touchpad. That's possible on the pinephone with a userland program that directly interfaces with evdev and uinput in a really simple way[1], taking the ability to run desktop software well beyond being a party trick. All it's missing is a scheme where single, double, or two-finger taps and drags are either relative or absolute to avoid having to switch modes. Or, because the touchscreen, like most, reports touch area, one might have a go at cloning force touch.
For swipe input, wvkbd[2] has experimental support that works amusingly well for how sucklessly it's implemented (see the readme), albeit only for long words or reduced dictionaries - so many possibilities, like having zsh write completions to a file. It does need a patch to not interfere with normal typing. Spacebar swiping would also be straightforward to patch (or on sxmo into lisgd instead*). Alternatively, a small key could receive a 4-way swipe gesture like the trackpoint that Windows 10 Mobile had. (btw, hey, the MessageEase patent expired...)
* <rant>I don't know how the devs tolerate the latency that sxmo_inputhandler.sh brings - handling basic OS shell gestures in a long shell script on a platform where every expansion piped to grep causes a noticeable increase in latency is very unsuckless!
Ah, I've tried to make my Steam Deck+sway setup usable when undocked, but there are bugs everywhere, from wvkbd mishandling seats or straight up drawing an invisible user interface, to sway not detecting gestures (lisgd seems to work well, though). There are plenty of physical buttons on that platform, but it seems the default kernel driver does not handle them (need to have Steam running, which then sends input through... X11).
It doesn't help that the interfaces for emulating and intercepting input are very rough or experimental (well, there's libei but wlroots doesn't support it). Maybe I should work on a libinput wrapper or my own "parent" compositor?
Anyway, all touchscreens should be able to sense 3D coordinates, which could be used to move a cursor around the screen while hovering. This could be very usable, especially the cursor is shown above the finger position so it isn't hidden. This alone would help a lot with the issues pointed out in the article.
I really wish I could bring myself to get into sxmo more. It seems like something I'd like, but I mostly find myself confused when I try to use it. Even if it's not perfectly suckless, maybe it's a bit more in the "suckless" dimension than I can bear--I did like dwm for a while, but I switched to i3wm and later Sway and never really looked back.
There's obviously a lot of small little projects that implement cool ideas, but one thing that is a little bit of a bummer is that there's really no obvious solution that you can flash onto a Pinephone and call it daily-driver ready. It would really be nice if standard-ish Linux desktop environments could be adapted to work well on phones; I mean, at this point, the proof-of-concepts are enticing enough for me to believe that it'd be worth the effort. That having been said, I've been wondering if maybe to get the Pinephone to a working state, if it'd be better to actually go for a very minimal base system and try to build a more or less non-standard usermode. Something like, musl, pipewire, eg25-manager, a custom wlroots compositor that implements most of the actual phone features directly, and some simple system software to go under it (file browser, terminal, etc.) The main thing I really want is to get the absolute best possible battery efficiency, something that can manage rtcwake to occasionally check for notifications and handle some basic alarm clock functionality, and intelligent enough system software to e.g. restart the EG25 when it seems to be stuck. (Usually on Phosh + Debian, restarting the eg25-manager systemd unit is enough, so apparently it'd be good enough to just find a way to detect when it's stuck and restart eg25-manager.) It's a lot of work, and I admit that it feels like you'd be going a tad in the direction of Android by ditching most of the standard userland. But on the other hand, there's so many damn projects involved with most functionality in the device that it is a bit difficult to even know where to start when it comes to troubleshooting, and in my opinion it'd still be nicer than Android if the userland was "standard" enough to still run typical desktop apps and run more-or-less stock kernels.
Then again, for now, I feel my frustration would be better spent trying to debug what's already there. I'm wondering if maybe it would be possible to improve the wake times, for example... I have a sneaking suspicion that most of the resume time is taken before the Linux kernel gains control, but maybe it'd be worth trying to get something like pmgraph running to see if there's any room for improvement.
> Why should we privilege an intentionally nerfed computing experience as the inevitable future?
The fact that the author worked at Google should be a hint. Advertising companies like Google want desktops and laptops to be left behind in favor of phones and tablets specifically because of the inferior, more locked down and consumption-focused computing experience they provide, and I'm guessing this idea is hammered into all their employees.
I agree with your general point but there's nothing inherently bad about the smartphones themselves. They are perfectly good computers, it's these big corporations who turn them into locked down approved content consumption machines. We shouldn't be dismissing these computers just because of that, we should be working to empower ourselves to use them to the fullest with projects like postmarketOS.
I'm waiting for the new Pixel to come out so I can buy it and run grapheneOS on it. Then I'll try to port postmarketOS to my current phone.
Just because something is designed for consumption doesn’t mean we should ignore text editing.
The fact that there are other devices that are better at text editing is a horrible reason to leave text editing on mobile devices as a terrible experience.
Phones, sure, but there’s plenty of good productivity and creation software available for tablets. And these days, most productivity software runs in the browser anyway.
Because using my phone while lying down in bed or sitting in my couch is so much more comfortable than using a bulky laptop or personal computer while sitting in a chair. I've written way too much code inside Termux. It's gotten to the point I only use my laptop to edit open street map now.
I'm looking for a way to build new Android apps inside Termux itself. Wonder if anyone here's managed it.
I wrote programs on my calculator in high school, an ostensibly narrowly focused device with computing power generously estimated at well under 1% that of your phone, made by a company that didn't really give two shits about their device's programming community.
Therefore, it's hard not to come to the conclusion that if someone as dedicated as you is struggling to write programs for your phone, on your phone, it's because you're being actively discouraged from doing so. Demand better! As a consumer the only voice you have is your wallet. Can you not find a real computer that's comfortable to use in bed?
> Can you not find a real computer that's comfortable to use in bed?
Real computers are not made for that. I don't think there's a comfortable way to use a keyboard and mouse while lying down.
I had better luck trying to find a bed for use with computers instead of the other way around. Even that was extremely difficult. I'd need an over bed or zero gravity workstation that's more expensive than the computer itself. I can't justify this cost for my recreational programming.
Nothing on mobile is okay. I know everyone loves their smart phones but they really are terrible computers in every way except for portability. Use a computer when you actually want to do things. Smartphones are just for when you're literally unable to use a real computer.
> Smartphones are just for when you're literally unable to use a real computer.
Which is literally extremely frequent, since we often need to do stuff on the go. Which is why an article like this is so important -- we should improve, not give up. There's no reason to think text editing on a phone has reached its final form.
The reason I think it won't get much better is that the size of smart phones will not change. It is size that's really the issue for text editing. No space for a real keyboard, no space for a real display.
Maybe we can look at it as an opportunity for inventing a fundamentally different approach to text editing on mobile?
In another topic on HN people are discussing how software gets more bloated over time, because throwing more powerful hardware at a problem is easier than optimizing the software.
As mobile devices aren't likely to get much bigger or get equipped with a mouse and keyboard anytime soon -- in other words, we don't seem to have a hw solution -- isn't it the perfect moment to try something novel and different?
Sure, it isn't a given someone will come up with a great, new approach soon enough. Still seems worth it looking at this as an opportunity.
There are novel approach ideas. I have many of them, and even saw a few implemented. The problem is, any sufficiently comfortable text entering/editing experience requires a non zero amount of training, which, considering the consumption devices smartphones are, discouraging for the average user. I believe in the end TTS with LLMs win and speaking in public to text will be accepted as a norm. Because it is literally the easiest possible experience bar brain implants.
Current mobile keyboards and screens are pretty much fine -- I have no issues with text input on a phone or with reading.
The problems are with editing only -- precise cursor positioning, precise selection, and then access to basic cut/copy/paste/undo operations that doesn't mess everything up.
My hunch is that we need a button on mobile keyboards to switch to a kind of "edit gesture mode". Some kind of swipe area to move the cursor, some kind of swipe area/mode to extend/contract a selection, some method to handle scrolling as necessary, some kind of magnifying zoom to select tiny things like narrow punctuation, and separate larger button areas for cut/copy/paste/undo. Maybe instead of swipes there are gestures in a kind of dedicated trackpad-type area of the screen, I don't know.
But I definitely think there's a ton of area for experimentation that hasn't been explored yet. The hold-spacebar-to-turn-keyboard-into-trackpad-to-move-cursor mode was a first step the iPhone took towards this, but I think it can go 20x further.
I think it's something that only Apple and Google are capable of developing right now though. I don't think there are enough API's exposed for third-party keyboards to directly control things like text selection, zoom, scrolling, cut/copy/paste/undo, and the like.
I remember the first Android phone. It was small, and had a real keyboard. It's not that it can't be done, it's that it's cheaper to not have one, and they can get away with it because of the lowest common denominator consumer.
Meh. I remember thinking "but why?" when the "high res" screen phones came out. It did the job, and I didn't really see the point in a higher resolution back then. Honestly, if I had the choice, I'd go back to that device in a heartbeat, low-res and all.
Do we though? It is a fair assumption, but I urge you to stop and inspect. We didn't "need" to do most things we do on the go 20 years ago, do we now? If we're being honest, most people using their smartphones on the go are not for critical situations.
except for bottom-of-the-barrel androids, most drcent smartphones cost equivalent to a decent computer. I refuse to accept the garbage UX on an equivalent device which charges me such money and then dumb it down with tracking and ad shenanigans over usefulness.
P.S. My own opinion and it is very OK to 150% disagree with me.
If slide-out keyboards were still a thing they wouldn't be nearly as bad but we had to get rid of those because being thinner is the most important thing in the world
And then because on screen keyboards suck so bad we had to make phones BIGGER which meant they needed to be THINNER which meant cameras needed to stick out, and here we are, in a world where an iPhone Pro Max takes up the full height of my jeans pocket and digs into my hip when I sit down. Yeyyy.
iPhone mini was a thing for a while. It still has the stuck out camera but otherwise only a bit larger than the iPhone 5 from 9 years earlier. I wish they hadn’t ended it on the 13, because I also feel phones are getting too large.
That’s for sure why it was discontinued. I’m sad it’s gone, but I do understand I’m a minority in that feeling.
I was just offering it as a counter point (from as recent as last year) to the onscreen keyboards necessitating huge phones that the comment I replied to was suggesting. Plus an option that is still modern enough to use. Using the mini keyboard isn’t much worse than the pro. It’s feasible, these minis type well, I don’t think it’s the reason for bigger, thinner phones with cameras popping out. I’d blame content more.
Don't think I'd even need a full keyboard. I had a much easier time writing on my old Nokia 3310 or similar than I've ever had on my newfangled "smart" phone (it's quite dumb).
I recently bought a PineTab2. Its a pretty rough around the edges experience (hardware is fine, but Plasma Mobile is still lacking compared to Android) BUT with the case with a keyboard it is far better for writing anything. Not as good as my desktop, but a lot better than most mobile devices.
I usually use Whatsapp web and the Signal desktop app at my desktop when I want to write more than sentence or two. A physical keyboard makes it so much easier. For example, I left making my comments here today until I sat down at my desktop.
If you must use a phone for typing more than a tiny bit, buy a keyboard for it. Even a little bluetooth keyboard is a lot better.
I am tempted by phones with keyboards. I used to have a Blackberry a long time ago and there are Android phones in similar formats.
They are better for taking photographs simply because they're smaller, but for every other aspect of photo viewing, management, and editing, they are worse. They have a tiny screen, insufficient local storage and compute, and no keyboard or mouse.
They're not better music players. Sound quality can be identical because phones support USB DACs, but again, everything else is worse. On a real computer I can easily search my library just by typing. There's no way to find specific tracks that fast on mobile.
I think they are also great for photo viewing, because they are easy to show to others. As the saying goes the best camera you have is the one you have with you.
Same reason I listed them as the best music players - I don't want to carry around a laptop to hear tunes. And while some audiphiles claim that they can hear a difference, I have never seen evidence that this is the case.
If you EQ your headphones, e.g. to match the Harman curve[0] to better replicate the sound of loudspeakers in a moderately treated room, you'll probably find they need a lot of bass boost. This means you have to turn everything down to avoid clipping, which hurts the signal to noise ratio. If you only listen to low dynamic range music you still might not notice it, but for high dynamic range music, like orchestral music with natural sounding mastering, you will probably be able to hear noise from low quality DACs in the quiet parts.
Fortunately, good DACs are cheap now, e.g. the Apple USB to 3.5mm adapter [1].
>Same reason I listed them as the best music players - I don't want to carry around a laptop to hear tunes. And while some audiphiles claim that they can hear a difference, I have never seen evidence that this is the case.
For playing music when I'm at work, etc., my phone sounds MUCH better than my laptop that's sitting at home.
Also, even if we really did compare apples to apples, I'm not sure how the sound hardware on a laptop is going to be any better than that on a phone.
I generally agree with you and the parent post. But touch screen is the best UI for photo cropping. This is awkward because it makes me want to split editing tasks between devices.
I've never used one. I know they're popular with artists but I have very little concept of how they're used. I would not have guessed that anyone uses them to crop images.
Given that what many touchscreen tablets (particularly e-ink) offer is a Wacom layer, the advantage is that you're getting far more precise indication of location. The difference between Wacom-in-display and Wacom-as-peripheral is much as with a mouse, in that you have to get accustomed to indicating actions off-screen which translate to on-screen movements, though in practice that seems to be a fast learn for most people. This can actually be better as depending on screen registration and depth of the display elements themselves, "direct registration" often isn't with offsets and parallax.
The precision of using a stylus and pen/pencil grip rather than a mouse, trackball, or trackpad is the primary win.
And yes, cropping is in fact a major function for graphic artists, though painting directly with the stylus + tablet is another option.
Assuming 24 bits per pixel, that's 1 Gigabit (128 MB) per image, or ~1,000 images on 128 GB storage.
Dedicated cameras have removable storage and professional photographers will carry multiple 500 GB -- 1 TB+ storage cards, swapping them out through a shoot.
I'll note that the iPhone 15 Pro specs list a 48 MP sensor, which is similar, though storage tends to be as jpeg rather than raw. Compression varies but sizes of 1/2 to 1/15 RAW at high quality are common.
>Assuming 24 bits per pixel, that's 1 GB per image
1Gb, not GB. And in practice it's not that high. Most digital cameras use a Bayer filter, with the pixel count specified as the number of grayscale pixels behind the individual filter elements. Raws are most often 12 or 14 bits per pixel, so about 68MB or 79MB for a 45MP sensor. This can be further reduced with lossless compression. Still a lot for a 128GiB phone, though.
Android even supports access to raw pseudorange data from the GNSS chipset now. Today's phones support all the constellations and new technology like GPS L5. There's even a way to turn off things like duty cycling to maximize accuracy at the expense of battery consumption. Should be possible to set up centimeter precision relative positioning with these things.
They're better for some types of photos. For example, you'd never want to take a photo of something you'd want to measure quantitatively with a smart phone. They hallucinate all the detail.
The smartphone is the second best device for pretty much every task. For the vast majority of things, in which we’re not specialists, they are pretty good.
It’s an interesting take. I had to stop and think for a second “is there anything I prefer a phone for”, and yeah the only positive of using a phone (for me) is portability and multi functionality (being a de facto camera/camcorder/recorder).
I've been using a little $220 laptop for the past 15 months or so. Wasn't even the cheapest one at the nearby big box store. Way cheaper than most smart phones. In that time I've learned Python and made like 20 apps, done all the job applications and Google Voicing I've needed to do. I don't even have a working mobile phone anymore.
Plenty of battery life, like 6 hours on full bright. Not hard to find ways to plug it in, draws like 20W from my Jackery 160 which is the smallest model yet big enough to charge it a few times. The computer is not much bigger than a big iPad. I just have a little backpack to carry it in, too small to look homeless (tho I am homeless).
Not that my experience will change the opinion of mobile addicts. It's an IYKYK kinda thing (if you know you know).
My "phone" is really just a camera+GPS that my wife sometimes calls me on when she can't find me in a store. I dread typing on it, hell I dread _talking_ on it even.
"Smartphones are just for when you're literally unable to use a real computer."
When I am lying on a couch or in bed, I could use a laptop or tablet, but for just surfing a bit, a mobile phone is just more handy. But sure, smartphones or tablets are not really useful for anything productive.
Editing photos in lightroom mobile works fine. Interfaces that are about swiping and sliding - from match 3 games, through tinder, to scrolling feeds - are very responsive.
What an absolutely lazy assessment of technology. I’d say using those two appendages we call legs to move around while still being able to do many things you use a desktop computer for is a revolutionary piece of technology.
Do you really want to carry a Walkman around along with a backpack with a Rand McNally map to guide you so you don’t get lost in a neighborhood you’ve never visited? Maybe watch a portable DVD player on your way to/from work on the subway to pass the time? Carry some quarters so you can call someone at a pay phone or pay for a parking spot?
> I’d say using those two appendages we call legs to move around while still being able to do many things you use a desktop computer for is a revolutionary piece of technology.
The above poster did acknowledge that smartphones absolutely win out in portability.
Obviously you're stuck with mobile UIs for things like navigation, directions, location information, and a lot review apps, ordering a car, etc. Consequently, the user experience for those things is often better on a mobile phone than it would be on a desktop (if a desktop UI is even provided)
For anything involving typing more than an address, you're almost always better off with a Desktop application. There's nothing "lazy" about pointing this out
> Obviously you're stuck with mobile UIs for things like navigation, directions, location information, and a lot review apps, ordering a car, etc
The "etc" in your statement could be expanded to 100 additional useful things in everyday life. Given that, how can you agree with their original posters statement that mobile devices are terrible for everything except portability? It's such a ridiculous understatement.
let's say only large few lines of text are okay, and some images are... okayis. everything else is a magnificent display of massive UI failure, sadly it took us 20 years to actually start talking about this seriously.
I’m so much faster with touch than I ever was on my BB. But your comment made me realize how much I’d love a jog wheel where my right index finger rests (top right side) while my thumbs are typing. I’d use it for adjusting the cursor location or press in to select a word for replacement.
What you're describing is how my very first cellphone (a Sony PCS phone, model CM-B1201SPR) operated. It was awesome, and I distinctly remember the mini-grieving I did when I replaced that phone with a Nokia. Snake was cool, but I missed that jog wheel!
And yeah, imagining it already feels right to me as well. I just picked up my iPhone and imagined how cool it would be if Apple took their digital crown, flipped it sideways, and submerged it into the edge of the phone. Sure, waterproofing might be tricky, but they figure that out for other penetrations through the case, so I have faith.
No mention of "undo"? That's usually my biggest bug bear, no way to undo accidentally deleted/garbled edits. Oh and the fact that intended deletion of half a sentence via backspace often ends up deleting far more than intended.
I've been wondering what I'm supposed to do with my third thumb. /s
I just tried using this in a text screen. My first attempt was on the keyboard which added random characters and brought up the little thing at the top to undo, my second attempt was above the keyboard, but my most recent message was a link so I got taken to safari, and my third attempt in a small sliver of the message screen in the upper left worked.
Obviously if you're familiar with it, it's easier to do, but I rarely have three fingers available while I'm writing text on a phone.
I tried that with an iPad once - if there's a way to do so without looking faintly ridiculous I'm all ears. But I'm an Android person as far as phones go, lack of a standard system wise undo command isn't quite enough to send me back to Apple.
My solution to this would be to enable the Assistive Touch button and set its double tap action to 'shake'. If you don't want the button all the time, create a personal automation shortcut which runs when a specific app opens and closes, setting Assistive Touch on and off as you wish.
Alternatively have Voice Control enabled and say "undo that".
Seriously, it is beyond ridiculous that the default keyboard doesn't have an undo button anywhere. Copy/paste take way too many taps too. I like the changes to cursor motion proposed in the article.
It’s clunky, you look utterly ridiculous whilst doing it, but it is occasionally very useful. I’ve found that the UI gets better about putting undo arrows everywhere, though. But then even that is not really elegant.
There was a company, Tactus, who developed a touch screen that had little physical bumps pop out of the screen to represent your keyboard. I’ve always wondered why it never caught on, but I’m sure the technology came with a host compromises. Anyhow, it always seemed like a neat idea. Perhaps it would even allow for a denser button arrangement, giving users their arrow keys back.
In other news, I typed this on my phone, having to edit parts of it, and failing to simply drag my cursor to the end!
Apple had a solid solution with 3d touch -- press a little bit harder and it was near instant jump to right position/highlight. I felt much faster inputting on my iPhone then. I miss it every day.
> Haptic Touch is a feature on the iPhone XR (but not the iPhone XS) and later iPhone models replacing 3D Touch. The touchscreen, which no longer has a pressure sensitive layer, distinguishes between a tap and a long-press using a timed delay to activate certain 3D Touch features (only ones for elements that do not have an action assigned to long press). This feature was added to the iPhone SE (1st generation) with the iOS 13 update and to any iPad capable of running iPadOS 13. As of watchOS 7, only Haptic Touch is recognized, and Force Touch is discontinued on all subsequent Apple Watches.
The whole point is that, it enables a whole new type of interaction, that is: press to get into cursor mode -> hover over a word -> press again to select it. This is simply impossible without 3d Touch.
The fact that I've see this exchange many times ("But cant you long press to get a cursor?" / "Yes but 3d touch is faster"), even from people defending it, is so fun to me because it means the technology really was misunderstood, and explains a lot about why Apple killed it...
If I'm remembering correctly the main difference is that it was fast -- the space bar cursor is fine but I do feel like I lost something that made my input on mobile better.
Before: I'm typing quickly, and I want to move cursor, I press slightly harder.
Now: I'm typing quickly, and I want to move the cursor, I press and wait.
iOS17 has a "faster" haptic touch setting and it helps, but with 3d touch I was much faster.
When in an input field, you simply tap to place the cursor. If you need finer placement you use the spacebar. The tap works 90% of the time since you’re usually editing whole words.
Also, iOS has kept a software implementation of force touch around, even after removing the hardware: a hard press goes directly into cursor placement, it takes a bit more force than it used to. I imagine it’s a mix of gyroscopes and detecting a growing touch surface.
Yes. It’s a long press, though, not 3D Touch or whatever they used to call it. Admittedly this got much better with the new shorter delay to activate long press in iOS 17.
It’s restricted to the space bar, too. IIRC it was much better on my old 6S.
3D Touch capable models (6s to x) had actual pressure sensors embedded in the screen assembly, which meant there was another input channel (how hard you pressed) that apple mapped to an extra commands, like a right click. To highlight text with it, you could press slightly harder anywhere on the keyboard and that would start cursor movement mode (like holding on the spacebar on newer models), then, with the cursor on the word you wanted to start highlighting from, you pressed a bit harder and it would highlight from that word on until you released the keyboard.
One thing this article does not mention and what makes mobile text editing slightly more bearable: Using the volume buttons to control the text cursor.
It doesn’t support desktop modifiers like Ctrl or Shift, but at least you can properly place the cursor where you want without going mad. For some reason, only LineageOS has that feature, and not even every other ROM. I once tried a different ROM and switched back because the feature was missing, horrible.
Use a keyboard app with arrows. My choice is "Hacker's keyboard" [0] but I think there are others. (Search reddit for best keyboards with arrows.) I can't say more about it, text editing is not a problem on mobile. It's ok. I installed, then I forgot.
AnySoftKeyboard works on Android 12 and 13 and has arrow keys available in one of the default layouts. I’m actually using my own customized layout since it was pretty easy to make one a few years ago. If you want a layout that is customized to your needs and has all the exact characters you need, I would recommend looking into it. I think it was just editing an XML file and building the layout in Android Studio (the layouts are installable as separate add-ons for the main app).
I still use Swype, the button to select the word the cursor is at is irreplaceable. No other keyboard I know of has it. There are also more suggestions and you can scroll them. Google's keyboard only shows 3, and two of them are basically the same with different capitalization. It's just horrible.
The other benefit is that you can quickly copy/cut/paste/select all with a single swipe action (from the swipe button to the c, x, v, a keys respectively).
There is just an single downside: no swipe on the space bar to move the cursor. But with another gesture you can turn the keyboard into an arrows one.
I don't know why there aren't any good keyboards. SwiftKey came close, but as it grew it became slower and slower and now it's just ossified and abandoned, with a few really annoying bugs that surely should have been easy to fix (like "forget this word" actually forgetting a word).
I was slightly surprised by the author’s example experiments. Deleting x from the middle of a word does not sound like a common editing task to me and I think it would be much more common to just select the whole word and retype it. If that means the UI is more optimised for other interactions than deleting letters from the middle of words, I think that’s fine. It seems pretty important to have the tests be realistic or their results won’t relate much to the real world.
It is possible to connect a keyboard to a phone though obviously this is a bit silly. But it does seem obvious that people don’t care that much about the text-editing experience. And iPhones tried being better at text editing and apple dropped those changes, so it does seem they at least didn’t feel the change was worth it:
- you used to be able to select a word and pressing shift would toggle the suggested correction between regular, capitalised, and uppercase. That feature was dropped
- with the pressure-sensitivity feature, you could press on the keyboard to turn it into a kind of touch-pad for moving the cursor, this meant your finger didn’t get in the way of the text so you could see where the cursor was going. You could press harder to begin a word-by-word selection then move to select more or press harder to upgrade to sentence/paragraph selection. Apple got rid of the pressure-sensitive screens in newer phones and give them a long-press space bar gesture, but this doesn’t allow for selection (as far as I know) and makes it hard to move the cursor down.
Deleting an 'x' was MEANT to be hard (and not typical) the whole point was to see how easy it was to target.
As to your "it does seem obvious that people don’t care that much about the text-editing experience" You are exactly the type of person that I'm trying to get through to! As I said in the post, for most social media tasks, text editing isn't a problem. However, if you believe that mobile will replace desktop, then you've got a problem as sophisticated text composition is quite hard on mobile.
There is this website, nngroup.com that used to be a popular reference on UX matters. They wrote lots of articles about how they tested using computers by mouse instead of keyboard and that mousing was loads better and had wide-ranging conclusions about UX from this. But the studies turned out to have actually been things like ‘replace all letter ‘e’s in this sentence with ‘|’ symbols’ and using the keyboard meant arrow keys and backspace rather than using a find-and-replace function. So sure, mice are better than cursor keys at selecting lots of letter ‘e’s and editing them, but one would hope that such a description is not representative of actual computer use and so not a good foundation on which to base general UX conclusions.
It is in this sense that I worry about the test scenario described being unrealistic. If one is measuring (and so implicitly optimising for) things that don’t matter, one is potentially rejecting solutions that would improve things that do matter without helping things that don’t.
More realistic scenarios could be:
- successfully format 10 lines or so of haskell into a hacker news comment. Obviously this is not relevant to most people but I give it as a more realistic scenario where typical mobile text-input mechanisms struggle.
- edit a misspelled name in an email of a few paragraphs.
- reorder two paragraphs in an email then edit for clarity.
- add full stops to a bulleted list (struggling with this right now).
I am sad that text editing on mobile is hard. But it does seem that efforts to make it better were not appreciated. My reading of the 3d-touch thing and other comments on this article is that the problem was appreciated by Apple but people didn’t particularly care[1] when Apple then dropped 3d-touch, for which improved text editing was its ‘killer app’.
[1] I have seen people complain about the loss of this online but I think the complaint is quite niche.
> give them a long-press space bar gesture, but this doesn’t allow for selection
If you put a second finger on the keyboard when you're doing long-press-space, it'll do selection but it'll stop when you lift your finger off space (if you've run out of display when selecting downwards) and won't pick up where you left off - you have to move to the little selection handles instead.
I fully agree with the author on that text editing is nothing but cumbersome.
But instead of modifying and (hopefully) improving on what we have now, I would actually prefer an entirely different solution; one that disables any touch input on textboxes.
In place of touch, I'd prefer a new keyboard screen containing a joystick to move the text cursor with.
On the opposite side of the keyboard, you could have all the context buttons, together with a 'select' button which can be held while moving the joystick to make a selection.
Add a toggle button to the existing keyboard to switch from and to these new input options and you're all set.
Whether this solution is intuitive enough for the average mobile user is up for discussion.
Blackberry 10 had really nice text selection for their phones with physical keyboards. The keyboard itself doubled as a touchpad so one was able to double tap the physical keyboard anywhere in a text box to bring up a loupe and hold the virtual shift Shift key + drag one's fingers along the surface of the keyboard left/right/up/down to highlight text (dragging the selection handles via the screen was also possible but the keyboard method was faster).
It made intuitive sense coming from a desktop environment and didn't suffer from having one's fingers obscuring the caret or handles.
I still miss this method of highlighting text in Android.
That seems like a simple idea to test in a keyboard (well the loupe might be hard) but moving the cursor and using SHIFT to select are fairly easy. Who's working on an Android keyboard here? ;-)
The KEYone and KEYtwo were Blackberry branded Android phones with a physical keyboard that had at least the scrolling ability. Unfortunately discontinued.
The fact the iOS delete button moves at snails pace then suddenly jumps to deleting words for about 4 words then suddenly starts deleting whole chunks of text. It’s objectively trash and a nightmare to use when just ramping up single character delete speed alone would be an improvement.
This is before we get into the silliness of autocorrect repeatedly autocorrecting the same thing when you delete it and retype it.
I wonder if the author is aware of the cursor positioning gestures on iOS (hold space, then drag)
I’m also not sure that the inline magnifier is such a good idea, he never gives an example with a fat finger overlaid, I think that would immediately show it’s just not feasible.
I’m also sure that Apple has dozens of user tests with all type of text editing strategies, Ken Kocienda even wrote a book about his process developing text input and auto-correct.
I am aware ;-) It's certainly helpful. My point is that it's not nearly enough. The goal of this post was to make it clear that the entire model of mobile editing is flawed and it's helpful to rethink it. However, I'm all for anything that can help!
I was not aware of this! This is a game changer for me, my single biggest complaint about text editing (which I was about to post here as something the authors missed) was that there's no means to perform fine cursor movement with gestures (I would expect the cursor to slow when the magnifying lens pops up). But apparently there is already a way!!
Long-form text-editing on a phone is going nowhere for me. I can't even type in my phone-number without having to correct 3 or 4 mistakes. Basically, I can't use a phone for anything that involves free-form data entry.
In practice, I only use the thing to receive SMS messages containing verification codes (which I can't receive on my laptop). The phone doesn't travel with me (i.e. it's not in fact mobile). It stays near the laptop, which also doesn't travel.
My eyesight isn't that great these days. And I have a distinct sense that my fingers and thumbs have got fatter with age; one thumb covers three "buttons" on the virtual keyboard.
You may say that I should get a bigger, more-modern fondleslab. But the buttons on the virtual keyboard are the same size on big fondleslabs as they are on my old one; they keep the keyboard as small as possible, so that app designers have more screen to play with.
My takeaway is tacit irritation at the size of the screen. If it were larger then it would be easier to position a cursor and edit a word. With a keyboard and mouse, of course, it would be very easy. Our fingers are pretty large compared to smartphone screens, and regrettably that can only be helped with a stylus you'll lose.
Is this because application developers are simply going where the people are? Or is there better data harvesting to be had on your average iOS/Android app with blindly accepted permissions than there is in the average desktop browser.
When my wife was super pregnant and overdue, friends were consistently asking for updates. I made a website called "is<wife's name>stillpregnant.com" and put a very very minimal website there. It was literally just route53 and cloudfront in front of an S3 bucket.
But that meant I needed to make updates from the hospital with my phone. I mean yes there were probably better options but I was a bit busy at the time to think of them.
Let me just say that writing raw HTML files using textedit for Android was not a great experience. It's just not the right interface for making complex text.
Maybe LLMs will help with this, allowing us to describe what we want at a higher level, through voice or text. But God help me I do not want to try to write valid HTML on a phone (after being awake for 35 of the previous 36 hours).
This is the kind of stuff you read on 4chan as memes about orange site.
While going through all that you decided to create a website of all things and update it by hand editing html.
Normal people, and I hate saying that, would create a facebook/WhatsApp group or something like that.
I'm not making fun of you or anything, not everyone has the same lives, I'm just amazed by the hn crowd.
I mean even if you are going to make a self-hosted site for this, like, there's software you can use where you're not using raw html. I use a static site generator that takes markdown. I don't find markdown too hard to write on mobile, as long as I don't need images.
I was about to comment that using an LLM to update a static site can be seen as overkill like using napalm bombing against mosquitoes,
and you post about those considering odd to have your own services to publish basic information...
> amazed by the hn crowd ... [whereas] Normal people...
I have seen members of the "madding crowd" amazed by people reading a book. And not one: many, in different increasingly bewildering capacities. (Including "law enforcement", calling it "suspicious behaviour".)
Be careful of what can be called "normalcy" nowadays.
Congratulations on parenthood. I thought the decision was very much overkill myself, but it's impressive you had the determination and skill to execute it while in the state you describe. I doubt I would be able to.
> While we should always take care of novice users first, we shouldn’t ignore proficient users.
I believe that the relation should be reversed: Make the best system possible, usable by well-trained users, and only then add easy modes, tutorials, child safe modes, etc. with paths for improvement at every level. But when using modern systems, I often feel like I’m reduced to using Fisher-Price toys with one (inoperative) button.
You've made the experience more tedious to create now. If we adhere to the article's advice, you work from the novice upwards. You create more and more freedom as you go up, you also have an easier time catching problems between levels this way.
In your proposal, we're expected to start from the top magically creating the best experience, and then jumping back down to novice users, and then scaling up back towards expert level to cover the gaps. It sounds quite jarring, and in practice it winds up leading to lackluster tutorials made by experts who overestimate everyone else's levels (see: "How to draw an owl: draw a circle, now draw the rest of the owl").
Creating a good system for advanced users is not magic.
In practice, what tends to happen (when things go well, that is) is that people follow the same course as Douglas Engelbart’s development of the NLS system, which he called “bootstrapping”, and which we today might call “agile” or possibly “devops”. I.e. the initial users are using the system from day 1, and developers are constantly giving users more options and features to aid the users, and since the developers are either in close cooperation with the users, or the groups simply overlap, the finished system (that is, when it starts to change more slowly) is one in which the system is quite complex, but all users are also advanced users who can use it at high speed to tremendous advantage.
This is the point at which you should go back and add intermediate levels, tutorials, easy modes, child proofing, etc.
About 15 years ago, in the era of dumbphones, I read an article in the news paper that in Japan's top 10 novels, 7 of them were written on a mobile phone.
It appeared as if writing and text editing (writing a novel is a lot of editing) for mobile phones was solved for Japanese, and it was a matter of time until it would arrive in Latin scripts.
Still waiting for that future. I wonder if things in Japane regressed too?
I can buy into the fact that text editing on phones is a pain, but the solutions presented in the video seemed very unconvincing for me. The "only magnify in place" option is obviously a horrible idea. I don't know about you but my thumb is not transparent. The whole reason for moving the magnified piece off to the side is so that you can actually see it. The problem isn't as evident in the video because there is no finger in the way, but I just was imagining this behavior on the comment I'm currently typing, and there is no way I'd be able to see the magnified content if it worked like the proposed solution in the video.
> The "only magnify in place" option is obviously a horrible idea. I don't know about you but my thumb is not transparent.
I think you misunderstood the proposed solution. If you look at the video, the magnification is offset. The improvement is that the cursor is only visible in the magnification.
But the only way to magnify the place where you just tapped (that is, inline, where the magnification shows where the text is on the page) as you drag would essentially be to not move the cursor as your finger moves (i.e. wait until your finger gets out of the way).
I don't even understand how this would work if you're dragging up (going to a previous point in the text).
you are right, thinking the same - the solutions are more in the same directions, but not actually something in the right directions.
the windows mobile had some interesing design decisions with text, and particularly nice large menus that... were so big they didnt fit the screen. best stuff I've seen in UI design for like ever since.
> Whenever I explain my research at Google into mobile text editing, I’m usually met with blank stares or a slightly hostile “Everyone can edit text on their phones, right? What’s the problem?”
These people must have never used a phone. This article hits 100%, there's quite a lot of these micro-issues on things we use constantly that could be major differentiators and extremely easy sells, but no one implements.
My very personal pet peeve is text reflow on mobile browsers. Just zooming into a long paragraph and having it wrap so you only have to scroll vertically and never horizontally to read it. It's a killer feature and almost no browsers have it!
Yes! I'd rather not use it because it doesn't support uBlock but it's a feature I can't browse without. I try every other browser at least once every year and they're all massively uncomfortable.
Ever tried editing the URL in the URL bar in iOS Safari? It's even worse than just editing as it involves horizontal scrolling in a tiny box. Why don't they expand the URL address bar to be a large text box when selected is beyond me.
My peeve with the URL bar in Firefox on Android is that there's no way to partially select. A tap places the cursor, and a long press selects the whole URL. There's no way to select the domain, or a value from a query string, etc. Simply supporting the normal text editing gestures here would allow this, but the developers have gone out of their way to make it impossible.
I saw Unexpected Keyboard recommended here, turns out it can do this as well among other things. Just tap Shift and drag left/right on spacebar. The main feature though is that you can enter the characters that aren't used commonly by swiping insted of holding on a key or switching to a different key view. You can also get Ctrl and Alt and you can like lock them as well which is useful in Termux.
I use my phone extensively for writing and I must say I dont experience that much hassle in writing. Only two points:
1. The autocorrect is the worst part and the only real struggle as it often change the word not logically with the previous words (softkey) or it doesn't know the correct conjugation and changes it. Finally, the menu shortcut is way too close to the letter 'a' and I can't stop hitting it.
2. On the ergonomy side, only 'select text' is problematic for me, but not the long press + cursors, that's okay, it's more about cursors themselves when they reach the edges of the screen when I select whole paragraphs. I often find that I can't select the first letters. Therefore, I always need to select some letters from the paragraph above and remove, or select less and rewrite.. Another problem with it is selecting very big chunks of text, I would love a click somewhere + 'shift' click elsewhere to select all the text. Finally, the 'Semantic' web doesn't help sometimes when I can't select several paragraphs because they are not in the same Semantic unit.
I the past I was carrying with me a small Bluetooth keyboard that could switch with a button between my phone and my tablet, but finally, writing on the phone was good enough.
I have way less good experiences with writing code on the phone or interacting with a ssh session.
Watched and read it all and find it amazing work. Text editing on mobile is embarassingly terrible and the proposals presented are very solid. Props to the author.
I tried this out for the past hour or so. It takes the 9 most common letters in English and makes them easily accessible through one touch down, whereas all the other letters require a touch+swipe.
Multi-lingual support is problematic, the 9 most common letters differ per language, even if you stick to those based on a latin script. So either you end up learning multiple layouts for multiple languages, or end up doing way more touchswipes than should strictly be necessary.
Granted, I am a pretty fast and accurate two-thumb typer on a qwerty keyboard (AnySoftKeyboard) on my phone, so it might not be for me. Perhaps someone who struggles more with accurately hitting individuals keys would benefit more from this simplified keyboard
Point is, I doubt this system will ever be faster (for me) than typing on the qwerty keyboard where all keys are available at a single touch.
I would also be interested if this method would improve my ability to type blind, but I think it still suffers from the lack of tactile feedback there, I still dont know where my finger is exactly.
There’s a lot here I agree with. I dread text entry on iOS and it never seems to get that much better. It seems in many ways to have regressed in iOS 17 as well with things like invisible/near zero contrast text selection, hit detection being generally broken, it likes to super aggressively interpret taps anywhere near a word as me wanting to select a word when I’m just trying to reposition the cursor (and generally a word that’s not even close), and all kinds of other annoyances and friction.
But in addition the keyboards just don’t seem to be very good. I can’t tell you for how long now I’ve been infuriated by entering a search in iOS Safari only to have every word separated by a ‘.’ because of how it overloads the right edge of the to have a period right where my thumb likes to go.
I’ve tried numerous third party keyboards and they are all some sort of combination of “bad”, so I always end up going back to the built-in one after a while.
These are all far from solved problems, it’s just that it seems like a lot of people are so accustomed to the friction that it just seems like one of those inevitable things.
So I read this article with my iPad and of course that made me immediately start exploring the text selection within the article - it appears that it has changed in iOS 17 Safari and applies several ideas from the article.
* Immediately start dragging. When you pause it immediately enters text selection. If you move slowly you get the loupe otherwise it starts jumping word by word in the text
* When you release you get the popup options to do something with your text
* Holding the end of the selection and moving slowly gives you fine control over the cursor.
* When editing this comment, it doesn’t immediately jump to selection but gives you cursor control right away after long press with the magnified loupe. Move fast enough and the magnifier disappears. This is different behavior from long pressing the space bar.
* Double tap and drag gives you the selection behavior.
* Double tap and no drag selects the word.
Actual coding on a touchscreen is still impossible even with the articles improvements. I’d like to see more work on that. I wonder if speaking to our devices inaudibly could be the long term way forward.
I'm the author of the paper. That first one though you called out is VERY interesting (the others seem quite close to what is already done today)
We tried pause-to-select and it user tested very poorly as users were thrown into selection far too easily. It was very irritating. However, I can't really criticize it until I've tried it. I'll upgrade my iPad and check it out. Thanks.
"Editing"? My most important computing application is KEdit, a terrific text editor.
Mobile? Since it seemed like highly inferior computing, I delayed getting one. Finally I did. Tried to do some little thing, and for four hours the fake keyboard came and went for no good reason, several times had to power off the device and log in again.
Conclusion: I hate that mobile device.
Result: I put the thing in a Faraday cage envelope and put that in a box in my car. I hope never to use it. My only intended use is to contact 911 in an emergency, and I hope never to have one of those.
To me, mobile is a big step backwards and very unwelcome.
I'm putting up with Windows 10 Home Edition but ASAP returning to Windows 7 Professional. Most important application -- KEdit.
Theme in computing: Mostly just text, in files, in the Windows hierarchical file system. Next, macros for KEdit. Next, command line scripts in Rexx. Next, software in the Microsoft .NET version of Visual Basic.
Main use of computing: My startup and its Web site. I have the Web site code, 100,000 lines of typing, all from KEdit, all in .NET Visual Basic, running, apparently as intended.
Latest Irritation from Mobile: Wanted to order some Chinese food for pick up. Used their Web site. The site assumed I had a mobile phone. So, couldn't order via the site and had to call the Chinese restaurant and give me order to a person, item by item.
The assumption of mobile for the Web site ruined the on-line ordering and much of the site.
The site was complicated and in the end all they needed was one page of HTML. That one page could easily be plenty good, and even a static page would be better.
Good example of computing and a Web site: Hacker News!!!!!
At the end of the article the author seems resigned and that it seems impossible to bring this to the end user. I don't understand why? Why can't this be a keyboard that I can switch to, like I'm able to switch from Samsung keyboard to Google keyboard? Or like an accessibility app?
Because the text editing process is baked in the base UI classes on Android (TextView), you cannot change it with an app. OEM manufacturers sometimes do because they ship different class code with their OS.
But the author works at Google, so can surely pass the message on up the chain that this design decision is stifling innovation in touch-based text editing?
I suggest focusing on the minimum viable product: an app with a single text field.
Maybe add the option to load and save txt files, but even clipboard I/O would be sufficient. Just let people play with the editor. If it's actually good, the next steps should become obvious.
I’ve had a growing suspicion for a few years now that the iOS keyboard developers are just taking the piss. Given the level of technical ability displayed in literally every other aspect of the iPhone, there’s no way that the keyboard can be THIS bad and getting worse except on purpose.
Arrow and modifier keys, problem solved. If we’re going to keep making the screens ridiculously huge, use the space for functionality instead of gimmicks.
I've been using Hacker's Keyboard in compact mode for years now for this reason. (And my screen is pretty small for today's standards.)
It's just so much easier to edit text with arrow keys, shift-select, ctrl-c/x/v, etc. than getting that teardrop handle to behave, even on a touchscreen.
I purchased a samsung fold 5 with the hope of it being big enough to make a significant difference in how/what I use my cell phone (for).
While the bigger screen is indeed a step in the right direction, most of the shortcomings from smaller screens are still here. Text input, as described by this post is just as silly on a phablet like it is on an iphone. I even went as far, as purchasing a logitech mobile keyboard (with a tablet/phone slot) so that I can reply to something longer than a short sentence.
For me its obvious, that without a physical keyboard, the phone will remain a device to scroll up and down and will not replace a pc, no matter how big the screen is.
One notable thing, I have linburg-comstock in my left hand, and this is apparently not uncommon (estimated to be around 30% of the population in one form or another, iirc).
This means I cannot curl my thumb without my pointer finger also curling. This is not something that can be stretched or relaxed or otherwise fixed without surgery- my tendons are cross-connected. The result is that typing on a cell phone keyboard ends up cramping my hand very quickly.
Off-topic: I want to murder the person who removed the beautiful magnifying glass in iOS when manually dragging a text and replaced it with some garbage mini reticle that often bugs out and disappears if I am dragging on multi-line text >:(
This problem was solved 20 years ago when devices like the Blackberry popularized the miniature physical keyboard. Those interfaces were much easier to type on than these crap touch keyboards we've been given. Phone manufacturers told us these are better, but in reality it's just cheaper to make a software keyboard interface than it is to make a bunch of tiny buttons.
The original IPhone "un-solved" the problem and then every phone company copied Apple because that's what they all do. And here we all are complaining about this like it's 2010 all over again.
Problem is that that feature is not very discoverable. But I agree. The author seems very Android focused because most issues don’t really exist for iOS users once you know how it works.
Do agree that this one is too hard to discover, Apple doesn't really highlight it at all, while it's so incredibly useful.
Luckily there are all the engagement farmers with their ios tips and tricks, maybe that is the official Apple strategy haha, and to be fair it kinda works as people do find out that way.
It's not discoverable. I learned about it through some "things you should know about iOS" listicle or something. Once I picked it up, I've used it a lot.
I loathe when apps interrupt usage to say "did you know you can?" but... this is one of those things where Apple should point the feature out on upgrade, or something.
It’s only a few characters wide, so you can’t edit large text in small input, such as… the URL, the most frequent usage of editing text on mobile. Along with tasks in the TODO app, typing a text message.
It’s only awful BECAUSE Apple is persuaded that we have fat fingers.
But I have kept a very old iPad from the pre-force space bar, and lightly touching any word would simply put the cursor between letters. Modern-day iOS selects the full word as if I had fat fingers.
You can drag around the whole screen btw. If you need to go further just hold your finger at the side of the screen and it will work as if you hold the arrow keys on a keyboard.
I believe it also responds to velocity, so quick swipes will get you quite far.
That doesn’t answer the question. Since the space bar is at the bottom, I can’t navigate lower than the initial position of the cursor. Aaaaaand since the keyboard is not wide, I can’t go laterally to a portion of a URL that is far away.
Just to prove it’s possible to do better: iOS used to have the two-finger drag (one force-press, one gently navigating wherever you want, like on the Touchpad) and it worked.
I know something that works and something that doesn’t.
I completely agree with author's view of the issue. Text editing on iOS and android have always been an afterthought. I mean, how much time did it take for the iPhone to get Copy/Paste ?
The weirder part to me was that the iPad didn't bring much improvement on that front. Even with a bigger screen and "desktop class" application, text editing only barely works with a keyboard/touchpad attached. The only improvement has been on moving the text cursor, yet it's still a real pain to select random text in the middle of a page.
On the solution though .... I think we should really do something simpler than introduce new paradigms and further dig the "that's how we do it on mobile" well.
Just give people damn arrow keys and a crontrol key: let them put a cursor on the middle of the text they want, move the cursor with ultimate precision exactly where they want it, and shift select the text they need.
Android does the arrow part right, Windows' on screen keyboard does both right. It's incredible how liberating it is to just select text with the arrow keys, hit ctrl+C, crl+V it elsewhere and be done. That's really not that much to ask, Apple not doing anything in this front feels like sheer laziness at this point.
> I mean, how much time did it take for the iPhone to get Copy/Paste?
It's kind of sad to reflect that the Apple Newton had this almost a decade prior[1] to the first iPhone and in an intuitive manner, by selecting text by highlighting it with the stylus then dragging it to the edge of the screen where it would remain in a truncated pill form for later retrieval (dragging out the pill to a document). No menus involved at all.
I have an iPad that’s about to go out of support and is starting to get some small cracks in a screen corner. Seriously wondering if I should replace it or just take an older, albeit heavier, MacBook for extended travel.
I don’t find myself using it at home much and media consumption on planes isn’t that important to me any longer.
I tried an older macbook for a while, and it was kinda frustrating on two fronts:
- reading is subpar
Having the ipad in portrait helped a lot for reading A4 size documents, where I'd be much more scrolling on the macbook.
The the form factor plays a lot when you're checking documents at an airport counter, reviewing your hotel reservation in a taxi, or looking at a map with others.
- I had to baby it a lot more
iPads and tablets in general can fit in cases and be lugged around without damage. The screen is also tough enough, where a macbook will be out of order if anything goes between the keyboard and the screen when you close it. That's fine at home, but that means it doesn't get out in any situation where it could be risky. In contrast I bring my tablet in the bathtub in a waterproof sleeve.
None of those are absolute deal breakers, I mean we used laptops anywhere we could for decades before tablet areived. But I definitely couldn't go back (disclaimer, I ended up with a Surface Pro. It's nice but as Panos is out of Microsoft now, I have no idea of the future of the lineup...)
That’s all fair. For now I have an out of support mini Chromebook I can throw in my luggage in addition to a tablet. Bad I know. Maybe I should get a new one—it’s cheap—and nurse my old tablet. I just try to avoid hauling too many electronics.
I 100% agree with this. My teen, OTOH, refuses to use a keyboard to write papers or do anything. She finds it completely normal and preferable to write on her phone. To me this feels like using a hammer to chop wood, but it's her native input method.
Allegedly this is standard in her age group. Makes me wonder about the next few waves of people entering the workforce and whether they're going to hate using keyboards for input.
This is exactly the point I tried to make in the paper. Teens are using the phone and that's fine, but if you watch them, it's horribly inefficient. I'm not saying stop using the phone, I'm saying your kid can be far more productive.
I mostly just use emacs in termux. I think vim makes more sense to be honest, since the modal edit mode is perfect for touch screens, but I do not know it well enough. Keyboard-combos in emacs are not great without a real keyboard, but at least I know what to press and it is often possible to navigate around using very few key-presses with some thinking and things like avy-mode.
Text editing done right requires either a true keyboard to manipulate it, or true AI that understands every small nuance one might want to apply to a text.
Unfortunately until AI catches up, the best tool we could use for text is a real keyboard, or a virtual one that doesn't suffer from the lack of usable space of mobile devices, which would probably make use of an external wearable, possibly BT connected, device. Years ago I thought of one shaped as an "ergonomic potato" that uses a few keys (3 or 4) and an accelerometer for hand orientation detection, so that one could use a mix of the key combination and hand position to rotate and find the right char/symbol on a virtual ball-printer-head-like "thing" shown on the screen. I never went beyond the "what if" stage; the contraption would probably work, but would also be too cumbersome to use.
> For highly proficient users, this gets even worse as their is no command key equivalents for cut, copy, or paste. […] Part of the unspoken reason desktop clipboard use is so high is the speed in which it can be used. Mobile has none of this.
I recently discovered [1] that iOS has several three-finger pinch/swipe gestures for copy, paste, cut, undo, and redo. Apparently these gestures were introduced in 2019/iOS 13 [2]. While they are certainly not the easiest gestures to reliably perform, they at least do not suffer from the issues of ambiguity described by the author.
[1]: accidentally, of course, because the discoverability of many features on mobile is quite poor
That's a fair point, thank you for bringing that up (I'm the author) While these are certainly helpful, the fact that almost no one knows about this proves they aren't really working all that well. Discovery is a key issue that can't be ignored.
> Mobile devices were originally designed for consumption.
They weren’t and then they were. Palm, Nokia, and Blackberry were all devoting around half their user-facing surface area to content creation with physical keyboards.
Near then end Nokia did make some really nice Linux-powered slide out keyboard devices. Best of both worlds if not for their thickness.
And before Palm had a keyboard, it had Graffiti, which could share screen space with display.
It took a short period of training but was actually quite good. Far better than onscreen keyboards.
I wouldn't recommend Graffiti for, say, trying to access a Linux shell. But for writing text, and editing it, it was serviceable in ways that current touchscreen devices aren't.
As a side note, text editing on desktop also isn't okay. Especially in the browser era, the fact that the "do command" manipulation from text editors and the "do command" manipulation from control panels collided violently in the browser as an application mean that when I do a simple click and drag to select text, I have no idea what's going to happen. Will I select the text? Will I start dragging the link underneath the text? Is the text an image so I start dragging the image? Will I try to activate the button that the text is actually wrapped by?
Browsers could honestly use a modifier (additional mouse button or key held down) that put them in a "text selection mode," in which all other meanings of click and click-drag are switched off. That would give the user full control to excerpt text from the page regardless of its context.
You definitely can with recent GBoard versions. If something is in the clipboard, tapping somewhere (including within a word) to move the cursor brings up a paste button in the prediction bar.
I've always dreamed of having Maya-like menus (1) on mobile. The edit menu they demonstrate in the article (2) reminds me of them.
On the topic of mobile text editing: I use an iPhone SE (2nd gen), which has a 4.7" screen. The other day, I needed to translate some text. In iOS's Notes, I couldn't type and see my source at the same time. So I did what any iOS user would do and opened vim (3), made a v-split for the source, then wrote and edited my translation.
Vim-users, do you also find vim good on a small touchscreen?
Excellent article. I have long thought that mobile has horrible UX, especially for typing. I don't know how other people aren't more annoyed. For me it's because I can touch type relatively fast on a real keyboard, and can switch between a mouse and a keyboard very fast. It's sooo frustrating not having the same speed on mobile while editing.
For writing prose, gboard is bad, but it's the least bad solution, its sliding gestures and predictions are good enough.
I use unexpected keyboard for programming on the go. Free and open source, originally developed for termux. It has arrow keys and modifiers, and undo also works (ctrl z). TBH it's not perfect for me as it doesn't have predictive text and its arrow keys are fiddly, so I use it sparingly.
> This creates our first ambiguity. The text handle is itself a tap target. Unfortunately, so is the text surrounding it. We now have two potential tap targets. When they are far apart, it’s fine. The problem only occurs when I want to tap just to the left or right of the text cursor. In this case, it’s unclear what the user wants: to move the cursor or to tap/drag the handle.
This used to work perfectly on iOS until iOS 17. You could hold and drag text handle or you could double tap to select text around it and then drag selection handles. Now select-dragging a handle in place was made impossible, I don't know why.
Not invisible for me. I do occassinally enter moderate amounts of text 3-4 paragraphs). I also like to iterate on my texts, so I do a lot of editing and entering at the same time.
The points mentioned in the article are painful, but something I could get adjusted to and get better at with time. There are a couple of things for me though, that I just can't work around, and which are subsequently the biggest pain points for me. These are: lack of vertical space in landscape mode and broken text selection feature. Let me elaborate. Note that I own a google pixel phone, so can't really speak for iphones or androids from other companies.
Pixels gravitate towards long and narrow displays. This can be great for portrait mode, but sucks for landscape most of the time. The issue is made so much worse that neither OS or most apps seem to try to optimize for this mode, further wasting the space available. For example, I'm typing this post in Firefox on my phone right now. About 1/3rd of the screen is taken up by the keyboard. This is fine and is actually the reason I use landscape mode - bigger keyboard allows for faster typing. Another 1/3rd however is taken up by a combination of notification bar at the top, gesture hint bar at the bottom and firefox's address bar. With some manipulation you can make the adress bar go away, but the other two bars can not be hidden on pixels period. Perhaps the biggest frustration is the fact that the "fullscreen" mode I want is already implemented on Android for games and such, there's just no way to turn it on for firefox in either the app itself or the OS.
Text selection tends to be broken when selecting several paragraphs of text, especially when selecting existing text from a web page - the teardrop just jumps around violently not letting you to put it where you want it. Sometimes you can wrestle it with multiple attempts, but other times it just won't let you leaving you with the only workaround of selecting in smaller blocks.
This reminds me there actually was one somewhat recent improvement to text editing on androids from few years back: ability to move cursor by sliding your finger over the space bar. That eliminated one big pain point I used to have.
By the way, a simple solution I have come across to make the experience slightly less painful: use an OTG adapter, and plug in a USB keyboard. Most offices/hotels etc have USB keyboards lying around, and the OTG adapter is a tiny device. This means that I can reply to emails when I am travelling, and I have found the experience satisfactory. It's not a truly mobile solution, though, since carrying a keyboard around yourself, is bulky.
Not having a global menu is a terrible hindrance. Understandable with the original iPhone screen size, but definitely not today with the phablets we all carry.
I also want multiple undo everywhere and consistently like any decent desktop.
With the magnifier no longer being offset - does your finger now not completely cover it? Definitely does raise some good points though - it’s so difficult to get to the select all button on iOS when you actually need it
On current Android, you can drag the cursor directly without seeing the teardrop, so your finger is directly over the cursor, requiring the magnifier to see what you are doing
With the demo shown here, you always tap on the teardrop, below the magnifier, so the magnification is more to help you see rather than show you what's under your finger
My first smartphone in 2000 had a very thin pen, that allowed for high precision typing, selecting text, etc. The OS was made for pen usage. Android and iOS are however not designed for pen usage. I tried a Galaxy note, but due to Android not being pen friendly I ended up using my fingers instead. If you want to be productive, just use a stationary computer and all the ergonomics that comes with it. Some phone models today have USB-C and you can dock the phone to a keyboard/mouse & big screen and the smartphone will go into "desktop" mode.
While all the same issues highlighted in this post still apply the WYSIWYG editor I'm using in my project, this has been my side project take on rapidly iterating on blog posts from my mobile phone:
It lets you combine speech to text and markdown editing.
Partly for fun and practice and partly because it helps me to quickly commence new posts and either iterate on the go or return to the open PR when I'm back at my laptop.
I find text editing on mobile to be fine, I'd have to try this to see if it's any better.
The article did skip over one important QOL feature that every modern keyboard has: Swiping the spacebar to move the cursor. It solves a lot of the things the author complains about.
The thing that causes me physical pain when writing something on mobile is the web. Browsers/websites misinterpret inputs into text fields all the time. Trying to scroll the text field while it has an active selection? Great, now your keyboard is closed and the selection is gone.
Nice effort to fix the broken fundamentals, though this doesn't go far enough.
Why no customizable menu buttons? Good note editing apps like Bear on iOS have them, and predictable position makes choice faster/more precise vs a near-cursor popup
A selection undo stack (and action undo tree stack) would also help in fixing mistapping and other mistakes
And maybe modal editing paradigm could also be a good source of learning?
Also fully customizable keyboards would help in text editing
(don't like their proposed animation , seems like distracting noise, though maybe in actual use it's not bad)
The author made a demo app, surely this could be evolved into a library? I could give an option to users of my app to switch all text inputs to use this instead of the system's default.
Something I do quite often with FlorisBoard on Android is holding down shift and dragging the space bar to create a selection, just like I would with shift and the arrow keys on a desktop
I'm actually rather surprised that we're still using qwerty layouts on touchscreen devices.
Of all the possible ways to input text with your thumbs, this is undoubtedly one of the worst.
I'm sure plenty of people have tried to come up with alternate keyboards, maybe a radial layout or something, but when I look, I don't see a whole lot of activity in this space.
I've been using a mobile device for programming since 2012 and I developed a mobile text editor and screen keyboard that I use myself. I think I have the richest experience here. The problem is that the people who create mobile input interfaces developed them on a computer in order to make a “product” for a company, not for people. This is also due to patents and other problems not related to this task like "It’s not the flashy feature that shifts your Net Promoter Scores"
Step back. Text /input/ on mobile isn't okay. It's a copy (for the most part) of desktop editors as well. There have been a few changes, such as dragging your finger around. A few keyboards have tried (and largely failed) to introduce new, mobile-first keyboards. It doesn't stick.
I have used a number of non-qwerty/typewriter style mobile keyboards. I have landed on MessagEase, but really loved a few others. Thankfully MessagEase still works, most died somewhere in the mid 10s.
For what it's worth, I do feel like text editing is a big problem on Mobile. This is why I got myself a samsung note-line phone, the stylus greatly improves the text editing experience by basically being about as precise as a cursor and having a second button.
I still wish text editing was better, I like taking notes on things usually on Obsidian and the entire mobile text writing and more importantly editing is just not too great, but as the demo shows it can be improved a lot... :/
This wouldn't even be hard to fix. Give me a modal button on the keyboard for cursor mode, like how we've the numerical keyboard mode. When pressed, shows arrow-keys, ctrl-arrow-keys, pageup/pagedown, home/end, a delete (not backspace) key, a shift-toggle, and a button that shows the longpress copy/paste menu.
There. No need to reinvent the wheel, just do the same stuff desktop does but with the actual keys desktop has.
The spacebar-drag trick is cool, but it's not enough.
Whenever I explain my research at Google into mobile text editing, I’m usually met with blank stares [...long and detailed description of the problems and how their research attempts to solve them...] Unfortunately, shipping something like Eloquent would be challenging.
What a strange attitude to take! What’s the point of the research if it’s not aimed at shipping anything?
Hopefully this is just a miscommunication and isn’t meant to be as passive-aggressive as it sounds.
Passive agressive? (I'm the author) I did research to find a better solution. I did and discovered a wide range of market forces (e.g. people don't like to change) that will make it difficult to ship.
But the last thing I intended was to imply giving up. I said very clearly in the post that my solution, Eloquent, was meant to be one possible solution. There are likely many others! My goal of this post was to clearly describe the problem so people a) understood b) knew there were alternatives and c) we could get fired up to actually solve this (even though it WILL be hard to change)
Aha, I’m glad to hear that! I did misinterpret it after all - thank you for the correction.
I think it was most likely the phrase “shipping something like Eloquent would be challenging”, implying it isn’t going to happen, that gave me that vibe.
Emacs on termux from a Logitech K780 works great for me, the benefits of not being mouse dependent pay off in life more often than you might expect. Almost all the "editor" apps (foss only app user here) are subpar compared to using emacs/vim. Web apps are obviously the "in-between" without much of a solution. Also, I do miss my Blackberry KeyOne sometimes and have heavily considered some of the newer physical keyboard phones.
Positioning the cursor on Android is a complete disaster. I remember it not being a great experience on iOS, and being shocked when I moved to Android and found out it could be even worse. But I'm even more surprised that in all these years it hasn't been fixed. I hope this matter gains some visibility and traction.
In the meantime if anyone knows a good "Graffiti" app keyboard for me to try out feel free to recommend.
1) I use the arrows in my keyboard to move the cursor. (Or letters in vi) Consider four new buttons to be able to move around once the cursor is active.
2) Love this concept. Editing this very post is too hard. This is a big change. Put together the progressive UI that adopts these changes over time. Do the research on user adoption and least impact. Move this from idealist end state, to practical sequence of steps to get there.
I still lament the effective death of physical phone keyboards, but it’s decided by the users already, apparently. The new generation got used to be typing on the screen and don’t even know what they are missing.
Same with gaming: the gamepad with horrible accuracy that mandates autoaim and limited number of buttons became standard. I can’t do fathom it, but people who grew up gamepads see no issue.
It's a long video but the segment I mean is only a few minutes, and it's really fascinating if you're interested in people having to work out surprisingly hard, invisible problems.
Yeah I use swiftkey with the arrows and number row. Also I have it set up so the keyboard appears in the middle of the screen where my thumb can actually reach. I have no idea how anyone can use one of today’s giant phones one-handed with the default keyboard.
Reading this article I realized I didn’t even know how to cut and paste on android, and I’d long ago resigned myself to deleting and retyping.
This came up on TikTok for me after visiting this post, so there must be a cookie or tracker that saw what I was reading (or I suppose, it’s complete coincidence). Whatever the reason, this is helpful: https://www.tiktok.com/t/ZT86HSvwL/
The irony is that despite their conceptual elegance and increasing ubiquity, touchscreens are a technological dead end as far as HCI is concerned. As dim as I am on the current slate (and near-future) of VR headsets, their gestural interfaces are much more promising in the long run (though still lacking in some ways compared to mouse and keyboard).
> (...) there is a completely new gesture, double tap. Wait, there is actually a third gesture long press, which also works.
TIL about the double tap -- always used the long press to select text. It is interesting that the author refers to long press as an alternative. Do most people use double tap for that?
I guess - it's just that nothing is mature yet. I can easily see this problem taking a century to "solve". Society has a lot of inertia and tech is always unintuitive - what even is a keyboard? Can I eat it? Can I burn it to cook the meat?
Please fix this. I mainly use my phone to write hn comments and it is already too much pressure on the current state of the typing experience for me. I could imagine myself doing all sorts of support and maintenance jobs from my phone if only I could interact better with code on it.
I feel like this is an Android specific problem. The author says iOS has the same issues but I don’t think this is entirely true.
I can move the cursor by pressing and holding the spacebar or 3D touching anywhere on the keyboard. I can then 3D touch harder in the same movement to select a word and drag to select more. One tap on that selecting brings up the menu. Double tapping selects a word, double tap and drag lets you select more. Here the menu appears instantly.
And the proficient users can even three finger pinch to copy and three finger zoom to paste. Or three finger tap to get a special menu.
——
This one might be specific to me but I can type on mobile nearly as fast as I can on my keyboard (I might just be a slow typer). I also sometimes feel like I’m the only one who highly appreciates autocorrect. It rarely makes mistakes (great, now I get roasted for every typo).
Still, editing on the computer is much better. I use a lot of shortcuts and vi modes where I can. But I don’t think this can be emulated on mobile. Please prove me wrong.
I honestly think this problem is unsolvable, like, you can only do so much with typing on screen.
Maybe with a device that read your mind and allowed you to type by thinking, but with that aside, I don't think there is a solution. I think this is a fundamental limitation of mobile.
I wonder if there is a way to unboil the frog and introduce features in a backwards compatible way.
One way could be to implement some or all of this along with changes to Android's "spacebar drag" cursor movement mode. Currently it can only move left and right unlike Apple's touchpad approach. Maybe I'm missing something but it feels like you could implement a lot of the features through a keyboard touchpad while keeping regular editing clunky but familiar. By expanding on an otherwise very limited feature it might be welcomed by power users without depriving anyone of their existing workflow and muscle memory.
One problem that comes to mind is that people are used to selecting from beginning to end and triggering the interaction from the bottom of the screen gives very little area go down while keeping 1:1 movement of the cursor. It could be solved by having the cursor hit the "roof" until the users finger matches the translated position then switch to a scrolling mode at the edges.
I'm the author. I agree, this could be a good way to do it. It's what I meant by a "slow burn" in the post. For example, just start with improvements to tapping and dragging, then add a better magnifier, etc. Just getting rid of some of these ambiguous bugs would be helpful.
I have termux (terminal emulator) installed on my android devices and make heavy use of sed(1) to edit then cat the text out, copy it to the clipboard and paste it into text messages, email, ... Works well for me.
Any Blackberry that had a trackball\trackpad was great at editing text - because it had the cursor movement and text selection cleanly separated from the touch screen.
Too bad that ship has sailed. I still miss my Bold every time I try to edit text on IOS
I recently upgraded from an iPhone 8 to a 13 mini. I thought I would miss the home button, but what I really miss is 3D Touch in the keyboard for moving the cursor. It doesn’t solve editing text on mobile, but it went a long way.
Thanks! Happy to know that feature is not completely gone. I actually asked at the apple store about this, but they thought there was no equivalent in the new phones.
Pasting is particularly hellish because of what essentially amounts to a zero-width tap target. Then add in poorly designed third party apps changing the interface (looking at you microsoft) and it's a nightmare.
GBoard has the text-editing view of the keyboard which is nice for longer/more accurate selection. But it would be nice if there was a universal standard and a way to bring that view up for non editable text.
Surely it's going to need hardware innovations (e.g the ability to "roll out" a larger keyboard) for text editing on mobile devices to even come close to the desktop computing experience?
I hate my phone for any sort of editing. I just always bring a laptop. Tablets are nearly useless. I remember dreaming of the days of ubiquitous computing. Now we live in it and user experience sucks.
The article doesn't mention two great features on iOS:
- Move cursor by tap-and-hold on soace bar
- tap-tap for word selection and tap-tap-tap for sentence selection.
You cannot edit much on a virtual keyboard imo. That's where BlackBerry and sidekick excelled but and touch screens would always be subpar. It's a tradeoff.
One of the issues is that most of us don't want to edit text on mobile. Beyond quick checks I'll wait till I have a big screen and a full size keyboard.
What they need is extra keys to move cursor by word, cut and paste. The tap and wait for menu sucks. You just need good old kb emulation for those things.
maybe voice commands, will after all, catch up properly with LLMs now. but who really wants to talk to Her on a daily basis, I doubt to many sane people.
Ima use this opportunity to advertise that I'm working on a mobile shader editor, an area even less adopted to the form factor and am using my structured code editing research[1] to smoothen the experience.
The post echoed many of my frustrations and inspired me to consider adopting the fisheye-zoom to my editor.
I'm looking for people interested in doing shader programming on the go to try this early version, hit me up if you are interested shade@dflate.io
when I started designing mobile applications in 2009, I was fully convinced they are inept for (keyed) text entry. No one would type more than a few characters on such devices.
Since I found me proven both wrong and right in some sense. Looks we rather sacrificed quality for immediate reaction and enthusiastically do bad text editing and sloppy writing on the go.
I'm the author. As the code is owned by Google, I'm a bit constrained in sharing it. I've got permission to talk about the work and show it, but that's about it.
We've all seen what happens when engineers try to do UI/UX. Maybe you're an engineer and not a designer, and maybe stay in your lane and leave usability to the people who, ya know, study it?
reminds me of once I saw a girl with the iPhone 1 when it first came out. during lecture, she was dictating the lecture on her iphone 1. she was so fast, I was wondering what she would be using today.
iOS text editing with the tap and hold to move cursor is a nightmare, and couple that with its extremely poor spelling correction which I have to go to google with the misspelled word amd have it predict instead
No, it's completely broken. And mobile is an absolute mess. The touch system is so utterly overloaded with functionality that it effectively makes all these wonderful things completely opaque and, in a lot of cases, not discoverable at all.
Ask any iOS user what single, double, triple clicks mean. Ask them if they know about long touch on the spacebar. Ask them to show you everything they know about how to operate the device.
What you'll learn is that the vast majority of people pretty know nothing about the function overloading of touch and gestures. They know simple touch interaction and that's about it.
The touch screen, whether it is on a phone or in a car, exists for one simple reason: It is a lot easier to make UI decisions when you don't have to make them. It's cheaper. It's faster to design. Manufacturing costs are lower. Testing is simpler.
Having design many physical control panels during my career, for products ranging from commercial through aerospace, I can tell you it is a difficult grueling process. In short, pain in the ass. And, if you make a mistake or change your mind, you can't just compile your way out of it. Hardware is hard (and expensive) for a reason.
Today's smart phone designers flat-out refuse to add physical UI elements to their phones, instead, choosing contorted ways to overload the touch interface.
Prior to iOS devices I owned some very interesting phones and PDA's by companies like Motorola, Blackberry and Palm. In the aggregate, these companies had interesting physical UI ideas that worked very well.
A stylus, for example, is brilliantly simple and better than your finger, particularly if more sophisticated, as seen on Wacom tablets.
Blackberry had a small trackball you could also click right on the keyboard (another very useful UI element). It was great for editing text and other use cases. Heck, you could use a browser with a proper desktop-style cursor.
Motorola had a click-wheel on a PDA-like add-on to their flip phones. Fantastic for navigation and super easy to use. The clip-on also had additional buttons, that made such things as single-handed almost-blind address book navigation very simple.
Also, touch screens are incredibly fragile. Example, during a call, any accidental touch will cause problems. I have resorted to switching away from the phone application during my call and, if I have to move around (while wearing a headset) placing the phone facing outward in my pant pocket. I've had too many cases of the phone doing stupid things during a call when in my pocket facing inward.
In short, touch screens are terrible for the consumer in a lot of use cases. They are fantastic for the manufacturer because they don't have to make any decisions, manufacture a single product and all is well so long as the torture they inflict on their customers is just below a pain threshold.
There are so many potentially interesting ideas that could be explored. Adding a set of user-programmable physical macro buttons somewhere (sides or back). Adding a small cursor-only touch pad to the back for, well, pointing control. Click and right click could be actual buttons. Blackberry's mini trackball was brilliant. And a real keyboard, as a first-class citizen add-on, not something left to third parties with no real support past the basics.
If someone wants to make heavy use of a device they would love a real full-width keyboard that clips-on and folds open with a Blackberry style trackball and a few extra keys and functions. Lots of people buy cases for their phones. I would absolutely buy an official fold-out keyboard/case if it was truly integrated so I could actually use the thing without having to remember what triple-click-press-slide-right-and-face-north f-ing means.
For me, the phone is unusable for anything beyond the trivial. I hate browsing the internet with it. I don't buy anything other than Uber rides with it. I don't use it to buy anything on Amazon. Text messaging for work is useless other than the very basic, because it is horrible to edit anything and there is no real desktop integration to track and search as you can with an email client.
Which brings me to the other problem with mobile:
It's 2023.
Why isn't my phone seamlessly integrated with my computer?
When I plug-in or otherwise connect my phone to my desktop or laptop, the phone's screen should appear on my desktop as a first-class citizen for full interaction without having to touch my phone. I should be able to edit text on my phone from my desktop in a phone window. I should be able to cut, copy and paste anything between desktop and mobile. I should be able to run applications on my phone and interact with them on one of my three 27 inch screens. I should be able to click on a phone number on the desktop and dial it on the phone. I should be able to use the same headphones I use on my desktop to run a call on the phone through the desktop interface. I should be able to move, copy and manage files on the phone from my laptop. Etc.
I don't really understand what mobile designers are doing. It seems that they stopped being creative and holding their user's best interests as their primary guidance. They slap a touch screen on an aluminum frame, use as few buttons as absolutely possible. Limit desktop integration to the absolute minimum (iTunes is garbage) and leave third parties to hack stuff together rather than develop exquisitely integrated solution.
I could be wrong on this. I firmly believe Microsoft still has a real shot at a next generation phone. Their Windows Phone attempt was a failure for many reasons. What is lacking today are the things I mentioned above, and more. As ridiculous as it may sound in 2023 --because it should have been done by now-- having a device that fully integrates with desktop in an exquisitely usable symbiotic relationship would be nothing less than amazing. That, I believe, would gain traction.
The author almost saw the fix for this, but didn't. Here's how you fix this:
Buttons.
Modern phones are horrifying examples of UX, largely because they don't have buttons. You get two buttons for volume, even though volume accounts for probably 1/10,000th of the UX of the user. You get one button for power, because how else do you turn it on? And if you're lucky you get a button where the fingerprint reader is. You also increasingly get an "assistant" button, which can often only be used for activating a shitty voice assistant specific to the make of phone you have.
Navigating text editing requires buttons. You can try to use touch tricks, but they're complicated, clunky and inaccurate. For ease of use you need buttons specific to editing. A mouse and keyboard have buttons and shortcuts for editing because a person needs specific functionality for specific use cases. So the phone needs more buttons to make this work.
For physical buttons used for editing, nothing I've seen has ever surpassed the clickwheel. It worked for iPod, it worked for Blackberry. You can open menus, scroll with precision, and select items. And a flat layout allows use as arrow keys, compared to the side-mounted version that only has two dimensions. You can emulate this on-screen, though it'll be inferior as haptic feedback is one of the important aspects of the precision of a clickwheel.
Arrow keys are another important button layout. They allow precise navigation within a text field so you can get to the place you want to make a specific change in, without the clumsy finger-tapping that never works right. On-screen these need to be dedicated buttons, as a virtual clickwheel can't easily differentiate between just clicking on the top of the clickwheel versus using the arrow functionality, and the UX suffers.
Another important button is the delete key, as opposed to backspace. You should be able to delete things in front of the cursor as well as behind it.
And then there's the "select characters" cursor modifier. You should be able to start selecting text where the cursor is, navigate using your arrow keys, and then stop the selection, and then click one of the modifiers for copy/cut/paste or just start writing to replace it.
And of course we need an undo button, in addition to buttons for cut/copy/paste so we don't need to long-finger-press to show an editing menu.
Basically all the buttons I've mentioned could easily fit within an editing box at the bottom of the screen, with the text in the top screen. Having the buttons right there for the entirety of the editing session is the right UX. It's actually faster than keyboard shortcuts, it's more intuitive, and more accurate, with less precision required for the user.
Maybe one day these stupid hardware vendors will get off their obsession with chasing Apple and finally introduce some useful buttons on the device again. Until then, fake buttons in an editing modal is the best choice.
> This, again, makes me lose all confidence toward humanity
Mobile text editing issues were what made you lose confidence? The fact that we have the luxury of nitpicking about such mundane issues (as opposed to, you know, starving) makes me realize how great humanity's innovations have been.
I looked at the video for Messagease, and I feel like this must be a joke. I can easily type their sample "the quick brown fox" example using Swiftkey-like tracing easily twice as fast as what they showed in their demo.
1000% agree with this post, wow. I can’t believe this is controversial.. it’s a disaster. I dread opening the mobile version of google docs, sheets, or even Notes for anything more than a grocery list.
The problem is the target audience for this is business customers, and particularly those in management, so it’s a small market. ICs aren’t up and down as much between meetings and can largely stick to their desktop setup. But the manager’s life is one of constant movement, reviewing and making comments on things, drafting, editing, tweaking, etc… and usually in blocks of time where it doesn’t quite make sense to sit all the way down (unless you’re really good at calendar management, which.. also sucks on mobile!)
> fixing text editing isn’t seen as important enough in the war between Android and iOS.
I think there’s a group of people for whom this would make all the difference. It’s just not your typical consumer.
EDIT: and before you say “sounds like you just want a tablet” - I don’t! A tablet is the difference between lugging around a briefcase with me everywhere, or not.
My only disagreement. I think many people don't know they want something better because they've never used something better. The video in the link describes how the users do short texts to get around the problem [0], including deleting whole blocks of text instead of editing. Teenagers, parents, and many others communicate heavily through texts and are often "up and down" like management/business users. On the whole I believe many people would benefit from a better text editing experience.
Yes, a big part of the problem is that so so many words need editing.
I updated to ios17 excited about the new transformer model, and it still predicts absolute BS words I’ve never used in my life.
I definitely remember typing working better on my iPhone 5 which was 20% smaller so I don’t know what’s gone wrong, but it’s gone really wrong and the fixes also suck.
I had the first SE, and it initially handled mixed language text correction phenomenally, but went downhill with each update, which was really puzzling
I agree although I have never used a Blackberry. I have used a dozen of smart phone models in the early 2000s, either with physical keyboards or resistive touch. And then the iphone came and everybody moved to capacitive touch. It might have some advantages, but precise working is not one of them. I still hate it 15 years later what shitty products we have to use despite of all engineering efforts buried in them.
Remember T9? I haven't much experience with it—I'm young. But it was a much more pleasant experience than the current experience with touch screen. Swiping works, but I feel it requires too much cognitive energy. As soon as a conversation is taking longer than this paragraph, I tend to call the person instead.
You could literally type the message without taking the phone out of the pocket because it was extremely predictable and you knew the menu structure by the gut.
I tried t9 when I in the name of simplicity went for a Nokia candy bar. It was utter shit. I think it only worked because there was no good alternatives back then.
Agreed I remember being faster than my friends who preferred the “abc” mode. Not sure if all phones did this but my phones would sort the words by frequency of use for each combination of button presses so T9 kept getting faster the more I used it.
This. Ever since I lost the physical keyboard of the blackberry, I have had to accept to live with a significant and constant productivity hit when it comes to writing more than a couple of words on my touchscreen phones with the need to avoid typing errors. I failed to master the act of writing on a cellphone, contrary to typing on a physical keyboard.
I have seen people type blazingly fast on touchscreen phones though, albeit I didn't run error stats on their output. I accept that I may lack some skills needed to become better at this, but seeing others highlighting that this is a problem to them is not surprising to me, based on anecdotal evidence.
E.g., I knew someone who added to his email signature a permanent apology for typing errors giving the justification that he was typing on a touchscreen device.
All in all, I agree that the problem is real, and havging had experience with a working solution (physical keyboard) I wonder why there appears to be no business case today for such devices.
That looks pretty cool, and others must’ve thought that as well because it’s sold out.
Some of the immediate things that came to mind were making a case that integrates the phone with the keyboard; and a small battery to extend the life of the phone since the keyboard adds to the formfactor anyway.
Bonus points if this creates a market for a small phone again (iphone 12 user here).
The way cursor movement by touch was implemented on BB10 Wwas also the best I've ever used on mobile. Some kind of adaptive zoom and tap to move by one position.
I want a modern touch pro from Google - slider keyboard, no bloatware or Samsung apps, nice big battery and modern screen, one nice camera, proper landscape support all over, and 5 years of updates.
I don't need 15 cameras or ai anything. Heck make it a battery + keyboard case that goes on to a pixel and does a decent typing job and I'd buy it.
> before you say “sounds like you just want a tablet”
One of my friends uses a GPD Pocket 3, which is an 8-inch computer that's small enough to fit in your pocket, with a full keyboard, touchpad, and touchscreen that swivels to become a tablet. It's pretty nifty. https://gpd.hk/gpdpocket3
I don’t think screen size is the limiting factor when editing, but the lack of a convenient keyboard. I’ve edited quite large texts with a Bluetooth keyboard connected to my phone. Currently have a Nuphy air 75 and keychron k15 pro, small sized, mechanical, ergonomic Bluetooth keyboards, both make editing pretty acceptable.
To be honest, I haven’t even been able to make a tablet really work for me as a serious laptop replacement for any halfway serious office productivity work. Except for a very short trip I bring a laptop because otherwise I’ll run into something I have trouble doing that I need to do.
True, but i feel the same way about laptops. For true productivity i need at least one big display, so will usually dock it with an external display, keyboard and mouse. Funny thing is that the modern iPads connected to an external display gives more or less the same experience as a laptop (as long as you're not programming...)
I have a multi-display desktop. I accept I’m not going to have full capabilities if I’m not at home. For travel an external display on an iPad is a nonstarter.
But then you need a table or some sort of surface to put the keyboard on, and then I might as well use my laptop if I have it. I want better editing for when I'm standing in line at the grocery store with one hand free.
I have a keyboard that I've attached a guitar strap to, in order to be able to type while standing, without a table to rest the keyboard on.
I miss the Sidekick, myself. Was the perfect device for writing while standing. Physical keyboard and a big-ish screen, with the form of a modern smartphone.
The swivel design made finding a decent case tough though. And there were few apps.
Which was an influence on the first Android device, the HTC Dream. Swivel keyboard AND a trackball. It was pretty cool but unfortunately didn't catch on.
Although they used the 'Pocket' brand for the Pocket 3, it is really a GPD MicroPC 2. The original MicroPC was 6 inches and is an even better candidate for "phone sized laptop". The crazy thing is that it's perfectly usable and much more productive than a phone - if it could make phone calls and had the same sensor suite as a smartphone, it would be superior along every axis.
Oh wow! I remember the 1st and 2nd Gen.! The 3rd one is so nice! I don’t need one any soon, but it looks like I’d love to have one later in the future. I’m very curious what would the next gens look like.
As a bit of a GPD fanboi I have both a pocket 2 and a pocket max. When I was a server admin, pocket 2 went everywhere with me in a pelican case. That was just this side of too bulky to carry without noticing.
Pocket max in the pelican was just on the other side of that threshold.
The benefit of the pocket 2 is the full windows/linux installation alongside the usable qwerty keyboard.
I love the concept... but in that price range, I'm no longer comfortable just carrying it around with me, which partially negates the reason for getting it.
It doesn't address this editing issue at all. It somewhat attempts to address multi-tasking, but this thread is specifically about how poor the text editing experience is, and I'd like to hear how a folding phone fixes that?
I have an iPad and even there without a mouse plugged in, the editing experience is poor.
You're correct. It isn't a direct solution. For the most part, it makes the selection experience easier, which then makes the editing experience easier.
But I wouldn't underestimate how big a difference it is. One difference between a folding phone and an iPad, is that I can reach every part of my screen with my thumbs still in their resting position. It feels a lot more ergonomic to have an 8 inch 4:3 screen than a larger 16:10 screen that requires hands gymnastics to touch type.
Hold to select word. Drag to extend selection. Drag pops up the edit menu by default, which takes little screen real estate and zero additional clicks to do a cut/copy. Lastly, a cut/copy automatically puts the snippet as the top recommendation on my auto complete bar on the keyboard. So paste doesn't need triggering the edit menu again.
Samsung doesn't have a double tap experience. So it removes that ambiguity.
I have the stubbiest fingers I know and I type a lot ; as my HN profile bears witness. So I have heavily user-tested this experience that I claim to love so much.
Shocker! touch is not the best at complex and precise input. But it’s great for general purpose use and buttons. Problem is we all at some point in our day, week, month need to do something complex on the touch device, which becomes a challenging task and experience.
I generally agree and also disagree with a few aspects of both your comment and the OP.
First of all, to the OP, maybe not for the leet kids, but I absolutely find large amounts of text input on a phone to be a pain. I can do it if I’m away from a computer and have to, but I don’t enjoy it and it takes more time and requires a lot of concentration.
As for cursor positioning, holding down the space bar on iOS works pretty well—although it took me a few years to learn about it.
I do think text editing could always be better and I actually think it’s a pretty broad use case. I’m also not convinced mobile is easily adapted to it, especially without really good text input [probably speech] which also isn’t suitable in a lot of situations.
> As for cursor positioning, holding down the space bar on iOS works pretty well — although it took me a few years to learn about it.
I did not know this and it’s a godsend. I have otherwise found iOS cursor positioning to be awful, especially when text selection is involved as well. Thank you.
Most operating systems have a lot of really useful shortcuts that most people don't use despite it potentially making their lives 10x easier like Ctrl/Cmd+C. Swipe to undo, holding down the key to access accented keys (also works on address bars as it prompts you to add common domains like .com or .net), slide to type, etc. An easy way to discover most of these (on iOS specifically) is just to go through the working with text section in the iOS Tips app.
I have been using this sine iphone 6/3d touch era and it is ok, but has gotten worse.
1. The space bar is a much smaller target than the original whole keyboard and I often find myself needing multiple press drags.
2. Something changed recently and it is harder to hit your placement target in text especially in single line inputs. When this happens your cursor ends up back where it was or at the start of the line. It is particularly difficult to edit or select a url in the safari menu bar.
Note: I used the space bar cursor placement at least 5 times writing this. It would have been much more difficult without.
Maybe I’m a leet kid but I type 65 WPM on my phone so I find actual typing on my phone on par with typing on desktop, except if special punctuation is involved.
…but the whole mobile experience is still terrible with a lot of scrolling, panning, and annoying text selection.
I’m aware most people don’t type that fast so they would never type on mobile but for those that do, it would be sweet if the mobile experience was better.
One thing I found out on iOS recently is that you can shake your phone for text undo.
Holding down spacebar and swiping side to side changes keyboard language for me. Have to hold stationary for quite a bit longer before it becomes a cursor mover.
The moment I opened the article and started reading an idea came to me before reaching the authors solution. Imagine if you could just double tap a word and suddenly it becomes focused and enlargened while slightly blurring the rest of the text. The word in question now has easily targetable tap targets that you can modify quickly, swipe up to delete, rearrange, select etc like a sliding puzzle game. Like everyone I've had the same problem but in a majority of the cases its to fix something that the phone suggested or accidentally added to the directory that I can't easily remove (thanks ios) and nothing overly complex.
An especially sucky part of editing text is changing tense or plurality. Changing "run" or "ran" or "running" is a exercise in frustration because the editor could easily change it for you by reading what you're typing.
Eg I have "bottles are falling" typed out, but I go back and edit it to say "bottle are falling", and then the editor offers to update "are" to "is" for me.
Swype keyboard used to have this feature built in. Put the cursor on a word, any word (made earlier by swype's edit keyboard, which had cursor controls), and tap the Swype key. You'd be presented with a list of alternates/stems, with the current word as the root. At minimum you got simple plurals
Gboard has a similar edit keyboard, although I find it's a tad harder to get to than Swype's
A lot of the problem (for me at least) is that phone text-selection UIs are way too stateful. It’s very easy to mistime a tap and get a word select instead of selecting a different spot. It’s also very easy to misposition a tap and select too much, or not enough.
Adding text-selection features is IMO not the best approach. I’d actually rather they reduced the feature set and made the effects of UI actions less dependent on the state of the text selection UI.
> I dread opening the mobile version of google docs, sheets, or even Notes for anything more than a grocery list.
I dread it even for grocery lists. For one the apps (Notion, Google Docs, you name it) all give me a stupid splash screen when I tap them, and I have to wait 5-10 seconds to see content. We're in 2023. Splash screens should be unnecessary. Opening a piece of text should take no more than 50ms after the tap on the app icon.
About half the grocery stores I frequent have no cell phone signal. Why the HELL can't a device in 2023, with all the machine learning we have today, deduce from my GPS track and time/day deduce that I'm probably headed to one of several grocery stores, record that the last time said grocery store had no signal, and pre-download anything that looks like a grocery list, or hell, labelled explicitly as one? And do whatever rendering it needs off screen so that it's no more than 25ms from tapping the app icon to viewing it? It has a GHz+ processor, 90Hz screen, 25ms isn't a tall ask to render some text. 5-10 seconds? What are engineers busy doing?
And this is why I still use post-its for grocery lists.
We've had this device for many years, you're just using bloated apps
Just tested Simplenote on a very old phone, reopens immediately in the list and I can type/check mark items right away (though first open takes longer)
Apple notes is also snappy
I like AnyList for this. Keeps the groceries synced to your phone, sticks things in relatively sane categories, is fairly customizable, supports multiple real time editors (so my wife and I can both check stuff off as we add it around the store), has a web interface, and until Google decided to ruin it, had a nice smart speaker integration
I don’t understand why a tablet requires lugging a briefcase around? Just carry it raw between meetings and put it down if you need two hands for something. If you already carry papers around, get a folding keyboard case. Maybe we’re just thinking of different working environments and I’m imagining an office whereas you’re imagining a job site or something.
The actual reason that a tablet isn’t the answer is that the Google docs app for iPads is incredibly bad (I’ve had one second per letter max typing speeds with it, though thankfully it buffered my key presses). And then the solution kinda sucks because you would want a light laptop (eg MacBook Air or windows equivalent) but closing the lid will add too large a delay so you’ll need to carry it around half-open, which isn’t great but also isn’t lugging around a briefcase.
I used Acer Iconia W4-821, a 8" tablet on Win8 for some time.
It was great for my use case, to tinker with PwSh/CentOS VM while commuting or ocassional meetings as a notetaking app. Sadly it wasn't the device even I (a big, tall guy with big hands) could haul around with ease. It really wanted some sort of case/minibag or whatever. It was perfectly fine with a backpack or a messenger bag, but how often do you see people with a messenger bag in the office or meeting?
EDIT: the best case for it would had been some sort of a pistol holster or a leg tacticool holster... even I would mutter 'Nerd' on sight of this.
> I don’t understand why a tablet requires lugging a briefcase around? Just carry it raw between meetings and put it down if you need two hands for something.
Being forced to carry the tablet every time you're moving during a work day is annoying. The big issue is not moving between meetings, it's every other movement during the day. For example getting lunch or getting to the office (especially on e.g. the subway or bike).
I think the target market is huge. My wife, for example, spends tons of time writing out stuff for Instagram posts, some quite long. And I see tons more across social media that is largely done on people's phones.
This is what folding phones are great for. The extra space is great for multitasking, reading spreadsheets, viewing calendars, reading documents, etc.
They don't really help with writing or editing text, unless you have big hands and need a larger keyboard. Most of the difficulty with text editing are related to the cursor, as the article points out, and a bigger screen doesn't really help. I suppose the extra real estate can be used for a docked or floating menu.
I consider my phone devices mostly read only. I do have an iPad and text editing is better, but still there are still stumbling blocks. Entering text is pretty easy, and making large scale cut/copy/paste edits is OK. It's the smaller edits, adding or changing punctuation, fixing typos, transposing words, or changing small words (‘the’ to ‘a’, for example), where I find difficulty with the behavior of the selection function.
Yeah, the bizarrely apologetic tone of this article was a real turn off. I don’t think any reasonable person would classify text editing on mobile as anything less than needing a full overhaul. Also, whose feelings are we sparing here by being apologetic for criticizing it?
Surprised it didn't mention the lack of a "Delete" key (at least in the standard Android keyboard). All that is provided is Backspace. On the desktop, I use Delete and Backspace equally, depending on where my cursor happens to be.
Can't speak for everyone but it took me a decade of computer usage to even "fit" the delete key in my text editing workflow and getting used to reaching for it. I still don't use it much, and it doesn't help that it's hard to reach on laptops and not simply shift+backspace. I think of the delete key as a power-user feature.
This is honestly why I carry my GPD MicroPC with me now, as its almost as portable as the phone, and unfathomably easier to do things like draft emails, do text editing and other basics, than my phone is.
that has to look hilarious in the grocery store, I just have to say
I need to get a replacement battery for mine after it drained too deeply and no longer will charge :( They didn't push out some kind of fix for that, did they?
Just to be clear, the problem here isn’t entering text, but with editing it. With better keyboards, voice transcription, and physical keyboards on many tablets, getting text into a device is not the problem it used to be. However, you will always want to edit your words afterwards.
I personally don’t like speech-to-text because then you’re broadcasting every word you write to everyone around you.
It might be better with a specially-trained model that uses stenographer-style compression that only you know, so drive-by eavesdroppers can’t translate your text entry on the fly.
Very nice ideas, please please please someone pick this up!
Doing anything on a mobile device is far more painful than people assume. A generation of people are now unfamiliar with using a PC and although it has its problems, the use cases are far easier.
Are these really solved cases?
- multiple tabs in a browser. Why not put tabs/shortcuts on the desktop (errr mobile top?)
- apps in general, why do they exist at all? Most are just slightly better web pages--ok I know it's just to give Apple&Google gatekeeper status and create a way to get paid for the app. but at a cost to users that is more and more annoying. Why have a mobile aware NYTimes.com AND an app? Ditch the apps where not really needed and do a decent one-page web app instead. No need to update, no special gatekeeper for install. If the first issue was solved, it would make mobile devices easier. I have no sympathy for the Mush burdened x-twitter but if they have a gripe with Apple do a decent web page and no gatekeeper is involved.
- &^&%$#$$% passwords. Partly because of apps in general, all your passwords are hidden from your password manager, which is of dubious value on a mobile to begin with. Typing in that auto-generated password created by a password manager is HELL on a mobile device, text being hell as this article points out.
My solution to text editing on a mobile device is:
- Use a hardware keyboard.
- Install Termux.
- Install and use vim (or emacs, or nano, or whatever) to edit. That is, use a goddamned fucking text editor designed for the task.
The fact that devices similar to the 1990s-era Psion Series 5 are rare as hen's teeth doesn't help. This was pocketable (given sufficiently large pockets), had basic office productivity apps preinstalled (word processor, spreadsheet, database, email, contact and diary manager), as well as its own scripting language. It was optimised for power consumption and battery life. I know at least one journalist who used the device to file copy on the road --- this was no toy.
(I did receive a highly intriguing email suggesting such a device might be in development but have had no further follow-ups.)
There's a delicate balance between size, usefulness, and portability which is challenging to get right. In virtually all cases even the worst netbook-type compact laptop is superior for any significant composing task than the best smartphone or tablet. Touch-only interfaces are ultimately a dead-end.
The section of Jenson's article on how taps can be misinterpreted should be tattooed to the forehead of every UI/UX designer, PM, and marketer on the planet, as well as stenciled to all office walls, windows, doors, and whiteboards. I've railed about the ambiguity of selection vs. movement before (see: <https://news.ycombinator.com/item?id=31771672>), but Scott highlights additional nuances. Does a screen touch: place the cursor? bring up the menu? start a drag? start a double tap? start a long press?
And what of when the user is disabled (Parkinsons, other motor/neuron disorder), tired, ill, drunk, in a moving vehicle (preferable not as the driver/operator), juggling children (don't drop), or in any other way at less than 100% focus, attention, ability, and dexterity?
I've had some variant of a smartphone or tablet since the mid-aughts. The two I've the fondest recollections of were the first (a Palm Centro, with a hardware keyboard) and my most recent (Onyx BOOX Max Lumi). Each has different strengths. The Centro was good for basic text entry, though its display was minimal. The BOOX is a large e-ink device which is excellent for reading and has Termux (a Linux userland for Android) installed. With an external keyboard it's passable local Linux system and can be used to SSH elsewhere, though the combined set is not something that can be crammed into a pocket. Slides easily into my messenger bag though.
As I'd listed out in table-form three years ago, there is no use-case for an emissive-display tablet for which it is superior to a different device, and the distinction is little changed for smartphones. Display + keyboard remains the ultimate productivity environment. For capability, privacy, and flexibility, I'd prefer separate audio, image/video, and comms devices, each of which individually can be quite small. Technological advances for each is slow enough that these need not be replaced frequently (once a decade or more should be sufficient). The main driver of late has been storage capacity, though with 1 TB being now widely available, that's no longer a driver for text, audio, or even to a large extent images, though video can of course consume prodigious storage.
The privacy advantages of airgapped generative / capture tools, with a separate comms-capable device also seem increasingly advisable.
I find it telling that one of the more popular addons for ipads are covers with a builtin keyboard. It's a way bigger device than an iphone. But yet the keyboard sucks enough that Apple sells covers with a keyboard. Of course, all the touchscreen keyboard problems that the ipad has are magnified on their iphone. Yet, they don't have a solution for that. And they also sell a stylus for the ipad. Because fingers lack precision. It's the same OS but there seem to be no such options for the iphone. Does the stylus even work with an iphone? Is that deliberate? It's not like people are going to be magically more precise on an iphone relative to a huge ipad. Conclusion, Apple just accepts that that's the way things are. And besides, Steve Jobs would turn in his grave if they dared to ship an iphone with a stylus.
Hardware keyboards on phones used to be a thing. I worked at Nokia back in the day. Really nice keyboards. Blackberries were popular too. People wrote lots of stuff on those things. I wouldn't mind a little pocket laptop. It's not like my pixel 6 is small or subtle in my pocket. It would be more useful with a slide out keyboard.