> This demo is constrained to only using one stroke per letter, but systems that permit multiple strokes still get into trouble when the strokes don't match what they are expecting--for example if you draw an X using 4 individual strokes outward from a central point.
Arguably, an X drawn this way should NOT be recognized as an X--that's not how an X is spelled.
If the task is communicating with the computer, then recognition of the gesture is a valid approach. Just as there are conventions regarding the spelling of words, there are conventions involved in the formation of letters. Why not use them? It would even seem incorrect to leave these out.
The human convention of written language it to interpret the symbols after they have been completed, not during the act of writing them.
A computer that interprets the behavior of writing, rather than the final symbols, is going to violate user expectations at some point.
Why? Because people do not always write as linearly as you might expect, especially when writing fast. They might drop or mis-write letters or words, then go back and fix it. Or quickly jot down just enough letters to remind themselves of what they heard, then go back and fill the rest in. A routine that interprets actions in order is going to have a hard time with actions that the user completes out of order.
"human convention of written language" is a bit much. Stroke order is almost as important as what the actual strokes are in the definition of a Chinese character, for example. Of course unless you literally watch someone write you observe the characters after they're written, but the most predictive latent mental representation of a character does include an order component. I know this because I made the mistake of memorizing many characters almost like bitmaps and have had to go back and learn how to reliably write/read hand written characters.
I don't know what to say other than that the entire purpose of written language is to carry information between people who aren't in a position to directly observe each other writing. (If they were, they could just talk and would not need to write.)
There exist counterexamples in the broader world. Historically in the Sinosphere it was commonish that two people might share a command of written Classical Chinese but not really be able to speak to each other. For a modern example, consider the paper below:
https://eric.ed.gov/?id=ED515291
Even learning English the stroke order can help considerably. Mostly it helps in just learning to write legibly, but that really is just a fancy way of saying it increases the accuracy of recognition. :)
I completely concede that it is possible to get the same results with other stroke orderings. However, there is a reason when teaching children how to write, we often get fairly prescriptive with stroke ordering, as well.
In calligraphy there is the notion of 'ductus', or how and in which order the strokes are written. It has a significant effect on the end result, and for each script, there is arguably only one "correct" ductus. A similar concept can be applied to normal handwriting.
I highly doubt people are using non standard stroke orders, unless they are very young or it isn't their first language. However, this scheme probably won't work for cursive, which is how people actually write.
Going back and fixing up is almost never as legible as getting it right the first time. Even for human readers.
If you truly want to write fast, you go with a shorthand system. I don't know many folks that have tried reading other's shorthands. It probably isn't as tough as you'd imagine, but most of those systems are more demanding on stroke order, not less. If only because the speed is gained by being very prescriptive.
I understand the idea of looking at completed characters and inferring the original order of strokes.
But are you saying that Japanese writing is only readable if you observe the writer during the act of writing? Because that's what some stroke recognition engines do.
Here's a tangible example. Imagine I write "h l o", pause, then go back and place an "e" in the first space, and an "l" in the second space, then hold it up to you. You're going to see "hello," right?
But an algorithm that tries to interpret the act of writing itself, might see "hlo el", because that's the order in which I wrote the characters.
>But are you saying that Japanese writing is only readable if you observe the writer during the act of writing? Because that's what some stroke recognition engines do.
Not exactly, but the situation, as I understand it, is somewhat related: Japanese writing is better (and thus more readable) if the writer observes (respects) a specific stroke ordering.
So, one doesn't have to observe a writer while he is writing to be able to better read what they wrote. But the ordering of strokes can have impact on readability, even when one sees the written words after they've been completed.
I've seen people write letters in all manner of unexpected ways. If the resultant marks on the paper look enough like the intended letter, then it's readable by a human, and if it's readable by a human, it should be readable by a machine.
Not that I don't think "meet me halfway" type approaches (like the Graffiti system) aren't worth using, but in this case we're talking about recognizing writing (the artifact), not writing (the verb).
Your comment illustrates why Palm Pilots found market success with handwriting recognition (as far as I know, the only product that ever did so).
The trick was that they treated "Graffiti" as a new alphabet that users had to learn. Thus, when the recognition engine failed, many users blamed themselves (i.e. their Graffiti fluency) rather than the product.
In contrast, when products that promised to recognize natural handwriting had a recognition failure, the users tended to blame the products.
It's a good lesson for product development--user satisfaction will depend in part on user expectations.
Wow, what a nostalgia trip! The Graffiti handwriting system was brilliant. I only owned a PalmOS device for a short time (I was very late to the party and they were already old hat) but I picked it up very quickly and still remember how to write most of the "letters".
I still think that Graffiti was very fast and efficient. It had real effect on my handwriting and I still see myself simplifying letters in graffiti way when I try to write fast :)
Yeah, when I was learning Japanese it was a useful thought of how order of strokes actually matters and there should be always 1 way to write a letter, but, no -- everything I can recognize computer should recognize as well. No matter how fucked up, if I can guess it -- the program should guess it. That's being good in handwriting recognition. Everything else will be perceived as subpar by the enduser.
Arguably, an X drawn this way should NOT be recognized as an X--that's not how an X is spelled.
If the task is communicating with the computer, then recognition of the gesture is a valid approach. Just as there are conventions regarding the spelling of words, there are conventions involved in the formation of letters. Why not use them? It would even seem incorrect to leave these out.