The WebARonARKit repo is simply a tiny little iOS app that make a WKWebView, hooks up a few message handlers, fires up ARKit, and sends messages back and forth, by the book.
But the WebARonARCore repo is a whole freaking clone of the entire Chrome browser!
Isn't there a much simpler way to extend an off-the-shelf Chrome component like they did with WKWebView on iOS, without rebuilding the entire honking browser?
I know there is a WebView chrome component for Android -- why didn't they simply use that, the same way as they did with WKWebView on iOS? Performance? Does it require features the WebView Chrome component doesn't support?
Yes, they could have done the same thing with an Android WebView. But the Chrome team is actively working on implementing WebVR (which also want to support many AR use cases) so they already have a native (= higher performance, lower latency, etc) implementation to modify with some smaller changes to use the ARCore API.
not yet. right now ARKit feature points are not 'persistent'. Meaning, they are only really used to help track 3d space frame over frame (with the internal gyro). Getting a 3d map that can be shared with multiple people is another layer of complexity not yet integrated into ARKit
No, the toy AR in smartphones doesn't really know depth or scale to the accuracy of a known marker. They just make assumptions and do basic plane-finding to avoid collisions. You don't have to know depth to make a hot dog sit on a plane, you just have to know it's a plane and how many pixels tall you want the hot dog.
It will be a long way before markers are obsolete. My guess is about 5 years.
Have you tried the "toy AR" in new Apple devices running iOS 11? it's quite impressive how accurately it's able to track most environments. Sure, having a known marker makes that process a lot easier, so you can avoid the issue entirely even where ARKit has problems (a large featureless surface, for example)...
But even with the current limitations, it does feel like the next evolution of AR -- one you can just start using with no setup. Printing a marker is not something a casual person, just interested in checking out a new app, would do (assuming they have access to a printer, or a second device, in the first place). I think that it's a different ball game, and agree with OP that it's the main point of the new generation.
That's not to say anything negative about this project specifically -- it's awesome, and really cool to see something like this perform well on the web in general, and mobile web in particular.
Yes, I develop monocular slam for a stealth startup, so I have directly dealt with the requirements for markerless AR. It's a well known fact in SLAM that a physical measurement is necessary to determine scale. IMUs get close, depth sensors are better but have rough resolution, seeing a known object (like a marker) in the space is best.
Because the toy AR displays on a small 2D screen, there's a number of shortcuts that can go mostly unnoticed.
They didn't add ARKit support to Chrome. ARKit only runs on iOS. Chrome isn't allowed to run on iOS. (The iOS Chrome app is just WKWebView with a Chrome branding wrapper, plus a few features bolted onto the side.) But even if somebody did build the real Chrome browser on iOS, Apple wouldn't allow anyone to publish it on their app store.
On iOS, WebARonARKit simply uses WKWebView's standard method of sending messages back and forth to the browser:
My question is why did they have to make a custom Chrome build on Android, to simply integrate the browser with a library like they did on iOS (WKWebView <=> ARKit)? Why didn't they use the off-the-shelf WebView browser component for Android instead of building their own monolithic custom Chrome app, just to integrate the browser with an Android library (WebView <=> arcore-android-sdk)?
This is super cool. I really appreciate the ability to use (albeit a more primitive version of) this technology on my iPhone 6. It isn't even 4 years old, and it seems to be one of the largest iPhone groups in terms of device share next to the 5S.
When iPhones 8 and X came out, I was thrilled more so with iOS 11 featuring ARKit, but bummed out my A8-powered device could not run their advanced augmented reality software. The price points are high right now for me, and what the devices provide are incremental, making me wonder when smartphones will taper off like PCs did in terms of delivering power across new generations of chipsets.
For the time being, software like this allows hobbyists like me to play around with concepts I'd love to but are otherwise locked out of for now. Thanks for sharing!
This depends on ARKit for iOS, so it won't work on any phones that ARKit doesn't support. On older phones, ARKit falls back to 3DOF "rotation only" mode that is not anything you can't already do in WebKit with DeviceOrientationEvent.
Nice work, With huge companies picking up AR and VR and making them more mainstream, I think soon we will see more developers in these fields and finally some very useful applications.
I think the project is cool, but with Google recently putting real work into this effort[1] I don't see how this project is going to succeed over the competition in the long-run. This is hard stuff. Marker-based AR is not enough. And as far as I can tell from the creators twitter account, he doesn't seem to have the necessary math background to tackle the research heavy papers needed to implement the really cutting-edge stuff. Hopefully some computer vision PhD can help out otherwise I would bet my development time on the stuff Google is creating.
So sick and tired of people who act like only Google/Facebook/etc. can do cutting edge work, or that single-contributor projects aren't useful.
Here's something you probably don't know, since you're so quick to dismiss AR.js. AR.js uses Google's WebAR where it is available. So no, WebAR is not competition for AR.js, anymore than Native ES7 support in Chrome is competition for Babel.
But the WebARonARCore repo is a whole freaking clone of the entire Chrome browser!
Isn't there a much simpler way to extend an off-the-shelf Chrome component like they did with WKWebView on iOS, without rebuilding the entire honking browser?
I know there is a WebView chrome component for Android -- why didn't they simply use that, the same way as they did with WKWebView on iOS? Performance? Does it require features the WebView Chrome component doesn't support?