This stuff has been a nightmare for us for a long time. We have to scan pdf417 barcodes from physical identification as part of our offering.
Recently, we stumbled upon some cheap USB keyboard emulator 2d scanners that can pick these up very reliably. For our product/customers, this is a viable path. We have tested probably 30 different webcams by this point. The only 2 that are as reliable as the handheld CCD scanner are the iPad and my Canon DSLR tethered to my PC in automatic mode with a macro lens.
Resolution is a factor, but so is frame rate and environment. You have to consider the full UX stack. Someone is required to physically position the barcode in a particular way with certain lighting constraints, etc. With a handheld CCD scanner, you get a laser guide for precisely what distance to go for, and an ergonomic experience that is much more conducive to successful scans. Most offer on board illumination, so even in a pitch black room you will get a successful scan.
We ended up going with the purpose-built barcode sleds from Infinite Peripherals. The Linea Pro, I think. Extra battery, onboard illumination/laser guide, _really_ fast barcode engine in any orientation, and a few pretty simple SDK options. We ended up using "our custom webkit browser will fire a custom JS event on scan", which worked perfectly for our purposes.
Everything else is measurably slower: waiting for focus in a dim room, low framerate, weird other issues.
We use the Linea Pro too w/ the SwipeTrack browser making them available in our web app. We've previously used both the Infinite Peripherals SDK w/ an iOS app, but SwipeTrack have a pretty good JS API [1].
Funnily enough, the Linea Pros have been going down hill. They swapped out the barcode engine and it is not fast / crisp. Sometimes faster to just use the iOS Camera (w/ Swipetrack) instead of the Linear Pro's sled.
Wow, I don't like the sound of that at all. The version we're using is an older.... 5? 6? So it would predate those issues but that's literally.... half the point of the thing! Lightning fast barcode engine + range + extended battery + pistol grip!
We also scan pdf417 from ID cards. In our case, it was much easier to just use hardware 2d scanners. We are a b2b SaaS and they are scanning IDs all day, so a hardware solution makes sense.
The weird thing about pdf417 is that there is a lot of helpful info our there, up until a point. It's like everyone is collaborative and asking/answering questions up until the point they really figure it out, but those last few steps never much make it into open code.
A couple times a year I go searching to see if anyone has implemented solid pdf417 in WASM yet. It seems like a natural fit to me, and I am sure people have done it, but I have yet to see anything publicly available.
I wonder how good the Chrome/Android PDF417 reader is? Is it just ZXing, or something else under the hood? For PDF417, ZXing is only OK. Unfortunately PDF417 is a pretty crappy format to decode with a sensor/camera compared to DataMatrix and QR which have much easier alignment targets and are designed for the sensor/frame based use case (instead of a 2D scanning laser). I wouldn't expect great performance out of BarcodeScanner / Android Chrome if it's just using Zebra Crossing under the hood.
Chrome is likely using ML Kit on Android under the hood on Android. [1]
Well if you try the demo I just made [2] you can find out. Works pretty well in Chrome Version 99.0.4844.51 (Official Build) (arm64) and on my Pixel 2. Naturally won't work on my iPhone because #SafariIsTheNewIE
I found the most reliable barcode scanner to be an old Symbol usb scanner.
I also found that I could scan barcodes off of a screen with a laser scanner if I put a translucent white plastic bag over the screen. Maybe someone with more knowledge about optics could give some insight into why that trick works. My understanding is that laser scanners should only be able to scan by reflection, and shouldn't work off of a screen at all.
Reading a linear barcode is quite easy, the shitty webcam of my 2010 macbook pro was enough for Delicious Library to pick most every article instantly, usually faster than I could even present the article.
Purpose built barcode scanners typically don’t fumble with autofocus have faster 0-latency autoexposure (adjusted during frame readout) and even the good 2D CMOS sensor ones will start decoding linear or stacked codes such as PDF417 while the frame is being transferred. That can’t be beaten for latency.
Idk but whatever library the NHS website is using to scan the QR codes on LFTs is flawless, as soon as the camera view opens it immediately scans, you'd miss it if you blinked.
>Secure context: This feature is available only in secure contexts (HTTPS), in some or all supporting browsers.
What's the point of this? I understand why you might want this for APIs that expose private user information, but this just performs some computation. You could do the same thing with a js library or webassembly. Is this just to punish http sites?
There might be more to the privacy characteristics of an API like this than meets the eye. For instance, if a user grants camera permissions scoped to http://insecure.website in order to scan a QR code and get it detected, it might be surprising to the user that on-path attackers would be able to see the user’s camera snaps. Making the API work in a degraded mode in insecure contexts (e.g. by disabling particularly privacy-sensitive data sources) might be a plausible alternative, but at this point it’s arguably cleaner and less risky to just make the interface HTTPS-only in order to avoid making developers and users reason through these nuances.
I don't get it. The privacy risk in there is from getUserMedia(), which is already (correctly) https-only. There is no privacy risk coming from the barcode API itself.
Basically, yes. Browser vendors have had an agreement for some time now to only expose new APIs in secure contexts in order to encourage HTTPS adoption.
I am in agreement with wyager, this feature looks like it could be heavily beneficial to a shipping company's internal web app for instance, or other applications that could be running completely isolated from the global network (or running on the machine itself for that matter).
Forcing https in these applications is a waste of time and money for orthogonal purposes. I guess it could be seen as a cost of dev. for any web based application from now on, but it's sad to me to see the barrier to entry get higher for these reasons.
I've had the same issue, but mostly for home lab type stuff. Even if you set up an internal CA and all the trappings, you still have a root cert problem. For consumer devices without easy access to certificate stores, it gets complicated fast.
Maybe the state of the art has changed since the last time I looked, but it would be really nice to have something as easy as Lets Encrypt for private tools while at the same time not exposing internal network details.
External wildcard cert? Maybe it’s different in smaller shops, but it’s just as easy for me to get an internal cert as it is to get an external cert. Only annoying part about our shop is a hard req fit EV certs, so no ACME.
You can use a public CA like LetsEncrypt then. Exposes you to the certificate log but you should be secured already anyways. Just have to use the DNS challenge (unless you wanna poke a hole for certbot) to grab it
And people wonder why ioshit devices are all so cloud dependent. I just want my microwave to talk to my refrigerator (or whatever). But they have to use https, because. And the cert has to expire every 90 days, because. So now I provision a kitchen full of stuff with AWS creds so they can respond to DNS challenges to get those certs.
So much simpler for everything to revert to client only mode and route all messages through a server 3000 miles away. Until they pull the plug and nothing works at all.
This is why I hate privacy first thinking. The IoT is a big deal. I own basically nothing that has electricity that wouldn't be slightly to majorly improved, at low cost, by IoT.
So... lets... make the tech that powers it not suck, so we can stop with all this analog business.
Why would you ever want your microwave to talk to your fridge? Imo the real reason IOT is such a shit show is because it's being crammed into every device without any actual benefit to the end user.
DNS challenge is easy to pass and automate if you can programmatically add txt records to your DNS zone. Every major cloud provider supports this with command line tools, so it's a matter of moving the DNS zone there and writing a shell script and a cron item in the worst case.
I had a chance to skim through the link you posted - they are doing the http challenge verification (in step 6) for some reason which involves forwarding their domain into their internal network.
The DNS methods we already mentioned does not involve any of that - just a simple zone file change or a few clicks in a web UI to add a new record.
For those whooshed by the reference, it's about the iPhone's antenna design flaw that Jobs poopooed on stage, to silently get it fixed on the next version.
It seems we're agreeing it's an actual issue that would merit to be fixed.
I'm not sure I feel great about gating feature A behind completely unrelated feature B just because manufacturers like feature B. Even if feature B is concretely pretty good (which is debatable for https as it exists now), this seems bad on principle.
It's not always necessary. Think fully offline networks that can't/won't use a CA anyway, or networks where physical/machine access is the intended layer of security (a web server running on localhost).
In these scenarios a self-signed certificate will rarely improve security because most users will click through the warning anyway in case of an MITM attack.
Users should not have to rely on the network being isolated for security.
Even when I have an offline network, I still use SSH whenever possible.
Yes, I don't benefit from initial verification, but I pin the certificate from then on.
I don't think that the current "all or nothing" paradigm that we use with SSL in browsers makes any sense.
I have been really disappointed over the last few years deploying network connected devices and trying to make their services available in a secure way.
It is not really possible for ipcams, routers, etc to offer services in HTTPS in a semi-online network. There should be a kind of 'encrypted but unverified' mode.
Even the worst failure mode is no worse than a plaintext connection.
Let’s say I sell a physical device that allows you to use a browser as its UI. You just plug an Ethernet cable and point your browser to its web server.
Security is done by physical access. Anything else is just extra complexity and points of failure, and if an attacker can get physical access to the LAN cable, he can just as well walk to the machine directly and change the settings on the control panel.
HTTPS, with its current UX, would be a net negative in this case.
> HTTPS, with its current UX, would be a net negative in this case.
I would just replace "HTTPS" with "SSH", and see if that statement is still reasonable.
Even though SSH is not perfect when you can't verify the initial connection to a host on a local network, using TELNET instead is not a solution.
I have yet to hear a reason why HTTP is better than self signed and pinned HTTPS; why possibly insecure is worse than insecure.
The only justifications I have heard are UX justifications, and those are really just a critique of the UX, not the protocol.
> Security is done by physical access. Anything else is just extra complexity and points of failure, and if an attacker can get physical access to the LAN cable, he can just as well walk to the machine directly and change the settings on the control panel.
That is an extremely fragile solution.
By default ethernet is very open. It is very easy to bridge it with other devices or routers to announce themselves and start routing. It happens all the time on what are supposed to be closed networks, I've seen it.
And many networks are semi-offline. You may need to have limited internet access. Or the network configuration may change in the future.
> By default ethernet is very open. It is very easy to bridge it with other devices or routers to announce themselves and start routing. It happens all the time on what are supposed to be closed networks, I've seen it.
The reason for using Ethernet and browsers is that both are built into pretty much every computer, don't require any drivers, admin privileges or persistent software.
The idea is to replace USB, serial or some other point-to-point link which would require platform-specific drivers and client software. Most peripheral connections for printers, etc don't employ encryption on the control cables and the world hasn't ended, so physical security here is good enough in practice for the majority of purposes.
Browsers make an exception for localhost and treat it as a secure context.
It's also possible to get a certificate from a CA using any public IP address, and then reuse that cert for LAN. Certs are bound to domain names, not IP addresses.
Why should an org expose details of private infrastructure publicly? For anyone with more than a few internal tools this becomes untenable or unreasonable quickly and sharing a wildcard cert has diminishing returns as its shared between apps, teams, etc.
If it helps, it really doesn’t care what certificate the page it signed with, whether that be localhost, Mozilla CA participants, or maybe even a future CA fork for Russia if that happens. Plaintext HTTP is just not advisable even on a private network when some inline network device like LAN turtle could be passively exfiltrating traffic.
The browser security model is trivially defeated without the usage of https. IT is not an orthogonal or unrelated feature.
Making all new API development available only when content is delivered via a mechanism that provides the foundation for the rest of web security is good engineering practice.
I'm not a fan of the whole secure context separation thing. A lot of cool P2P stuff is impossible because a service worker can't access an HTTP server, which would be very useful for web sites to talk to IoT devices.
If there was something sensitive in front of my camera, I wouldn't give any random site permission anyway.
File system access should definitely be secure only, the rest might not need it.
Of course if Mozilla actually did FlyWeb we could almost totally deprecate HTTP except for some public legacy sites.
pdf417 barcodes are commonly used on US driver's licenses, and contain private information like the person's address, height, eye color, full name, etc. For the API to do its work in detecting a valid barcode, it needs to be able to read the entire barcode to compute a checksum/checkdigit. Which means it has to read the data.
But this API doesn't allow access to the camera. The caller is responsible for passing in an image to be parsed. Saying it needs to be blocked for that reason is like saying parseInt() should be blocked.
Used this in a b2b saas for a few years. Worked well and helped remove complexity in a previous way we did it that relied on a native app called pic2shop. We lost a bit of functionality specifically a clear target zone, green scanning line and better low light support. It made up for it in ease of support and reduction in perceived jank.
I recently made a webapp for scanning barcodes using the phone camera for warehouse use. I used QuaggaJS. I disabled the barcode locator because it was too heavy, instead I just draw a box on the screen that the barcode is to be positioned in. The barcode detection and decoding happens entirely in the browser and it works very well.
I also built a web app for barcode scanning. For testing quaggaJS is fine but it’s highly inconsistent and requires an extreme degree of customization to get adequate results. I recommend scandit. It’s a good production ready library with a very nice web sdk.
My team was trying to get a JS solution working as well, using zxing-js. The barcode basically had to be perfect size, on a flat surface and no glare. we ended up swapping it out for ScandIT's Web SDK and have been very please with performance. As someone else said, it is quite expensive and being a public-facing web application, we had to get a per-scan license vs device license. We're going under contract for 1 year with a "shot in the dark" scan estimate and will renegotiate next year once we have usage statistics.
Would be nice to see if this new spec gets full adoption so we can avoid high licensing fees.
Same here. Scandit was the best thing we tested but they quoted us $50k/yr. We don’t have many users that actually need that feature so the next best option was compiling zxing to wasm. That performed a little better than using the js version.
As the page does not discuss the motivation behind the API, does anyone know why this is ”native” API and not just a generic image processing script implemented with JavaScript / WebGL / etc.?
Of course, it's a matter of perspective whether you want to see this as lovers of the free web helping webapps to feature parity with native apps; or Google churning out yet another low-effort standard that's easy for them and difficult for all their competitors.
At this point in time it's a pain point for me that Android devices don't usually have built-in QR reader software. I can tell iOS users to just "scan the code with the Camera" app but I have to tell Android users to find a QR scanner app on the app store.
My experience with an actual bottom-of-the-line prepaid phone was that the first QR scanner app I downloaded from Google Play worked right the first time but I had to try six different apps before I found one that worked with my Samsung Tablet.
A web-based scanner would be a possible answer, though really Google and the phone makers and the carriers should have prioritized a "just works" solution years ago.
I've been scanning QR codes with my Android phone for a long time. I know at least two native cameras work, Pixel's and Samsung's since 2019. But I am sure legacy phones with older android versions don't support it natively.
I have a flagship Xiaomi phone and the built in QR code scanner is garbage. I went to a restaurant with friends which only had a QR code for the menu (which turned out just to be a link to the root of their website - why not write the URL too?). In the end I had to take a picture of the QR code, then crop it, then open that in the scanner app.
I've had intermittent issues with my motorola's camera app. Sometimes it picks up qr codes at restaurants, sometimes it refuses, maybe because of lighting issues, though turning on the flash doesn't help. I ended up downloading BinaryEye from F-Droid and I just default to that now.
As always, some fragment of Android actually got there before Apple (to put it in the Camera app), but no, it's not standard.
My current phone (Nokia 3.4) doesn't have it, but previous one (Motorala One) did. Ironic, since I chose them for being about as close to stock as you can get, both in the 'Android One' programme (so also comparatively LTS).
Honest question: for what reasons do you ever need to scan a random QR code? I think I've used them to sign in to some apps (eg, WhatsApp) while logged in on a PC, and other than that exactly 0 times.
The Google camera app does support QR codes FWIW, but I don't know if any other vendors except Google use it.
> Honest question: for what reasons do you ever need to scan a random QR code?
In the past two years, due to COVID, a lot of restaurants and dining establishments got rid of paper menus and left people with "here is a QR code, scan it to see the menu". This is the most common situation I've encountered this in. It wasn't just a few restaurants that did this, it was heavy majority of places here (Seattle).
In general, assuming you have an easy-to-use QR scanner, it's easier to scan a QR code than type in a URL on a phone, especially a complicated one. A couple examples I've run into recently:
- Lately it's been pretty common for restaurants to replace their physical menus with a QR code on the table that links to their online menu.
- When I've taken COVID tests recently, the site gives me a sticker with a QR code that links to my results page.
- I encoded my wifi network/password into a QR code and printed it out, so guests can just scan it to join, rather than have to scroll and find the right network and type in a password.
They became very popular during the pandemic for restaurants to offer their menu as a scannable QR code instead of distributing and sanitizing hand-held paper menus. I think this is becoming less relevant now.
They're popular for street-level advertisements for (concerts, events, clubs, jobs) to provide a link to more information without the consumer having to type a url into the browser. This is more important if you don't want to pay for a pithy high-level domain as a landing page for the thing you are advertising.
I have also seen them for easily entering WiFi credentials to your device on a new network.
They can be used in museums to link to self-guided tour information on the artwork/exhibits you are looking at. This used to be done more with rentable/loanable walkman+headset kind of devices, but makes more sense to do on one's own mobile device now.
I print ‘three-sided cards’ that have a photo or art reproduction on one side, some documentation on the back, and more information if you scan the QR code. One series of the cards are ‘music cards’ that will play a song, another application is ‘constellations’ that let me stick anywhere from 3 to 40 cards on the wall (so far) and you can see the back sides and other narration by scanning a code.
So my use cases are (1) I am demoing the cards for people, and (2) I give the cards away for free (If it wasn’t for the supply chain crisis I’d be creating a new card design every day) or put up an installation somewhere and people interact with them. Either way I want it to be very easy for people.
In Argentina, QR codes is how you pay with your phone in super markets, pubs/bars, restaurants, everywhere - its their "apple pay", using Mercado Pago [0] app/service. Also, all the paper menus from restaurants are replaced with QR codes thanks to the pandemic.
Native code allows for performance gains that a script (even a performant one) may not be able to match. After all, this is fundamentally an image processing problem, one of the more computationally expensive things a browser has to do.
Isn't that the point of webassembly? Just seems like we are still moving in the wrong direction on trying to move the whole of NPM in to the browser web apis, rather than building a solid base you can build anything on.
High performance barcode decoding (e.g. scandit) uses the GPU.
After all, a modern cell phone could be producing 4k 30fps video, and high performance barcode decoders will detect linear barcodes with lines a single pixel wide, even in the presence of moderate noise/blur/overexposure (e.g. the barcode on a shiny can of coke)
Of course, I've got no idea if browsers will provide high performance barcode reading, or something simpler like zxing.
And I've got to say, it's not obvious to me why browsers would implement a barcode scanning API in preference to a more general accelerated image processing API. Plenty of other applications want to do real time video processing in the browser - such as video call background removal.
Do you have any examples of what a more general image processing API would feature? Everything between reading pixels and reading barcodes is a mystery to me.
I'm not a browser developer, but to wager a guess, it probably has something to do with the inertia of existing solutions being native, rather than in WebAssembly, which is still quite new as standards go.
It's a heavily standardized, very recurrent problem that can take a lot of advantage of native code and platform-specific optimization, so it's really an ideal target for a Web API.
To allow the implementation to take advantage of libraries provided by the operating system[0] (where available), without the web site developer needing to even know that it's happening.
Barcode detectors can be pretty big dependencies. Given that this will often just be forwarding directly so some kind of OS capability it makes sense to ship in browser.
Bit off-topic: can anyone recommend resources (videos/papers/tutorials) that could help me learn how QR barcode detection and scanning itself is implemented?
I've been thinking that building a basic scanner from scratch (i.e. only using vanilla cpp/python/whatever + image processing libraries, but nothing barcode-specific) could teach me a lot.
I forked another person's code and added bounding boxes, if you want to check it out. Seems like it works pretty well, though the bounding boxes are erratic.
Cool. Another feature that Firefox won't support. Same with the WebSerial API and others. I sort of understand their reasoning that they want to make a web browser and not an application platform, but that's where the web is shifting to ... or rather has already for a long time. And people wonder why every new tech/browser is based on Chromium all the time. Well, this is one of the many reasons.
I get what you are trying to bring up as an argument, but even Firefox doesn't argue this way. See their answers on issues related to these kind of APIs.
Does anyone happen to know where in the Chromium code base the implementation is hiding?
The android version seems to use a library that's part of Google Play Services:
/**
* Implementation of mojo BarcodeDetection, using Google Play Services vision package.
*/
and
// The vision library will be downloaded the first time the API is used
// on the device; this happens "fast", but it might have not completed,
// bail in this case. Also, the API was disabled between and v.9.0 and
// v.9.2, see https://developers.google.com/android/guides/releases.
I was interested in this because there are no really good open source DataMatrix decoders (to my knowledge) with the one in ZXing being basically unusable (last I checked) and libdtmx being ok but not comparable to commercial offerings.
A different context but related: a website plugin that I have built years ago for barcode scanning machines: JBarcode - https://github.com/veysiertekin/jbarcode
I didn't made any changes since then. But it can be easily ported into a more modern framework (React.js etc)
I’m happy to see things like this. My ISP used to ask for a picture of my docsis modem sticker because people kept making mistakes entering in the MAC address.
Unfortunately, the csr was just keying them in and keyed mine in wrong so I was without internet for a few days.
Something to automatically pick it out would be a lot better.
I’d love to see this in safari on iOS as well. The native barcode detection capabilities in iOS are substantially better than libraries like zximg, and I like barcodes as an interface between the real world and tech. I also wish text recognition were in safari.
Huh, TIL. Recognition performance is pretty modest (on Chrome/Mac at least) but it's fast and probably more than adequate for most close-up mobile device scans. It basically just wraps MacOS's Vision framework on the Mac, and ML Kit on Android.
I'm still looking for an open source python library that can decode Data-Matrix barcodes as seen on Digikey bags. Should I give up and switch to JavaScript+browser?
I just tried the example zxing-cpp Python module wrapper script and it worked with datamatrix.png from the pylibdmtx test suite. Hmm, but read_datamatrix from pylibdmtx detects two values while zxing-cpp detects one.
According to this documentation, yes, it returns the raw value encoded in the barcode.
What it does not appear to do is further decode the raw value into logical data. E.g. data matrices commonly contain GS1 element strings that are not trivial to decode. There are libraries for that, though.
Recently, we stumbled upon some cheap USB keyboard emulator 2d scanners that can pick these up very reliably. For our product/customers, this is a viable path. We have tested probably 30 different webcams by this point. The only 2 that are as reliable as the handheld CCD scanner are the iPad and my Canon DSLR tethered to my PC in automatic mode with a macro lens.
Resolution is a factor, but so is frame rate and environment. You have to consider the full UX stack. Someone is required to physically position the barcode in a particular way with certain lighting constraints, etc. With a handheld CCD scanner, you get a laser guide for precisely what distance to go for, and an ergonomic experience that is much more conducive to successful scans. Most offer on board illumination, so even in a pitch black room you will get a successful scan.