Here's the commercial version.[1] This machine is sorting peas by color. Peas. Individual peas. Each individual pea is examined by cameras for size, color, and looking like a pea. Rejects are kicked into the reject hopper by an air jet. There are machines like this for most fruit. Typical throughput is a ton per hour. Most fruit and berries go through such machines today. That's why the fruit at the supermarket is so consistent.
The process looks like magic. Color-mixed items go in, and single-color items come out, on a line going so fast that no human can see what's happening. It's amazing to see computer vision systems that fast.
These machines work by putting the items on a conveyor belt, then dropping them on a much faster conveyor to spread them out. The fast conveyor goes past cameras, and at the end, launches the items into free flight for a few inches. While in flight, computer-controlled air jets knock out the rejects.
A friend of mine works for a consulting company specializing in optics. They invented a machine for one client which sorts wheat grains. The machine does spectrography of each individual grain, looking for moulds and fungi as well as heavy metals and more. It's a big rotating drum with small chambers for each grain. After doing the spectrography it shoots each grain off into either the selected or rejected direction.
But it doesn't just reject every bad seed, instead it will optimize to keep within legally accepted limits.
> But it doesn't just reject every bad seed, instead it will optimize to keep within legally accepted limits.
This is just... I guess the emotion you feel when reading the last paragraph defines you as either technical guy (outrage / resignation, depending on your age and experience) or managerial guy (pure delight).
I am guessing there is a switch that makes it sort properly, you know, for VIP?
In all fairness, wasting food needlessly can have important environmental impact issues, and if the legal limits are well calibrated, getting the "allowed" dose of fungus might be a good thing from an immunological perspective. Heavy metals not so much...
Arguably, separating out grains, etc., that would be actually unsafe to eat is not really wasting food. Throwing out food that is perfectly edible is wasteful.
In the 1970s, the USA sold a large amount of wheat to the USSR in a famous deal.
The contract stipulated that the wheat would contain something like <1% sand. US wheat at the time had effectively no sand at all, so they mixed in 1% of pure sand, staying just within the contract terms.
(I heard this from a friend whose friend knew something about it; in searching I can’t find a source online, so the story might be apocryphal. I also might be off on the precise percentage.)
Grain elevator contracts contain an allowed foreign material allowance. Farmer's bringing in their grain get docked for any foreign material. Then the elevators add foreign material to the grain as train cars are loaded. It's crazy but that's just how the industry has always worked.
"Recombine or add broken corn and broken kernels to whole grain of the same kind, provided, that no dockage or foreign material, including dust, has been added to the broken corn or broken kernels;"
In the Google SRE book they say that if a service has reliability much better than the stated SLO they artificially introduce errors to get closer to the error budget.
This is to prevent over-reliance on the measured SLO rather than the stated SLO in upstream services.
My understanding is that they don't bring the service to exactly the SLO... To prevent overreliance on a service, it can be sufficient to introduce some level of failure, which may still be well above the SLO.
One major issue and you stop artificially inserting errors, which are inserted at a rate such that you could turn off the error inserted within some timely manner and still stay within budget.
That doesn't sound like a very nice thing to do. I wonder if capitalism is to blame, as I would certainly blame capitalism, or good old fashioned egotistical rivalry that I'm told was common in many areas of life then between the USSR and the US.
I'd say that feeding humans or even livestock is more important than profits. Or at least, it ought to be.
I just happen to have bought some sand recently, and it isn't that cheap. Especially not the type of sand that you'd feel comfortable mixing into food.
Where I am, a ton of sand aggregate using in construction is ~$100 per ton. The better sand used in gardens, the brown sand (it's used to repel water so you don't over-water crops) is more than than.
Considering the wheat/ton spot price is $160 per ton at the moment, I don't think the actual prices would be too different.
Though we'd want that to be okay. The Russians might then reject saying you sent us 999 kgs of wheat and 1 kg of sand.
Also, diplomatically, this might be more difficult to defend. Although allowing this would've been in the best interest of both nations. Cue the human condition :)
Speaking of legally accepted limits, a college friend told me of his first engineering job as a chemical engineer where the factory refined sugar and graded it based on impurities sold at different prices. Sometimes a batch turned out too good so his manager would insist on pouring an amount of sand into the mix to make sure it would meet the agreed upon impurity target.
Well, kind of both. As an engineer I applaud the fact that it optimizes for maximum usage while remaining in-spec. I'd still hope they preferentially discard the really nasty stuff. A few slightly discoloured grains per kg is much better than an entire dead mouse every 100kg.
I have relatives in Australia who worked for the sugar industry.
Turns out that unrefined sugar is kept in gigantic open barns, and moved around with bulldozers --- not a clean process. Of course, birds and animals get in, and frequently die there, and then the sugar does... sugar things... to them.
As a result there is apparently a legal limit as to the maximum permissible number of crystallised parrots per tonne in Australian sugar.
It seems far more reasonable to me that increasing the true positive rate (bad grains rejected) will also increase the false positive rate (good grains rejected), so it's not the value of the bad grains included (which is almost certainly very small) but instead the value of the good grains which would accidentally be destroyed.
According to my friend, no; it could definitely do a "better" job to select the best grain. But obviously no one is interested in rejecting grain that could be legally sold.
Further, I'm guessing that there will always be some trace amounts of heavy metals, poisons and such in any grains so it's necessarily about setting a limit and optimizing the mix for the largest clean yield (in itself an interesting problem since the regulations are not for each grain but instead for all of the grain taken together).
There's a similar issue, IMO, with new home construction. There's a minimum legal quality limit for having a home be sellable, and anything past that basically doesn't get done. It's made worse by the home evaluation metrics - square footage, bathrooms/bedrooms, and location are most of what matters for getting a mortgage (and thus for bidding on and pricing a house).
Like, if you look closely at a 100 year old house, you'll find details like "the awnings over the window are just long enough to shade the window in the summer and short enough to get sun in the winter" that basically don't make it into modern homes.
I don't really think this has to do with legal limits; you could probably sell a hut as a "house" in most places if you wanted to, as long as it was compliant with the building codes. The problem isn't that the government won't let you call it that, the problem is that nobody will buy just a hut, they want an actual house.
While it may or may not be the case that 'house quality' has declined over the decades, that's driven by economic factors: do house buyers want to pay 1.2x or whatever for those extra details? I would conjecture that home developers are not stupid, and that they have tried adding those (presumably expensive) details, and found that they were unable to recoup those costs in an increased price.
>as long as it was compliant with the building codes
Huts aren't compliant with building codes - and regardless, there's specific criteria you have to fulfill to get various statistics (square footage, bedrooms, bathrooms, kitchen). Plus there are zoning restrictions on minimum building sizes, on top of regulations on minimum room sizes.
>do house buyers want to pay 1.2x or whatever for those extra details?
Even if they do, they often can't. If the bank appraises it as a $200k house, that's often going to mean that the buyer can only ever get a mortgage for $180k with $20k down. If the buyer wants to pay 1.2x, that means that they have to come up with $40k in cash.
>I would conjecture that home developers are not stupid, and that they have tried adding those (presumably expensive) details, and found that they were unable to recoup those costs in an increased price.
They aren't stupid, and they have tried adding details. What you wind up with are granite countertops, because that sort of thing impresses average home-buyers (and is an easy thing to point to for getting your appraisal adjusted).
The big issue is that American homebuyers, as a class, have lost the ability to distinguish good home design from bad. "Why" is a long essay, but the "just trust me on this" is that the vast majority of home-buyers have little to no experience in the sort of work that goes into building a home.
> The big issue is that American homebuyers, as a class, have lost the ability to distinguish good home design from bad. "Why" is a long essay, but the "just trust me on this" is that the vast majority of home-buyers have little to no experience in the sort of work that goes into building a home.
and therefore this is not a legal issue, but an economic one. (or aesthetic, or whatever you want to call it, but not legal)
My house has a number of details that would never be put in a spec house, because only someone like me would be willing to pay for them. I expect when this house is sold, it will garner a 0% premium for those features.
For example, it has GFCI breakers for all circuits, not just the bathroom ones. For another, it has a stainless steel sill plate (which keeps wood boring insects from coming up through cracks in the foundation).
that seems implausible to me, for two major reasons:
1. you're suggesting that the machine has a zero percent false positive rate, which I believe given the circumstances is physically impossible. for this type of machine, there must necessarily be some increasing function relating false positive to true positive rate. perhaps doubling true positives from (making up some numbers) 0.0001% to 0.0002% only wastes 0.01% of the available grain, but either way, I refuse to believe that increasing the true positive rate is truly "free".
2. you're effectively stating that the grain processors take out most of the bad grain, then dump it back in. given that the allowable percentages of "bad material" are (as far as I know) quite low, I don't really see why they would bother reducing the amount thrown away from, say, 0.0001% to 0.00008% to save that tiny amount of money.
The legal limits probably don't specify rejection criteria on a per-grain basis - they probably say something like "No more than x.x% trace of fungus Y in a given batch", so being able to calibrate on the fly is pretty important.
I think it probably has to be your first gig as a technical guy to not feel resignation. I mean, "good" grains and "bad" grains? They don't come labeled. It's a noisy, messy spectrum.
Because a field of wheat isn't uniform. The goal is to make all the wheat perfect, but nature foils that by spoiling some of the wheat with mold/fungi; other parts of the field may grow over soil that has a higher heavy metal content. As truckloads of this non-uniform wheat are processed, the machinery sorts the grains to get a mostly uniform end product, rejecting grains that fall outside of an acceptable threshold.
I took a tour of a company where they build machines like this last year, it's amazing. One guy was telling about a customer who bought several machines from them, then buys rejected products from big manufacturers who have less sensitive machines, and find the good product out of those rejects and sells that again. Almost literally finding needles in haystacks, and making money from it (and reducing food waste). (not sure if the guy was supposed to talk about this, so keeping it vague on what the product was)
I worked on a cranberry farm one summer where they used a cranberry grader to sort remove the fruit from the vines and separate the good from the bad. It is basically a bunch of boards at a slight angle so that any damaged or very small cranberries would bounce out, away from the good cranberries. Next the fruit rolls down a large comb like board where it is sorted by size. It is generally sold at that point and Ocean Spray would sort it by color depending on the final product. It was amazing how well such a simple and easy to repair machine worked
That's incredibly fast. The precision timing to actuate the puff of air from above is super impressive, a bit too long or too short or mistimed and a whole pile of good stuff gets rejected or bad stuff ends up in the product. Amazing video, thank you for posting that.
You feed the whole grape clusters, machine remove each individual berry from stem and then rejects all the unripe/damaged berries and little debris. All in speed of many tons per hour.
All of those are incredible machines, I wish I can buy it for my winery, but even small machines costs something like mid sized house.
The fact that the air jet can target individual bad peas out of that torrent is incredible. Computationally, probably not that difficult, but the timing required must be ridiculous.
Long ago I visited a company that made agricultural sorting equipment. They were trying to develop a new machine for sorting ears of corn. Among the requirements, the machine had to be capable of rejecting a dead rat.
My partners and I were bidding to design the machine... we didn't get the job.
https://www.youtube.com/watch?v=Lz88nsWL4kw there's also similar sorting machines for larger items (tomatoes in this case) that use mechanical paddles instead of air bursts to shoot down unwanted items.
Speaking of air bursts, here's an example of the sort of valve that gets used to actuate (with precise timing constraints) the air burst:
I just fell into a serious rabbit hole watching every video I could find on this. The speed and accuracy of these things is wild, thanks for sharing it.
There is a technology similar to this for sorting cells based on biological properties and fluorescent markers called Flow Cytometry. The machines typically allow "user defined" gates and schemes through interactive software.
Coincidentially I'm working on something similar at the moment only with an order of complexity that is several magnitudes larger than the one on display here (39000 different shapes, several 10's of possible colors). But my contraption doesn't nearly look as nice as this one and definitely is not ready for any kind of production.
I've been working on this for the last two years or so, it has just about every bit of my skills exercised (optical, mechanical, software, electronics) and every time there is a minor breakthrough I feel like throwing a party.
Likely this piece of gear will never see the light of day in a commercial setting but it's the most fun I've had in a long long time.
Disillusioned with web programming (security really spoiled the fun I used to have making web stuff) I figured I should do something that will make programming fun again and at least on that count I have succeeded.
And on another note, I've gained a lot of respect for the visual cortex and it's preprocessing capabilities.
> [...] and every time there is a minor breakthrough I feel like throwing a party
Yes!
It's become a deliberate strategy of mine to set the subgoal complexity in my personal projects such that I get a small "yes! it works!"-rush about once per week or so. It's what keeps me going on larger projects, even if occasionally some part takes longer to do.
Networking this particular device is useless so not planning on that, but yes, you're 100% right. Many producers of SCADA stuff and industrial controllers that worked just fine as long as they were isolated have found this out the hard way over the last couple of years. And I don't doubt that there are many still to come, insecure protocols, world-open ports whose only protection is that probably nobody knows what sits behind that port.
I was hoping it was Lego sorting! Every kid's dream machine. I got to talk with Nathan Sawaya at an event where he brought boxes of mixed Lego for us to play with. (including some of the giant Lego boulder from the Mythbusters episode) He said paying his assistants to sort bulk used Lego was more expensive than ordering new bricks en masse. (In the context of his sculpture projects at least)
> He said paying his assistants to sort bulk used Lego was more expensive than ordering new bricks en masse.
That's roughly the economics of it. There is a cottage industry where people will sort lego in bulk by hand, effectively it's probably better to flip burgers on a $/hr base.
No, if you multiply by colors it gets much worse. 100's of thousands of possibilities...
Of course not all parts exist in all colors so that helps (a bit), but it is quite an interesting problem to work on. Every assumption you make will be challenged.
Prints and stickers are counted separately but that's not a really huge number and they should be correctly identified (so the surface decoration matters as well in the classification).
That's a really hard problem. In many cases there are subtle hints (color, the writing on the studs).
Spectrography might give a hint here due to the different formulation of the plastics but some of the knock-offs are now so good it can be very hard to tell them from the real thing.
I'm not really sure if 'counterfeit' is the right term, the companies selling these are not making pieces labeled 'lego', and in fact the Lego brand started out by copying an English product.
It might not be true at the piece level, but some sets on some websites(ali, etc.) are exact replicas of lego sets, bar the brand. Down to the manual. Hence the use of "counterfeit".
Already part of it. It's all lego pieces, not just the bricks, basically any part lego ever produced (hence the approximately 39k shapes).
The problem is anything but trivial but I'm making good headway, proof of concept took a few months and for the last year or so I've been slowly making progress with a more robust and capable version.
It's a superficially trivial idea, any toddler could do it but to have a machine that does this with any degree of reliability is fairly complex.
If anybody is ever going to try something like this I'd give you just one piece of advice: control your inputs.
Anything you have to deal with in software can eat up capacity very fast so by conditioning your inputs the software can get simpler in ways that really matter.
Now that green skittles have changed from "lime" to "apple" (my lawsuit regarding calling the flavor assortment "original" will be filed any day now), I need a machine like this to sort the green ones into the trash where they belong.
> I started working on this machine in May ’16 and only finished it in December. It took a lot of time to design and build the machine, and I kept optimizing the parts and software after the first prototype was done. Including all prototypes and spare parts, I spent nearly €500 on this machine. Well worth it, considering everything I learned.
Fantastic job! Functional and beautiful at the same time.
This is the kind of stuff that I love to read about in the mornings.
I know this machine will not solve world hunger or bring about world peace but I know this would bring peace to my family.
I apologize in advance but I am going to "borrow" your design and work with my kids to recreate this.
Thank you for giving me something worthwhile and productive to do with my kids today. Atleast give them something inspiring and fun to look forward to.
This is really cool! I love how it lights up with the color of the candy it's sorting each time.
I'm curious, have you tried putting both Skittles and M&M's in the same batch? I'd be interested to see if it determines that the purple Skittles and the brown M&M's are the same color, for example
This is still an awesome project, but anyone else notice the machine makes a mistake? @ 1:10, in the background, you can see a purple skittle with green ones.
(Technically it we don't see it make the mistake, but it probably had made a mistake...)
I wish I had a link, but this reminds me of a machine that was at the OMSI Maker Faire in Portland last summer that measured the shininess of pennies and then directed them into a large board with columns (kind of like a giant connect-four board).
The machine sorted the pennies to match a greyscale image given as input, so that the final output is a penny mural ready to be encased in epoxy.
I've often idly thought about such a machine but this is much more beautiful than anything I would come up with, well done! I expect you'll be working on some statistical analysis on M&M color distribution now that you've got this part finished?
Also, what's the difference between processing M&Ms and Skittles? Is it just the expected colors?
You could probably sell one of these into every concert theatre in the world. I'm wracking my brain and exercising my google-fu, but I can't seem to find/remember which famous musician wanted only one color of candy on their rider. Or was it everything except the green ones. Or something like that.
It was just a test for the venue provider, which is a really clever way of sneaking in a kill-switch as well.
> So, when I would walk backstage, if I saw a brown M&M in that bowl ... well, line-check the entire production. Guaranteed you're going to arrive at a technical error. They didn't read the contract. Guaranteed you'd run into a problem. Sometimes it would threaten to just destroy the whole show. Something like, literally, life-threatening.
Indeed, I had these as a kid in the 80s. Coins are simply inserted in the top right and roll down before falling into a stack. Hours of fascination!
https://uk.pinterest.com/pin/303781937345711623/
The process looks like magic. Color-mixed items go in, and single-color items come out, on a line going so fast that no human can see what's happening. It's amazing to see computer vision systems that fast.
These machines work by putting the items on a conveyor belt, then dropping them on a much faster conveyor to spread them out. The fast conveyor goes past cameras, and at the end, launches the items into free flight for a few inches. While in flight, computer-controlled air jets knock out the rejects.
[1] https://www.youtube.com/watch?v=JyGR6A5MWG0