Sorting by color is useless unless you want to go and sort directly into sets (it can do that too but that's very much experimental, also it would take lots of bins and there are some details that are important to get right that are almost invisible to the camera).
A full speed video is super hard to shoot because you can't follow the parts as they move, they basically disappear because of the air puff being so short that a part is there one frame and gone the next. Right now classification takes about 30 ms, that's the limiting factor because that belt keeps moving during that 30 ms so you need to be 'back' at the camera before it moves so far that you can't stitch the next image to the previous one.
Another limiting factor is the relationship between the two belts, the second belt can only go so fast before the precision of the puffers starts to be insufficient to aim the parts into the right bin, they also carry too much forward momentum, and the second belt needs to go many times faster than the first in order to spread out the parts sufficiently. Yet another problem related to that: if you look at the video you'll notice that one of the little parts grabbers on the belt can push ahead of it quite a bit of stuff, if fortune is against you all of that lands on the belt in one go. By making that belt go slow it creates just enough pause between parts to be able to separate them with air without pushing the wrong part off the belt. It pays off to leave some safety margin there so I tend to set the second belt a bit faster than optimum and the first one a bit slower. That way the accuracy goes up quite a bit.
It took lots of experimenting and tweaking to get to this stage.
About 1 part per second is a practical upper limit right now, it can go way faster than that but then it starts dumping stuff all over the room :)
I'll see if I can shoot a video at a higher speed than the one in the post right now.
>also it would take lots of bins and there are some details that are important to get right that are almost invisible to the camera).
I wonder if you could build something that would use a small number of bins (maybe just two) and multiple passes.
The first pass could just get an inventory and just show you what sets it could build toward (with emphasis on sets that are the most profitable, closest to completion, or your favorites). The second pass would split off legos that are a part of the set(s) that are trying to be built.
You could also pass along what you think was a completed set again at the end to check for mistakes.
The number of bins is of course directly related to the number of passes. 7 is the largest that I can still fit on my desk so that's why it is 7 right now, I'd love to take it up to 13 that would make things much faster. Two would be so slow that it probably would be quicker to sort by hand because of the number of passes you'd have to make.
Thanks for writing this up and adding all these detailed notes! Super awesome work on this, it's very cool.
The air puffs seem like the most unreliable parts at the moment?
Maybe if you could have some manual labour, one thing to do would be to mount a project above the setup to shine a graphic you generate onto it.
If you have bins labeled, you could project the label next to or on top of the part as it's moving on the conveyor. That way if you have human pickers, you're not constrained by the number of bins with each needing its own air-puff solenoid.
> The air puffs seem like the most unreliable parts at the moment?
Yes, absolutely. There are ways to improve on that but nothing simple.
> Maybe if you could have some manual labour, one thing to do would be to mount a project above the setup to shine a graphic you generate onto it.
I don't follow you, what do you mean?
> If you have bins labeled, you could project the label next to or on top of the part as it's moving on the conveyor. That way if you have human pickers, you're not constrained by the number of bins with each needing its own air-puff solenoid.
Ah I see, no, the airpuffs are reliable enough for now even if they are the most unreliable part, besides humans would not keep up with the machine at speed. But it is a solvable problem, just not the most interesting one at the moment. The most interesting one is the hopper feeder belt because if I can make that work just a little bit better it will speed things up by a factor of 4 to 10 depending on how well it will work.
A full speed video is super hard to shoot because you can't follow the parts as they move, they basically disappear because of the air puff being so short that a part is there one frame and gone the next. Right now classification takes about 30 ms, that's the limiting factor because that belt keeps moving during that 30 ms so you need to be 'back' at the camera before it moves so far that you can't stitch the next image to the previous one.
Another limiting factor is the relationship between the two belts, the second belt can only go so fast before the precision of the puffers starts to be insufficient to aim the parts into the right bin, they also carry too much forward momentum, and the second belt needs to go many times faster than the first in order to spread out the parts sufficiently. Yet another problem related to that: if you look at the video you'll notice that one of the little parts grabbers on the belt can push ahead of it quite a bit of stuff, if fortune is against you all of that lands on the belt in one go. By making that belt go slow it creates just enough pause between parts to be able to separate them with air without pushing the wrong part off the belt. It pays off to leave some safety margin there so I tend to set the second belt a bit faster than optimum and the first one a bit slower. That way the accuracy goes up quite a bit.
It took lots of experimenting and tweaking to get to this stage.
About 1 part per second is a practical upper limit right now, it can go way faster than that but then it starts dumping stuff all over the room :)
I'll see if I can shoot a video at a higher speed than the one in the post right now.