So simple and yet highly complex: Grasping from the box

Clearing out boxes – no problem! It gets difficult when robots take over this task instead of humans. Where we can quickly find our way around and intuitively reach for the right thing, a machine has to learn by control what needs to be done.

Our list of 14 unexpectedly complex requirements for bin picking automation proves that this is quite a lot.

1.  Chaos and disorder in the bin – Where should the robot reach? Where are the objects located in the first place?
2. The problem with edges: Objects perfectly separated from each other in the crate are unlikely to make it out of the delivery to the factory. Canting, overlapping and partial concealment are standard. Automated machine vision, however, no longer recognizes the predefined shapes – because they cannot be seen. This affects 2D cameras in particular, which lack the depth of the image.
3. If objects are stubbornly tilted, a human can work with both hands and solve the task logically. In robotics, this case currently still involves the detour of dumping the box onto, for example, an assembly line.
4. Everything is upside down? This is not uncommon in unsorted crates. If there is no uniform orientation, the gripper may grab the wrong part and the part is sent to further production in the wrong direction.
5. Move there, grab, put down. Unfortunately, it is not so easy to determine the gripping points. Finding the best and safest gripping point for a piston, for example, while taking the robot kinematics into account, makes this process much more difficult.
6. Have you considered the ergonomics and gripping properties of weighty objects? Just as we literally struggle with this, so does an incorrectly configured robot when reaching out of a crate.
7. Speaking of boxes: What kind of container is it anyway? Large or small? Fluted, corrugated, latticed? Is it even easy to see at the bottom? Are there unpredictable dents and deformations? We unconsciously adapt to these new conditions, but a machine cannot.
8. Let’s not build any accidents! Collision avoidance is particularly important in bin picking, because industrial robots have incomparably greater forces than humans. The robot arm should therefore not drive into walls, crates or other safety areas without braking. For use as a cobot, i.e. without a confining robot cell, the complexity of collision avoidance increases many times over.
9. How the robot arm can move also determines path planning. Safety areas, collisions with other workpieces or walls must be included for this. The closest route is not always the best one.
10. This in turn leads directly to the issue of time. At what speed does the overall process run? Which subsections are currently particularly time-consuming and can be shortened? Different image acquisition and processing methods differ significantly in this respect.
11. To shorten these times, it also depends on the lighting conditions of the shooting situation. Changing light should be avoided. Today, additional lighting is standard to sufficiently illuminate dim production environments, as well as to reduce noise and artifacts during shooting.
12. The simpler the camera, the more important the right camera parameters. They have to be set according to the task – so that the depth of field is right, for example.
13. A manufacturer who supplies you with all the components of a bin picking cell from his own production? Better not! Because each sub-step of the task requires maximum competence and precision. It is more helpful to pay attention to good integrability of the selected components. An integrator can provide you with competent support in this regard.
14. Finally, the last challenge: Is the system used reliable and does it answer all the questions already mentioned? Depending on the gripping task, some points have more or less weight. You should also consider this when reaching into the box.