The famous bin pick: What is no problem for us, poses great challenges for machines: detecting, locating, classifying, grasping, depositing– and proceeding fast enough. Is there a bin picking system that meets all of these requirements and thus enables the perfect grip into the box?
Core components of a bin picking system
Bin picking describes the process of grabbing parts from a crate or bin and then placing them in the desired position. A bin picking System consists of:
- an evaluation unit, e.g. an industrial PC,
- a sensor camera,
- and a robot.
The evaluation unit acts as the brain, the sensors as the eyes of the system. First, the camera collects information about the workplace. In doing so, it localizes the target object. The industrial PC then receives the necessary data for evaluation. This includes the localized parts and the working environment. The PC plans the robot’s movement on the basis of the possible gripping points defined in advance. In the process, the PC calculates the risk of collision with other objects, the edge of the crate and objects that cannot be grasped. Based on this, the PC selects possible gripping points and a collision-free path. The PC forwards the information to the robot controller or an intermediate PLC, whereupon the robot executes the movement and grips the parts. The gripper either deposits the part for further processing or performs a quality inspection.
Requirements for bin picking Systems
Where is bin picking used?
Functioning bin picking systems relieve workers of monotonous, physically demanding tasks. The systems are used in industrial production or logistics, among other areas. This automation frees up additional resources – employees can be deployed in other areas instead. In addition, bin picking is time efficient and reduces costs within a two-year period.
Positioning of parts
In bin picking, experts differentiate between structured, semi-structured and random bin picking. In structured bin picking, parts are located in predictable patterns, e.g. in sorted stacks. As a result, the system recognizes them reliably and picks them without problems. Image recognition works best with geometrically symmetrical parts. In semi-structured bin picking, the parts are in a rough order. This certain order facilitates the recognition and ability to grab the parts. Random bin picking, on the other hand, describes working with parts in completely random positions. The parts overlap, are placed in different angles, and may even be wedged into each other. This makes recognition more difficult. In addition, the collision problem of the gripper with the edge of the box or other parts poses a major challenge in random bin picking.
Robot and gripper
Choosing the right robot is important for the success of the bin picking process. Individual robots have advantages and disadvantages. Based on the shape and weight of the parts, the appropriate gripper attachment and payload are selected. The more degrees of freedom or axes a robot has, the better it can grab workpieces that are difficult to reach. However, the cycle time also increases with the number of degrees of freedom. In random bin picking, the robot is sometimes unable to empty a bin, if parts are in a corner or wedged together despite selecting the right robot and gripper.
Effect of lighting
Lighting is important for part recognition. The appropriate brightness depends on the surface texture and the contrast between parts and background. The light should not be too bright to avoid reflections. The lighting conditions should also remain the same throughout the process. Depending on the situation, polarizing light or area lighting is therefore used.
Bin picking Solutions
2D or 3D vision?
There is a difference between 2D and 3D vision systems. A 2D sensor does not recognize the height or the exact shape of an object. Changing backgrounds or lighting conditions therefore affect the accuracy of the system. Reflecting, moving or dark parts are less likely to be recognized by the system. 2D vision systems work well with clear contrasts between parts and the environment. Thus, they are often used in structured bin picking. Due to the less complex algorithm, these systems respond faster. They are also less expensive.
A 3D vision system detects the depth of the Z-axis in addition to the X- and Y-axis and thus has a three-dimensional view of the scene. Due to the extended viewing angle, the system can blank out the background. This makes it possible to delineate parts that lie on top of each other. New parts are entered into the software once through CAD models and are then easily recognized. Thanks to their own light source and regulation, 3D systems are less dependent on external light. Compared to 2D vision systems, they offer a more reliable solution even for complex parts, arrangements or different backgrounds. They are also used for semi-structured and random bin picking. Therefore, 3D bin picking solutions are widely used today.
What is the ideal camera system?
Furthermore, a distinction is made between laser, light field, stereo and time-of-flight cameras. The light field systems often offer the best data quality. The light field approach often consists of a lens or multi-camera array. Thus, it is able to detect shiny surfaces. They process information reliably and the commissioning is faster. Because of the high quality of the data, this approach also works for metallic and shiny surfaces.
Stereo systems work on the basis of structured illumination: the system projects patterns onto a scene, thereby generating a 3D image. Stereo cameras capture clear images – but only with the right lighting conditions. Poor lighting or reflective objects can produce distorted results. They can only be used for distances of up to two meters. Very short cycle times can be achieved with stereo cameras.
Laser systems, on the other hand, achieve higher accuracy. However, laser systems are often scanning systems, which is why it is not possible to detect moving objects. Laser systems also have a danger zone that must be taken into account in safety regulations. Nevertheless, they offer a low-cost entry into vision technology.
Time-of-flight sensors use infrared light to determine the distance between the target object and the sensor. Therefore, they provide very accurate results even in low light conditions. Time of Flight sensors are also dependent on the right lighting conditions – reflective objects therefore pose a problem. While they record very accurate data, the potential range of the application is small.
All camera systems offer both advantages and disadvantages in terms of cycle time or depth accuracy. Advanced technologies such as 3D analysis and 3D image processing are needed to overcome demanding picking challenges.
Outlook
It is for a reason that experts refer to bin picking as the supreme discipline of machine vision. Numerous degrees of freedom, challenging workpieces and specific setups challenge today’s systems. Although new technologies for image acquisition and processing are continuously pushing the boundaries of what is possible, there is still no all-round talent in bin picking. However, with AI-based bin picking, there are already promising approaches in research and initial applications. In the future, this could make it possible to recognize unknown and different objects, automatically determine the point of grip, and reliably pick up objects that are difficult to recognize with just a single system.
