Industry 4.0, Industrial IoT or Smart Factories. No matter which term you use, all of them refer to the same goal: increasing efficiency, reliability, security and safety on the factory shop floor. At NXP, we do not only design and produce innovative chips that are at the core of the next generation of Industry 4.0 applications, we also integrate Industry 4.0 concepts to increase the efficiency of our factories even further. In this blog, we will show you how NXP is leveraging its own technology in conjunction with Basler’s advanced embedded vision technology to create a new visual inspection concept for our factories.
NXP has silicon wafer factories around the world. Within each facility, quality and efficiency are at a very high level. However, we always seek ways to exceed the requirements of the already demanding semiconductor market. A typical silicon wafer manufacturing process ends in a delicate packaging process of the wafers in a transport box before they leave the factory, as Figure 1 illustrates.
Figure 1 – Last stage in silicon wafer manufacturing
Once the transport box is closed, before wrapping, an operator performs a visual inspection. Amongst others, the inspection intends to detect any incorrect positioning of the wafers within the box. This check is crucial in order to avoid issues downstream in the supply if wafers are shipped in the wrong position.
The error rate in the boxing process is very low. As a consequence, an operator rarely encounters an error. With such low hit rates, natural human behavior leads to a higher chance that the rare error is not spotted by the manual inspection. As part of our continuous improvement activities, the wafer factory is testing a new concept for automated visual inspection. This automation adds value in terms of efficiency. Also, it will provide evidence of the status of the transport boxes before they leave the factory.
To make this initiative a reality, we joined forces with our official partner Basler AG, a leading global manufacturer of computer vision applications. Building on the close collaboration in the development of vision-based edge devices, we were able to take advantage of the strong synergy between our companies. Widely recognized for their extensive expertise on factory automation and embedded vision solutions, Basler was capable of building the complete vision application at hand. In addition to delivering a cost-efficient and powerful vision system, Basler also provides the complete application software to perfectly complement our use case. The powerful industrial-qualified i.MX 8 Family of applications processors from NXP ensures the high performance needed to efficiently run the algorithms in real time.
For this specific use case, we integrated a Basler Add-on Camera Kit daA2500-60mci-IMX8-EVK, which comes with a 5 Megapixel dart camera module, cable, lens and a BCON for MIPI to Mini SAS adapter, supporting up to 750 MB/s of data transfer. Additionally, Basler provided optimized drivers specifically designed for easy integration with various i.MX 8M evaluation boards.
As processing platform, we chose the NXP i.MX 8M Mini evaluation board 8MMINILPD4-EVK, exhibiting an industrial qualified i.MX 8M Mini Quad applications processor with 4x Arm® Cortex®-A53 @ 1.6 GHz as main CPU platform and 1x Arm Cortex-M4 @ 400 MHz as low power/security CPU, making this board an excellent candidate for efficiently computing image processing routines. Besides, the i.MX 8M Mini evaluation board provides the right peripherals to meet the requirements of this proof of concept. Not only does it support a MIPI-CSI Camera mini-SAS connector; it also comes with a 1Gbit Ethernet port allowing direct communication with the factory’s manufacturing execution system (MES). It is a power efficient board and exposes enough digital IO signals capable of interacting with other devices on the shop floor.
Figure 2 Basler Add-on Camera (daA2500-60mci-IMX8-EVK) with i.MX 8M Mini Evaluation board (8MMINILPD4-EVK)
The i.MX 8M Mini evaluation board, together with the Basler Add-On Camera Kit, was installed at the station where the visual inspection takes place. The applications processor was already running an image processing algorithm tuned by Basler for the specific illumination conditions expected in the clean room. The algorithm uses “classic” image processing methods, such as segmentation and blob analysis. To communicate with the MES backend via Ethernet, the application sends/receives JSON messages to/from an MQTT broker.
To start the automatic visual inspection, the transport box with the wafers is positioned on top of an industrial-grade backlight platform, as Figure 3 shows. The Basler camera is located approximately one meter above the transport box and is connected to the i.MX 8M Mini board. The applications processor starts the checking process once it receives a message from the MES backend’s MQTT broker, or when the operator manually activates a digital input connected to the evaluation board.
Figure 3 – Wafer box finished – Proof of Concept
Once the image is captured and pre-processed by the Basler camera, the actual image processing algorithm searches the box in the image and aligns it for the subsequent steps (just in case the box is tilted).
Figure 4 Wafer box centered and aligned with regards to subsequent processing steps, even if box was placed tilted on the backlight
In the following processing steps, the algorithm looks for empty slots as well as for slanted wafer disks or disks not fully inserted into their slots. Additionally, the disks in the box are counted. As a result of this image processing routine, the i.MX 8M Mini sends an MQTT message to MES with the conclusion of the inspection as follows:
Figure 5 – Automatic identification of position errors
Finally, the 16-bit images of the analyzed wafer are sent to the MES and stored for the traceability of the factory’s processes.
The use of embedded vision technology in the field of factory automation applications is a rising trend as the benefits are obvious. Due to the form factor that continuously grows smaller, power consumption getting lower and growing processing performance, it becomes much easier to substitute big PC-based vision systems with embedded technology—ensuring a lean and cost-efficient solution.
We are closely collaborating with our partners to bring innovative solutions of machine vision for factory automation. This automated visual inspection system is only one example of what the strong synergy of NXP and Basler technology can solve in the area of factory automation. We encourage you to visit our webpage and learn more about our portfolio at:
Felipe Fernández is a business development specialist in NXP’s Industrial System Innovation team based in Hamburg, Germany. He has 10 years of experience in the areas of cybersecurity and factory automation, having worked in different regions such as Latin America and Europe. He holds Masters and Bachelors degrees in Mechatronics Engineering and an MBA from the Northern Institute of Technology Management. Find him on LinkedIn at www.linkedin.com/in/jfelipefg.
