AISA (AI based Situational Awareness) is a Business Finland funded co-innovation project belonging to a Nokia-led Veturi program. The technical focus of AISA is on artificial intelligence and sensing technologies, especially video, image and audio processing.
The AISA-related research and development at Top Data Science focuses on solving the key problems related to scalable deployment of industrial computer vision applications.
Although several generic computer vision platforms are available in the market and vendors claim universal solutions for industrial needs, the reality is mostly different. The extensive experience of Top Data Science from industrial computer vision applications indicates that achieving the desired levels of performance in actual production environments often requires careful selection and fine tuning of a combination of constantly developing leading edge AI technologies.
In many cases the right computer vision application consists of a modular pipeline of several AI models, each trained to solve a specific part of the problem. Industrial processes keep evolving and the models may need to be maintained to accommodate for the changes such as new products or changing production quality criteria.
Industrial computer vision applications also need to be integrated into diverse IT landscapes, often implemented in hybrid (on-premise, edge, cloud) platforms. The environments and pre-existing systems may dictate the input data availability as well as camera selection. Furthermore, any incidents or anomalies detected by a computer vision -based quality control application typically need to be stored for analytics, used to trigger specific corrective actions and/or communicated to right stakeholders. Besides core computer vision knowledge, this requires competencies in application integration, deployment and scaling.
In AISA, Top Data Science has made significant progress in our capabilities for flexible, scalable and cost-efficient development, deployment and scaling of customised industrial computer vision applications. Some of the related technical aspects are:
- Camera integration: applications must be easily deployable with a standard inexpensive industrial camera as well as able to efficiently utilise standard image and video stream formats from cameras already installed at production lines.
- Deployment: applications often need to run on inexpensive edge hardware while maintaining near real-time performance for the computer vision tasks. On the other hand, applications have also to be easily deployable to all popular cloud environments.
- Model updates: the AI model pipeline needs to be maintainable and support easy model updates.
- Flexible actions: especially in quality control applications it is often necessary to quickly trigger corrective actions, therefore an efficient and scalable way of defining and implementing triggers and communication events is needed.
- Integration: standard interfaces such as simple REST APIs and OPC UA must be available.
The capabilities are embodied in a software framework supporting the development and deployment of modular computer vision pipelines including high performance video processing. It also provides building blocks for business logic, event storage, event-based communication and outcome visualisation. The framework-based applications are containerized for easy deployment in the cloud or on edge devices.
Demonstrating quality assurance with Computer Vision
The recent AISA work at Top Data Science has concentrated on the Computer Vision Application Framework. The starting point of the development work was a thorough analysis of a number of industrial use cases implemented for our customers. The results were demonstrated as a live real-time quality assurance application at Hannover Messe 2024 in April.
The full anomaly detection functionality seem in the video clip is composed of four separate inference steps:
- Perspective correction: the corners of the field of view are detected to compensate for perspective distortion. This feature is useful in various real-life settings, as it enables flexible camera positioning in relation to the objects of quality control. Perspective correction is implemented by placing apriltags in the corners of the detention area.
- Segmentation: the individual quality controlled objects are detected and accurate object masks are created in the video stream. Detection and segmentation are implemented with RTMDet. Besides its performance, one of the practical advantages of RTMDet over other popular models is its relatively unrestricted open source licensing model.
- Diameter measurements seen in the output stream are generated using the outputs from the previous steps (corrected perspective with accurate masks).
- Anomaly detection is performed for each detected object. It uses the unsupervised FastFlow model. The lack of need for supervision means that the application does not need to know the types of anomalies before they occur, which is a great practical benefit.
Running on the edge
As seen in the video clip, the full demo is running on a relatively inexpensive Jetson Orin NX computer with a small form factor. The application includes GStreamer-based video signal processing together with the CV model pipeline and is containerized for simple portability. It provides an annotated, H.265-encoded RTSP stream as the main output.
The video stream is captured by a basic industrial camera from Basler providing a 60 fps frame rate with 5MP resolution.
Next
AISA project will continue until the end of 2024. For the remaining part of the project, Top Data Science will concentrate on further generalising the framework-based technology foundation based on our constantly accumulating experience from customer projects. This will further improve our implementation productivity and flexibility in future solution deployments and project deliveries.
Author:
Otto Pulkkinen, Head of Industry Solutions
