Our new Feedback Loop feature enables continuous improvement of AI-based anomaly detection through targeted user annotations and intelligent prioritization of uncertain cases. In this way, we combine human expertise with data-driven optimization and ensure stable, reproducible inspection results during ongoing operations. In the following article, we show how this feature sustainably improves our customers’ quality control and systematically reduces incorrect decisions.

Image 1: User interface of the Feedback Loop

+++ Create your first AI model for quality inspection within 5 minutes – free and directly in your browser with preML Vision Lab! +++

With all complex systems, we know the situation: once they are up and running, it takes time and money to engage with them on a regular basis. In quality control, however, this is exactly what ensures auditability and continuous process improvement. That is why we encourage our customers to actively use the systems, understand them, and work with them in a targeted way.

Our new Feedback Loop feature was developed to intensify interaction with our inspection systems. It enables systematic monitoring of model decisions with the help of human expertise, early detection of misclassifications, and long-term safeguarding of training and test data quality.

The tool can be used with all preML systems that are currently based on anomaly detection. Anyone who would like to try it can create a free account for the web browser version of Vision Lab.

The idea behind the feature is to have conspicuous images from the inspection system reviewed by a human in order to retrain and thereby improve an existing model through targeted intervention. When the human reviewer, for example the quality manager, selects the “Feedback Loop” function, images are displayed one by one for independent evaluation. After reviewing all images and labeling them as “Normal” or “Anomaly,” the reviewer can actively trigger model retraining. The new evaluations are then incorporated into the training process, leading to long-term improvements in model performance.

Image 2: Flow diagram

There are various reasons why images from an inspection system may be selected for the Vision Lab Feedback Loop.

1. Uncertainty

For anomaly detection models, uncertainty is defined as the difference between the anomaly score and the optimal threshold. Images with a small difference—indicating high model uncertainty—are particularly relevant and are therefore prioritized in the Feedback Loop.

2. Conflict with Human Annotation

Images for which the model prediction contradicts the existing label are guaranteed to be included in the Feedback Loop. This ensures that the model never remains in conflict with a human evaluation.

3. Random Selection

In addition, a small random subset of images is included in the review process to ensure broader coverage of the dataset.

For all three criteria, it should be noted that only images for which a current evaluation by the deployed model is available can be included in the Feedback Loop.

More technical details about the preselection process can, of course, be found in the Vision Lab documentation.

Video 1: Process of Human Review of the Inspection System Using the Vision Lab Feedback Loop

With the Feedback Loop, a semi-automated process is created that checks the plausibility of data on a sampling basis. This approach, often referred to as “Human in the Loop” or “Active Learning”, represents, in our view, the most pragmatic solution for enabling continuous improvement of anomaly detection while ensuring traceability and quality control at the same time. The Feedback Loop thus forms the central interface between live inspection and human expertise combined with data-driven optimization.

For us, implementing one of our customers’ most requested features was an important milestone. Stay tuned for further developments and exciting insights into the world of AI-driven machine vision systems.

Cheers!

David (Fehrenbach) and Paul (Hartwich)

Some developments in the area of the Feedback Loop at preML GmbH were funded within the SMILE4KMU project as part of the German Federal Ministry of Education and Research (BMBF) funding initiative “KMU-innovativ.”