Warehouse Damage Detection
Deploying AI technology to detect storage abnormalities, ensuring safer warehouses with 90% on-device LLM accuracy.
The Opportunity
Our client provides automation tools to large warehouse providers around the world. Many of their clients require significant manual labour to check shelving and stock to ensure it is safe, secure and accurately inventoried. Using their existing imaging equipment, they wanted an on-device solution to detect quality and safety risks.
What we did
✔︎ Deployed a range of open-source, multimodal Large Language Models (LLMs) on an Nvidia Jetson GPU. ✔︎ Created a pipeline to evaluate the performance of different LLMs across a range of use-cases. ✔︎ Developed a UI to display the performance of the system across the models and use-cases.
90%
True positive rate - most issues were detected
How we did it
To host the on-device, open-source LLMs, we used an Nvidia Jetson - a rugged GPU designed to run at lower power (60W) than consumer GPUs, so perfect for robotics applications. We designed a pipeline to quickly switch between models to evaluate their performance on a range of image classification tasks. We also developed a UI to display and compare the results. Using these LLMs, we experimented with a range of warehouse specific use-cases, including detecting damage to shelving and boxes, identifying hazardous content labels on merchandise and detecting all risk. We achieved up to 90% accuracy on-device, and in some use-cases were able to match the accuracy of state-of-the-art, cloud-based LLMs, such as GPT 4o.
