Automotive Damage Detection
Reducing engineer review time by creating an AI system to detect and identify damage on used cars.
The Opportunity
Our client assesses thousands of used cars at garages across the country to estimate market price. Currently, engineers review photos from camera rigs to manually identify any damage to the vehicles. We were asked to create an AI system that can identify damage on cars from existing images without human intervention.
What we did
✔︎ Ingestion and analysis of image data to create a validation set to evaluate system performance ✔︎ Implementation of a multimodal Large Language Model (LLM) pipeline to ingest images ✔︎ Prompt engineering and experimentation to optimise the LLM pipeline and maximise detection accuracy
The Results
Our AI system achieved 87% accuracy in detecting and identifying car damage using 20 images per vehicle at an estimated 7p per vehicle. This significantly reduced engineer review time, enabling simultaneous image processing. Additionally, we provided recommendations to enhance camera hardware and technical stack for improved performance and production readiness.
87%
Accuracy achieved using existing data
How we did it
Despite a large corpus of existing images, these were not labelled and required us to construct a validation set to evaluate the performance of any system we developed. The lack of existing labelled data lent itself to the use of a state-of-the-art Large Language Model (LLM), in this case GPT-4o, which was accessed via Microsoft Azure. We created an image ingestion pipeline feeding multiple images of the same car, at different angles, to the LLM. Through experimentation with the model version, image selection and prompt we were able to create an LLM classification system that could not only detect damage, but also state where on the car the damage occurred. In addition to providing the results and code to the client, we offered detailed recommendations to enhance the camera hardware and technical stack, further improving performance and moving towards production.
