Learning pavement surface condition ratings through visual cues using a deep learning classification approach

Pavement surface condition rating is an essential part of road infrastructure maintenance and asset management, and it is performed manually by the data analyst. The manual rating requires cognitive skills built through training and experience, which is quantitatively challenging and time-consuming. This paper first analyses the complexity of the current manual visual rating system. This paper then investigates the suitability and robustness of a state-of-the-art convolutional neural network (CNN) classifier to automate the pavement surface condition index (PSCI) system used to rate pavement surfaces in Ireland. The dataset contains 3735 images of flexible asphalt pavements from Irish urban and rural environments taken from a video camera mounted in front of a van. The PSCI ratings were applied by experts using a scale of 1-10 to indicate surface conditions. The classification models are evaluated for different input pre-processing variations, image size, learning techniques, and the number of classes. Using 10 PSCI classes, the best classifier achieved a precision of 57% and a recall of 58%. Adjacent combination of classes (e.g., ratings 1 and 2 combined into a single class) to form a 5-class problem produced a classifier with a precision of 70% and recall of 77%. Given the complexity of the problem, classification using CNN holds promise as a first step towards an automated ranking system.