Railway condition monitoring using a hybrid multi-view stereopsis and real-time kinematic geolocation pipeline for dense reconstruction

Abstract

Continuous, effective condition monitoring and measurement of railway infrastructure remains a cornerstone of effective maintenance practices. Nonetheless, the potential benefits associated with the rapid advancements in technology, and its adoption in the railway sector, are undeniable - and unavoidable. The aim of this research was to develop an autonomous, non-contact, optical measurement technique, termed Spooroog, that rivals the measurement performance of comparative, state of the art instrumentation with the objective of providing quantitative condition assessments of the track superstructure, namely the vertical mid chord offset measurements and classification of the rail profile. Spooroog is derived from the Afrikaans nouns “Spoor“ (rail) and “oog” (eye). The research methodology comprises a thorough review of existing literature and state of the art research developments, numerical estimations of camera geometry for robust structure-from-motion solutions, development of a low cost, cm accurate geolocation service, an end to end, dense point cloud reconstruction pipeline, development of the Spooroog prototype and accompanying post processing software implementation (Spoorpyp; “-pyp” (pipe)) and experimental work, followed by the analysis, interpretation and discussion of the results from which the conclusions and recommendations were composed. A hybrid reconstruction technique was developed, consisting of a traditional structure-from-motion pipeline for the sparse reconstruction of the 3 camera configuration, working in conjunction with a neural network-based, multi-view stereopsis model tasked with the dense reconstruction. MVSNet was trained on custom, domain specific, synthetic datasets embedding characteristic geometric and material features associated with the railway environment. The diffuse and specular material properties do not conform to traditional Lambertian constraints employed by traditional photogrammetric pipelines. A low-cost, cm accurate geolocation service was realised, providing the requisite geolocation priors necessitated by the poorly defined, linear camera sequence, ensuring convergence of the structure-from-motion pipeline. Spoorpyp in turn transforms the acquired datasets into accurate, high-fidelity, rectified, georeferenced dense reconstructions (point clouds) for geometry extractions. The viability and efficacy of exclusively training neural networks on synthetic datasets was successfully demonstrated with the deployment of a trained image classifier on a mobile smartphone, providing real time inference performance. Image segmentation required both sample averaging and genuine data samples to be effective. Combining both synthetic and genuine datasets for depth inference, proved the most effective. The performance evaluation of the real-time kinematic geolocation service illustrated comparative to commercial solutions, achieving a horizontal and vertical precision of 23 mm and 87 mm, respectively. Similarly, a horizontal and vertical accuracy of 4 mm and 15 mm, respectively, was achieved after surveying the antenna. The qualitative and quantitative performance of Spooroog was compared against six other established measurement techniques: aerial photogrammetry, a geometry measurement system (KRAB), light detection and ranging (LiDAR; Hovermap), digital levelling, visual simultaneous localisation and mapping (vSLAM) and real-time kinematic antenna surface mapping (RTK-ASM). An alternative track quality index to the established running roughness - the running range roughness - was developed for comparing the measurement techniques. The running range roughness accuracy of the vertical mid-chord offset measurements attained by Spooroog varied between 2.6 mm and 3.9 mm, outperforming state of the art LiDAR (6.00 mm), aerial photogrammetry (35 mm) and RTK-ASM (40 mm). Dedicated, instrumented track vehicles (TLV, 0.5 mm accuracy) and geometry measurement systems (KRAB, 0.5 mm precision) still outperform Spooroog’s accuracy by a significant margin. However, the georeferenced point cloud density of Spoorpyp’s reconstruction ranked second only to vSLAM - without the requirement of any surface preparations or scanning geometry limitations - with the benefits of cm accurate georeferencing, rectification and superimposed colour information. The results substantiate the versatility and dexterity of the Spooroog prototype as an alternative, supplementary measurement technique for accurately assessing the condition of the railway environment, without relying on mechanical contact with the rail profile.