Virtual-reality blast scene modelling and reconstruction

Abstract

Virtual Reality (VR) allows the visualisation and exploration of detailed 3D environments. Recent advances in technology have made the use of VR more accessible for research and industry applications. VR is especially applicable to the mining industry, as hazardous environments or events can be experienced in a safe and intuitive form. One such application is the reconstruction and simulation of mining blasts. Scene modelling and reconstruction within the context of VR is not far removed from standard 3D scene modelling, as VR is typically merely an extension of standard 3D rendering. The areas of 3D reconstruction, modelling and data processing are large and active fields of research with extensive applications, however, many methods are not typically applied to natural, monochromatic and highly-unstructured data such as a mining blast, and as such the exact limitations and applicability of standard 3D modelling techniques are unknown. The research is based on multi-perspective, overlapping image data sequences. The 3D reconstruction of both large scale static environments and temporal mining blasts was achieved by means of the Pix4DMapper [1] photogrammetry suite. The optimisation of reconstruction parameters is dataset dependent. Significant limitations were encountered with 3D dataset preparation due to the non-metric 3D reconstructions obtained as a result of a lack of measured external camera parameters for the research image datasets. Camera calibration biasing, inverse external parameter transforms and manual alignment was used to prepare the 3D dataset for VR visualisation in lieu of these limitations. A custom asset management pipeline was implemented to allow large, complex 3D datasets to be imported into the Unity 3D game engine at runtime. A VR environment and blast visualisation application, with intuitive environment exploration mechanics, was developed in Unity. The use of shader-based level-of-detail (LOD) scaling, custom vertex buffer objects (VBOs) and cube map rendering ensures adequate VR performance on modern high-end hardware. The extraction of blast parameters from a combination of 2D image data and reconstructed 3D models was investigated. The possible applications of blast parametrization include special effects, projectile trajectory estimation, predictive models and safety planning. Blast parametrization was hindered by data anomalies encountered due to the setup and methodology used to capture the research dataset. A methodology was proposed for the capturing of blast datasets that addresses the limitations encountered. Methods were developed to enable reconstruction scale relative parameter extraction, as well as single-perspective 2D to 3D data association within the context of photogrammetry. A rudimentary dust cloud particle simulation was implemented as a demonstration of the possible applications of blast parametrization. The use of blast parametrization and special effects for blast visualisation augmentation was found to be highly dataset dependent and impractical without improved dataset availability. The research dataset limitations and the use of dataset dependent methods allows the VR blast modelling and reconstruction pipeline to be only partially automated.