Artificial intelligence has been widely used in reliability analysis for industrial equipment. The gear transmission systems are the most common components in mining machines. A simple fault in the gearbox may break down the mining machine for couple of days, resulting in enormous economic loss. Condition monitoring techniques can prevent unscheduled failures in the gear transmission systems. Although many techniques have been developed for gearbox fault diagnosis, one challenging task that the condition monitoring still faces is how to extract quantitative fault indicators. To this end, this paper proposes an improved quantitative recurrence analysis (IQRA) based on artificial intelligence theory. This new method takes advantages of chaos and fractal properties of the gear transmission system to obtain the recurrence of the system. The characteristics of different gear faults can be observed through the visualization of recurrence. Quantitative parameters can be then calculated from the recurrence plots. Experimental data acquired from a gearbox under variable working conditions was used to evaluate the proposed method. The analysis results demonstrate that the proposed IQRA method is able to effectively quantify different the gear faults.