Gaussian kernel support vector machine recursive feature elimination (GKSVM-RFE) is a method for feature ranking in a nonlinear way. However, GKSVM-RFE suffers from the issue of high computational complexity, which hinders its applications. This paper investigates the issue of computational complexity in GKSVM-RFE, and proposes two fast versions for GKSVM-RFE, called fast GKSVM-RFE (FGKSVM-RFE), to speed up the procedure of recursive feature elimination in GKSVM-RFE. For this purpose, we design two kinds of ranking scores based on the first-order and second-order approximate schemes by introducing approximate Gaussian kernels. In iterations, FGKSVM-RFE fast calculates approximate ranking scores according to approximate schemes and ranks features based on approximate ranking scores. Experimental results reveal that our proposed methods can faster perform feature ranking than GKSVM-RFE and have compared performance to GKSVM-RFE.