Hyper-parameter tuning is a core part of automatic machine learning (AutoML), which aims to automatically configure machine learning systems in deployed applications. Previously, hyper-parameter tuning is usually formulated as a black-box optimization problem, for which derivative-free optimization (DFO) solver is often employed. Such solvers often suffered from low-efficiency. Thus experienced DFO was proposed, which utilizes historical optimization process data to guide the optimization on new problems. However, the effectiveness of experienced DFO is sensitive to the relevance between the experienced tasks and the target tasks. Relevant experience can accelerate the convergence, while irrelevant experience could injure the convergence. This paper proposes an adaptation mechanism for the experienced DFO. It learns a set of experience models to guide the DFO processes, and exams these models on a few labeled samples from the target task. By comparing model predictions with the ground-truth labels, it adaptively learns the relevant experience by weighting those models. The experiments on synthetic tasks verify that the proposed method can effectively adopt the relevant experience for a range of target tasks. Furthermore, we apply the proposed method to the tasks of configuring LightGBM hyper-parameters. The empirical results show that the proposed method effectively selects the relevant experience and significantly improves the performance of hyper-parameter tuning in only a few iterations.