This letter proposes a novel post-processing method for self-similarity based super-resolution (SR). Existing back-projection (BP) methods enhance SR images by refining the reconstructed coarse high-frequency (HF) information. However, it causes artifacts due to interpolation and excessively smoothes small HF signals, particularly in texture regions. Motivated by these observations, we propose a novel post-processing method referred to as middle-frequency (MF) based refinement. The proposed method refines the reconstructed HF information in the MF domain rather than in the spatial domain, as in BP. In addition, it does not require an internal interpolation process, so it is free from the side-effects of interpolation. Experimental results show that the proposed algorithm provides superior performance in terms of both the quantity of reproduced HF information and the visual quality.