One-loop string scattering amplitudes computed using the standard D0-brane conformal field theory (CFT) suffer from infrared divergences associated with recoil. A systematic framework to take recoil into account is the worldline formalism, where fixed boundary conditions are replaced by dynamical D0-brane worldlines. We show that, in the worldline formalism, the divergences that plague the CFT are automatically cancelled in a non-trivial way. The amplitudes derived in the worldline formalism can be reproduced by deforming the CFT with a specific ``recoil operator'', which is bilocal and different from the ones previously suggested in the literature.