This study systematically analyzed the effects of Fe³⁺ ion doping and annealing treatment on the properties of ammonium dihydrogen phosphate (ADP) crystals. X-ray photoelectron spectroscopy (XPS), inductively coupled plasma mass spectrometry (ICP-MS), and energy-dispersive X-ray (EDX) spectroscopy confirmed the concentrations of Fe³⁺ ions in the pyramid (PY) and prism (PR) face, with doping levels ranging from 10 ppm to 30 ppm. Raman spectroscopy analysis indicated that the ADP crystal's fingerprint peak at 924 cm⁻¹ shifted to a lower wavenumber as the Fe³⁺ doping concentration increased from 10 ppm to 30 ppm, illustrating lattice distortions caused by doping. Fourier-transform infrared (FTIR) spectroscopy further confirmed these changes, with the crystal fingerprint peak at 967 cm⁻¹ shifting in two opposite directions due to doping with Fe³⁺ and annealing. The results from positron annihilation lifetime spectroscopy (PALS) indicated that the τ₁ values for the annealed samples decreased from 0.135–0.153 ns to 0.123–0.147 ns, suggesting that annealing reduced the concentration of small-sized defects. Meanwhile, τ₂ values for the untreated samples decreased from 0.635–0.671 ns to 0.589–0.696 ns, indicating that annealing reduced the number of large-sized defects. Ultraviolet-visible–near-infrared spectroscopy analysis showed that in the wavelength range of 240–800 nm, the average reflectance of the untreated samples was 15–30% higher than that of the annealed ADP crystals, with the reflectance of the latter converging and the variation reducing to below 5% after annealing. These findings suggest that by controlling the growth concentration of Fe³⁺ ions and optimizing the annealing process, the internal defects within the crystals can be effectively controlled, thereby improving their optical uniformity and mechanical properties, which in turn enhances the performance in nonlinear optical applications.