The contraction in atomic radii along the lanthanoid series ($\mathrm{La}$ to $\mathrm{Lu}$) is called lanthanoid contraction. It arises mainly because:

The 4f orbitals are diffuse and penetrate poorly to the nucleus, so they shield outer (5d, 6s) electrons very inefficiently from the increasing nuclear charge across the lanthanoid series. The net result is a steady, small contraction in atomic and ionic radii from La to Lu. A consequence is that the radii of 4d and 5d transition elements in the same group are very similar (e.g. Zr ≈ Hf), which makes their separation chemically difficult.