We present a derivation of the exact expression for Pulay forces in density-functional theory calculations augmented with extended Hubbard functionals, and arising from the use of orthogonalized atomic orbitals as projectors for the Hubbard manifold. The derivative of the inverse square root of the orbital overlap matrix is obtained as a closed-form solution of the associated Lyapunov (Sylvester) equation. The expression for the resulting contribution to the forces is presented in the framework of ultrasoft pseudopotentials and the projector-augmented-wave method, and using a plane wave basis set. We have benchmarked the present implementation with respect to finite differences of total energies for the case of NiO, finding excellent agreement. Owing to the accuracy of Hubbard-corrected density-functional theory calculations - provided the Hubbard parameters are computed for the manifold under consideration - the present work paves the way for systematic studies of solid-state and molecular transition-metal and rare-earth compounds.