Gravity and rotation drag the magnetic field in high-mass star formation