YSES 2b is a background star: differential astrometric M-dwarf measurements in time