Gravitational test beyond the first post-Newtonian order with the shadow of the M87 black hole