VIS^3COS. II. Nature and nurture in galaxy structure and morphology