The CORALIE survey for southern extrasolar planets. XVIII. Three new massive planets and two low-mass brown dwarfs at greater than 5 AU separation