Curing ill-conditionality via representation-agnostic distance-driven perturbations