From interstellar dust to comets - A unification of observational constraints