The evolution of the baryons associated with galaxies averaged over cosmic time and space