High-energy gamma rays and the large-scale distribution of gas and cosmic rays