Excess quantum noise is colored