Polarization degenerate solid-state cavity quantum electrodynamics