On the effect of regularization in policy mirror descent