Only unref the repository if it's actually set.