It's similar in concept to 'in power'. The government holds office, rather than it being in power. It's more to say that the power isn't derived from the government itself, but the office they are elected into.

Every government to hold office = every previous party that was elected and formed a government