It's only irrelevant to their job in office if they just admit to it. It becomes a liability if they try to keep it a secret.
I said early on that Clinton's best move with Gennifer Flowers was to make an announcement like "Yeah, we're sleeping together, Hillary doesn't care and neither should anyone else." Then Monica wouldn't have been any big deal, because we'd already have known that he had an essentially open marriage.