Ah no I wasn't reading too much into it! It was meant as the joke I interpreted:
"When the Fifth Doctor asks if the Master "still has that rubbish beard," the Tenth replies, "No ... well, he had a wife." This was interpreted by some fans as being a gay culture reference; a "beard" is a slang term for a member of the opposite sex who joins a homosexual individual in a marriage or other relationship in order to mask the fact that one or both partners is gay. Moffat confirmed it as a gay joke in Death Ray Magazine in 2008, when he proudly claimed "I've got the record for gay jokes. I've got the gayest joke of all time in Doctor Who - I've got the "beard" joke about the Master". However, Moffatt referring to it as a joke highly suggests that this is not intended to suggest at the Master's orientation."
(From
http://tardis.wikia.com/wiki/Time_Crash_(TV_story)#Plot)