I find it quite disrespectful to others that they have such a problem with that label.

As someone with a laundry list of mental illnesses and who is quite firm that mental illness is mental illness, it’s not disrespectful for trans folk to want to not get tarred with that brush. Trans folk are hindered by primarily cultural prejudices.