I suspect it’s more that people only have the room to care about Moralist-politics when they themselves are comfortable, with full bellies and a roof over their heads. If things are tight for them, all of a sudden somebody else’s minority sexual orientation or gender details doesn’t look important anymore. Whilst immigration can still be portraid as something with a negative economic impact (“they’re taking our jobs”) for people who themselves are economically hurting and thus can still be used by the Far-Right for fearmongering amongst the worse off, sexual orientation and gender definitelly cannot (there’s no “gays/transsexuals are stealing our jobs”).