The challenge we have is to dissect our thinking very carefully and pull away the thorns that have hurt us over the years without destroying everything we have built. The older one is, the more challenging this exercise becomes which is why if one is to transition it is best done when young.
I know of many cases of successful transitions late in life, but they do not come without considerable loss. It is inevitable that living a large chunk of your existence in one gender and then flipping a switch is not going to be easily digested by a society so intrinsically defined by gender.
Yes, we are people first, but the world counts gender identity among its most valued elements. It is why violence is resorted to by some when it is not respected.