It took me a long time to understand that my identity was not solely defined by these feelings and they were simply part of the experience of being trans. I needed to dig deep down to realize that even as the libido went down, the gender identity only grew in significance and not the other way around. Remember that you cannot be separated from your sexuality because it is an intrinsic aspect of who you are.
Having a male body and either a full or partial female identity only serves up a cocktail of confusion which can lead you to erroneous conclusions; it can even make you question your sanity. But as I discovered, the more I became accepting of my identity, the more relaxed and at ease with myself I became. This signaled to me that I was on the right track.
Today there is at least a basic understanding of what makes me tick even if that does not include a perfect roadmap to my creation as a being that is different from most others. The fact that my gender identity does not line up perfectly with my biology does not make me part of the majority, but I am certainly in very good company.
Do not let those sexual feelings mislead you. Instead, examine them closely and decide what they mean in the larger context of who you are. If they turn out to be the primary driver for your dressing, then so be it but there may be more, and that journey is up to you to discover. Most of us who are older end up getting to know ourselves at some point and then we relax into that role.
Hopefully this happens to you sooner rather than later.