I spent a very long time trying to find the answer to this same question. I enjoy it as well (obviously) but spent years covered in shame and have purged twice now. Everyone is different but for me, I feel it was best summarised by another user’s experience and is all based around Having no intimate female energy in my life

"
Now with us being straight, we start out desperately seeking that connection with the feminine, so in our loneliness and out of desperation and necessity as we crave that feminine interaction, turn to a substitute, womens clothes. We begin by trying on a few items, it is a rush to feel all these fabrics and textures that a woman gets to experience daily, the cycle repeats and it escalates until we have quite the feminine wardrobe.
This is all well and good and satiates our desire for interaction with femininity and feminine energy.
"
I think there must have always been that curiosity i guess. not every guy who is in my position ends up doing this. But now, knowing the above, its much easier to live with, accept, control and even enjoy. it doesn't leave this room. it doesn't hurt anyone.