That coloring is permanent and hard to separate from true reality because the story we have sold ourselves about our lives is in a way a form of self-protection. To look in the mirror honestly is a demanding thing but if we are to grow as human beings it must be attempted.
Imagine trying to love someone else through the fog of a prism that bends the light in a way that we have preset. Anything that tries to enter at a different angle is rejected because we are not primed to handle it. This is how turmoil begins as each lover only sees things through a lens that has been built over their lifetime.
I think this is one reason why partnering later in life becomes more arduous. Yes, there is baggage but also an expectation that the other person will fit into a vision that we have fashioned for ourselves and has been perfected through being burned by life a few times. It is not the exercise of identifying primary colors on a page but more like naming shades of each one; which is much more of a very delicate and subjective thing.
Perhaps objectivity is too painful because we need that gentler narrative about ourselves and the person we love to be about an ideal we have in our mind’s eye. If only we can get to that goal we will be that much closer to perfection.