I've done something wrong. I have a secret you know nothing about. You don't really know me. And as a result the guilty person lives each interaction as though she owes a debt to the other person.
Therefore, guilt erodes boundaries. Guilt tips the balance of power in relationships toward the other person. Guilt makes you a doormat. Guilt makes you lie and construct a facade so that other people can't get to know the real you. And guilt works its way out of you in the strangest ways. True Story: I lived through an entire decade of my life when I didn't check voicemail because of my guilt. Now when I'm not checking my voicemail I do a guilt-check on my heart. That is so strange that you can't make it up.
The practical response is to confess, but not just to God, to whomever you have affected. And if you think that sin only affects yourself then you are lying to yourself and are in a deeper place of deception than you thought.
"But I'll lose their respect." And you probably will. But they are only respecting the persona you have built, not the real you.
Why is it that confession has such a cleansing effect on your soul? Jesus went to the cross so that the penalty that was due to us for our rightful guilt was taken upon himself. He bears our guilt, not us. And so when we keep our guilt to ourselves we are keeping it out of the hands of Jesus. We keep it secret and safe because we are comfortable with it. We feel we deserve to be guilty for what we've done. We feel we deserve to be punished for what we've done. But Jesus doesn't. He bore our punishment so that we wouldn't have to.
The guilty person thinks to himself "I owe you because of my secret" But the gospel brings us to the place where we can say to God, "I owe you because of your Son." When you grasp how guilty you have been you will experience the pain you need to move you to confess. And when you grasp how loved you have been you'll have the security you need to confess.