What I have found in relationships is that in most cases, when faced with uncertain set of facts, most people will take the easy route and just assume the worse. In having experienced this a couple of times I have made a decision a while back to not worry about how my actions are perceived, but to simply do what's best with the knowledge I have and with the best intentions, and let all other 'cards' fall where they may.
In most cases the people who assume the worse about others from the beginning actually prefer the negative dialogue they have in their head to facts. It might be a protective mechanism they use to assure they do not get hurt. They might also jump directly to the worse conclusion about you if they do not like you. These people may be 'waiting in ambush' for some action on your part to misinterpret. I both cases, when a dubious action of yours comes their way, they pad and weave together some clever set of thoughts about you and rationalize the negative conclusions. When this happens the best thing to do is walk away, far away. Ultimately, breaking someone else's negative assumptions about you are not your responsibility or within your power.
I have come to trust through experience that the plain truth of our actions and of who we clearly are will be revealed through the passage of time only. Bad assumptions are better left alone; trying to dispel them is futile and may only serve to further feed the wrong assumption.