It's so easy to worry about what other people think. I do it all the time. When in conversation with someone I inevitably find myself confronted with a thousand worries: 'Does this person think I'm a good guy?', 'Am I not intelligent enough for them?', 'Am I too intelligent?', 'Does he think I'm an idiot?'.
These questions arise and we react to them and use them as a basis for our relationship with the other person. If we feel inferior to the other person we do our best to impress them. If we feel superior we do our best not to make them feel too inferior, or if we are insecure we relish the novelty of our superior position and endeavour to triumph over them in all aspects.
All of these reactions are actually really rather silly. You see, you can never ever hope to know what another person thinks about you (unless we develop some kind of psychic technology that is!). My advice is to stop being a mind-reader and just react honestly. You don't know what they think so why bother trying to find out? You'll just end up assuming a load of fictional crap. Be honest and then deal with the circumstances. It's just so much simpler than skipping around a bunch of assumptions. Afterall, when you think you know someone in actual fact all there is is an idea of that person, nothing else, and an idea of a person is really just an assumption.
Stop assuming and just go with whatever bubbles up. You didn't ask for those bubbles. Why should you feel responsible if the other person doesn't like them? Their disliking is just another bubbling. All human life is just the bubbling up of fate anyhow. Everything is one and everything is predetermined. That's a relaxing fact to take in isn't it?