Is this too straightforward?

Okay, so this is a follow-up to my previous question. My new friend is totally embarred of me and never wants to hang out because of his fear of what his other friends will think. Would it be too harsh of me to just be like "why are we even friends if you're too embarred to hang out with me?". I mean, he's a nice person, his friends are just evil.