At the most basic level, leaving aside all issues other than social practicality, inviting a vegan to a social event means that the gracious host must either serve vegan food to all the guests, or offer a variety of main courses and side dishes, making sure that the vegan guests understand which foods are which. If only some of the food is vegan, then a food-based apartheid is created. If all of the food is vegan, than one guest has decided the menu criteria for everybody.

I've tried being vegan, and also vegetarian, and the greatest difficulties for me arose when my hosts did not know about my self-imposed dietary limitations. For example, when a coworker baked a surprise cake for me for my birthday. I decided the most polite thing to do in those cases was to eat what the host offered, thanking them for whatever they offered, rather than saying, "I'm sorry, I can't eat that." Once I made that decision, I was no longer a vegan or a vegetarian according to the standard criteria.

I think some of the friction, and resulting ill will, between some vegans and some omnivores is caused by the strict nature of the vegan identity. Non-vegans can try to be polite by offering vegan alternatives, and vegans can also try to be polite by occasionally eating animal-based dishes that were prepared in good faith by their hosts. When the entire group's eating decisions are subject to the strict objections of a minority, that's going to cause friction.