Often times it's less about who/what you're joking about than the message behind the joke. Sometimes comedians tell jokes people just genuinely don't find funny or they even find them offensive or mean spirited and then the comedian will get defensive and say something to the effect of, "Oh, you can't take a joke/can't joke about ____?!" No, dude. They just didn't think that joke was funny and now you're coming off like an asshole.