I don't really think that's a reasonable way of looking at it ... not because you're expecting others to be nicer, but because it isn't something that logically should be offensive. It's the same as your momma jokes to people who are fat ... etc etc etc. However, our emotional responses do not have to make sense, and sometimes it is better to acknowledge that people feel uncomfortable from certain things rather than question it. I don't think I've ever used it on here, but I'll be sure to never use it from now on.
It's not really a slur anyway. It's a very popular meme that isn't even meant to be that offensive.
If you demand respect or gratitude for your volunteer work, you're doing volunteering wrong.