This is pretty much the same as vegetarianism for me. I mean, I fail to see the logic of not harming some animals, harming others and freely harming plants. Sure, plants may not feel, plants may not record any pain. Why does that give us some moral right to take their lives and play around with them?
Honestly, as humans, we have to take a very cruel "care only for other humans and nothing else" stance if we don't want to completely fail as a species.
I mean, I feel bad about all those endangered species, abused animals (I love dogs so it kills me personally to see those advertisements and such) and similar animal-related problems, but to really care about those and then proceed to swat a fly and dig into vegetables feels hypocritical.