And communists use the hammer and sickle which are just implements that farmers use/used.
That doesn't stop conservatives from recognizing that the hammer and sickle is communist symbolism.
Your "meme" is how all of language and symbolism work. People use words and symbols in a particular way (as a joke or not, it doesn't matter), and people start recognizing how people are using those words and symbols.
ADL has nothing to do with the government, by the way...
This may come as a surprise to you. It's not only conservatives that associate that symbol for communism. But you are correct in that symbols can mean anything. But using it to stigmatize a particular group of people for using it is dumb when you don't even know what the hell they're using it for.
Just like someone wanting to use the Charlie Chaplin mustache that can be associated with Hitler. If someone wants to use it, so be it. Don't go jumping to conclusions that they are racist for doing so. You're gonna sound like some evangelical from the Bible Belt.