I do both. My own standard is something along the lines of, "would I buy this if I couldn't download it for free?" If yes, then I usually buy it. If no, then I have little problem downloading it. In that situation, there is no "victim".
And yes, there are instances where I really want to support the artists themselves, like when I bought some RJD2 songs I already had, or Jaymay's entire album (rare for me). A lot of what I download is live stuff that has no official release, which doesn't bother me. On the other side of the coin, I don't really want to support Kanye West, so I don't mind downloading his music, or the music of someone deceased.