I don't mind filler quests as long as they don't get in the way (forced to play them or otherwise you lag behind the enemy level). Quests overall could use some refinement, though. Considering the amount of terrible filler quests Fable 3 had, there was that D&D one that was utterly brilliant, and I wondered why more quests couldn't be like that.
Personally, I have a very hard time making myself skip quests. I know that doesn't apply to everyone, though, and it's hardly the developer's fault. So, yeah, if you aren't really penalized by skipping them then there shouldn't be a problem--but most games give a lot of exp for quests, so by skipping them, you kind of are being hurt, at least in games where exp opens up new skills. In Dead Island, for example, you want to get Sam's final Stomp skill as soon as possible because it's so damn good, which means you want exp.
Personally, I'd like less blatant filler quests, even if it does mean a slightly shorter game. Instead, I'd prefer if the filler quests the game does have are better in quality. Using Dead Island as an example once again, I've done two separate quests where I have to go get insulin for someone and another where I have to go get an inhaler. That's kind of annoying. I'd rather they cut one of those out and make one insulin quest that's really good.
Also, multiple filler quests that you get at the same time, which are in roughly the same area, aren't nearly as bad as quests that make you back-track.