I would argue that no, one need not feel any obligation to comment when downvoting a post.
First of all, voting is, by design, anonymous, and comments are not. If we don't want the site to have an anonymous negative feedback mechanism, we should request that feature be removed. Having an anonymous negative feedback mechanism in place that no one is supposed to use is rather absurd.
Secondly, with regard to answers, the downvote button is labeled "This answer is not useful." We discourage people from posting comments like "thanks, this worked for me"; rather, we encourage them to simply click the up arrow. Similarly, we don't need a host of comments saying, "this answer doesn't work." Voting communicates information; we don't need to duplicate it in comments.
A downvoted question is more ambiguous: it could mean the question does not show research effort, it could mean it is unclear, and it could mean it is not useful. A comment would certainly add clarity. But even for questions a vote by itself communicates useful information.
Now if one wants to offer constructive information to the poster beyond "this answer is not useful," a comment might be quite helpful. But that added information is a bonus; we should neither require nor expect people to post that.
Most of the time users don't need to downvote anything. Upvote what is good and the rest will filter out. But sometimes one encounters a question or answer that needs to be actively marked as not working. While a vote and comment might communicate even more, a vote by itself should never be considered inappropriate.
I expect some people will think it clever to vote against this answer. If you disagree with my point, you of course can do so. And voting on meta is different than on the main site. That said, I would urge people to refrain from pressuring downvoters to comment. We have far too little voting on the site to begin with; let's not discourage people from participating!