Huh. Really? I use whom all the time, and most of my friends do as well. Granted, a) we're all snobs, and b) we don't use it in every case we should, but we still use it.
"Whom" makes you sound a bit dated. Etymologists will get the joke. ;-)
(I suppose I should explain it. "Whom" comes from a dative case, and would is not descended from a word that would be used for direct objects, but only for a subset of indirect objects and prepositional objects. However if elative uses relate to elated things, then surely dative uses in English relate to dated things, right?)
So ask yourself the question, why not "thee", "thou" and "thy" (and relevant possessive forms)? I'm not saying "whom" is wrong only that it is archaic. Reviving just that word seems rather capricious and arbitrary no?
Maybe we're part of this little post 2000 resurgence: http://books.google.com/ngrams/graph?content=whom&year_s...