Yep. In my experience it tends to miss a fair bit of stuff, though. I think you'd want to be more aggressive about sniffing out files loaded by javascript or linked from CSS. Wget could probably serve as part of an MVP though.

I don't think the crawler is actually that hard. The killer feature would be the execution and the UX. It should be usable by anyone.