The only really noticeable impact I've seen is that it blocks Disqus, which too many people seem to use for comments. That's usually not a problem, and I can always disable it in the rare cases where I really want to comment on something. What sites are breaking for you?
It's been a long time since I used Ghostery, but the most common problem were JS trigger events from clicking a button that didn't fire. It made it a bit of a pain to debug.
