It's not necessarily that you "can't" do a lot of things -- it's just harder. Recently I was trying to play around with Xmonad. I managed to get it up and running in X11 on OS X but it always was a little "off". I wanted it full screen, but sometimes the dock would appear, sometimes not, sometimes the OS X menubar along the top would appear, sometimes not. Sometimes various applications would lose focus but stay in front of X11. It was just not very great.

Then I ran into some problems getting gvim working in that setup. I'm not completely facile with all things Linux and usually Google for some help setting things up, but in this case I couldn't find anyone else who wanted to install gvim on OS X using X11, so I was out of luck there.

In the end I just fired up a Linux VM and it was all as easy as could be.