Nice setup. I've used vnc before, not x11vnc so I'm not sure if it's similar. But vnc seemed to require a lot of bandwidth to run. But it's definitely nice.
x11vnc allows you to see the desktop the user is seeing. I think vnc by default creates its own sessions.
So for example, my mom would say "I am not sure where to find the pictures". I would move the folder to her Desktop from ssh, hoping the would see it. But if not I ssh in, then launch x11vnc then connect to it with a vncclient and then move around her desktop to see what the problem might be.
She would see the mouse move and buttons getting clicked.
Bandwidth is not a problem because, we have have something like 50Mpbs connections, latency is usually the issue as the signal has to go half way around the world.