You don't need to use the same language to use the same serialization library, however. If the latency of HTTP/JSON is truly too high, there's Protocol Buffers, Thrift, and a host of others in that space.
HTTP/JSON is nice, however, for the wealth of tooling that already exists and the relative ease of standardizing API design with REST.
If you like the REST architecture it works just fine with the message body being Thrift or whatever. I think of the ElasticSearch API which lets you use Thrift or XML or JSON.
The interesting distinction between binary serialization formats is if the schema is separated from the data.
For instance, if both sides of the system know you are encoding a 24 bit color value you can send 3 bytes and that is it; coding and decoding can be very quick and even possibly done on a "zero copy" basis.
If you are using something like JSON you not only have the waste involved with converting "255" to an #FF byte, but you also have to embed the schema in the sense of "here is an array of three integers (which happen to be bytes)" or "here is key 'red' and value R,..."
Thus, JSON is not "schema-free", it is "schema embedded in the data" and this inevitably bulks up the data and slows down encoding and decoding. Yes, general-purpose compression eats some of the storage/network encoding overhead, but you'll get it even tighter if you eliminate that fat before you put it through the compressor.
Now, separating the schema from the data means you need to make both sides aware of it, which is why you need some strategy for handling this in a systematic way rather than hoping things will work out OK without a plan.
HTTP/JSON is nice, however, for the wealth of tooling that already exists and the relative ease of standardizing API design with REST.