1

I search a Python library which supports a JSON-like format which supports binary data.

Required features:

  • Nested data structures like JSON
  • Support for binary data (without additional base64 encoding/decoding of binary parts)
  • Open source library (not GPL) for reading and writing for Python and Java
  • If no binary data is transferred the format on the wire should be valid JSON.
Improve this question
1
  • I think your last requirement means no such format exists. You would have to create it. However it should be pretty easy since there are plenty of invalid characters you could use to indicate "raw binary data follows", e.g. instead of {"foo_base64": "a34b3f5723"} have {"foo": #<raw binary data>}. Or something like that.
    – Timmmm
    Mar 1 '20 at 17:05
1

I think that the best option for you is BSON:

  • BSON is based on JSON so it will be an easier transition from JSON than Protocol Buffers.
  • This page links to implementations in both python and Java (and any other language you may want)
  • Here is a Python specific implementation
  • I do not used Java so I do not have any instinct as to which library to recommend for that. There are 5 listed on the implementations page linked above.

I believe that this was developed by MongoDB for their storage engine. I have not had to programmatically interact with BSON since the mongoDB drivers handle that conversion from JS objects automatically. As far as the fourth requirement "If no binary data is transferred the format on the wire should be valid JSON." BSON libraries make everything binary data for transfer on the wire.

Improve this answer
0

Google's Protocol Buffers should do the trick. The linked page contains the Google Documentation and tutorials for Python, Java, C++, Go, C#, and Dart.

All of their (including Java and Python) implementations are contained in this github. Releases are here. This completely replaces JSON and transfers everything as binary data over the wire.

Improve this answer
2
  • 1
    Please link to an open source Python implementation for both, thanks! Also, a brief explanation of how it works would be great. Feel free to split into two answers, that will get you more points :-)
    – Nicolas Raoul ♦
    Dec 11 '18 at 2:33
  • 1
    Also, please state explicitly how they satisfy the 4th requirement "If no binary data is transferred the format on the wire should be valid JSON", thanks!
    – Nicolas Raoul ♦
    Dec 11 '18 at 2:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.