If you just need the binary data in the strings and can recover the boundaries between the individual strings easily, you could just write them to a file directly, as raw strings.
If you can't recover the string boundaries easily, JSON seems like a good option:
a = [b"abc\xf3\x9c\xc6", b"xyz"]
serialised = json.dumps([s.decode("latin1") for s in a])
print [s.encode("latin1") for s in json.loads(serialised)]
will print
['abc\xf3\x9c\xc6', 'xyz']
The trick here is that arbitrary binary strings are valid latin1, so they can always be decoded to Unicode and encoded back to the original string again.
The Hitchhiker's Guide to Python
docs.python-guide.org › scenarios › serialization
Data Serialization — The Hitchhiker's Guide to Python
Serializing Text · Simple file (flat data) repr · ast.literal_eval · CSV file (flat data) YAML (nested data) JSON file (nested data) XML (nested data) Binary · NumPy Array (flat data) Pickle (nested data) Protobuf · Documentation overview · Previous: Image Manipulation ·
Videos
Real Python
realpython.com › python-serialize-data
Serialize Your Data With Python – Real Python
December 4, 2023 - As you’ve learned, pickle is the standard data serialization format in Python, and it’s capable of representing a wide range of native data types as well as user-defined classes. It’s a binary format specific to Python, which requires no schema definition and can handle data in almost any shape or form.
MachineLearningMastery
machinelearningmastery.com › home › blog › a gentle introduction to serialization for python
A Gentle Introduction to Serialization for Python - MachineLearningMastery.com
June 21, 2022 - Hierarchical Data Format 5 (HDF5) is a binary data format. The h5py package is a Python library that provides an interface to the HDF5 format. From h5py docs, HDF5 “lets you store huge amounts of numerical data, and easily manipulate that data from Numpy.” · What HDF5 can do better than other serialization formats is store data in a file system-like hierarchy.
GitHub
github.com › explosion › srsly
GitHub - explosion/srsly: 🦉 Modern high-performance serialization utilities for Python (JSON, MessagePack, Pickle)
14 hours ago - This package bundles some of the best Python serialization libraries into one convenience package, with a high-level API that makes it easy to write code that's correct across platforms and Pythons. This allows us to provide all the serialization utilities we need in a single binary wheel.
Starred by 481 users
Forked by 38 users
Languages Python 99.3% | Shell 0.7%
Python
docs.python.org › 3.3 › library › pickle.html
12.1. pickle — Python object serialization — Python 3.3.7 documentation
The pickle module implements binary protocols for serializing and de-serializing a Python object structure.
Tutorialspoint
tutorialspoint.com › python › python_serialization.htm
Python - Serialization
Deserialization, or unpickling, is the reverse process, converting the byte stream back into a Python object. We can serialize an object using the dump() function and write it to a file. The file must be opened in binary write mode ('wb').
Python
docs.python.org › 3.4 › library › pickle.html
12.1. pickle — Python object serialization — Python 3.4.10 documentation
June 16, 2019 - The pickle module implements binary protocols for serializing and de-serializing a Python object structure.
SSOJet
ssojet.com › home › serialize and deserialize binary in python
Serialize and Deserialize Binary in Python | SSOJet
March 1, 2025 - This article dives into Python's built-in `pickle` module, demonstrating how to serialize (convert objects to byte streams) and deserialize (reconstruct objects from byte streams). You'll learn best practices for handling binary data, ensuring reliable data transfer and storage for your projects.
Diveintopython3
diveintopython3.net › serializing.html
Serializing Python Objects - Dive Into Python 3
This is still in Python Shell #1. Use the open() function to open a file. Set the file mode to 'wb' to open the file for writing in binary mode. Wrap it in a with statement to ensure the file is closed automatically when you’re done with it. The dump() function in the pickle module takes a serializable Python data structure, serializes it into a binary, Python-specific format using the latest version of the pickle protocol, and saves it to an open file.
Problemsolving
diveintopython3.problemsolving.io › serializing.html
Serializing Python Objects - Dive Into Python 3
We cannot provide a description for this page right now
GitHub
github.com › metachris › binary-serializer
GitHub - metachris/binary-serializer: Minimalistic binary serialization protocol (a la protocol buffers) with Python and Java implementations.
November 3, 2022 - Minimalistic binary serialization protocol (a la protocol buffers) with Python and Java implementations. - metachris/binary-serializer
Author metachris