Notesencoding🌱 seedling

Serialization

Also: Marshalling · Serialisation · Deserialization · Decoding

Serialization turns in-memory data (objects, structs, pointers) into a self-contained byte sequence that can be stored or sent, and deserialization turns it back. The format you pick decides who can read the bytes, how big they are, and how safely the data can change.

Why it exists

Pointers and memory layouts mean nothing outside the process that made them. Anything written to disk, a network, a queue or a cache has to become bytes, and some other program, often a later version of yours, has to make sense of them.

How it works

  • Language-native formats (Java Serializable, Python pickle, Ruby Marshal) dump object graphs directly. Easy, but tied to one language and dangerous: decoding can instantiate arbitrary classes, so untrusted bytes can mean remote code execution.
  • Textual formats (JSON, XML, CSV) are readable and universal, with weak typing and optional schemas.
  • Schema-driven binary formats (Protocol Buffers, Apache Avro, Thrift) share a schema between writer and reader, so the bytes carry only values.
  • The hard part isn't the first encoding. It's reading data across versions: Backward and Forward Compatibility.

Trade-offs

Choice Gains Costs
language-native zero effort lock-in, security holes, no versioning
JSON readable, universal large, vague numbers, no enforced schema
schema-driven binary compact, typed, checkable evolution needs tooling; not human-readable

In practice

  • Python: the pickle documentation states it is not secure and to only unpickle data you trust.
  • Java: 2015 research showed common classpath libraries (e.g. Apache Commons Collections) could be chained into remote code execution through deserialization.

Pitfalls

  • Using a language-native format for anything that outlives the process or crosses a trust boundary.
  • Assuming the reader runs the same code version as the writer. In a Rolling Upgrade it doesn't.

Recall

These cards are also in your review deck, scheduled with spaced repetition.