If all your state is already json-serializable, yeah. But that's just as expensive as copying if not more, hence what cjbgkagh said about flatbuffers.

oh nvm, that doesn't solve this either

Perhaps flatbuffers would be better?

I love learning from folks on HN -- thanks! Will check it out.

Take a look at https://capnproto.org/ as well, while at it.

Neither solve the copying problem, though.

Ah, I forgot capnproto doesn't let you edit a serialized proto in-memory, it's read-only. In theory this should be possible as long as you're not changing the length of anything, but I'm not surprised such trickery is unsupported.

So this doesn't seem like a versatile solution for sharing data structs between two Python processes. You're gonna have to reserialize the whole thing if one side wants to edit, which is basically copying.

let me introduce you to quickle.

What’s the point? The whole idea is to share an object, and not to serialize them whether it’s json, pickle, or whatever.

I mean, the answer to this is pretty straightforward -- because we can, not because we should :)

That’s even worse than pickle.

pickle pickles to pickle binary, yeah? So can stream that too with an io Buffer :D