Python iconPythonSep 30, 2026 ~4 min source read

Memory Buffer Protocol: proposals for safer concurrent access and custom data types

At the Python Language Summit 2026 Nathan Goldbaum outlined changes to the Buffer Protocol: better documentation, buffer leases for coordinated reads/writes, and a namespace for custom data types. Draft PEPs and prototype work are available; NumPy interoperability and ABI constraints were discussed.

Python Insider: Memory Buffer Protocol (Python Language Summit 2026)

Share this story

Send the public story page.

Useful takeaways from this story.

Add buffer leases (PyBufferLease and PyBufferAccessState) so consumers can request SHARED_READ or EXCLUSIVE_WRITE access without blocking, enabling safe coordination of concurrent reads and writes.

Document the full Buffer Protocol grammar and provide a HOWTO with complete C examples to reduce reliance on PEP 3118 and third-party implementations for details.

Introduce a namespace-based extension for custom data types in the format string (syntax using square brackets and $) so exporters can declare and safely reject unknown custom types without parsing the buffer.

Protocol defines how code accesses an object's underlying memory (for example, bytes, bytearray, and array.array). Nathan Goldbaum's summit talk focused on three areas: consolidating documentation, adding protocol-level support for safe concurrent access, and enabling custom data types in the format language.

The Buffer Protocol supports a struct-style format language that covers records, field names, subarrays, byte order, alignment, and complex numbers. Parts of this grammar are underspecified across PEP 3118 and third-party implementations like NumPy. Goldbaum proposed documenting the full grammar in the official Python docs and publishing a HOWTO for exporters and consumers. The HOWTO would include complete C examples covering validation, cleanup, ownership, and safe thread use.

Current protocol facilities expose stable storage addresses but do not coordinate concurrent reads and writes at the protocol level. A readonly flag exists per view, but writable access cannot be made exclusive across multiple views.

This proposal leaves existing APIs (PyObject_GetBuffer, PyBuffer_Release, etc.) unchanged. Exporters would advertise available access modes via PyObject_GetBufferAccessModes(), and consumers would request leases only when the exporter supports them. An implementation similar to this idea is already being developed, and Kumar Aditya is working on NumPy changes to interoperate with the proposed PEP.

Custom data types in the format language

Community questions and constraints

Draft PEPs and prototype implementations are available for review. The proposal aims to provide clearer documentation, safer concurrency for buffer access, and a practical path for interoperable custom data types while avoiding changes to the existing Stable ABI.

More context around this story.

Keep reading in the app

Open the app view to save this story, compare related coverage, and continue from the same source.

Open in app