async_spi: SPI transfers for asyncio by lynt-smitka · Pull Request #11489 · adafruit/circuitpython

lynt-smitka · 2026-10-02T09:38:07Z

Adds the async_spi module, based on @kamocat's work in #11040. Part of #10856.

async_spi.SPI is configured and locked like busio.SPI, but write(), readinto() and write_readinto() are awaited, so other asyncio tasks run during a transfer. Cancelling a task stops its transfer. Tasks that share the bus can use an asyncio.Lock.

busio on raspberrypi gets async reads, full-duplex transfers and cancel, next to the async write from displayio: send one area buffer while composing the next #11468. A DMA interrupt marks the transfer as done, and asyncio then resumes the waiting task.
supervisor/shared/awaitable.c lets C functions return awaitables, so other async modules (I2C, UART) can use it later.
Only raspberrypi enables the module so far. It adds about 1.9 KB there; other ports are unchanged.

Tested on a Pico W with MOSI jumpered to MISO: data reads back correctly at up to 62.5 MHz, other tasks keep running during transfers, and cancelling transfers works while audio plays.

tannewt

Thanks for picking this up! I'd prefer it to be more like a new module than changing the implementation in busio.SPI().

lynt-smitka · 2026-10-03T17:40:38Z

I would like to clarify why the PR looks the way it does:

The blocking transfers, displayio's async write and async_spi share one DMA path. A separate module would either carry a second copy of the DMA code for the same SPI block, or busio would have to call into the new module.
Reusing busio's constructor and configure()/try_lock()/unlock() keeps the module at about 1.9 KB. With its own bindings it is about 0.5 KB more.
The lock is busio's. An earlier version had an awaitable lock() that polled - in a test with two tasks it let one of them run 790 transfers while the other got 1. Without a native lock, tasks use asyncio.Lock, which costs nothing in flash.

What I'm working on:

Moving the RP2 driver out of busio into a port file (peripherals/spi.c), with busio.SPI and async_spi.SPI as thin layers over it. busio's bindings stay unchanged.
async_spi.SPI as its own type: constructor, configure(), deinit(), frequency and the three awaited transfers, no lock methods. Tasks share the bus with asyncio.Lock.
A Zephyr implementation tested on native_sim. That needed zephyr-cp: fix asyncio and busio.SPI on native_sim, add an SPI loopback for tests #11504 first: asyncio tasks from create_task() did not run there, and busio.SPI crashed.
The current result is about +2 KB on RP2 boards, and other ports get a few bytes smaller. It passes on a Pico W and a Pico 2 W so far. I'll push once zephyr-cp: fix asyncio and busio.SPI on native_sim, add an SPI loopback for tests #11504 is in.

Questions before I push

Is it ok to keep the shared driver in peripherals/, so busio does not depend on the Python module?
Is it ok for async_spi to reuse busio's argument tables? Same arguments, 160 B less flash, but a link between the two modules.
Is it ok that configure() and deinit() wait for a running transfer instead of raising?

tannewt · 2026-10-05T16:39:48Z

busio would have to call into the new module.

This is what I was thinking. Build busio on top of async under the hood.

Reusing busio's constructor and configure()/try_lock()/unlock() keeps the module at about 1.9 KB. With its own bindings it is about 0.5 KB more.

The constructor can be shared by building on the async portion.

I don't think we want busio's locking. Instead, we likely want something async friendly. (Like an async lock)

Questions before I push

* Is it ok to keep the shared driver in peripherals/, so busio does not depend on the Python module?

Yup, that's fine. Code in peripherals shouldn't have any python specific code though.

* Is it ok for async_spi to reuse busio's argument tables? Same arguments, 160 B less flash, but a link between the two modules.

The reverse but yes. Have it in async and reuse it in busio.

* Is it ok that configure() and deinit() wait for a running transfer instead of raising?

Yes, but I image they too should be async then.

buffer_slice() turns a buffer object and element bounds into a byte slice, as busio.SPI's write(), readinto() and write_readinto() did inline. busio.SPI gets a little smaller, and async_spi will use it too.

busio.SPI's port code becomes rp2_spi_t and rp2_spi_*() in peripherals/spi.c, with common-hal/busio/SPI.c as a thin layer over it, so that another module can drive an SPI bus the same way. No change in behaviour.

rp2_spi_read_start() and rp2_spi_transfer_start() start the other two transfers the way rp2_spi_write_start() starts a write. rp2_spi_stop() stops a transfer, or finishes it if it is already done; it does nothing once the bus has started a later transfer. rp2_spi_configure_start() applies new settings once the running transfer has finished, and rp2_spi_deinit() stops a running transfer. With CIRCUITPY_ASYNC_SPI, a DMA_IRQ_0 handler, shared like the audio and rp2pio ones, sets the done flag when the RX channel finishes, so a waiter does not need to call rp2_spi_end() to find out.

CIRCUITPY_DEFINE_ASYNC_FUN_OBJ_KW(obj, n_args_min, fun) defines a method that returns an awaitable. As with a CPython coroutine, its arguments are stored and only parsed when it is awaited. While an operation runs, the task waits on asyncio's I/O queue: the awaitable is a stream that polls readable once the operation's done flag is set, possibly from an interrupt. Based on the awaitable in adafruit#11040.

async_spi.SPI is configured like busio.SPI, and configure(), write(), readinto() and write_readinto() are awaited: other tasks run during a transfer. Tasks that share the bus can use an asyncio.Lock. Enabled on raspberrypi, where it drives the bus through peripherals/spi.c. A manual test is in tests/circuitpython-manual/async_spi. Co-authored-by: Marshal Horn <175229+kamocat@users.noreply.github.com>

busio.SPI takes the same arguments as async_spi.SPI, so where that module exists its tables are used. Saves 160 bytes on RP2040.

Transfers use spi_transceive_cb() where the driver has an async path, and otherwise run on the system work queue. Zephyr cannot stop a transfer, so cancel waits for it. Enabled on boards with an SPI bus; tested on native_sim through the loopback device.

The driver's result reaches the await as an exception. cancel() and deinit() wait for the transfer without running background tasks, since they may run from a finalizer during GC. async_spi needs asyncio, not only async/await, so raspberrypi enables it with MICROPY_PY_ASYNCIO.

…O_SPI Unused, so a build error, on boards without busio.SPI.

busio.SPI is always built, async_spi only on some ports, so the shared tables belong to busio. This drops the CIRCUITPY_ASYNC_SPI conditionals from busio's bindings. Firmware size is unchanged.

tannewt

Thanks for working on this. I've got a couple suggestions. I'm being extra picky on this change because I expect to apply these patterns to more ports and APIs.

The driver takes a volatile bool * for a transfer's done flag instead of circuitpy_async_flag_t, so peripherals/ does not depend on the supervisor. common-hal/busio/SPI.h includes the flag header again, as before the driver moved.

common_hal_async_spi_spi_*_start() return the context that end() and cancel() get, so an implementation can keep per-operation state. Both ports return the SPI object, as the bindings did before.

tannewt

Getting closer thanks! Please resolve or reply to inline comments.

The context is for state of one operation; start() returns NULL on both ports, which keep that state in the bus object. end() and cancel() take self and the context as separate arguments, and the macro takes the end and cancel functions by name.

shared-module/busio/SPI.c implements busio.SPI over common_hal_async_spi: each blocking call starts a transfer and waits for its flag, the lock sits on top, and FourWire's write_start/end map onto the async start and end. A port turns it on with CIRCUITPY_BUSIO_SPI_OVER_ASYNC, which also builds its async_spi common-hal without the Python module. A driver error reaches busio as false and an await as an exception, through transfer_finish() and transfer_end(). raspberrypi and zephyr-cp use it, so their own busio.SPI goes away. The RP2 driver is always async now and loses its blocking entry points; Zephyr keeps a fixed-device constructor for board.SPI() and reuses the fill buffer of reads. Pico 898680 -> 898760 bytes, Pico 2 W 1375564 -> 1375500; other ports are unchanged.

bus->spi.spi.peripheral read as a typo.

Without a TX buffer the nRF SPIM driver sends its over-read character, 0xFF, so readinto() with the default write_value clocked out 0xFF instead of 0. Seen on a Feather nRF52840; native_sim's loopback sends zeros, which hid it.

lynt-smitka · 2026-10-14T19:57:17Z

Tested on a Feather nRF52840 under zephyr-cp too: busio and async_spi loopback pass from 250 kHz to 24 MHz. It showed one old bug: without a TX buffer the nRF SPIM sends 0xFF, so readinto() with the default write_value clocked out 0xFF instead of 0 (the previous zephyr busio did the same); fixed in 4546345 by always sending write_value from a buffer.

close() or throw() before the first await left the state NEW, so a later await started the transfer. It is now finished, as with a closed coroutine.

configure_start() checked for a running transfer and then stored the waiting configure(); a transfer_done() between the two left it waiting forever.

Errata RP2350-E5: an enabled channel can retrigger after the abort.

…ransfers "ticker ran" once also passed with a blocking transfer; now it has to run more than 20 times during 10 transfers of 4 KB at 1 MHz (about 330 ms).

tannewt

Getting better and better. Thank you. Some more simplifications to what we have.

A port that has it turns it on in its mpconfigport.mk; busio.SPI is then always built on the same implementation and FourWire's write_start() follows. The separate CIRCUITPY_BUSIO_SPI_OVER_ASYNC is gone.

Like touchio: busio/SPI.c goes to SRC_SHARED_MODULE_ALL on a port with async SPI and to SRC_COMMON_HAL_ALL otherwise, instead of filtering the expanded list.

The wait for the transfer or configure checks the interrupt character, cancels the operation and raises the KeyboardInterrupt instead of finishing it.

busio.SPI is built on it, so the rp2_spi_* layer in peripherals/ had one user. Its functions are the common_hal_async_spi_spi_* ones now and its fields are in the object.

It waits for a running transfer, then applies the settings; nothing to await, and a single task needs no lock for configure() followed by a transfer. Tasks sharing a bus serialize their transfers anyway, with an asyncio.Lock around configure() and the transfer. The pending-configure state in both ports is gone.

Lock is a type in the awaitable layer, embedded in the SPI object. __aenter__ is an awaitable that takes the lock or waits in line; the awaitable itself is the queue node, linked through a next pointer, so nothing is allocated. __aexit__ hands the lock to the next waiter and returns the lock, which awaits as already finished. A task cancelled while waiting leaves the line; one cancelled right after the hand-off passes the lock on. Transfers and configure() raise "Function requires lock" without it, as busio does; there is no try_lock, has_lock or unlock.

tannewt

Thank you for continuing to work on this. I think it's getting close. busio_spi will be in the place it is currently but it should be easy to switch displayio to async_spi internally in a follow up.

The lock holder is the only task that transfers, and a transfer only runs while that task awaits it, so configure() no longer waits for one. A transfer still running means another task skipped the lock; configure() then raises OSError (EBUSY) instead of reconfiguring the bus under it, which on RP2 would stall the DMA transfer for good.

common-hal owns the object's fields, so the binding asks it for the lock.

try_lock, has_lock and unlock work on the lock inside the async_spi object, so a bus used from C and from asyncio has one lock.

A transaction ends its transfers before the bus is reconfigured.

The async_spi object already holds the bus settings, so busio no longer keeps a copy.

A task waiting in `async with spi.lock` would otherwise wait forever once the bus is deinitialized; now it gets the lock and its transfer raises the deinitialized error.

tannewt

Three minor things and then we'll be ready to merge.

tannewt · 2026-10-20T18:30:32Z

+    if (self->async_active) {
+        mp_raise_OSError(MP_EBUSY);
+    }


I don't think you'll hit this case because the lock is checked by shared-bindings. If internal code does this incorrectly you won't want to raise anyway. So, I'd remove this.

tannewt · 2026-10-20T18:31:36Z

+    // End the previous transfer first: an async write may still be receiving into one_byte.
+    _end(self);


You shouldn't need this because a transfer can't start without being awaited.

tannewt · 2026-10-20T18:34:08Z

+// skipped the lock.
+void common_hal_async_spi_spi_configure(async_spi_spi_obj_t *self, uint32_t baudrate,
+    uint8_t polarity, uint8_t phase, uint8_t bits) {
+    if (self->done != NULL && !CIRCUITPY_ASYNC_FLAG_IS_SET(self->done)) {


tannewt requested changes Oct 2, 2026

View reviewed changes

Comment thread ports/raspberrypi/common-hal/busio/SPI.c Outdated

Comment thread shared-module/async_spi/SPI.h Outdated

Comment thread tests/circuitpython-manual/async_spi/async_spi_loopback.py

lynt-smitka mentioned this pull request Oct 3, 2026

zephyr-cp: fix asyncio and busio.SPI on native_sim, add an SPI loopback for tests #11504

Merged

lynt-smitka and others added 7 commits October 8, 2026 20:34

shared/runtime: add buffer_slice() and use it in busio.SPI

c1d61d2

buffer_slice() turns a buffer object and element bounds into a byte slice, as busio.SPI's write(), readinto() and write_readinto() did inline. busio.SPI gets a little smaller, and async_spi will use it too.

raspberrypi: move the SPI driver to peripherals/spi.c

4719ff9

busio.SPI's port code becomes rp2_spi_t and rp2_spi_*() in peripherals/spi.c, with common-hal/busio/SPI.c as a thin layer over it, so that another module can drive an SPI bus the same way. No change in behaviour.

busio: reuse async_spi's argument tables

4f18be7

busio.SPI takes the same arguments as async_spi.SPI, so where that module exists its tables are used. Saves 160 bytes on RP2040.

zephyr-cp: async_spi

8bbdf9b

Transfers use spi_transceive_cb() where the driver has an async path, and otherwise run on the system work queue. Zephyr cannot stop a transfer, so cancel waits for it. Enabled on boards with an SPI bus; tested on native_sim through the loopback device.

lynt-smitka force-pushed the async-spi branch from 5e39446 to 8bbdf9b Compare October 8, 2026 20:38

lynt-smitka added 3 commits October 8, 2026 21:10

busio: keep the SPI constructor's argument table under CIRCUITPY_BUSI…

9aaf7f5

…O_SPI Unused, so a build error, on boards without busio.SPI.

Merge remote-tracking branch 'upstream/main' into async-spi-v3

f2b40a4

tannewt reviewed Oct 12, 2026

View reviewed changes

Comment thread shared-bindings/busio/SPI.c Outdated

busio: own the SPI argument tables, async_spi uses them

7fe5f7a

busio.SPI is always built, async_spi only on some ports, so the shared tables belong to busio. This drops the CIRCUITPY_ASYNC_SPI conditionals from busio's bindings. Firmware size is unchanged.

tannewt requested changes Oct 12, 2026

View reviewed changes

Comment thread ports/raspberrypi/peripherals/spi.h Outdated

Comment thread ports/raspberrypi/common-hal/busio/SPI.c Outdated

Comment thread shared-bindings/async_spi/SPI.c Outdated

lynt-smitka added 2 commits October 12, 2026 20:19

async_spi: start functions return the context

b1de0e7

common_hal_async_spi_spi_*_start() return the context that end() and cancel() get, so an implementation can keep per-operation state. Both ports return the SPI object, as the bindings did before.

tannewt requested changes Oct 13, 2026

View reviewed changes

Comment thread shared-bindings/async_spi/SPI.h Outdated

Comment thread ports/raspberrypi/common-hal/async_spi/SPI.c Outdated

tannewt mentioned this pull request Oct 14, 2026

Implemented asyncio SPI for rp2040 & rp2350 #11040

Closed

lynt-smitka added 3 commits October 14, 2026 17:34

busio: name the embedded async_spi object async_spi

cff9b57

bus->spi.spi.peripheral read as a typo.

lynt-smitka added 3 commits October 14, 2026 20:50

async_spi: close() on an awaitable that was never awaited finishes it

6f5d10a

close() or throw() before the first await left the state NEW, so a later await started the transfer. It is now finished, as with a closed coroutine.

zephyr-cp: register a waiting configure() under irq_lock()

5178374

configure_start() checked for a running transfer and then stored the waiting configure(); a transfer_done() between the two left it waiting forever.

raspberrypi: clear the DMA enable bits before an SPI abort on RP2350

c96badb

Errata RP2350-E5: an enabled channel can retrigger after the abort.

zephyr-cp: the async_spi test requires the ticker to run during the t…

629e8a4

…ransfers "ticker ran" once also passed with a blocking transfer; now it has to run more than 20 times during 10 transfers of 4 KB at 1 MHz (about 330 ms).

tannewt requested changes Oct 14, 2026

View reviewed changes

lynt-smitka added 7 commits October 15, 2026 07:16

build: one flag, CIRCUITPY_ASYNC_SPI, for a port with async SPI

b3d34bd

A port that has it turns it on in its mpconfigport.mk; busio.SPI is then always built on the same implementation and FourWire's write_start() follows. The separate CIRCUITPY_BUSIO_SPI_OVER_ASYNC is gone.

build: pick busio/SPI.c's home through the source lists

2e3aa3e

Like touchio: busio/SPI.c goes to SRC_SHARED_MODULE_ALL on a port with async SPI and to SRC_COMMON_HAL_ALL otherwise, instead of filtering the expanded list.

busio: a blocking SPI call stops on Ctrl-C

271634b

The wait for the transfer or configure checks the interrupt character, cancels the operation and raises the KeyboardInterrupt instead of finishing it.

raspberrypi: the SPI driver lives in common-hal/async_spi

5f96f7b

busio.SPI is built on it, so the rp2_spi_* layer in peripherals/ had one user. Its functions are the common_hal_async_spi_spi_* ones now and its fields are in the object.

async_spi: fix the lock docstring's literal block

8dbae29

tannewt requested changes Oct 19, 2026

View reviewed changes

Comment thread shared-bindings/async_spi/SPI.c Outdated

Comment thread shared-bindings/async_spi/SPI.c Outdated

Comment thread shared-module/busio/SPI.c

Comment thread shared-module/busio/SPI.c Outdated

Comment thread shared-module/busio/SPI.c

Comment thread shared-module/busio/SPI.h Outdated

lynt-smitka added 6 commits October 20, 2026 06:14

async_spi: the lock comes from common-hal

447d787

common-hal owns the object's fields, so the binding asks it for the lock.

busio: use the async_spi lock

c679a9a

try_lock, has_lock and unlock work on the lock inside the async_spi object, so a bus used from C and from asyncio has one lock.

busio: configure() no longer ends a pending write_start

1a8015b

A transaction ends its transfers before the bus is reconfigured.

busio: polarity and phase come from async_spi

f9e0cf6

The async_spi object already holds the bus settings, so busio no longer keeps a copy.

async_spi: deinit() wakes the tasks waiting for the lock

cf470ae

A task waiting in `async with spi.lock` would otherwise wait forever once the bus is deinitialized; now it gets the lock and its transfer raises the deinitialized error.

tannewt requested changes Oct 20, 2026

View reviewed changes

		// End the previous transfer first: an async write may still be receiving into one_byte.
		_end(self);

Conversation

lynt-smitka commented Oct 2, 2026

Uh oh!

tannewt left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lynt-smitka commented Oct 3, 2026

Uh oh!

tannewt commented Oct 5, 2026

Uh oh!

Uh oh!

tannewt left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tannewt left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

lynt-smitka commented Oct 14, 2026

Uh oh!

tannewt left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tannewt left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tannewt left a comment

Choose a reason for hiding this comment

Uh oh!

tannewt Oct 20, 2026

Choose a reason for hiding this comment

Uh oh!

tannewt Oct 20, 2026

Choose a reason for hiding this comment

Uh oh!

tannewt Oct 20, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants