Skip to content

Block Processor: Fix extraction after an inner void block - #14104

Closed
sirreal wants to merge 19 commits into
WordPress:trunkfrom
sirreal:block-processor/extract-void
Closed

sirreal wants to merge 19 commits into
WordPress:trunkfrom
sirreal:block-processor/extract-void

Conversation

@sirreal

@sirreal sirreal commented Oct 8, 2026 •

Copy link
Copy Markdown
Member

Trac ticket: Core-66138.
Follow-up to r60939.

WP_Block_Processor::extract_full_block_and_advance() advances past void blocks but stops on the closing delimiter of other blocks. A subsequent next_block() therefore skips the block after a void block:

$processor = new WP_Block_Processor( $html );
while ( $processor->next_block() ) {
	echo $processor->extract_full_block_and_advance()['blockName'], "\n";
}
$html Prints
<!-- wp:a --><!-- /wp:a --><!-- wp:b /--> core/a, core/b
<!-- wp:a /--><!-- wp:b /--> core/a

At the top level, a caller can work around this: after extracting a void block, the processor is already on the next token. Nested extraction has no workaround. The method's own loop advances past the token after an inner void block, so the returned tree drops it:

$html Inner blocks of g, before After
<!-- wp:g --><!-- wp:v --><!-- /wp:v --><!-- wp:w /--><!-- /wp:g --> core/v, core/w core/v, core/w
<!-- wp:g --><!-- wp:v /--><!-- wp:w /--><!-- /wp:g --> core/v core/v, core/w

The loop no longer advances after extracting an inner void block. Extraction still stops on the closing delimiter, on the freeform HTML span, or on the token after a void block. A block whose last inner block is void now follows that rule; before, it stopped on the token after its closing delimiter. The top-level next_block() loop is unchanged. The redundant HTML check after recursive extraction is removed, so incomplete input no longer copies an inner block's trailing HTML into its parent.

Tests cover nested void blocks, incomplete input, and the stop position for each kind of block. 5 of the 14 new cases fail on trunk.

Trac ticket: https://core.trac.wordpress.org/ticket/66138

Use of AI Tools

AI assistance: Yes
Tools: Claude Code, Codex
Models: Claude Opus 5.5, GPT-6
Used for: Investigation, implementation, tests, and description editing. Codex reviewed the diff and edited the description.


This Pull Request is for code review only. Please keep all other discussion in the Trac ticket. Do not merge this Pull Request. See GitHub Pull Requests for Code Review in the Core Handbook for more details.

`WP_Block_Processor::extract_full_block_and_advance()` stopped on the token after a void block, but on the last token of every other block. The next advance then skipped that token: a `next_block()` loop dropped the block or freeform HTML after a top-level void block, and extraction left the block after an inner void block out of the result.

Extraction now stops on the last token of the extracted block in every case. Remove the check for an HTML span after an inner extraction: it handled the old void case and copied inner HTML at the end of incomplete input into the parent block. Update the docblocks to state where extraction stops.

See #66138.
@github-actions

github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown

Test using WordPress Playground

The changes in this pull request can previewed and tested using a WordPress Playground instance.

WordPress Playground is an experimental project that creates a full WordPress instance entirely within the browser.

Some things to be aware of

  • All changes will be lost when closing a tab with a Playground instance.
  • All changes will be lost when refreshing the page.
  • A fresh instance is created each time the link below is clicked.
  • Every time this pull request is updated, a new ZIP file containing all changes is created. If changes are not reflected in the Playground instance,
    it's possible that the most recent build failed, or has not completed. Check the list of workflow runs to be sure.

For more details about these limitations and more, check out the Limitations page in the WordPress Playground documentation.

Test this pull request with WordPress Playground.

The class docblock's depth walk continued while `get_depth() > $depth`, so it stopped on the closing delimiter of the first inner block. Starting on `gallery` in `<!-- wp:gallery --><!-- wp:image --><!-- /wp:image --><!-- /wp:gallery -->`, it stopped on `<!-- /wp:image -->` with the gallery still open. Continue while `get_depth() >= $depth` so the walk stops on the block's own closing delimiter.

See #66138.
When the input ends in a partial delimiter, `extract_full_block_and_advance()` can leave the processor on the final HTML span, and the extracted block can omit content at the end of the document. The docblock said `get_span()` returns `null` whenever the document ends before the block closes. Qualify that statement for incomplete input, and return the original content from both splice examples when `get_last_error()` reports an error. Without the check, the examples rewrite `<!-- wp:gallery -->y<!--` as `<!-- wp:gallery /-->`.

The incomplete-input test now asserts the error, the stop position, and that the starting block was found.

See #66138.
Add the missing semicolon after continue so the documented example compiles.

See #66138.
@sirreal
sirreal requested a review from dmsnell October 8, 2026 18:16

@dmsnell dmsnell left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sirreal would you mind speaking the change of intentional behavior, where the _and_advance() is communicated that this skips past the open block.

I’m not saying it’s wrong, but I think it’s changing the contract rather than fixing the defect (that this doesn't advance past open blocks).

I think the root cause is that once we reach the closing block delimiter, we are past the open block; it’s closed before we pause at it.

granted, I think the spirit of this PR upholds that; the goal if I remember was to be able to call next_token() immediately after extract_full_block_and_advance(), but I’m kind of surprised at the scope of the fix here, thinking it shouldn’t be this big.

I wonder if we do something like advance to a zero-width HTML span.

also, I think the current guidance is that one ought not call extract_full_block_and_advance() on VOID block delimiters. that may not be great, but allocate_and_parse_attributes() gives everything and there’s no need to do more.

perhaps the best realization of this fix is simply to remain at the same spot on a VOID block delimiter after extracting it.

either way I think we need to be careful here to be intentional about the fix. this is already too confusing as is. thank you!

* $after_gallery = $processor->get_span()->start;
* if ( null !== $processor->get_last_error() ) {
* return $post_content;
* }

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for this and above, I wonder if this level of error-checking is more distracting than helpful.

* them in place.
*
* Once this function returns, the parser will be matched on token following the close
* of the given block.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this has always been confusing to me, but this PR seems to be changing behavior, claiming that VOID block delimiters are wrong, but according to this old description, the VOID block delimiters are the only ones that are right

Extracting a void block again stops on the token after it, as in 6.9, instead of on its delimiter. Blocks with inner content still stop on their closing delimiter, and freeform HTML on its span.

Extraction skipped the token after an inner void block, because the inner extraction had already advanced to it and the loop advanced again. The loop now does not advance after extracting an inner void block. A group's inner blocks include every block after a void inner block, and extraction of a group that ends in a void inner block stops on the group's closing delimiter.

A `next_block()` loop still skips the token after a top-level void block, so those cases are removed from the `parse_blocks()` comparison. The splice examples take the end of a void block from its own delimiter.

See #66138.
Add stop-position cases for a block whose input ends after a void inner block: at the end of the document, and before incomplete input. In both, the loop does not advance after the inner void extraction, and must still stop.

See #66138.
@sirreal sirreal changed the title Block Processor: Stop extraction on a void block's delimiter Block Processor: Fix extraction after an inner void block Oct 9, 2026
Revert this branch's changes to the class and method docblocks, their examples, and the inline comment on the advance flag. The comment block for the removed HTML check after recursive extraction stays removed with that check. No code changes.

See #66138.
Add `void` return types to the two new tests, and the `array` return type and row shape to their data provider.

See #66138.
@sirreal

sirreal commented Oct 9, 2026 •

Copy link
Copy Markdown
Member Author

Good feedback, thank you. The behavior on void blocks could probably use an example of how to loop properly, but I agree that we don't want to change behavior or contracts.

I've reworked this to be a minimal change only fixing the defect and not changing any public behaviors.

@sirreal
sirreal requested a review from dmsnell October 9, 2026 14:17
@sirreal
sirreal marked this pull request as ready for review October 16, 2026 09:51
@github-actions

github-actions Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs.

Core Committers: Use this line as a base for the props when committing in SVN:

Props jonsurrell, dmsnell, westonruter, lancewillett.

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.


Props Bot commands

You can trigger Props Bot actions by commenting on this PR:

  • @propsbot add <username> to give props to a GitHub user. Requires write access.
  • @propsbot remove <username> to leave a user out of the props. Requires write access.
  • @propsbot refresh to refresh this list. Adding the props-bot label works too.

@sirreal
sirreal requested a review from dmsnell October 27, 2026 12:21
@github-actions

github-actions Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

⚠️ PHPStan Static Analysis

PHPStan reported 3 errors when analyzing 62310be (workflow run). This comment is updated as new commits are pushed.

Errors

These are also shown as annotations on the changed files. They do not fail the check, but they should still be fixed before the pull request is committed. An error that is a false positive may instead warrant being added to a baseline.

Location Error Identifier First reported
src/wp-includes/class-wp-block-processor.php:1300 Method WP_Block_Processor::extract_full_block_and_advance() should return array<array>|null but returns array<string, list<string|null>|string|null>. return.type 2f580c0
src/wp-includes/class-wp-block-processor.php:1318 Method WP_Block_Processor::extract_full_block_and_advance() should return array<array>|null but returns array<string, array|string|null>. return.type 6dea7ce
src/wp-includes/class-wp-block-processor.php:1343 Method WP_Block_Processor::extract_full_block_and_advance() should return array<array>|null but returns array<string, array|string|null>. return.type 2f580c0
Resolved errors (2)
Location Error Identifier First reported Resolved
src/wp-includes/class-wp-block-processor.php:1318 Ignored error pattern #^Method WP_Block_Processor\:\:extract_full_block_and_advance\(\) should return array\<array\>\|null but returns array\<string, array\|string\|null\>\.$# (return.type) in path /home/runner/work/wordpress-develop/wordpress-develop/src/wp-includes/class-wp-block-processor.php is expected to occur 1 time, but occurred 2 times. ignore.count 6dea7ce 2f580c0
src/wp-includes/class-wp-block-processor.php:1330 Negated boolean expression is always true. booleanNot.alwaysTrue 93aada3 257d291

Running PHPStan locally

To run the analysis on your own machine, use npm run typecheck:php, or composer phpstan outside the Docker environment.

@dmsnell

dmsnell commented Oct 28, 2026

Copy link
Copy Markdown
Member

@sirreal I pushed 93aada3 to this branch after spending some time trying to understand the fix, and I think I finally have a way to express what the expectation should be for this function.

I flipped the flag so that we’re making a positive check rather than negating one. initially I renamed $should_advance to $skip_past_closer, inverted the while condition, and then struggled to understand why we should only skip past the closer if we’re not on a void delimiter.

Then I realized I had the condition backwards in my head. We’re not speaking about the closer so much as we’re asking if we are already where we need to be for the next loop iteration, and short-circuiting next_token(). This is of course logically the same as saying ! $should_advance || $this->next_token() but I find this other construction more natural $skip_next_advance || $this->next_token() (either we stay put, or we advance).

Maybe we could still rename this $stay_put

Then I didn’t like the split assignments throughout the loop so I lifted it to the first line. Now it only needs a single assignment and is hopefully more difficult to miss in any follow-up work.

@dmsnell dmsnell left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sirreal you can revert my change or leave it. I have reviewed this and consider it good to go; best to get it in for WordPress 7.2

@westonruter thanks for the impure patch. I am so sad every time I see that, but I don’t want to hold up this patch on account of it.

@dmsnell

dmsnell commented Oct 28, 2026

Copy link
Copy Markdown
Member

oops: I found another failing case by trying to use this for WP_Block_Parser::parse() — will post an update soon. I think it’s the same problem, but we only applied the fix for inner blocks and it should apply to top-level voids as well

dmsnell and others added 3 commits October 28, 2026 15:34
Remove redundant wrapper checks and repeated recovery-mode cases. Strengthen mail assertions to check observable message content.

Developed in: WordPress#15023

Props huzaifaalmesbah.
See #65819.


git-svn-id: https://develop.svn.wordpress.org/trunk@64983 602fd350-edb4-49c9-b593-d223f7449a82
@dmsnell

dmsnell commented Oct 28, 2026

Copy link
Copy Markdown
Member

@sirreal I returned to my previous comment and updated the tests you added, which brings the behaviors back to something we discussed, but was awash in the evolution of this patch

the way it stands now, the class is designed to support this exact flow

	public function parse( $document ) {
		$this->document = $document;
		$this->offset   = 0;
		$this->output   = array();
		$this->stack    = array();

		$this->block_processor = new WP_Block_Processor( $document );

		while ( $this->block_processor->next_block( '*' ) ) {
			$this->output[] = $this->block_processor->extract_full_block_and_advance();
		}

		return $this->output;
	}

and I have this passing all tests locally when I replace the WP_Block_Parser code with that. I think this is right, and the original goal of the method, as evidenced by the test code in test_extracts_equivalent_parses_as_parse_blocks()

wporg-sync pushed a commit that referenced this pull request Oct 28, 2026
When the Block Processor was introduced, the `extract_full_block_and_advance()` method was provided in order to support backwards compatibility with `parse_blocks()`, specifically when wanting to extract the structural representation of a subset of blocks within a large post.

Unfortunately, when extracting void blocks, it was advancing to the next token, which, if called from within a loop that was already advancing, would lead to a skipped block delimiter.

In this patch the behavior is normalized such that, after a call to `extract_full_block_and_advance()`, the calling code should call `next_token()` to continue processing the next block or top-level freeform HTML in the post.

Developed in: #14104
Discussed in: https://core.trac.wordpress.org/ticket/66138

Follow-up to r60939.

Props dmsnell, jonsurrell, westonruter.
See #66138.


git-svn-id: https://develop.svn.wordpress.org/trunk@64994 602fd350-edb4-49c9-b593-d223f7449a82
wporg-sync pushed a commit to WordPress/WordPress that referenced this pull request Oct 28, 2026
When the Block Processor was introduced, the `extract_full_block_and_advance()` method was provided in order to support backwards compatibility with `parse_blocks()`, specifically when wanting to extract the structural representation of a subset of blocks within a large post.

Unfortunately, when extracting void blocks, it was advancing to the next token, which, if called from within a loop that was already advancing, would lead to a skipped block delimiter.

In this patch the behavior is normalized such that, after a call to `extract_full_block_and_advance()`, the calling code should call `next_token()` to continue processing the next block or top-level freeform HTML in the post.

Developed in: WordPress/wordpress-develop#14104
Discussed in: https://core.trac.wordpress.org/ticket/66138

Follow-up to r60939.

Props dmsnell, jonsurrell, westonruter.
See #66138.

Built from https://develop.svn.wordpress.org/trunk@64994


git-svn-id: http://core.svn.wordpress.org/trunk@64144 1a063a9b-81f0-0310-95a4-ce76da25c4cd
@dmsnell

dmsnell commented Oct 28, 2026

Copy link
Copy Markdown
Member

Merged in r64994
30b32ec

@dmsnell dmsnell closed this Oct 28, 2026
@sirreal
sirreal deleted the block-processor/extract-void branch October 29, 2026 10:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants