HTML API: Fix void tag nesting with next_token

When `next_token()` was introduced, it introduced a regression in the HTML
Processor whereby void tags remain on the stack of open elements when they
shouldn't. This led to invalid values returned from `get_breadcrumbs()`.

The reason was that calling `next_token()` works through a different code path
than the HTML Processor runs everything else. To solve this, its sub-classed
`next_token()` called `step( self::REPROCESS_CURRENT_TOKEN )` so that the proper
HTML accounting takes place.

Unfortunately that same reprocessing code path skipped the step whereby void
and self-closing elements are popped from the stack of open elements.

In this patch, that step is run with a third mode for `step()`, which is the
new `self::PROCESS_CURRENT_TOKEN`. This mode acts as if `self::PROCESS_NEXT_NODE`
were called, except it doesn't advance the parser.

Developed in WordPress/wordpress-develop#5975
Discussed in https://core.trac.wordpress.org/ticket/60382

Follow-up to [57348]

Props dmsnell, jonsurrell
Fixes #60382


Built from https://develop.svn.wordpress.org/trunk@57507


git-svn-id: https://core.svn.wordpress.org/trunk@57008 1a063a9b-81f0-0310-95a4-ce76da25c4cd

Author: dmsnell

wp-includes/html-api/class-wp-html-processor.php

@@ -431,7 +431,7 @@ public function next_token() {
 		$found_a_token = parent::next_token();
 
 		if ( '#tag' === $this->get_token_type() ) {
-			$this->step( self::REPROCESS_CURRENT_NODE );
+			$this->step( self::PROCESS_CURRENT_NODE );
 		}
 
 		return $found_a_token;
@@ -513,7 +513,7 @@ public function step( $node_to_process = self::PROCESS_NEXT_NODE ) {
 			return false;
 		}
 
-		if ( self::PROCESS_NEXT_NODE === $node_to_process ) {
+		if ( self::REPROCESS_CURRENT_NODE !== $node_to_process ) {
 			/*
 			 * Void elements still hop onto the stack of open elements even though
 			 * there's no corresponding closing tag. This is important for managing
@@ -532,7 +532,9 @@ public function step( $node_to_process = self::PROCESS_NEXT_NODE ) {
 			if ( $top_node && self::is_void( $top_node->node_name ) ) {
 				$this->state->stack_of_open_elements->pop();
 			}
+		}
 
+		if ( self::PROCESS_NEXT_NODE === $node_to_process ) {
 			while ( parent::next_token() && '#tag' !== $this->get_token_type() ) {
 				continue;
 			}
@@ -1781,6 +1783,15 @@ public static function is_void( $tag_name ) {
 	 */
 	const REPROCESS_CURRENT_NODE = 'reprocess-current-node';
 
+	/**
+	 * Indicates that the current HTML token should be processed without advancing the parser.
+	 *
+	 * @since 6.5.0
+	 *
+	 * @var string
+	 */
+	const PROCESS_CURRENT_NODE = 'process-current-node';
+
 	/**
 	 * Indicates that the parser encountered unsupported markup and has bailed.
 	 *

wp-includes/version.php

@@ -16,7 +16,7 @@
  *
  * @global string $wp_version
  */
-$wp_version = '6.5-alpha-57506';
+$wp_version = '6.5-alpha-57507';
 
 /**
  * Holds the WordPress DB revision, increments when changes are made to the WordPress DB schema.