General: Introduce output buffering for template enhancements.

This introduces an output buffer for the entire template rendering process. This allows for post-processing of the complete HTML output via filtering before it is sent to the browser. This is primarily intended for performance optimizations and other progressive enhancements. Extenders must not rely on output buffer processing for critical content and functionality since a site may opt out of output buffering for the sake of streaming. Extenders are heavily encouraged to use the HTML API as opposed to using regular expressions in output buffer filters. 

* A new `wp_before_include_template` action is introduced, which fires immediately before the template file is included. This is useful on its own, as it avoids the need to misuse `template_include` filter to run logic right before the template is loaded (e.g. sending a `Server-Timing` header).
* The `wp_start_template_enhancement_output_buffer()` function is hooked to this new action. It starts an output buffer, but only if there are `wp_template_enhancement_output_buffer` filters present, or else if there is an explicit opt-in via the `wp_should_output_buffer_template_for_enhancement` filter.
* The `wp_finalize_template_enhancement_output_buffer()` function serves as the output buffer callback. It applies `wp_template_enhancement_output_buffer` filters to the buffered content if the response is identified as HTML. 
* The output buffer callback passes through (without filtering) any content for non-HTML responses, identified by the `Content-Type` response header.
* This provides a standardized way for plugins (and core) to perform optimizations, such as removing unused CSS, without each opening their own ad hoc output buffer.

Developed in https://github.com/WordPress/wordpress-develop/pull/8412.

Props westonruter, nextendweb, dmsnell, flixos90, jorbin, peterwilsoncc, swissspidy, DrewAPicture, DaanvandenBergh, OptimizingMatters, tabrisrp, jonoaldersonwp, SergeyBiryukov.
Fixes #43258.

Built from https://develop.svn.wordpress.org/trunk@60936


git-svn-id: http://core.svn.wordpress.org/trunk@60272 1a063a9b-81f0-0310-95a4-ce76da25c4cd

wp-includes/default-filters.php

@@ -422,6 +422,7 @@ add_action( 'do_all_pings', 'do_all_trackbacks', 10, 0 );
 add_action( 'do_all_pings', 'generic_ping', 10, 0 );
 add_action( 'do_robots', 'do_robots' );
 add_action( 'do_favicon', 'do_favicon' );
+add_action( 'wp_before_include_template', 'wp_start_template_enhancement_output_buffer' );
 add_action( 'set_comment_cookies', 'wp_set_comment_cookies', 10, 3 );
 add_action( 'sanitize_comment_cookies', 'sanitize_comment_cookies' );
 add_action( 'init', 'smilies_init', 5 );

wp-includes/template-loader.php

@@ -113,6 +113,15 @@ if ( wp_using_themes() ) {
 	 */
 	$template = apply_filters( 'template_include', $template );
 	if ( $template ) {
+		/**
+		 * Fires immediately before including the template.
+		 *
+		 * @since 6.9.0
+		 *
+		 * @param string $template The path of the template about to be included.
+		 */
+		do_action( 'wp_before_include_template', $template );
+
 		include $template;
 	} elseif ( current_user_can( 'switch_themes' ) ) {
 		$theme = wp_get_theme();

wp-includes/template.php

@@ -823,3 +823,144 @@ function load_template( $_template_file, $load_once = true, $args = array() ) {
 	 */
 	do_action( 'wp_after_load_template', $_template_file, $load_once, $args );
 }
+
+/**
+ * Checks whether the template should be output buffered for enhancement.
+ *
+ * By default, an output buffer is only started if a {@see 'wp_template_enhancement_output_buffer'} filter has been
+ * added be the time a template is included at the {@see 'wp_before_include_template'} action. This allows template
+ * responses to be streamed as much as possible when no template enhancements are registered to apply.
+ *
+ * @since 6.9.0
+ *
+ * @return bool Whether the template should be output-buffered for enhancement.
+ */
+function wp_should_output_buffer_template_for_enhancement(): bool {
+	/**
+	 * Filters whether the template should be output-buffered for enhancement.
+	 *
+	 * By default, an output buffer is only started if a {@see 'wp_template_enhancement_output_buffer'} filter has been
+	 * added. For this default to apply, a filter must be added by the time the template is included at the
+	 * {@see 'wp_before_include_template'} action. This allows template responses to be streamed as much as possible
+	 * when no template enhancements are registered to apply. This filter allows a site to opt in to adding such
+	 * template enhancement filters during the rendering of the template.
+	 *
+	 * @since 6.9.0
+	 *
+	 * @param bool $use_output_buffer Whether an output buffer is started.
+	 */
+	return (bool) apply_filters( 'wp_should_output_buffer_template_for_enhancement', has_filter( 'wp_template_enhancement_output_buffer' ) );
+}
+
+/**
+ * Starts the template enhancement output buffer.
+ *
+ * This function is called immediately before the template is included.
+ *
+ * @since 6.9.0
+ *
+ * @return bool Whether the output buffer successfully started.
+ */
+function wp_start_template_enhancement_output_buffer(): bool {
+	if ( ! wp_should_output_buffer_template_for_enhancement() ) {
+		return false;
+	}
+
+	$started = ob_start(
+		'wp_finalize_template_enhancement_output_buffer',
+		0, // Unlimited buffer size so that entire output is passed to the filter.
+		/*
+		 * Instead of the default PHP_OUTPUT_HANDLER_STDFLAGS (cleanable, flushable, and removable) being used for
+		 * flags, the PHP_OUTPUT_HANDLER_FLUSHABLE flag must be omitted. If the buffer were flushable, then each time
+		 * that ob_flush() is called, a fragment of the output would be sent into the output buffer callback. This
+		 * output buffer is intended to capture the entire response for processing, as indicated by the chunk size of 0.
+		 * So the buffer does not allow flushing to ensure the entire buffer can be processed, such as for optimizing an
+		 * entire HTML document, where markup in the HEAD may need to be adjusted based on markup that appears late in
+		 * the BODY.
+		 *
+		 * If this ends up being problematic, then PHP_OUTPUT_HANDLER_FLUSHABLE could be added to the $flags and the
+		 * output buffer callback could check if the phase is PHP_OUTPUT_HANDLER_FLUSH and abort any subsequent
+		 * processing while also emitting a _doing_it_wrong().
+		 *
+		 * The output buffer needs to be removable because WordPress calls wp_ob_end_flush_all() and then calls
+		 * wp_cache_close(). If the buffers are not all flushed before wp_cache_close() is closed, then some output buffer
+		 * handlers (e.g. for caching plugins) may fail to be able to store the page output in the object cache.
+		 * See <https://github.com/WordPress/performance/pull/1317#issuecomment-2271955356>.
+		 */
+		PHP_OUTPUT_HANDLER_STDFLAGS ^ PHP_OUTPUT_HANDLER_FLUSHABLE
+	);
+
+	if ( $started ) {
+		/**
+		 * Fires when the template enhancement output buffer has started.
+		 *
+		 * @since 6.9.0
+		 */
+		do_action( 'wp_template_enhancement_output_buffer_started' );
+	}
+
+	return $started;
+}
+
+/**
+ * Finalizes the template enhancement output buffer.
+ *
+ * Checks to see if the output buffer is complete and contains HTML. If so, runs the content through
+ * the `wp_template_enhancement_output_buffer` filter.  If not, the original content is returned.
+ *
+ * @since 6.9.0
+ *
+ * @see wp_start_template_enhancement_output_buffer()
+ *
+ * @param string $output Output buffer.
+ * @param int    $phase  Phase.
+ * @return string Finalized output buffer.
+ */
+function wp_finalize_template_enhancement_output_buffer( string $output, int $phase ): string {
+	// When the output is being cleaned (e.g. pending template is replaced with error page), do not send it through the filter.
+	if ( ( $phase & PHP_OUTPUT_HANDLER_CLEAN ) !== 0 ) {
+		return $output;
+	}
+
+	// Detect if the response is an HTML content type.
+	$is_html_content_type = null;
+	$html_content_types   = array( 'text/html', 'application/xhtml+xml' );
+	foreach ( headers_list() as $header ) {
+		$header_parts = preg_split( '/\s*[:;]\s*/', strtolower( $header ) );
+		if (
+			is_array( $header_parts ) &&
+			count( $header_parts ) >= 2 &&
+			'content-type' === $header_parts[0]
+		) {
+			$is_html_content_type = in_array( $header_parts[1], $html_content_types, true );
+			break; // PHP only sends the first Content-Type header in the list.
+		}
+	}
+	if ( null === $is_html_content_type ) {
+		$is_html_content_type = in_array( ini_get( 'default_mimetype' ), $html_content_types, true );
+	}
+
+	// If the content type is not HTML, short-circuit since it is not relevant for enhancement.
+	if ( ! $is_html_content_type ) {
+		return $output;
+	}
+
+	$filtered_output = $output;
+
+	/**
+	 * Filters the template enhancement output buffer prior to sending to the client.
+	 *
+	 * This filter only applies the HTML output of an included template. This filter is a progressive enhancement
+	 * intended for applications such as optimizing markup to improve frontend page load performance. Sites must not
+	 * depend on this filter applying since they may opt to stream the responses instead. Callbacks for this filter are
+	 * highly discouraged from using regular expressions to do any kind of replacement on the output. Use the HTML API
+	 * (either `WP_HTML_Tag_Processor` or `WP_HTML_Processor`), or else use {@see DOM\HtmlDocument} as of PHP 8.4 which
+	 * fully supports HTML5.
+	 *
+	 * @since 6.9.0
+	 *
+	 * @param string $filtered_output HTML template enhancement output buffer.
+	 * @param string $output          Original HTML template output buffer.
+	 */
+	return (string) apply_filters( 'wp_template_enhancement_output_buffer', $filtered_output, $output );
+}

wp-includes/version.php

@@ -16,7 +16,7 @@
  *
  * @global string $wp_version
  */
-$wp_version = '6.9-alpha-60935';
+$wp_version = '6.9-alpha-60936';
 
 /**
  * Holds the WordPress DB revision, increments when changes are made to the WordPress DB schema.