, bounded: bool, diagnostics: array} Prepared source. */ public static function prepare( WP_Post $post ) { $diagnostics = array(); $raw = is_string( $post->post_content ) ? $post->post_content : ''; if ( strlen( $raw ) > self::MAX_BYTES ) { /* * Over the bound the content is not parsed at all. Article still * generates from post metadata; FAQ and HowTo need parsing and so * are unavailable, which is what the diagnostic records. */ return array( 'source' => '', 'outside' => '', 'regions' => array(), 'bounded' => false, 'diagnostics' => array( self::DIAGNOSTIC_TOO_LARGE ), ); } $source = strip_shortcodes( $raw ); /** * Filters the bounded source string the schema generator parses. * * Receives the stored post content after shortcode removal, never * rendered output. A filter must return a string no longer than * `SchemaSource::MAX_BYTES`; anything else is ignored and the unfiltered * bounded source is used, with a `source_filter_ignored` diagnostic. * * @since 1.3.0 * * @param string $source Bounded, shortcode-stripped source content. * @param int $post_id Post being evaluated. */ $filtered = apply_filters( 'aive_schema_source_content', $source, (int) $post->ID ); if ( ! is_string( $filtered ) || strlen( $filtered ) > self::MAX_BYTES ) { $diagnostics[] = self::DIAGNOSTIC_FILTER_IGNORED; } else { $source = $filtered; } $split = self::split_regions( $source ); if ( $split['ignored'] ) { $diagnostics[] = self::DIAGNOSTIC_MARKERS_IGNORED; } return array( 'source' => $source, 'outside' => $split['outside'], 'regions' => $split['regions'], 'bounded' => true, 'diagnostics' => $diagnostics, ); } /** * Splits a source string into valid marker regions and everything else. * * The marker grammar is `` / `` * and the same for `howto`. Comments were chosen because they survive the * Classic editor, the Code editor and a Gutenberg Custom HTML block without * any of them rewriting the markup, which a custom tag or a shortcode would * not. * * Pairing is positional and deterministic: markers are read in document * order and a start pairs with the next end of the same type. A start * followed by another start of any type before its end is crossed or nested * and both are discarded; an unmatched start or end is discarded; a region * over `MAX_REGION_BYTES` is discarded. "Discarded" means discarded *as a * region* - the content stays in the outside stream and automatic detection * can still use it, which is the behaviour the contract asks for. * * Splitting on byte offsets can cut an element in half when a marker sits * inside one rather than between two. The resulting fragment is malformed, * libxml recovers what it can, and a candidate whose required structure the * recovery cannot establish is skipped - the same path any other malformed * content takes. * * @since 1.3.0 * * @param string $source Bounded source content. * @return array{outside: string, regions: array, ignored: bool} Regions and the remaining source. */ private static function split_regions( $source ) { $markers = self::find_markers( $source ); if ( array() === $markers ) { return array( 'outside' => $source, 'regions' => array(), 'ignored' => false, ); } $regions = array(); $ignored = false; $open = null; foreach ( $markers as $marker ) { if ( 'start' === $marker['edge'] ) { if ( null !== $open ) { /* * A second start before the first closed. Which of the two * the author meant is unknowable, so neither is honoured: * the open one is dropped and this one is not opened, which * leaves the following end with no start to pair with and * discards it too. Both nesting and crossing land here. */ $ignored = true; $open = null; continue; } $open = $marker; continue; } if ( null === $open || $open['type'] !== $marker['type'] ) { // An end with no open start of its type. $ignored = true; continue; } $inner_start = $open['end']; $inner_end = $marker['start']; $length = $inner_end - $inner_start; if ( $length > 0 && $length <= self::MAX_REGION_BYTES ) { $regions[] = array( 'type' => $open['type'], 'html' => substr( $source, $inner_start, $length ), 'start' => $open['start'], 'stop' => $marker['end'], ); } else { $ignored = true; } $open = null; } if ( null !== $open ) { // An unmatched start. $ignored = true; } return array( 'outside' => self::outside_source( $source, $regions ), 'regions' => self::without_offsets( $regions ), 'ignored' => $ignored, ); } /** * Returns the source with every valid region and every marker removed. * * Valid regions leave the outside stream entirely: their content belongs to * the explicit category that claimed it and must not supply an automatic * candidate as well. Markers of discarded regions are removed too, because * the contract requires markers to be gone before text extraction whether or * not they delimited anything. * * @since 1.3.0 * * @param string $source Bounded source content. * @param array $regions Valid regions with their offsets. * @return string Source outside every valid region, markers removed. */ private static function outside_source( $source, array $regions ) { $outside = ''; $cursor = 0; foreach ( $regions as $region ) { $outside .= substr( $source, $cursor, $region['start'] - $cursor ); $cursor = $region['stop']; } $outside .= substr( $source, $cursor ); return self::strip_markers( $outside ); } /** * Removes every AIVE schema marker comment from a string. * * @since 1.3.0 * * @param string $html Source fragment. * @return string Fragment without marker comments. */ public static function strip_markers( $html ) { return (string) preg_replace( self::marker_pattern(), '', (string) $html ); } /** * Returns every marker occurrence in a string, in document order. * * @since 1.3.0 * * @param string $source Bounded source content. * @return array Markers in document order. */ private static function find_markers( $source ) { $found = preg_match_all( self::marker_pattern(), (string) $source, $matches, PREG_OFFSET_CAPTURE ); if ( ! $found ) { return array(); } $markers = array(); foreach ( $matches[0] as $index => $whole ) { $type = strtolower( (string) $matches[1][ $index ][0] ); $edge = (string) $matches[2][ $index ][0]; $markers[] = array( 'type' => self::REGION_HOWTO === $type ? self::REGION_HOWTO : self::REGION_FAQ, 'edge' => $edge, 'start' => (int) $whole[1], 'end' => (int) $whole[1] + strlen( (string) $whole[0] ), ); } return $markers; } /** * Returns the marker pattern. * * `aive`, `start` and `end` are matched exactly; the inline `(?i:...)` group * makes only the type word case-insensitive, which is what the contract * specifies. Whitespace inside the comment is tolerated because editors add * it, and nothing else is. * * @since 1.3.0 * * @return string Regular expression. */ private static function marker_pattern() { return '//'; } /** * Drops the byte offsets from region records. * * @since 1.3.0 * * @param array $regions Regions with offsets. * @return array Regions without offsets. */ private static function without_offsets( array $regions ) { $clean = array(); foreach ( $regions as $region ) { $clean[] = array( 'type' => $region['type'], 'html' => $region['html'], ); } return $clean; } /** * Parses an HTML fragment with networking off and diagnostics collected. * * Separate from `Helpers::parse_html_fragment()` because the schema module * needs to know whether libxml recovered from anything, and the shared * helper deliberately discards that. Neither one renders or executes the * fragment. * * @since 1.3.0 * * @param string $html HTML fragment. * @return array{document: DOMDocument|null, recovered: bool} Parsed document and whether libxml reported problems. */ public static function parse( $html ) { $html = (string) $html; if ( '' === trim( $html ) ) { return array( 'document' => null, 'recovered' => false, ); } $document = new DOMDocument(); $previous_state = libxml_use_internal_errors( true ); libxml_clear_errors(); /* * The XML declaration forces UTF-8 without libxml guessing from a meta * tag a content fragment does not have. LIBXML_NONET is the flag that * makes an external entity reference fail rather than fetch. */ $loaded = $document->loadHTML( '' . $html . '', LIBXML_NONET | LIBXML_NOERROR | LIBXML_NOWARNING ); $errors = libxml_get_errors(); libxml_clear_errors(); libxml_use_internal_errors( $previous_state ); return array( 'document' => $loaded ? $document : null, 'recovered' => array() !== $errors, ); } }