[go: up one dir, main page]

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RecursiveCallbackFilterIterator regression in 8.1.18 #11972

Closed
TimBozeman opened this issue Aug 14, 2023 · 4 comments
Closed

RecursiveCallbackFilterIterator regression in 8.1.18 #11972

TimBozeman opened this issue Aug 14, 2023 · 4 comments

Comments

@TimBozeman
Copy link
TimBozeman commented Aug 14, 2023

Description

I think I've found a regression that was introduced in php-8.1.18. The following test case completes in 8.1.17, but hangs silently in 8.1.18 or any greater version. The use case is that I'm going through a really large render array in Drupal and leaving bread crumbs so that I can later find them in a RecursiveCallbackFilterIterator to break out of infinite loops.

The following simplified test case:

<?php

class RecursiveFilterTest {

  public function traverse(array $variables): array {

    $array_iterator = new \RecursiveArrayIterator($variables);
    $filter_iterator = new \RecursiveCallbackFilterIterator($array_iterator, [
      $this, 'isCyclic',
    ]);
    $recursive_iterator = new \RecursiveIteratorIterator($filter_iterator, \RecursiveIteratorIterator::SELF_FIRST);
    $recursive_iterator->setMaxDepth(20);
    foreach ($recursive_iterator as $value) {
    
      // Does stuff here...

      // Avoid recursion by marking where we've been.
      if (\is_object($value)) {
        $value->{'#override_mode_breadcrumb'} = TRUE;
      }
      if (\is_array($value)) {
        $value['#override_mode_breadcrumb'] = TRUE;
      }
    }
    return \iterator_to_array($recursive_iterator);
  }

  public function isCyclic($current, string $key, \RecursiveArrayIterator $iterator): bool {
    // Skip closures.
    if ($current instanceof \Closure) {
      return FALSE;
    }

    ########### Debugging code ################
    static $i = 0;
    $i++;
    if (is_array($current)) {
      // When $i > 319 calling isset causes the process to hang and you have to restart the webserver to get out of it.
      $this_fails = isset($current['#override_mode_breadcrumb']);
    }
    ###########################################

    // Avoid infinite loops by checking if we've been here before.
    // e.g. View > query > view > query ...
    if (\is_array($current) && isset($current['#override_mode_breadcrumb'])) {
      return FALSE;
    }
    if (\is_object($current) && isset($current->{'#override_mode_breadcrumb'})) {
      return FALSE;
    }
    return TRUE;
  }

}

$classes_removed_variables = unserialize('a:17:{s:7:"element";a:45:{s:5:"#type";s:6:"select";s:8:"#options";a:1:{s:16:"Other workspaces";a:3:{s:36:"eb57d455-4dba-4f29-a356-1ef8d75f976f";s:17:"Foundation Models";s:36:"7d3bd93b-f1c5-439a-a60b-fce9fddcf516";s:9:"John Test";s:5:"stage";s:5:"Stage";}}s:14:"#default_value";s:0:"";s:11:"#attributes";a:5:{s:8:"onchange";s:19:"this.form.submit();";s:20:"data-drupal-selector";s:17:"edit-workspace-id";s:5:"class";a:1:{i:0;s:11:"form-select";}s:2:"id";s:17:"edit-workspace-id";s:4:"name";s:12:"workspace_id";}s:19:"#wrapper_attributes";a:1:{s:5:"class";a:1:{i:0;s:16:"container-inline";}}s:6:"#input";b:1;s:9:"#multiple";b:0;s:13:"#sort_options";b:0;s:11:"#sort_start";i:0;s:8:"#process";a:4:{i:0;a:2:{i:0;s:33:"Drupal\\Core\\Render\\Element\\Select";i:1;s:13:"processSelect";}i:1;a:2:{i:0;s:33:"Drupal\\Core\\Render\\Element\\Select";i:1;s:15:"processAjaxForm";}i:2;a:2:{i:0;s:45:"Drupal\\inline_form_errors\\RenderElementHelper";i:1;s:14:"processElement";}i:3;a:2:{i:0;s:38:"Drupal\\bootstrap\\Plugin\\ProcessManager";i:1;s:7:"process";}}s:11:"#pre_render";a:2:{i:0;a:2:{i:0;s:33:"Drupal\\Core\\Render\\Element\\Select";i:1;s:15:"preRenderSelect";}i:1;a:2:{i:0;s:40:"Drupal\\bootstrap\\Plugin\\PrerenderManager";i:1;s:9:"preRender";}}s:6:"#theme";s:6:"select";s:15:"#theme_wrappers";a:1:{i:0;s:12:"form_element";}s:15:"#value_callback";a:2:{i:0;s:33:"Drupal\\Core\\Render\\Element\\Select";i:1;s:13:"valueCallback";}s:8:"#context";a:0:{}s:5:"#icon";N;s:14:"#icon_position";s:6:"before";s:10:"#icon_only";b:0;s:16:"#defaults_loaded";b:1;s:5:"#tree";b:0;s:8:"#parents";a:1:{i:0;s:12:"workspace_id";}s:14:"#array_parents";a:1:{i:0;s:12:"workspace_id";}s:7:"#weight";i:0;s:10:"#processed";b:1;s:9:"#required";b:0;s:14:"#title_display";s:6:"before";s:20:"#description_display";s:5:"after";s:7:"#errors";N;s:3:"#id";s:17:"edit-workspace-id";s:5:"#name";s:12:"workspace_id";s:6:"#value";s:0:"";s:15:"#ajax_processed";b:0;s:5:"#ajax";N;s:24:"#autocomplete_route_name";N;s:12:"#input_group";N;s:19:"#input_group_button";N;s:7:"#sorted";b:1;s:18:"#smart_description";b:0;s:6:"#cache";a:2:{s:4:"tags";a:0:{}s:7:"max-age";i:-1;}s:9:"#attached";a:0:{}s:9:"#children";s:0:"";s:16:"#render_children";b:1;s:10:"#has_error";b:0;s:13:"#field_prefix";N;s:13:"#field_suffix";N;}s:19:"theme_hook_original";s:6:"select";s:12:"title_prefix";a:0:{}s:12:"title_suffix";a:0:{}s:12:"db_is_active";b:1;s:8:"is_admin";b:1;s:9:"logged_in";b:1;s:9:"directory";s:49:"profiles/tacos/your_sites_profile/themes/my_theme";s:7:"options";a:2:{i:0;a:3:{s:8:"selected";b:1;s:4:"type";s:6:"option";s:5:"value";s:0:"";}i:1;a:3:{s:4:"type";s:8:"optgroup";s:5:"label";s:16:"Other workspaces";s:7:"options";a:3:{i:0;a:4:{s:8:"selected";b:0;s:4:"type";s:6:"option";s:5:"value";s:36:"eb57d455-4dba-4f29-a356-1ef8d75f976f";s:5:"label";s:17:"Foundation Models";}i:1;a:4:{s:8:"selected";b:0;s:4:"type";s:6:"option";s:5:"value";s:36:"7d3bd93b-f1c5-439a-a60b-fce9fddcf516";s:5:"label";s:9:"John Test";}i:2;a:4:{s:8:"selected";b:0;s:4:"type";s:6:"option";s:5:"value";s:5:"stage";s:5:"label";s:5:"Stage";}}}}s:8:"is_front";b:1;s:6:"#cache";a:1:{s:8:"contexts";a:1:{i:0;s:17:"url.path.is_front";}}s:5:"theme";a:30:{s:4:"name";s:8:"my_theme";s:4:"type";s:5:"theme";s:11:"description";s:21:"Sub-theme of houstons";s:7:"package";s:5:"Other";s:24:"core_version_requirement";s:10:"^8.8 || ^9";s:10:"base theme";s:8:"houstons";s:9:"libraries";a:1:{i:0;s:15:"my_theme/global";}s:18:"libraries-override";a:1:{s:14:"houstons/modal";s:14:"my_theme/modal";}s:7:"regions";a:15:{s:3:"top";s:11:"Top of Page";s:10:"navigation";s:10:"Navigation";s:22:"navigation_collapsible";s:24:"Navigation (Collapsible)";s:17:"navigation_search";s:17:"Navigation Search";s:4:"hero";s:11:"Hero Banner";s:6:"header";s:7:"Top Bar";s:11:"highlighted";s:11:"Highlighted";s:4:"help";s:4:"Help";s:7:"content";s:7:"Content";s:13:"sidebar_first";s:7:"Primary";s:14:"sidebar_second";s:9:"Secondary";s:10:"pre_footer";s:18:"Pre Footer Content";s:6:"footer";s:6:"Footer";s:8:"page_top";s:8:"Page top";s:11:"page_bottom";s:11:"Page bottom";}s:16:"libraries-extend";a:1:{s:29:"core/drupal.dialog.off_canvas";a:1:{i:0;s:19:"my_theme/off-canvas";}}s:20:"ckeditor_stylesheets";a:2:{i:0;s:43:"css/libraries/ckeditor/ckeditor-backend.css";i:1;s:35:"css/libraries/ckeditor/ckeditor.css";}s:17:"core_incompatible";b:0;s:9:"lifecycle";s:6:"stable";s:5:"mtime";i:1690914479;s:6:"engine";s:24:"component_library_engine";s:8:"features";a:5:{i:0;s:7:"favicon";i:1;s:4:"logo";i:2;s:17:"node_user_picture";i:3;s:20:"comment_user_picture";i:4;s:25:"comment_user_verification";}s:10:"screenshot";s:64:"profiles/tacos/your_sites_profile/themes/my_theme/screenshot.png";s:7:"version";N;s:3:"php";s:5:"7.3.0";s:16:"libraries_extend";a:0:{}s:18:"libraries_override";a:0:{}s:12:"dependencies";a:1:{i:0;s:8:"houstons";}s:14:"regions_hidden";a:2:{i:0;s:8:"page_top";i:1;s:11:"page_bottom";}s:3:"dev";b:0;s:10:"livereload";s:0:"";s:4:"path";s:49:"profiles/tacos/your_sites_profile/themes/my_theme";s:5:"title";s:23:"Your Sites Custom Theme";s:8:"settings";a:86:{s:5:"_core";a:1:{s:19:"default_config_hash";s:43:"EBskbP-6knEscPxbKQU9fI6FcAYffuwyrdEPRHD8LT4";}s:7:"favicon";a:4:{s:8:"mimetype";s:24:"image/vnd.microsoft.icon";s:4:"path";s:0:"";s:3:"url";s:62:"/profiles/tacos/your_sites_profile/themes/my_theme/favicon.ico";s:11:"use_default";i:1;}s:8:"features";a:4:{s:20:"comment_user_picture";b:1;s:25:"comment_user_verification";b:1;s:7:"favicon";i:1;s:17:"node_user_picture";b:0;}s:4:"logo";a:3:{s:4:"path";s:0:"";s:3:"url";s:0:"";s:11:"use_default";i:1;}s:7:"schemas";a:3:{s:9:"bootstrap";i:8000;s:8:"houstons";i:8000;s:8:"my_theme";i:8000;}s:15:"button_colorize";i:1;s:14:"button_iconize";i:1;s:11:"button_size";s:0:"";s:15:"fluid_container";i:0;s:28:"forms_has_error_value_toggle";i:1;s:24:"forms_required_has_error";i:0;s:24:"forms_smart_descriptions";i:1;s:37:"forms_smart_descriptions_allowed_tags";s:33:"b, code, em, i, kbd, span, strong";s:30:"forms_smart_descriptions_limit";s:3:"250";s:18:"image_lazy_loading";i:0;s:16:"image_responsive";i:1;s:11:"image_shape";s:0:"";s:14:"table_bordered";i:0;s:15:"table_condensed";i:0;s:11:"table_hover";i:1;s:13:"table_striped";i:1;s:16:"table_responsive";s:2:"-1";s:10:"breadcrumb";s:1:"1";s:15:"breadcrumb_home";i:0;s:16:"breadcrumb_title";i:0;s:14:"navbar_inverse";i:0;s:15:"navbar_position";s:10:"static-top";s:12:"region_wells";a:13:{s:10:"navigation";s:0:"";s:22:"navigation_collapsible";s:0:"";s:6:"header";s:0:"";s:11:"highlighted";s:0:"";s:4:"help";s:0:"";s:7:"content";s:0:"";s:13:"sidebar_first";s:0:"";s:14:"sidebar_second";s:0:"";s:6:"footer";s:0:"";s:3:"top";s:0:"";s:17:"navigation_search";s:0:"";s:4:"hero";s:0:"";s:10:"pre_footer";s:0:"";}s:13:"modal_enabled";i:1;s:22:"modal_jquery_ui_bridge";i:1;s:15:"modal_animation";i:1;s:14:"modal_backdrop";s:4:"true";s:17:"modal_focus_input";i:1;s:14:"modal_keyboard";i:1;s:17:"modal_select_text";i:1;s:10:"modal_show";i:1;s:10:"modal_size";s:0:"";s:15:"popover_enabled";i:1;s:17:"popover_animation";i:1;s:18:"popover_auto_close";i:1;s:17:"popover_container";s:4:"body";s:15:"popover_content";s:0:"";s:13:"popover_delay";s:1:"0";s:12:"popover_html";i:0;s:17:"popover_placement";s:5:"right";s:16:"popover_selector";s:0:"";s:13:"popover_title";s:0:"";s:15:"popover_trigger";s:5:"click";s:15:"tooltip_enabled";i:1;s:17:"tooltip_animation";i:1;s:17:"tooltip_container";s:4:"body";s:13:"tooltip_delay";s:1:"0";s:12:"tooltip_html";i:0;s:17:"tooltip_placement";s:9:"auto left";s:16:"tooltip_selector";s:0:"";s:15:"tooltip_trigger";s:5:"hover";s:12:"cdn_provider";s:0:"";s:11:"cdn_version";s:5:"3.4.1";s:9:"cdn_theme";s:0:"";s:22:"cdn_cache_ttl_versions";i:0;s:20:"cdn_cache_ttl_themes";i:0;s:20:"cdn_cache_ttl_assets";i:0;s:21:"cdn_cache_ttl_library";i:0;s:10:"cdn_custom";s:275:"https://cdn.jsdelivr.net/npm/bootstrap@3.4.1/dist/css/bootstrap.css https://cdn.jsdelivr.net/npm/bootstrap@3.4.1/dist/css/bootstrap.min.css https://cdn.jsdelivr.net/npm/bootstrap@3.4.1/dist/js/bootstrap.js https://cdn.jsdelivr.net/npm/bootstrap@3.4.1/dist/js/bootstrap.min.js";s:18:"include_deprecated";i:0;s:28:"suppress_deprecated_warnings";i:0;s:5:"table";s:0:"";s:16:"use_classic_logo";i:0;s:19:"use_classic_favicon";i:0;s:24:"reset_http_request_cache";s:0:"";s:11:"navbar_dark";i:1;s:11:"font_weight";s:5:"heavy";s:19:"container_max_width";s:0:"";s:20:"navbar_space_between";i:1;s:18:"use_compact_search";i:0;s:17:"navbar_background";i:0;s:27:"navbar_background_screen_xs";s:0:"";s:27:"navbar_background_screen_sm";s:0:"";s:27:"navbar_background_screen_md";s:0:"";s:27:"navbar_background_screen_lg";s:0:"";s:27:"navbar_background_screen_xl";s:0:"";s:17:"navbar_full_width";i:0;s:20:"autocomplete_enhance";i:0;s:12:"navbar_color";s:4:"dark";s:19:"local_task_position";s:0:"";s:13:"navbar_sticky";N;}s:14:"has_glyphicons";b:1;s:12:"query_string";s:6:"ryrmpu";}s:11:"input_group";b:0;s:6:"prefix";N;s:6:"suffix";N;s:10:"is_multple";b:0;s:22:"theme_hook_suggestions";a:0:{}}');
$test_class = new RecursiveFilterTest();
$test_class->traverse($classes_removed_variables);
echo "Completed.\n";

Resulted in:

The system hangs silently and you have to restart the process to get it going again.

But I expected this output instead:

Completed.

The reason we think it's a PHP bug is because it seems to hang on isset($current['#override_mode_breadcrumb']). Similar things like empty cause it to hang too.

PHP Version

8.1.18

Operating System

No response

@nielsdos
Copy link
Member

This regressed in 49b2ff5 @Girgias @NathanFreeman

@Girgias
Copy link
Member
Girgias commented Aug 14, 2023

Oh god, not this sort of issue again ;-;

I will have a look if @NathanFreeman is not available.

@Girgias
Copy link
Member
Girgias commented Aug 22, 2023

Can you produce a simpler, reproducible?

I really cannot comprehend what is going on in that code due to the size of the array and I have never done anything with Drupal.

Especially as the recursion code seems to work as expected with a simple case such as:

<?php

$arr1 = [];
$arr2 = [$arr1];
$arr1[] = $arr2;

$std1 = new stdClass();
$std2 = new stdClass();
$std1->prop1 = $std2;
$std1->prop2 = $arr1;

$std2->prop2A = $arr2;
$std2->prop2B = $std1;
$arr1[] = $std1;
$arr2[] = $std1;

$test_class = new RecursiveFilterTest();
$test_class->traverse($arr1);

@TimBozeman
Copy link
Author
TimBozeman commented Aug 22, 2023

Sorry about that. That was a lot. 😓 Thank you very much for looking into the issue. I think I've narrowed the test case down a bit, but it's still weird.

$test_array['e']['p'][] = ['a', 'a'];
$test_array['e']['p'][] = ['b', 'b'];
$test_array['e']['p'][] = ['c', 'c'];
$serialized = serialize($test_array);
$unserialized = unserialize($serialized);

$test_class = new RecursiveFilterTest();
$test_class->traverse($unserialized);

I seem to need to serialize it and unserialize it for the system to hang. I guess the array I’m dealing with in production is serialized because of some caching.

This conditional breakpoint $i === 13 is right before it hangs for me.
Screenshot 2023-08-22 at 11 02 58 AM

@nielsdos nielsdos self-assigned this Aug 30, 2023
nielsdos added a commit to nielsdos/php-src that referenced this issue Aug 30, 2023
When you do an assignment between two zvals (no, not zval*), you copy
all fields. This includes the additional u2 data. So that means for
example the Z_NEXT index gets copied, which in some cases can therefore
cause a cycle in zend_hash lookups.
Instead of doing an assignment, we should be doing a ZVAL_COPY (or
ZVAL_COPY_VALUE for non-refcounting cases). This avoids copying u2.
nielsdos added a commit that referenced this issue Aug 30, 2023
* PHP-8.1:
  Fix GH-11972: RecursiveCallbackFilterIterator regression in 8.1.18
nielsdos added a commit that referenced this issue Aug 30, 2023
* PHP-8.2:
  Fix GH-11972: RecursiveCallbackFilterIterator regression in 8.1.18
nielsdos added a commit that referenced this issue Aug 30, 2023
* PHP-8.3:
  Fix GH-11972: RecursiveCallbackFilterIterator regression in 8.1.18
nielsdos added a commit that referenced this issue Aug 30, 2023
When you do an assignment between two zvals (no, not zval*), you copy
all fields. This includes the additional u2 data. So that means for
example the Z_NEXT index gets copied, which in some cases can therefore
cause a cycle in zend_hash lookups.
Instead of doing an assignment, we should be doing a ZVAL_COPY (or
ZVAL_COPY_VALUE for non-refcounting cases). This avoids copying u2.

Closes GH-12086.
nielsdos added a commit that referenced this issue Aug 30, 2023
* PHP-8.1:
  Fix GH-11972: RecursiveCallbackFilterIterator regression in 8.1.18
nielsdos added a commit that referenced this issue Aug 30, 2023
* PHP-8.2:
  Fix GH-11972: RecursiveCallbackFilterIterator regression in 8.1.18
nielsdos added a commit that referenced this issue Aug 30, 2023
* PHP-8.3:
  Fix GH-11972: RecursiveCallbackFilterIterator regression in 8.1.18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants