[go: up one dir, main page]

Optimize `has_ambiguous_refs?`.

What does this MR do and why?

Optimize has_ambiguous_refs?. With large numbers of branch or tag names the regular expression can get very expensive. This revised approach uses set intersection to check for overlap.

References

This performance issue was discovered while investigating the root cause of 500 errors as reported here: #516086 (closed)

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Screenshots or screen recordings

Profile captured via Gitlab::Profiler on a repo with apx. 1,500 branches and 60,000 tags using the original implementation:

Gitlab::Profiler.profile('https://example.com/group/project/-/blob/main/path/file.txt?ref_type=heads', user: user, profiler_options: { out: '/tmp/profile.dump' })

$ stackprof /tmp/profile.dump
==================================
  Mode: wall(1000)
  Samples: 63258 (0.11% miss rate)
  GC: 15 (0.02%)
==================================
     TOTAL    (pct)     SAMPLES    (pct)     FRAME
     62857  (99.4%)       62857  (99.4%)     Regexp#match?
        68   (0.1%)          68   (0.1%)     PG::Connection#exec
     63054  (99.7%)          26   (0.0%)     Repository#has_ambiguous_refs?
        15   (0.0%)          15   (0.0%)     (sweeping)
        14   (0.0%)          14   (0.0%)     String#to_sym
        14   (0.0%)          13   (0.0%)     Module#module_eval
        13   (0.0%)          12   (0.0%)     RequestStore.store
        12   (0.0%)          12   (0.0%)     IO#wait_readable
        12   (0.0%)          12   (0.0%)     String#prepend
        35   (0.1%)          11   (0.0%)     Gitlab::Utils::StrongMemoize#ivar
     62883  (99.4%)          11   (0.0%)     Array#any?
        11   (0.0%)          10   (0.0%)     RedisClient::RubyConnection::BufferedIO#gets_integer
        12   (0.0%)           9   (0.0%)     RequestStore.[]=
         9   (0.0%)           9   (0.0%)     Regexp.escape
        55   (0.1%)           8   (0.0%)     Gitlab::Instrumentation::RedisBase.read_bytes_key
        18   (0.0%)           8   (0.0%)     RequestStore.[]
        98   (0.2%)           7   (0.0%)     Gitlab::Instrumentation::RedisHelper#measure_read_size
         7   (0.0%)           7   (0.0%)     String#byteslice
         6   (0.0%)           6   (0.0%)     Module#define_method
        47   (0.1%)           6   (0.0%)     Gitlab::Utils::StrongMemoize#strong_memoize
        91   (0.1%)           6   (0.0%)     Gitlab::Instrumentation::RedisBase.increment_read_bytes
        32   (0.1%)           5   (0.0%)     RedisClient::RESP3.parse_blob
         5   (0.0%)           5   (0.0%)     Symbol#to_s
         4   (0.0%)           4   (0.0%)     PG::Connection#exec_params
         4   (0.0%)           4   (0.0%)     Module#method_defined?
         6   (0.0%)           4   (0.0%)     ActiveRecord::DynamicMatchers::Method.match
         4   (0.0%)           4   (0.0%)     BasicSocket#__read_nonblock
        14   (0.0%)           3   (0.0%)     ActiveModel::AttributeMethods::ClassMethods#define_proxy_call
        39   (0.1%)           3   (0.0%)     ActiveModel::AttributeMethods::ClassMethods#define_attribute_method
         3   (0.0%)           3   (0.0%)     String#%

Revised profile with this proposed change:

==================================
  Mode: wall(1000)
  Samples: 924 (6.57% miss rate)
  GC: 78 (8.44%)
==================================
     TOTAL    (pct)     SAMPLES    (pct)     FRAME
       101  (10.9%)         101  (10.9%)     PG::Connection#exec
        41   (4.4%)          41   (4.4%)     (sweeping)
        41   (4.4%)          41   (4.4%)     String#split
        71   (7.7%)          40   (4.3%)     BasicObject#instance_eval
        37   (4.0%)          37   (4.0%)     (marking)
        25   (2.7%)          25   (2.7%)     PG::Connection#exec_params
        24   (2.6%)          24   (2.6%)     Module#module_eval
        27   (2.9%)          24   (2.6%)     Hash#hash
        17   (1.8%)          17   (1.8%)     IO#wait_readable
        30   (3.2%)          16   (1.7%)     Ripper#parse
        12   (1.3%)          12   (1.3%)     String#prepend
       182  (19.7%)          11   (1.2%)     Repository#has_ambiguous_refs?
        88   (9.5%)          11   (1.2%)     Gitlab::Instrumentation::RedisBase.increment_read_bytes
        17   (1.8%)          11   (1.2%)     RequestStore.store
       837  (90.6%)          10   (1.1%)     Array#each
       259  (28.0%)           9   (1.0%)     Class#new
        27   (2.9%)           9   (1.0%)     Gitlab::Utils::StrongMemoize#ivar
         9   (1.0%)           9   (1.0%)     String#scan
         8   (0.9%)           8   (0.9%)     Regexp#match
        98  (10.6%)           8   (0.9%)     Gitlab::Instrumentation::RedisHelper#measure_read_size
        46   (5.0%)           8   (0.9%)     Gitlab::Utils::StrongMemoize#strong_memoize
        20   (2.2%)           8   (0.9%)     RequestStore.[]
         7   (0.8%)           7   (0.8%)     Hash#merge
         7   (0.8%)           7   (0.8%)     Module#method_defined?
        11   (1.2%)           7   (0.8%)     RedisClient::RubyConnection::BufferedIO#gets_integer
         8   (0.9%)           7   (0.8%)     ActionDispatch::Journey::Path::Pattern#requirements_for_missing_keys_check
        58   (6.3%)           6   (0.6%)     RedisClient::RESP3.parse
         6   (0.6%)           6   (0.6%)     Kernel#instance_variable_defined?
         6   (0.6%)           6   (0.6%)     String#to_sym
         6   (0.6%)           6   (0.6%)     GRPC::Core::Call#run_batch
Edited by 🤖 GitLab Bot 🤖

Merge request reports

Loading