Optimize `has_ambiguous_refs?`.
-
Please check this box if this contribution uses AI-generated content (including content generated by GitLab Duo features) as outlined in the GitLab DCO & CLA. As a benefit of being a GitLab Community Contributor, you receive complimentary access to GitLab Duo.
What does this MR do and why?
Optimize has_ambiguous_refs?.
With large numbers of branch or tag names the regular expression can
get very expensive. This revised approach uses set intersection to
check for overlap.
References
This performance issue was discovered while investigating the root cause of 500 errors as reported here: #516086 (closed)
MR acceptance checklist
Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
Screenshots or screen recordings
Profile captured via Gitlab::Profiler on a repo with apx. 1,500 branches and 60,000 tags using the original implementation:
Gitlab::Profiler.profile('https://example.com/group/project/-/blob/main/path/file.txt?ref_type=heads', user: user, profiler_options: { out: '/tmp/profile.dump' })
$ stackprof /tmp/profile.dump
==================================
Mode: wall(1000)
Samples: 63258 (0.11% miss rate)
GC: 15 (0.02%)
==================================
TOTAL (pct) SAMPLES (pct) FRAME
62857 (99.4%) 62857 (99.4%) Regexp#match?
68 (0.1%) 68 (0.1%) PG::Connection#exec
63054 (99.7%) 26 (0.0%) Repository#has_ambiguous_refs?
15 (0.0%) 15 (0.0%) (sweeping)
14 (0.0%) 14 (0.0%) String#to_sym
14 (0.0%) 13 (0.0%) Module#module_eval
13 (0.0%) 12 (0.0%) RequestStore.store
12 (0.0%) 12 (0.0%) IO#wait_readable
12 (0.0%) 12 (0.0%) String#prepend
35 (0.1%) 11 (0.0%) Gitlab::Utils::StrongMemoize#ivar
62883 (99.4%) 11 (0.0%) Array#any?
11 (0.0%) 10 (0.0%) RedisClient::RubyConnection::BufferedIO#gets_integer
12 (0.0%) 9 (0.0%) RequestStore.[]=
9 (0.0%) 9 (0.0%) Regexp.escape
55 (0.1%) 8 (0.0%) Gitlab::Instrumentation::RedisBase.read_bytes_key
18 (0.0%) 8 (0.0%) RequestStore.[]
98 (0.2%) 7 (0.0%) Gitlab::Instrumentation::RedisHelper#measure_read_size
7 (0.0%) 7 (0.0%) String#byteslice
6 (0.0%) 6 (0.0%) Module#define_method
47 (0.1%) 6 (0.0%) Gitlab::Utils::StrongMemoize#strong_memoize
91 (0.1%) 6 (0.0%) Gitlab::Instrumentation::RedisBase.increment_read_bytes
32 (0.1%) 5 (0.0%) RedisClient::RESP3.parse_blob
5 (0.0%) 5 (0.0%) Symbol#to_s
4 (0.0%) 4 (0.0%) PG::Connection#exec_params
4 (0.0%) 4 (0.0%) Module#method_defined?
6 (0.0%) 4 (0.0%) ActiveRecord::DynamicMatchers::Method.match
4 (0.0%) 4 (0.0%) BasicSocket#__read_nonblock
14 (0.0%) 3 (0.0%) ActiveModel::AttributeMethods::ClassMethods#define_proxy_call
39 (0.1%) 3 (0.0%) ActiveModel::AttributeMethods::ClassMethods#define_attribute_method
3 (0.0%) 3 (0.0%) String#%
Revised profile with this proposed change:
==================================
Mode: wall(1000)
Samples: 924 (6.57% miss rate)
GC: 78 (8.44%)
==================================
TOTAL (pct) SAMPLES (pct) FRAME
101 (10.9%) 101 (10.9%) PG::Connection#exec
41 (4.4%) 41 (4.4%) (sweeping)
41 (4.4%) 41 (4.4%) String#split
71 (7.7%) 40 (4.3%) BasicObject#instance_eval
37 (4.0%) 37 (4.0%) (marking)
25 (2.7%) 25 (2.7%) PG::Connection#exec_params
24 (2.6%) 24 (2.6%) Module#module_eval
27 (2.9%) 24 (2.6%) Hash#hash
17 (1.8%) 17 (1.8%) IO#wait_readable
30 (3.2%) 16 (1.7%) Ripper#parse
12 (1.3%) 12 (1.3%) String#prepend
182 (19.7%) 11 (1.2%) Repository#has_ambiguous_refs?
88 (9.5%) 11 (1.2%) Gitlab::Instrumentation::RedisBase.increment_read_bytes
17 (1.8%) 11 (1.2%) RequestStore.store
837 (90.6%) 10 (1.1%) Array#each
259 (28.0%) 9 (1.0%) Class#new
27 (2.9%) 9 (1.0%) Gitlab::Utils::StrongMemoize#ivar
9 (1.0%) 9 (1.0%) String#scan
8 (0.9%) 8 (0.9%) Regexp#match
98 (10.6%) 8 (0.9%) Gitlab::Instrumentation::RedisHelper#measure_read_size
46 (5.0%) 8 (0.9%) Gitlab::Utils::StrongMemoize#strong_memoize
20 (2.2%) 8 (0.9%) RequestStore.[]
7 (0.8%) 7 (0.8%) Hash#merge
7 (0.8%) 7 (0.8%) Module#method_defined?
11 (1.2%) 7 (0.8%) RedisClient::RubyConnection::BufferedIO#gets_integer
8 (0.9%) 7 (0.8%) ActionDispatch::Journey::Path::Pattern#requirements_for_missing_keys_check
58 (6.3%) 6 (0.6%) RedisClient::RESP3.parse
6 (0.6%) 6 (0.6%) Kernel#instance_variable_defined?
6 (0.6%) 6 (0.6%) String#to_sym
6 (0.6%) 6 (0.6%) GRPC::Core::Call#run_batch
Edited by 🤖 GitLab Bot 🤖