Backend: Improve CI Linter performance through parallelizing HTTP calls

Summary

Some users have been receiving a timeout error when using remote includes. In an effort to help the CI linter perform better, research has suggested that the problem could be resolved by requesting HTTP external resources in a sequential way.

Solution

By parallelizing the HTTP calls when using remote includes, it will improve the performance of running a pipeline, and having the calls done in parallel should decrease these occurrences of timeout errors. Leveraging the MAX_INCLUDES attribute could also be considered.

Implementation

Work Type	Description	Issue link
	NOTE: 🚨 All below issues can be done in parallel
backend	Backend: The gitlab-ci.yml is limited to 100 includes	#207270 (closed)
backend	Backend: Remove N+1 for Gitaly requests when fetching includes	#344829 (closed)
backend frontend	Improve the error messaging when fetching remote includes are timing out	#351168 (closed)
backend	Backend: Improve CI Linter performance through parallelizing HTTP calls	👈 You are here
backend	Backend: Caching includes to improve performance when using remote includes	#351252
backend	Backend: Batch request calls to Gitaly when fetching include	#382531 (closed)
backend	Backend: Group files by projects in config_file_project_validate_access	#382751 (closed)