High memory usage while importing snapshot of a flattened context

The memory consumption while importing a snapshot targeting a block with a "flattened" context of mainnet (i.e. > 1,916,929), that is to say after the activation of Hangzhou, is too high. Several users have reported issues while importing such a snapshot, where a 4Gb ram + 4Gb swap was not enough.

After monitoring the import, it appears that the snapshot import reaches a ~8Gb ram usage at ~20% of the import: logs are frozen during a while on Writing context: 2497K/11601K (21%) elements, 421MiB read., corresponding to the memory burst. The memory usage drops afterwards and remains acceptable. Could this be related to the load of the "contracts" Irmin's node which is quite a big entry?

Here are the graphs of a monitored snapshot import (TEZOS_MAINNET-BLr3yct277uX9ptq2WxG6ZUChKKCH8CPGBjuEfK6UnYxZtHyMso-1921968.full).

Update

MR !4076 (merged) is a quick fix to improve the memory consumption of the snapshot import. It reduces the memory usage by a factor of 2. The real fix, which is to completely rework the snapshot format, should come later.