[go: up one dir, main page]

DAL/Node: cannot handle of large amount of shards

With an experiment on GCP with 48 producers and 2 bakers, we noticed that the DAL attester nodes do not manage to process all the shards: they only process 4-8k shards instead of the expected 12k.

Disabling the shards_handler function make the experiment go through. As the shards handler mainly only writes the shard on the disk, we suspect that this is the main issue. We note however that an attester DAL node only needs the shards for a rather short amount of time (namely the attestation lag), and therefore could store the received shards in memory.