Draft: Separate appendedLSN and committedLSN

This MR is extracted from !7385 (closed).

Recently, the TransactionManager uses two indices which are appendedLSN and appliedLSN. It increases the value of appendedLSN after it accepts a transaction. When the log entry is applied, the transaction manager increases appliedLSN accordingly. Any log entries prior to appliedLSN could be removed if there are no pending transactions referring to them.

We are working on adding Raft consensus to the TransactionManager. When a log entry is accepted by the current node, it also needs to be transferred to and accepted by the majority of cluster members.

To make it easier for later steps, the TransactionManager needs to track two indices independently:

• appendedLSN: this index tracks the LSN of the log entry accepted by the current node, but not yet accepted by others.
• committedLSN: this index tracks the committed log entries. They are safe to be applied to the repositories. Eventually, other nodes apply those entries in the same order.

	// ┌─ oldestLSN                      ┌─ committedLSN
	// ⧅ ⧅ ⧅ ⧅ ⧅ ⧅ ⧅ ■ ■ ■ ■ ■ ■ ■ ■ ■ ■ □ □ □ □ □ □ □
	//                 └─ appliedLSN                   └─ appendedLSN

The transaction manager is allowed to override entries from committedLSN + 1 to appendedLSN. As this MR concentrates on refactoring, the committedLSN is then bumped and reaches the appendedLSN instantly. In some later MRs, Raft will intercept the flow and seek acknowledgment from other nodes.

This MR also introduces a new non-subtle change: committedLSN needs to persist in KV DB, next to appliedLSN. Before, as soon as a log entry is flushed to disk, that log entry will be applied, eventually. After splitting the appending stage and committing stage, things might go wrong after a log entry is appended but is not committed. This is particularly true if Raft enters the picture and adds network latency to the log entry proposal. As a result, WAL needs to persist the new committedLSN and re-load them after restart.

Thus, the transaction manager needs to persist the committedLSN to avoid applying for uncommitted entries after a crash. After restarts, it resumes from the latest committedLSN, applies unapplied entries, if any, and removes obsoleted appended entries, if any.