[Feat] Database checkpoints

[ljedrz]

This PR exposes the functionality introduced to snarkVM in ProvableHQ/snarkVM#2742 via the REST API.

Creating a checkpoint at location /home/aleo_ledger_checkpoints/1:

curl -X POST http://localhost:3030/mainnet/db_backup?path=/home/aleo_ledger_checkpoints/1

Rolling back to (or, more precisely, switching to) the aforementioned checkpoint:

snarkos --storage=/home/aleo_ledger_checkpoints/1

Alternatively, the original ledger can be removed, and manually substituted with the checkpoint.

In addition to the root issue:

• the checkpoint's location can have any name - it only needs to be a non-existent one
• (almost - depends on the hard link limits of the filesystem) any number of checkpoints can be created locally

Filing as a draft, pending community feedback wrt the scope.

[ljedrz]

note: the CI issues appear to have been caused by #3630

[vicsn]

Can you revive and rebase: #3592

Can you write a short manual in the top PR message for how to backup (call endpoint) and recover (???)

[vicsn]

This is ready for rebase and opening

[ljedrz]

I stopped the devnet CI job, as it wasn't stopping on its own; let's see if it was a fluke.

Edit: yep, it was a fluke - there wasn't an issue locally either.

[vicsn]

I'm wondering if we can get .circleci/db_backup_ci.sh in CI without having to spin up yet another CI job. Could we just run them one after the other? It shouldn't be an issue if state bleeds from one network to the other.

          name: "Run devnet test"
          timeout: 20m  # Allow 20 minutes total
          command: |
            echo "Starting db_backup_ci.sh"
             ./.circleci/db_backup_ci.sh << parameters >>
            echo "Starting devnet_ci.sh"
            ./.circleci/devnet_ci.sh << parameters.validators >> << parameters.clients >> << parameters.network_id >> << parameters.min_height >>

And to be clear, I think we should not migrate to this other script yet, we can evaluate that PR after this is through.

[ljedrz]

Could we just run them one after the other? It shouldn't be an issue if state bleeds from one network to the other.

Hmm bleeding state would be a problem, as the checkpoint script expects very specific block heights; it should be fine if snarkos clean is run first, though 🤔.

That being said, database checkpoints are quite isolated from the other functionalities, so I wouldn't expect them to be fragile enough to require a test run after every PR.

[vicsn]

Hmm bleeding state would be a problem, as the checkpoint script expects very specific block heights; it should be fine if snarkos clean is run first, though 🤔.

Well I'm suggesting to run db_backup_ci.sh first before devnet_ci.sh. But doing snarkos clean at the beginning or end is also fine. Up to you.

[ljedrz]

Seems to have worked just fine; a tiny modification, too.