Users should be able to set numToExecute bootstrapping count at startup

[rektbuildr]

The bootstrap procedure currently consists of

1. Fetch the latest accepted tip from peers and set this state to N
2. From 0 to N fetch the blocks/tx/vx
3. From 0 to N execute the blocks/tx/vx

While bootstrap/storage.go races towards N = numToExecute, all APIs except info will respond "node is not bootstrapped"

This makes no sense, because N is static and since it was fetched, there's already been new accepted blocks. Therefore N is not the latest block on the network, it is an arbitrary checkpoint that was chosen because the latest accepted state is a good choice for a bootstrap procedure.

The point of this suggestion is I believe point N should be configurable by users at startup time

Users could set N to an arbitrary value like 2 million and the node would catch up to N and then allow us to use the API while it continues to catch up with the network. Users could also set it to zero, so the API can be immediately used and the node can start relaying some calls that don't require it to be fully bootstrapped. For example, if I sign an offline TX and send it later via any available RPC endpoint. The node could relay that TX without being fully bootstrapped.

In this scenario, the node admin deliberately wants a node that is bootstrapped to a previous point N and not the latest. A config file key like bootstrap-num-to-execute or command line option like --bootstrap-num-to-execute could be set by the user. The node would execute blocks up to bootstrap-num-to-execute and then allow APIs to be called.

The isBootstrapped call would still return false, P2P gossip would be limited, the node would not be able to vote with synced nodes, etc, just as it is today, but local users could query the indexer and other APIs in the meantime

[StephenButtolph]

This makes no sense, because N is static and since it was fetched, there's already been new accepted blocks. Therefore N is not the latest block on the network, it is an arbitrary checkpoint that was chosen because the latest accepted state is a good choice for a bootstrap procedure.

N is not static. The bootstrapping procedure happens multiple times until the number to execute during the bootstrapping rounds is not decreasing significantly round-by-round.

For example, if I sign an offline TX and send it later via any available RPC endpoint. The node could relay that TX without being fully bootstrapped.

AvalancheGo only gossips valid transactions, it likely won't be able to validate the transaction if the node is currently bootstrapping. Additionally, there is a lock held during bootstrapping on the execution state, so removal of the isBootstrapped check would likely just result in a request that blocks until bootstrapping finishes (would likely timeout first).

Of course AvalancheGo could be modified to not perform these checks and gossip (potentially invalid) transactions, but that is not a change I would personally agree with.

In this scenario, the node admin deliberately wants a node that is bootstrapped to a previous point N and not the latest.

We certainly could modify avalanchego to sync to a given point if that were desirable. However, I don't think this aligns with the long-term maintainability goal of having all chains performing state-sync to the current tip.

IMO: Historical execution shouldn't be tied to implementing a client for advancing the current state of the chain.

[rektbuildr]

Thank you @StephenButtolph

I assumed N was static, therefore the rest of my points are mute

What about APIs which don't need consensus, could avalanchego be adapted to allow those to respond while it's being bootstrapped for the consensus-dependent functionality? E.g. index queries for already accepted containers and txs which will never change?

[StephenButtolph]

We definitely could modify the API server to allow handling to be determined on a method-by-method basis.

What APIs specifically would you be interested in allowing to use during bootstrapping? Are these C-chain APIs? P-chain?