Some questions about the architecture

Technical Discussion
#1
  1. Any reason why ZeroMQ was chosen as the notification system vs. others?
  2. Any particular reason to why the rest server was done as an entirely separate project from the node code?
  3. All accounts eligible to vote are expected to vote. Voters that violate these expectations are considered Byzantine and might be punished in the future. What happens now if there are less than ⅔ of voters remaining online in a public network?
  4. It is said that in deterministic finality mode there can be deep rollbacks. Is there an estimation of the depth of such rollbacks in blocks and in minutes?
  5. There is a fixed voting set with fixed value of importance and stakes for each Harvester in each epoch. In which state of the blockchain are the values of stakes and importances calculated? As I understand, by the start of a new epoch each Harvester has its opinion about the last finalized block, so the values of voting set, importance and stake can differ from a harvester to a harvester.
1 Like
#2
  1. We picked ZeroMQ because it was entirely implemented in a library and had C++ and JS bindings. The catapult ZeroMQ code is implemented entirely in a catapult extension and can be swapped out and replaced with a catapult extension implementing a different notification system.
  2. We wanted to minimize potential attack vectors in the core server. So, if the web server crashes, the entire blockchain doesn’t go down. This is the same reason ZeroMQ and MongoDB are both implemented as catapult extensions. A peer node doesn’t load either of these third-party dependent extensions and is a more “bare metal” node that is more robust due to fewer attack vectors.
  3. If fewer than 2/3 voting stake (not voters) is offline, finalization will stall until a supermajority of voters are back online (and in agreement).
  4. In normal conditions when the network is regularly finalizing blocks, finalization progress should be made every 4 minutes. If the network is under attack and finalization cannot progress, there is always the potential for a rollback to the last finalized block. The depth of this is dependent on the duration of the attack.
  5. All nodes must agree on all finalized blocks. That is the point of finalization! Therefore, at the end of an epoch, they should all agree on the blocks in that epoch and its terminal state. The voting set is derived from that terminal state.
4 Likes
#3

Thanks for your quick and detailed reply @Jaguar0625

I’ve some more questions:
6. In normal conditions, finalization progress should be made every 4 minutes. – Could we reduce the finalization time? What is the lowest limit that could be set instead of 4 minutes?
7. Different Harvesters can have different finalized block heights at the same time. E.g., harvester A has finalized block #10 but harvester B has already finalized block #12. – How much the discrepancy of block heights can be? Is there any estimation of how many harvesters have finalized the same height at the same time?

1 Like
#4
  1. stepDuration = 5m so finalization is every 10 minutes, you need to balance various things:
    • you want to give voters time to actually exchange messages
    • you don’t want voters to vote too often (as you have chance of not progressing within a round - waste of time)
    • you want the chain to actually progress few blocks before next round happens

I can’t give you good answer (and in private deployment, depending on non-tech factors, you probably don’t need finalization at all)

  1. sooner or later all voters will finalize on block at the end of epoch (votingSetGrouping = 1440).
    • if node is actively voting, it shouldn’t be further than few blocks behind
    • if node is peer and non-voting (but with finalization extension enabled), it will recv votes anyway
    • if node is api and not-voting, there’s unfinalizedBlocksDuration = that should be non-zero, and forces it to pull some “fresh” data about finalization (otherwise, with 0 value it will only have info about finalizations at epoch ends, which as above is 1440blocks in mainnet ~= 12h)
3 Likes
#5

Why do you suggest that there is no need for finalization in private blockchains? Is it because of the absence of intentional double spending? Or something else?

#6

Because in private deployments it’s usually the entity, that is running the chain, and there are no outsiders that can join the network (and well you can have two currencies, one that is used for payments other for harvesting purposes).
Also in private deployments you’ll probably want some kind of monitoring if things are going the way you intended (and in case of network split, there might be some more complicated scenarios needed - like rearranging blocks).

P.S. Sorry for late response, missed the message.