Catapult Migration - Should be "One chain with two network protocols and API Layers"

General Discussion
#1

The NEM Foundation’s proposal of a Catapult migration involving splitting XEM into two chains and tokens is a complete joke that will disrupt the ecosystem and destroy XEM’s market value just as any contentious fork has done with other coins.

A proper seamless migration should only concern Supernode users having to upgrade from the NIS daemon to a Catapult daemon and nothing else. This could be done by writing a wrapper/proxy layer on the Catapult daemon to support the NIS API and network protocol.

Supernode users then upgrade from NIS to Catapult before a specific block height/date or forfeit the centrally issued supernode reward. Providing this economic incentive is enough to get an upgrade of 51+% of supernodes (it has to date with NIS upgrades) the majority can then blacklist network communications with the old NIS supernodes.

This method of providing a NIS network/API proxy through Catapult means you don’t need to worry about upgrading the entire eco-system as old wallets and applications would continue to work through the Proxy layer handled by the Catapult daemon.

The Foundation leadership should be ashamed of themselves as they still haven’t recruited a CTO who would advise how a seamless migration could be done and is done in the enterprise world.

As an end user we demand a Catapult migration being; One Chain with Two network protocol and API layers for a seamless migration!

3 Likes
#2

I actually see this as the ideal situation as well. I posted this here already: Catapult (v2) Update + Community Feedback

Scenario
1- Make a snapshot of every balance at a specific block number in the old chain (including every mosaic with the correct timestamp) and start the new catapult chain with these nemesis balances at block height 0.
2- In the new catapult nodes, keep the whole pre-catapult chain available, where node owners can turn the loading of the old chain on or off in a setting. And make using API calls to the old chain available.
3- Make sure this update is mandatory for supernodes a while before the actual snapshot so the whole network updates before catapult goes live, +90% of supernodes upgrade because of the rewards

3 Likes
Migration Committee Community Update #1
#3

1- Make a snapshot of every balance at a specific block number in the old chain (including every mosaic with the correct timestamp) and start the new catapult chain with these nemesis balances at block height 0.

Any funds currently held on NIS multi-signature accounts would then be mirror to simple accounts, putting funds at risk (some multisig private keys are surely lost due to the characteristics of NIS multisig).

2- In the new catapult nodes, keep the whole pre-catapult chain available, where node owners can turn the loading of the old chain on or off in a setting.

One part here is not feasible as blocks in NIS are not compatible with blocks in Catapult, also we are starting with a new nemesis. It is not a fork like there is with Bitcoin and therefore NIS history cannot be prepended to Catapult blocks.

Another part here sounds somewhat feasible where having a Catapult plugin available to allow data queries in NIS-style (smooth upgrade plugin) is totally feasible but adds on development.

3- Make sure this update is mandatory for supernodes a while before the actual snapshot so the whole network updates before catapult goes live, +90% of supernodes upgrade because of the rewards

I would like to point at 2 points here: 1) supernodes are a subset of NEM nodes, they are not “the network”. 2) supernodes do not affect consensus more than other nodes. The fact that supernodes find consensus on a topic is likely to affect the network (because they control many funds) but the migration should not be done as such that supernodes get super powers.

1 Like
#4

The NEM Foundation’s proposal of a Catapult migration involving splitting XEM into two chains and tokens is a complete joke that will disrupt the ecosystem and destroy XEM’s market value just as any contentious fork has done with other coins.

This is not a NEM Foundation proposal. The Foundation has representatives in the migration committee but does not influence decisions more than any other entity present in the committee.

One Chain with Two network protocol and API layers for a seamless migration!

This feedback will be taken into consideration.

The Foundation leadership should be ashamed of themselves as they still haven’t recruited a CTO who would advise how a seamless migration could be done and is done in the enterprise world.

The Foundation went through a transformation over the start of this year and filling positions should not happen in the wild.

How it is done in the enterprise world is very well an input of our migration committee. Migrating over a distributed ledger protocol to a completely new distributed ledger protocol with keeping history of the old chain and same blocks just doesn’t apply for the Catapult upgrade. We are not talking about the same blockchain storage, different signature schemes and features that are only available since NEM (i.e.: onchain multisig).

Please, prove me wrong: I doubt that you will find a perfect fit for this position to fill before migration happens, that will also permit to keep on track for the announced launch period. Mind also that there needs to be a clear path forward post-launch. Foundation is addressing missing positions as needs be - I would like to see a more constructive feedback on your end.

5 Likes
#5

Blocks are just a protocol layer that can be decoded into the ledger entries stored in Catapult’s database so there is no reason why you can’t have Catapult start with NIS genesis and sync its block history to create the ledger. Then once the majority of supernodes have upgraded to the Catapult daemon with NIS proxy/API layer the network can then switch over and drop the NIS protocol layer at a specific block height, but continue to support the NIS API layers used by the ecosystem used by applications and wallets.

Supernodes “are” the part of the network that matters with regards to the migration as they create the blocks and commit the ledger entries. Other parts of the network ecosystem don’t matter as they interface with the Supernodes and if a migration is done seamlessly then only Supernodes need to be effected. Furthermore the economic incentives to run a Supernode are centralised by issuance from the NIS FUND, to think we are dealing with a Decentralised network (like Bitcoin) is a gross misrepresentation of the facts.

Proposing a two chain/token migration just to keep to an announced date made without any basis of technical consideration is terrible way of destroying NEM in a contentious manner. Your better off admitting that the launch date was made up just to sell a pretty picture for the Foundation to get further funding at the start of the year, put together a proper seamless migration plan, and announce a new “real” release date.

#6

Supernodes “are” the part of the network that matters with regards to the migration as they create the blocks and commit the ledger entries.

Blocks are not only harvested by supernodes, they are also harvested by simple nodes.

Other parts of the network ecosystem don’t matter as they interface with the Supernodes

This is a wrong assumption, other parts of the network do not rely on Supernodes to function. You can run your own node and let people harvest on it just as well as on a supernode.

then only Supernodes need to be effected

It is true that supernodes can and probably should/will help during the migration process.

to think we are dealing with a Decentralised network (like Bitcoin) is a gross misrepresentation of the facts.

Very simple question to answer this: Who owns the private keys to multi-signature account cosignatories? Only these are allowed to execute updates to the contracts and only these are allowed to re-create the contract on Catapult.

Proposing a two chain/token migration just to keep to an announced date made without any basis of technical consideration

Who says there is no technical consideration? The technical aspects of the migration has partly influenced our decisions with the migration committee.

2 Likes
#7

I think there is a misunderstanding of how much NIS1 is not compatible with Catapult.

One major blocker is the fact that private/public key pairs are calculated differently in NIS1 vs Catapult. Yes, the blocks are just ledger entries containing account information and transaction information. However without knowledge of a private key, it is impossible to do a one to one match of corresponding public key in NIS1 to public key in Catapult. Much more, not all accounts have known public keys, some only have known addresses. So if the address in NIS1 belongs to a private/public keypair in NIS1, how do we guarantee that the same address belongs to the same private/public keypair in Catapult?

There are also other uses for addresses, eg. burn addresses, mosaic sink address. To which address should they be assigned to in Catapult?

And the accusation that this migration plan was announced without any technical consideration is an unfair statement. If you had been paying attention in the Slack channel or even this GitHub issue (https://github.com/nemtech/NIP/issues/22), you can see that this plan has been in discussion for months.

6 Likes
#8

@gevs has given a very complete answer to the technical sides (and others) of these questions. Jag has also tweeted a much shorter version of it:

image

So we can debate the technical compatibility or not but the people who designed and wrote it have said it is incompatible, publicly.

@gevs has also set the challenge to prove it incorrect, if it can be proven and shown to be possible, not hypothesised to be possible, its may be an option to revisit but right now the understanding from all involved is it is not possible.

Also to confirm his answer, this is not a NEM Foundation proposal, as per the announcement it is a cross section from the various entities publishing a recommendation (Foundation, Ventures, Studios, Tech Bureau, that has been reviewed by the core devs for viability technically prior to release).

1 Like
#9

If NIS1 does not migrato to catapult, can’t install Catapult on NIS1

#10

I am saying that Catapult should have a protocol level and API wrapper/proxy layer built into it so it can support the encryption scheme of NIS transactions. This way after it has validated the transaction it can update the ledger stored by Catapult and you don’t need to worry about your question.

By seamlessly supporting the existing infrastructure you can give the ecosystem a much longer time to change their wallets and applications over to Catapult protocols and API’s. Whilst as developers you can build new features on top of Catapult code which act as an incentive for the ecosystem to upgrade.

The survey you did asking how any exchanges are needed for a second Catapult token, if desktop, mobile wallets are required for launch etc is really telling of how short sighted the migration committee is in seeing how crippled the launch of Catapult will be in its own right as arbitrary launch dates are being prioritised over everything else in a way that it destroys everything about NEM with a two chain approach.

#11

As mentioned above a wrapper layer in Cataput could support the processing of NIS transactions. Its only a difference in encoding of transactions for NIS and Catapult, once each is decoded and validated they both have the same result on the Ledger.

They don’t need to be assigned anything on Catapult, they remain using NIS addresses.

#12

I recall seeing that back in May, the lack of comments and discussion on Github issue itself is quite telling. If all the discussion has happened in Slack then my bad as I don’t use that platform.

EDIT: If the discussion has occurred in Slack then why no update to the official NIP-0008 document?

1 Like
#13

As NIS is closed source, how can it be proved or disproved?

#14

so it can support the encryption scheme of NIS transactions

I do see the advantage of this implementation but would like to remind about 1) the deadline (which is not randomly picked), 2) catapult has been open sourced last year but we havent seen such contribution, 3) it is not possible to sign the transactions being added without owning the private key of accounts making it complicated to include history (or have it unverifiable in nemesis block…) and 4) Catapult incompatibility is so that transactions would need to be signed again.

As NIS is closed source, how can it be proved or disproved?

Just as a FYI, I do not have access to the code more than you.

Also, as mentioned by dave above - we would love to see a process definition that is realistic and can be tackled now, not just hypothesis but a real plan. I will point at the deadline again here as this influenced our migration recommendations a lot.

You have to understand that catapult has been in the making for X years, I tend to agree with you on that keeping the deadline over everything is maybe not so important to the general public but it happens to be one of the important variables of this launch.

discussion on Github issue itself is quite telling

Agreed. The github issue has not seen comments and contributions and we would gave appreciated more feedback. But discussions did not stop at NIP8, we indeed discussed a lot through slack groups and committee calls.

NIP8 was an attempt at opening the decision making process, and it did right in its job - it pushed us to move forward on which data can be ported, and how that data can be ported over.

#15

  1. That deadline is not important if it compromises the ecosystem. There is no technical urgency for Catapult as the existing network is more than enough to handle the load for the foreseeable future. Migrating to Catapult (in terms of a feature) isn’t going to do anything to the market price of XEM as the crypto market is hibernating.

  2. I am under the impression the Foundation got 210 million XEM funding and was shifting to being product focused on Catapult. This means they were hiring and paying developers to do this work, so the excuse of this being an open source project lacking “free” contributions doesn’t stand.

  3. I don’t see what this has to do with anything. Only the token holder signs the transaction when they want to spend it. The supernode then verifies the transaction is valid against consensus rules and then it gets added to the ledger. The ledger would indicate if it was a NIS or Catapult type transaction so you could replay the appropriate networks encryption and transaction schemes for verifying from the genesis block.

  4. Not at all, its simply a matter of implementing a logic switch for each of the transaction types, i) NIS, and ii) Catapult. Then you can apply the appropriate encryption and consensus rules to each transaction and update the Ledger for token ownership. One would need to facilitate a type of bridge address to allow for interaction between NIS and Catapult transactions. This would involve all Catapult type wallets generating both Catapult and NIS pub/priv keys with addresses, so it can receive funds from NIS holders that would then give the receiver the option to spend those coins against Catapult addresses.

Obviously old NIS only wallets could not spend to Catapult addresses, however this would be one of the feature incentives to upgrade, but they are not forced to upgrade as the Catapult key owner can still receive the funds from the dual NIS address/keys associated with the Catapult address.
Catapult addresses can spend to NIS addresses by signing with the NIS keys.

#16

I don’t see what this has to do with anything. Only the token holder signs the transaction when they want to spend it.

Point taken, I was thinking about adding data to catapult but you have the intention to let it be on NIS. In that case, I dont see 3) as a valid argument but would want to note that what you are trying to do would effectively gives Catapult networks power to manage funds on NIS? Why do you want that to even happen? Even worse if this is only possible through supernodes. I want to point at that supernodes currently dont fill any more features than simple nodes, that would change with your idea.

Furthermore I believe we should be pushing interoperability as well but I do not agree that NIS funds should be made “spendable” from catapult.

–

Need to jump off quickly morning routine…, will come back later today :ok_hand:

#17

That is a point of difference, why do you not agree with Catapult supporting NIS transactions?

I think it should to allow for a seamless migration by supporting both NIS and Catapult network protocols through the new Catapult daemon. Therefore only the core network infrastructure needs to be complicit in the migration as the rest will still be supported by Catapult’s NIS compatibility layer.

Once the majority of the core network has upgraded at a specific block height parts of the system could switch over, such as changing over from NIS blocks to Catapult formatted blocks. NIS and Catapult transactions can coexist on their separate network protocols whilst both are processed by the new Catapult daemon which maintains the single chain/ledger set.

#18

Any funds currently held on NIS multi-signature accounts would then be mirror to simple accounts, putting funds at risk (some multisig private keys are surely lost due to the characteristics of NIS multisig).

What is the difference between taking the opt-in multisigs and putting them in the genesis block and just take all the multisig accounts and put them in the genesis block? In the current proposal all the opt-in multisig account are put in the nemesis block are they not?

I would like to point at 2 points here: 1) supernodes are a subset of NEM nodes, they are not “the network”. 2) supernodes do not affect consensus more than other nodes. The fact that supernodes find consensus on a topic is likely to affect the network (because they control many funds) but the migration should not be done as such that supernodes get super powers.

I do understand that, but if we incentivize the supernodes to already run the catapult software before the snapshot, we are sure that a majority of the node network is ready for the catapult nemesis. Other, normal nodes can make their own choice.

#19

What is the difference between taking the opt-in multisigs and putting them in the genesis block and just take all the multisig accounts and put them in the genesis block?

Difference is that on catapult, any cosignatory must optin to being a cosignatory. Meaning if you want to create the contract you need cosigners to sign it. Nemesis could do it, but this would be an unverifiable account convertion (cannot be validly signed because we dont have cosigner private keys.)

the current proposal all the opt-in multisig account are put in the nemesis block are they not?

Thats only possible because endusers will provide with a signed convertion transaction and this transaction will be executed at time of nemesis.

I do understand that, but if we incentivize the supernodes to already run the catapult software before the snapshot, we are sure that a majority of the node network is ready for the catapult nemesis.

Totally agree here, supernodes will be a help for a smoother launch. Just wanted to point at the fact that they should not have any extra powers compared to different NIS nodes.

2 Likes
#20

Part of it is open and part of it is closed the tx format is and always has been public, the required part of the code to prove/disprove is open.