Symbol Supernode Programme - community testing

DaveH · May 3, 2021, 11:49am

All,

The Symbol Supernode issues have been resolved and tested in both a development environment and NGL nodes on the public testnet. We are now ready for community testing.

This allows us to ensure the different hosting/OS/config settings in the wider network behave the same as our reference platforms.

We encourage anyone who wants to get involved in the testing, and particularly those who intend to enrol a Supernode on Mainnet, to please follow the guides below and let us know if there is anything missing, there are any problems etc (via Github for problem reports).

If you have Bootstrap already running, you will need to update to the latest version which includes the new agent.

Step by Step Guide: Enrolling to the Supernode program — Symbol Documentation - please ensure you use the TESTNET commands
Bootstrap v1.0.5 has been released Release [1.0.5] - May-3-2021 · nemtech/symbol-bootstrap · GitHub

Notes

Testnet has been refreshed since launch and the enrolment process is now different, so anyone who enrolled via the previous process or on older Testnet will need to re-enrol.

To upgrade an older Bootstrap you will need to run the command below:

npm install -g symbol-bootstrap

If you have an existing node running via bootstrap then you should add the --upgrade option to your start command after upgrading Bootstrap - this will pull down the new agent etc for you.

Next Steps

Following community testing on Testnet, we will schedule a launch date for Mainnet, this is dependent on any issues found on Testnet and how many/how quickly community nodes enrol on Testnet. At this stage we expect to require minimum 1-2 weeks of a reasonable number of community nodes running on Testnet before scheduling launch

Any questions please do let us know

freigeist · May 3, 2021, 8:22pm

Ok thanks for the info.
Here it is 1st problem at 1st step

I have run the following command:
npm install -g symbol-bootstrap

but is does not seem to update the older version.
I do not get any errors it seems to be installed correctly.
When i then issue symbol-bootstrap help or symbol-bootstrap --version
i get the following:
VERSION symbol-bootstrap/0.4.5 linux-x64 node-v14.16.0

which is the old version i had a month and half ago.

BTW: i have used the command symbol-bootstrap clean
to remove the target folder before i have called the npm install -g symbol-bootstrap

Any ideas what could be wrong here?!
Thanks.

C2daB · May 3, 2021, 8:42pm

To update the version of symbol-bootstrap, use the following commands:

symbol-bootstrap stop
cp -r target target_backup
sudo npm install -g symbol-bootstrap
symbol-bootstrap start -p mainnet -a dual -c custom.yml --upgrade -d

Check the version by running:

symbol-bootstrap --help

garm · May 3, 2021, 8:53pm

why “sudo” ?

C2daB · May 3, 2021, 9:04pm

Whenever a user tries to install, remove or change any piece of software, he has to have the root privileges to perform such tasks. The sudo command is used to give such permissions to any particular command that a user wants to execute once the user enters a user password to give system based permissions.

garm · May 3, 2021, 9:16pm

please have a look here :

fboucquez · May 3, 2021, 9:16pm

Sudo when installing global npm may or may not be needed. It depends if node js has been installed globally or user locally. It’s fine either way.
The import point is that once installed, boostrap is used using the local user.

klimgeran · May 4, 2021, 6:12am

Русская версия:

Программа Symbol Супернод — тестирование сообщества

ИЗМЕНИТЬ

https://nemnews.io/programma-symbol-supernod-testirovanie-soobshhestva/

DrCryptos · May 4, 2021, 12:54pm

sudo npm install -g symbol-bootstrap

garp · May 4, 2021, 5:21pm

The guide for Enrolling to the Supernode program mentions two methods, the symbol-bootstrap method and the manual method.

The symbol-bootstrap method does not specify how to proceed if you want the agent to use another port and how to configure that other agent port in the custom preset file.

That is particularly disappointing as the same issue was reported before launch and the question was not answered with instructions or documentation then. At least I would have expected an official solution and answer by now.

It’s even worse. Then, it was already mentioned there is a conflict with the NIS SN Servant that already uses port 7881 and that using another port would be required in some configurations.

Why is the default port and the corresponding documentation not changed to 7882 for example?
And if it stays port 7881 by default as mentioned in the documentation now, why is the alternative to use another port not mentioned and described at the same time.
Why do we test? Why do we report issues?

I’m not shooting at the messenger, but these situations unfortunately keep repeating and are symptoms of problematic behaviour towards community members.

Xavi · May 4, 2021, 6:47pm

Hi @garp,

I’m @Xavi the guy who wrote the supernode enrollment guide. Let me try to clear up some of the confusing points.

That is on purpose, to keep the guide simple. The manual section of the guide, which is meant for more knowledgeable users does explain how to change the port (without using Symbol Bootstrap).

I’ll update the guide to add this information, since it seems to have become a regular paint point, but I think the need for it is diminished as explained below. Please bear with me a bit more.

The problem was that the Symbol agent and the NIS1 agent both used port 7880 and clashed. This was detected by the community and we are infinitely grateful for that!
The quick workaround was to change Symbol’s port via configuration. The better solution, which has already been implemented and deployed, was to change Symbol’s default port to 7881. I am afraid this has not been communicated to the community and we (NEM Group) are to blame for that.

So the current situation is that, by default, Symbol Bootstrap uses a different port than NIS1, so no clash should occur. And that’s why there is no mention of the issue in the documentation.

There is still a mechanism to change Symbol Bootstrap’s default port for the agent, though, and I am going to add it to the documentation shortly, in case it helps in other situations.

Thanks for reporting, as usual!

freigeist · May 4, 2021, 9:24pm

Hello.
It’s me again. I just wanted to inform you that I found out what the problem was
on my system and why the symbol-bootstrap did not update after installation.

1st reason was that i was using the command npm install -g symbol-bootstrap instead of
calling npm update -g symbol-bootstrap before install.

2nd reason was that i had a globally installed older version which i installed with sudo command
before i made changes that @garm have posted above.
The action from the point 1 did not have any effect because the pointer (symlink) was pointing to the old version so that calling symbol-bootstrap --version was always showing the old version.

I then uninstalled the older global version with sudo and the new global version without sudo.
I have then reinstalled the the new global version without sudo and made
manually the symlink from the /usr/local/bin/symbol-bootstrap to my /home/$USER/.npm-packages/bin/symbol-bootstrap

Thank you all for your help.
I will now try to proceed further by following the instructions.

AnthonyLaw · May 5, 2021, 6:09am

I like to use npm install -g symbol-bootstrap@latest so it will always point to the latest version.
not sure it’s helpful for you.

garp · May 5, 2021, 7:03am

Thank you Xavi for clarifying, appreciate it.
So the two previously needed and undocumented paramaters in the custom config file for symbol-bootstrap are no longer needed. Very good.
Again, thank you for confirming.

Xavi · May 5, 2021, 9:07am

They have been turned into unneeded and documented

rigel · May 5, 2021, 4:27pm

Almost two hours has passed and the transaction isn’t confirmed.
Low fees or testnet problems?

$ symbol-bootstrap enrollRewardProgram --useKnownRestGateways
                         _             _         _                    _         _                      
  ___  _   _  _ __ ___  | |__    ___  | |       | |__    ___    ___  | |_  ___ | |_  _ __  __ _  _ __  
 / __|| | | || '_ ` _ \ | '_ \  / _ \ | | _____ | '_ \  / _ \  / _ \ | __|/ __|| __|| '__|/ _` || '_ \ 
 \__ \| |_| || | | | | || |_) || (_) || ||_____|| |_) || (_) || (_) || |_ \__ \| |_ | |  | (_| || |_) |
 |___/ \__, ||_| |_| |_||_.__/  \___/ |_|       |_.__/  \___/  \___/  \__||___/ \__||_|   \__,_|| .__/ 
       |___/                                                                                    |_|    
? Enter password to use to encrypt and decrypt custom presets, addresses.yml, and preset.yml files. When providing a password, private keys will be encrypted. Keep this password in a secure place! 
2021-05-05T13:23:37.518Z warn     Password has not been provided (empty text)! It's recommended to use one for security!
2021-05-05T13:23:37.566Z info     Looking for the best node out of: http://ngl-dual-501.testnet.symboldev.network:3000, http://ngl-dual-601.testnet.symboldev.network:3000, http://ngl-dual-001.testnet.symboldev.network:3000, http://ngl-dual-101.testnet.symboldev.network:3000, http://ngl-dual-201.testnet.symboldev.network:3000, http://ngl-dual-301.testnet.symboldev.network:3000, http://ngl-dual-401.testnet.symboldev.network:3000, http://ngl-dual-502.testnet.symboldev.network:3000, http://ngl-dual-602.testnet.symboldev.network:3000, http://ngl-api-001.testnet.symboldev.network:3000, http://ngl-api-101.testnet.symboldev.network:3000
2021-05-05T13:23:39.081Z info     Connecting to node http://ngl-dual-501.testnet.symboldev.network:3000
2021-05-05T13:23:42.457Z info     Node's minFeeMultiplier is 85
2021-05-05T13:23:43.481Z info     Creating enrolment transfer with message 'enroll https://51.178.44.174:7881 LS0tLS1CRUdJTiBDRVJUSUZJQ0FURSBSRVFVRVNULS0tLS0KTUlHU01FWUNBUUF3RXpFUk1BOEdBMVVFQXd3SVFXZGxiblFnUTBFd0tqQUZCZ01yWlhBRElRQmtaVkxZeVlOYwpCYnp4VDM0cjVoTW1ockc2VUxiYnNJYmduNXozRlIyNWtxQUFNQVVHQXl0bGNBTkJBR2JySzZTRFFvd1NDR2E0CjYzU1lqSUJ6Q1ZUWXVOYW00Z21JYUI5ekFCYXRSZlBhWDZYU3FCTk9pZ1ZkYjBuY0Npc3g0VDEvZ0p1bStaWUsKeWdpSTd3VT0KLS0tLS1FTkQgQ0VSVElGSUNBVEUgUkVRVUVTVC0tLS0tCg=='
? Do you want to announce TransferTransaction - Hash: F4A911FE34D9DECD4B640D9E66C22751F3ACAD4C0878C40EAC810EDE00E9DB30 - MaxFee 0.047855? Yes
2021-05-05T13:23:47.715Z info     Announcing TransferTransaction - Hash: F4A911FE34D9DECD4B640D9E66C22751F3ACAD4C0878C40EAC810EDE00E9DB30 - MaxFee 0.047855

Ok, --maxFee 10000000 succeded.

garp · May 6, 2021, 8:45am

The symbol-bootstrap method seems to simplify a lot of the needed steps, so it is becoming more and more the preferred method for most. However, for various configuration options, some remaining questions remain about how to use/apply or activate these options with the symbol-bootstrap method, without being obliged to edit the generated docker-compose.yml manually before starting the node.

If one would like to use the docker-compose option “mem_limit” for one or more containers, is it possible to specify that in the CustomPreset.yml? How?
If one would like to use the docker-compose option “cpus” for one or more containers, how to specify that in the CustomPreset.yml.
If one would like to have docker restarting all containers automatically at restart (after reboot f.e.), one of the possibilities and the recommended option is to use “restart: always” for each container in the docker-compose.yml. However, symbol-bootstrap uses "restart: ‘on-failure:2’ " on most containers. I suppose this current default behaviour needs to be modified for an automatic restart. How to modify CustomPreset.yml to achieve that?
(Edit) Another question. If one registers a node for the Supernode reward program and the node is also registered in the Ecosystem reward program, how to use the Rewardprogram parameter in CustomPreset.yml? Twice?

Some of these possibilities might already be supported, but remained unnoticed. In that case an example could clarify. Thanks in advance

fboucquez · May 6, 2021, 7:55pm

Hi @rigel

Almost two hours has passed and the transaction isn’t confirmed.

The current default max fee calculated with the min network multiplier is too cheap and slow (0.047855 just not fast enough). I’ve created Use max fee of 0.1 Xym by default in enroll and link · Issue #241 · nemtech/symbol-bootstrap · GitHub and the 0.1 XYMs max fee default by fboucquez · Pull Request #242 · nemtech/symbol-bootstrap · GitHub PR wil increase the default MaxFee to 0.1 Xym. PR will be merged soon

Hi @garp

If one would like to use the docker-compose option “mem_limit | cpus” for one or more containers, is it possible to specify that in the CustomPreset.yml? How?

As we know, changing anything inside the target folder is not recommended. You need to use the custom preset to tune the configuration. If you don’t know how to change something ask the question or raise an issue or pull request.

github.com

nemtech/symbol-bootstrap/blob/main/docs/presetGuides.md#add-docker-compose-properties-to-services

# Custom presets

It's the way you can tune the network without modifying the code. It's a yml file (`--customPreset` param) that could override some or all properties in the out-of-the-box presets.

Most people would use the out-of-box preset or tune a few attributes.

The file is a hierarchical yaml object. If an attribute is defined at root level, it overrides the default value for all the affected configurations.

The attribute can also be defined in a lower level object just affecting one component (node, gateway, nemesis, etc).

The best way to validate your configuration is by inspecting the generated configuration and preset.yml files in the target folder

**If you are trying new configurations, remember to reset the previous one by running --reset (-r) or by removing the selected target folder (./target by default)**
 
## Examples

### Custom Rest image and throttling

```yaml
symbolRestImage: symbolplatform/symbol-rest:2.3.2

This file has been truncated. show original

If one would like to have docker restarting all containers automatically at restart (after reboot f.e.), one of the possibilities and the recommended option is to use “restart: always” for each container in the docker-compose.yml. However, symbol-bootstrap uses "restart: ‘on-failure:2’ " on most containers. I suppose this current default behaviour needs to be modified for an automatic restart. How to modify CustomPreset.yml to achieve that?

The custom preset overrides what ever is default at shared, network (testnet/mainnet) or assembly (dual, peer, api) level

github.com

nemtech/symbol-bootstrap/blob/main/presets/shared.yml#L163


transactionDisruptorMaxMemorySize: 20MB
transactionElementTraceInterval: 10
enableDispatcherAbortWhenFull: false
enableDispatcherInputAuditing: false
maxTrackedNodes: 5'000
enableAutoHarvesting: true
fileDatabaseBatchSize: 1
maxUnlockedAccounts: 10
delegatePrioritizationPolicy: Importance
dockerComposeVersion: '2.4'
dockerComposeServiceRestart: 'on-failure:2'
dockerComposeDebugMode: false
minPartnerNodeVersion: 1.0.0.0
maxPartnerNodeVersion: 1.0.255.255
serverVersion: 1.0.0.0
outgoing_connections_maxConnections: 10
outgoing_connections_maxConnectionAge: 200
outgoing_connections_maxConnectionBanAge: 20
outgoing_connections_numConsecutiveFailuresBeforeBanning: 3

incoming_connections_maxConnections: 512

In this case, you can use the custom preset:

dockerComposeServiceRestart: always

(Edit) Another question. If one registers a node for the Supernode reward program and the node is also registered in the Ecosystem reward program, how to use the RewardProgram parameter in CustomPreset.yml? Twice?

Use rewardProgram: Ecosystem instead of rewardProgram: SuperNode at nodes level

rigel · May 6, 2021, 7:58pm

I’d like to set up some kind of alert when my node fails a round.

Unfortunatly the page Symbol Block Explorer can’t be parsed because contains no data if you don’t run JavaScript in your browser.

Is there some static page that can be parsed?
Something like NEM Node Rewards where I just have to search for the string “FAIL”.

fboucquez · May 6, 2021, 8:02pm

Hi @rigel , are you a supernode owner?