With huge thanks to the all the development and test teams who have been working night and day for weeks now; we are able to make the announcement below.
All the issues from testnet have now been resolved and various patches, config and deployment improvements have been made, we are ready for release and Testnet reset.
Shortly we will be performing the below (target today):
Full Server release including new Bootstrap and network configs
Public Testnet reset
Commmunity nodes on Testnet
Community testing, and particularly NEMTus testing
Further stress tests with the nodes distributed
The exact timing will depend on various build jobs and validations but likely to be late in the day UTC time today (Fri morning Japan time).
An announcement with full instructions will be made when it is complete.
A new version of Core-Server, Rest, SDKs, Bootstrap and various supporting components will be made, this contains all the patches and config fixes found from the issue resolution over the past 3-4 weeks, plus some that could not be applied on the old Testnet from before this issue occurred.
The new release also contains significant memory savings/optimisations both on the core server and the MongoDB. There are ongoing memory optimisation investigations as well which may or may not be incorporated in future patches depending on what is found.
Assuming no significant issues are found (P0 or P1 type issues) this will be the last major release prior to Mainnet and resolves all issues found in the last stress test. Some minor patches may be made, with full testing (see below).
A classification and resolution process will be written up so it is easy to communicate publicly. It will be posted soon to explain how the test team are have been approaching and will continue to approach issue classification and resolution between here and Mainnet.
It will be necessary to reset Testnet as communicated previously. There are several changes that cannot be cleanly applied to the old Testnet. Finality has also stalled due to a deployment issue during the soak test.
This reset gives us a clean environment with the Mainnet code that will be deployed at launch.
To perform this reset, we have been running a new 500 node network internally for ~1 week, which has had a significant number of transactions pushed into it (almost the same as the old Testnet) as part of the stress testing. This data is being left there on purpose (i.e. not reset to an empty chain) to ensure the remaining soak period is performed on a chain that has a substantial data in it.
That does mean it currently takes 3-5 hours to synchronise from zero. We are working on a quicker synchronisation approach via Bootstrap which will be available as soon as appropriately tested. It will download a copy of the chain and retrospectively validate/verify it, similar to the NIS1 approach that some will remember.
Short term it will be necessary to take a few hours to synchronise a fresh node and is an important test that we would like community nodes to perform.
@GodTanu we will try and fix the Link issue on bootstrap from the other thread in time for the release, but I cannot be certain until it is looked at
The Testnet has now had the following tests performed and passed:
- Automation/Regression testing on each patch
- 100tps soak test - passed - meets stated Mainnet target
- 150tps soak test - passed - exceeded stated Mainnet target for 12 hours with no issues
- 400tps soak test - passed - throttled at ~130-200tps and when load stopped, recovered
- A final regression test was happening overnight on 2 minor patches, no issues are anticipated.
As communicated previously, the next steps are:
Perform a release (today)
EDIT: This has now occurred - NEM Symbol 0.10.0.5 Release Announcement & Testnet Reset
Bring community nodes onto the new Testnet
To organise community testing, and particularly NEMTus testing
NGL reruns the stress tests with a distributed network
At that point we will then be able to plan a launch date, I am confirming in this message that launch will occur AT LEAST one month from the point of the release. It is depends heavily on the points above and no definitive date will be selected until they are complete and any issues that are raised have been assessed.
For the avoidance of doubt, I am therefore not confirming the launch is 1 month from today, I am saying it will not be less than that time and until we complete testing, the date will not be known.