NEM Symbol Wallets - plan for release by Nov 20th

This is an update for the community on the current state of the Wallets and how we intend to resolve the situation, there has been a lot of conversation on Twitter, Telegram etc and this post is to address it all.

It has been posted similar to the Single Source of the Truth thread with comments disabled so that we can post updates and it reads easily, a second post will be created to allow discussion and gather input. We do want input, we also want an easy information point for people so will split the two threads.

Summary

The summary is the Wallets are late and what has been released with every Testnet so far (including the 3 month soak version) is not good enough, this is a problem that goes back some 12-18 months and pre-dates NEM Group who took over the SDK/Rest from NEM Studios and Wallets from the NEM Foundation in late May. We know it is a problem, the community knows it is a problem and we need a strong push to fix it and have a plan.

There are some mitigating circumstances we could present but we do not want to dwell on what has come before in this post or waste energy that can be better used resolving difficult conversations and issues.

Current State

The current state of the wallets can be summarised as:

  • Mobile: the code/architecture that has been created for this project in the past is not of good quality, that causes problems particularly in the Ui/Ux and ability to get it to display consistently across devices, rebrand and reuse components between screens. This code base has already started being refactored and the Ui/Ux work has also started. Most of these issues did not become apparent until the teams tried to fix the Ui/Ux for the Opt In Mobile Application which uses the same framework, that is what caused the mobile opt in release to be later than desktop.

  • Desktop: the structure of the code is reasonable, the way features and functions have been implemented into it is not as good and there are a lot of bugs/issues in the current version. Additionally the Ui/Ux is not great and there are features we consider must haves that are not present and have not been started. By our estimation, there is approximately 90-100 days (total, not elapsed time) of test/dev time to bring it up to where it needs to be so a team of several developers, managed well under and agile process should manage to deliver this in the next ~4 weeks, with interim releases before the final one.

  • Requirements: Neither of the above wallets had until recently an appropriate requirements document, which makes it very difficult to both ensure they are all present or to test that they work. NEM Group discovered this issue to our surprise in late July/early Aug and have been working with various people to retrospectively compile a reliable one and then work out which gaps exist. This has had to happen in parallel with the focus on getting Testnet launched on time, Opt-in released and a multitude of other things for launch. We accept this should have been spotted sooner given we have owned since late May/early June, it took 2 months to get to be able to realise the gap and fix it, however this was in parallel with a lot of other launch critical work. It has the majority of the tech team’s focus since it was identified as yet another recovery job that should have been completed by previous owners given the dates that were being communicated for a public launch date last year.

  • Timelines: We have heard a couple of times recently that wallets did not have a defined timeline or set of requirements to meet so they are not late/under quality. We do not agree with this statement, they at minimum should have been ready for Testnet in Sept 2020, but they should have been ready by Q3 2019 with the exception of new features added in the protocol since then (Finality, Voting, BLS etc) given the timelines communicated last year. We consider them to be significantly late and this applies to Desktop, Mobile and Hardware wallets.

The Plan

The plan from here. Let’s face it, this is the important bit and the one everyone wants to know about. We have spent the past week with various internal and external teams working out a plan and exactly what needs to happen to try and release a production ready wallet(s) by 20 Nov, so that there can be a few weeks usage on Testnet and avoid a launch date impact.

The below is what will be happening, it will all be as public as possible and we will want community input on most of it:

  • We will reply on this post with a copy of the requirements document that is being worked from, we are targeting all Must and as many of the Should requirements as possible - we think all are achievable in the desktop, mobile we are still assessing. We encourage feedback on the discussion thread if there are things that are not present in it
  • The initial Ui/Ux refresh designs will be finalised by late this week, with a view to publishing it for community discussion, likes/dislikes etc, we don’t commit to being able to include all feedback in the production version, but will do our best. Anything that doesn’t make it for launch will be looked at in a roadmap item soon after launch. This will be polishing for Desktop due to timescales, Mobile will be matched more with the Opt In mobile app experience that some have seen already.
  • A sprint started this week and will run for 2 weeks, it will be followed by another 1 week sprint after that, then a stabilisation/bug fix sprint of 1 week after that (4 weeks)
  • Each sprint will have a planning session focussed on being sure on what the sprint is delivering and these will published on the forum, on this thread, as soon after the session as practical, along with views of our Asana project boards where possible
  • Each sprint will have testing and a release of the wallet, ideally a usable public release if at all possible to allow community use, testing, issues, feedback etc
  • Github - it will be committed to our public github, so you can see the work going on in plain sight
  • A CI/CD job will be set up to create nightly releases of the desktop wallet for those interested, these will not be fully tested or release ready but will include the features as developed and issue fixes for anyone that wants to try the release out
  • We are prioritising the Desktop wallet but expect to have an acceptable MVP of the mobile wallet by the same date. We also expect to have outstanding features to be delivered shortly after launch if necessary but nothing too major.

We want to be very clear about the fact that in our view the wallets have been behind for far too long, we do not accept responsibility for the previous failings but we fully accept responsibility for not spotting it for 6-8 weeks after taking over.

We now have a plan that we believe will result in a usable wallet for the Testnet for the final few weeks of the soak test period. We are also cognisant that there is a lot of work to be done to right the problems, we have a blended team of developers and testers devoted to it and intend to do as much in public as possible with a view to hitting the 20th of November.

There is a risk this may not be achieved, as with all project work, but we are mitigating it as far as possible, if it is not possible to deliver this by on/around 20th Nov, then we will be into the territory of discussions about a launch delay due to the Wallet not having enough time to be reliably soak tested with Testnet. We are not there yet and will try and avoid this but it is a large risk and we want to be as open as we can about that fact that it exists.

The next communications to expect will be:

  • Requirements Document (within 24 hours)
  • Sprint Plan for 1st Sprint (24-48 hours)
  • Ui/Ux refresh (late this week or early next week).

We will also communicate here how the CI/CD can be watched and reviewed as soon as it is available, it may take a few days.

The team is listening to all feedback and commentary even at times it is hard to reply to it all, we are focussed on rapid delivery of the Wallets to make up the time that has been lost in the past 12-18 months. I trust the above all makes sense and look forward to continue communicating on this topic as openly and frequently as practical

JAPANESE

15 Likes

The discussion thread can be found here:

7 Likes

Symbol Wallets Requirements Document

As noted above, there are a few documents we would like to share as soon as practical to show what is being worked on. The first of these is the Requirements Document.

We have made a copy of it which is publicly accessible, I have tried to configure it so it should be able to be translated using google translate but please let me know (direct message due to comments being disables) if it is not working correctly.

This is not expected to be a hugely interesting read but it gives an idea of the features we are targeting (all Must and all Should if possible).

The next communications will be the Sprint Plan calendar for the first sprint which has now started.

Edited: original doc uploaded had a few errors in the mobile must vs should, now corrected

4 Likes

Symbol Wallets Sprint Plan

The link below shows the initial sprint plan for the next few weeks for Symbol Desktop and Mobile Wallets.

This plan is a working document and is based from an Asana board of tasks that the team are moving multiple times a day, we will try and produce an updated version of this at the start of each sprint. It also has a visual roadmap on the last slide. Things may still come in and out of scope as the work happens but broadly this is the target.

I have put text summaries on the slides at the bottom of the image which I think will allow google translate to work, if it doesn’t please let us know on the discussion thread.

Symbol Wallets Nightly Build Previews - now available

The wallet development team have set up a CI/CD (Continuous Integration/Continuous Deployment) server which is a publicly accessible hosted version of the wallet. You can just click the link below and use it (including importing any profile you have on Testnet already):

We will not intend to to deploy the wallet in this way for Mainnet or any production private chains but it is the easiest way to give access to regular builds without uninstall/reinstalling etc.

You will see that the wallet has a red banner at the bottom of the page which indicates it is not for production use, the profile may or may not work from one day to the next because it is being rebuilt with new commits everyday so backing up the profile if you create one is a good idea because you may need to re-import it.

It is connected the live Testnet so anything you do on chain should still be present between the nightly builds, even if the wallet itself changes in some way.

The primary purpose of this is to allow the community to see updates as they are being made - we will still create a proper version release at the end of each sprint but if there is a bug or a feature you see on Github get closed that you particularly want to try or see, this is an way to do it.

6 Likes

Symbol Desktop Wallet Ui/Ux Mockups

The link below shows the expanded and refreshed Ui/Ux for the desktop wallet. It should be noted that these are the intended designs at the start of the sprint delivery, they show the expected direction of travel but some of the details may be tweaked as the team go through delivery. We do not expect massibe deviations but they are also not 100% definitive.

If the community has input, likes/dislikes etc please leave comments on the discussion thread

9.2 Symbol Desktop Wallet Ui-Ux

6 Likes

(Posting this for @DaveH due to forum restrictions stopping another reply)

We are pleased to announce that Sprint 1 of the revised wallet plan has completed as planned and the teams have made very solid progress, this sprint has resolved a variety of issues, started to revise the UI and laid the framework to allow Sprints 2 and 3 to go smoothly.

The first sprint was a 2 week one, we are now into weekly sprints with a release at the end of each sprint:

The release was recently tagged on Github as per the plan and the team have now started on Sprint 2 which will release on 9th November 2020.

As a reminder, Sprint 2 is primarily focused on Delegated Harvesting, you can continue to see progress by monitoring the nightly build server here:

At this time the wallet delivery is proceeding to plan for delivery by 20th Nov 2020 as communicated recently, there is no change to that date or launch at this time

6 Likes

Symbol Desktop Wallet v0.13.3 Release

The second planned release of the revised Symbol Wallet plan has just been released on 9th Nov 2020 as promised.

Progress this Sprint has been strong and the team have achieved a lot in 1 week since the last release. Below is a link to the Github tagged release:

The most prominent features that this one addresses are probably:

  • Delegated harvesting activation/deactivation
  • Multi-sig account creation/editing
  • Address book

The next release is targeted for 16th Nov and is primarily targeted on aggregate transactions, with various other smaller features, fixes and tidy up.

As always, please do let us know if there are any issues by logging them on Github.

The release note is below:

[0.13.3][v0.13.3] - 09-Nov-2020
Milestone: [email protected]

Added
Added AddressBook to the user profile.
Added AddressBook backup function.
Added AddressQR feature to symbol-qr-library.
Added Sender and Recipient filter in transaction history.
First part (1/2) of Delegated Harvesting release. Enabled delegated harvesting activation and deactivation on existing known nodes only. Activation status and full peer nodes selection will be released in the next drop.

Changes
Recipient input can now use known contacts from AddressBook
Contact names (known contacts from AddressBook) are shown in the transaction history list.
Restyled news page.
Removed modal close buttons.
Restyled general modals structure.
Aligned dropdowns with input sizes.
Restyled confirm buttons on modals.
Fee dropdown hover styles.
Fix multisig add cosignatories was overlapping.
New layout in settings modal.
Refactored navigation links for settings and account.
Styled contact list.
Styled address book details page.
Disabled encrypting / decrypting message for multisig account.

Fixed
Fixed minApproval and minRemoval showing the wrong values issue.
Blocked users form adding currently selected account as a cosigner for itself.
Fixed multisig graph showing on non-multisig account issue.
Set overlay component position to block transactions forms properly in multisig accounts
Fixed issues in transaction history export csv.
Various UI issues fixed.

6 Likes

Symbol Desktop Wallet v0.13.4 Interim Release

As most are aware the interim release yesterday (v0.13.3) had some issues with Delegated Harvesting that were not evident until it was tried on Testnet. These have been resolved and a new release has been created and tested.

Thanks to everyone in the community who worked hard to both find and diagnose the issues, it has allowed the team to pick them up and fix them quickly

The release is tagged here:

A reminder from the release yesterday:

  • This release is part 1/2 in terms of harvesting, the second part should come in the current sprint.
  • This release allows activation/deactivation of harvesting,
  • The exact harvesting status of an account relies on a change to REST server so will not function until both the final sprint deliverable and the next Testnet patch (part 2/2).

One thing has changed since yesterday - it will only be possible to connect to known NGL nodes for the next week or two, one of the issues found with certificates needs to be resolved at the REST level so will also come in the next REST release and Testnet patch. Until then, the selection of which node to harvest on is controlled by a predefined drop down list of ones we know it can support. There is a chance that this means the slots fill up on those nodes, however it is the only way to ensure it activates reliably until part 2/2. We will try and keep an eye on those slots but if you get a “slots full” type error - try another node in the list for now. This decision was taking to allow delegated harvesting to start as soon as possible, without waiting for the Core Server + REST release for the patch to Tesnet.

Thankyou to the various members of the community for finding and helping with diagnosis of the issues in particular, a special shout out to @tomotomo9696 and @GodTanu along with various members of the Japanese community on the little endian issue.

The dev team have done a great job of turning round the issue and issuing an unplanned patch release without impacting the current sprint plan.

That means the desktop wallet is still on track for full release on or before 20th Nov, as always we will update if any of that changes.

A final reminder - this is an INTERIM release, it has been through some internal testing, it is still a work in progress, it is possible further issues/bugs are found. This is expected and is why the delivery process has been set up in sprints - to allow us to release incrementally, get feedback and fix issues quickly while we work on recovering the Symbol Desktop Wallet delivery timeline which started ~3 weeks ago.

Please let us know of any problems, issues, questions as before, please do let us know

The release note is below:

[0.13.4][v0.13.4] - 11-Nov-2020

Milestone: [email protected]

Fixed

  • Fixed delegated harvesting activation issues.
  • Added existing NGL harvesting nodes to delegated harvesting node selector for function preview purposes.
  • UI fixed on the delegated harvesting activation confirmation page.
8 Likes

Symbol Wallet Release Update (18th Nov)

I wish to give massive recognition and thanks to the to the wallet dev teams of NGL and Peersyst who have worked tirelessly for the past 4-5 weeks to achieve this announcement.

It is with great pleasure that I am able to give this update and begin to close out the wallet release plan.

The short version:

The last sprint finished on 16th November and the teams have been finalising, testing etc between then and now. The features all work as they should in the development environments and have been smoke tested on the Testnet. There are some known bugs/minor issues which will be worked on over the next few weeks (see release note and Github), any issues raised by the community will be worked on along with those.

The features that exist in the releases will be the features we go live with for Symbol, subsequent releases before launch will be bug fix/patch releases. However, new feature development will continue with a view to adding more features post launch.

Further detail on this will be provided in the next few weeks

Symbol Desktop Wallet

A short while ago, the release 0.13.5 was tagged, this version fixes various issues and completes the harvesting features, it also adds Metadata and Aggregate Transaction support.

It is worth noting that the harvesting status and node selection does rely on a new feature in REST, that won’t be present until the Testnet is patched with the new core server + rest release, until then, you will need to use the drop down selector for the nodes as with the last wallet release.

We expect to make patch releases but no new feature releases in the coming weeks, there are known issues on Github and the team will begin clearing those down in the coming days. We expect to see the issue board much more organised and a smaller list very shortly after some issues that date make many months.

[0.13.5][v0.13.5] - 18-Nov-2020

Milestone: [email protected]
Added
  • Added Delegated Harvesting (Part 2/2).
    • Split Delegated harvesting activation into 2 steps: Key linking & Persistent delegation message sending.
    • Activation is now supporting multisig accounts by sending key link & persistent delegation message in AggregaredBonded transaction for cosigning.
    • Allow the user to manually input the remote node’s public for the node key links.
    • Extended harvest node selector to load all peers. Node public key is loaded automatically for all Dual Nodes with latest catapult-rest changes implemented (Due to be released with the next full server release and Testnet patch, note this will not be fully functional until the release, until then please use the known nodes from the drop-down list or provide the node public key manually).
    • Activation status has been changed to using catapult-server’s unlockedAccount diagnostic API (Due to be released with the next full server release and Testnet patch, note this will not be fully functional on all peer nodes until the release).
    • Added delegated harvesting eligibility check (10K stake)
  • Added MeataData for Account / Modaic / Namespace.
    • Metadata creation with randomly generated scope key.
    • Existing metadata modification will be implemented in the next release.
  • Added Aggregate Transaction support.
    • Currently support 3 types of inner transaction: Transfer / MosaicDefination / NamespaceRegistration.
  • Added AddressBook import from JSON back up.
  • Added logout button under the settings page.
  • Added AddressQR to contact details.
    • AddressQR currently cannot be exported. We acknowledged this issue (not logged in github issues). This will be fixed in the next release.
Fixed
  • Fixed account name not showing properly after the update.
  • Fixed bugs in transaction CSV download.
  • Fixed bugs in account deletion from multiple profiles.
  • Fixed invalid node URL bug.
  • Fixed the bug of transaction date in transaction history.
  • Fixed harvesting blocks UI.
  • Improved aggregate transactions UI
  • Accounts restyled. The separation between private keys and seed accounts
  • Aligned all dropdowns.
  • Download paper wallets on account creation.
  • Network settings showing a list of nodes.
  • Fixed private key overflow.
  • Changed transaction modals styles.
  • Other UI fixes.

NOTE: We have known issues that have been logged into github and will look for fixing them in the next release.

Android Mobile Wallet

Feature work is complete, the Ui/Ux is being polished up and some final issues worked on, the intention is to release this as a prebuilt APK and tagged release on Github on Fri 20th November.

It will have been tested by the NGL team, however with mobile apps it is really only when wider user bases start to use them, install them on different devices etc that we see the how it really fares in the wild. So any assistance or usage by the community will hugely help the final end product.

We do expect to make patch releases but no new feature releases

Apple Mobile Wallet

Is is in a very similar position to the Android Mobile Wallet, however in order to “release” a test version we need to employ a tool called Test Flight. This requires sign ups with Apple IDs and email addresses etc to be whitelisted for testing. Further details will be provided in the next few days but it is expected this will work approximately as below:

  • Friday 20th Nov: Announce it is released, add a small number of known users to test flight to ensure it is working
  • Monday 23rd Nov: Create a task on NemHub for a closed beta of ~20 users to expand the test base, keep that running for 1-2 weeks
  • Open up to a wider open beta - this requires approval from Apple to be possible so is awkward to predict

The exact timings may alter slightly but the basic plan is as above for the testing


That ends a rather long update and I would like to close by again thanking the development team for their momentous effort in achieving a very aggressive target - and not only achieving it, but actually coming in early on the desktop wallet.

We will from next week be asking the development team involved to scale back and recover for a few days/couple of weeks, they will still be working and progressing issues, but for people’s health and sanity I will be asking them to drop to lighter workloads and to recover so everyone stays healthy and looking forward to Symbol launch when it comes.

9 Likes

This is a dummy post to reset @DaveH reply count so he can respond on the thread with the mobile releases shortly.

Standby…

2 Likes

Thanks @DavidM

Symbol Mobile Wallets Release v0.4

As noted above, it is with great pleasure that we can announce the release of the Android and iOS mobile wallets today. The combined team from NEM Group and Peersyst have done a fantastic job to pull things back in such a short time frame. The codebase was almost completely refactored and Ui/Ux is now very different to what it was when we started this 4 week recover plan. The end result is something the team should be very proud of.

There are few remaining issues (as with the desktop wallet) which will be resolved in the coming weeks while it is live on testnet, but the features are complete and now frozen. It is likely we will make a couple more patch releases as issues are found and fixed. The apps have been tested by our team, however mobile apps always respond better in the real world with different handsets, languages, mobile networks, other apps - all that stuff. So the testing period is very likely to find issues which we will resolve before launch.

The testing approach for each is slightly different, due to the operating systems and how you install apps.

Android

We have built a separate custom APK that will install alongside the Symbol Opt In application, it is not listed on the Play Store for now until it goes through the first round of community usage, just in case it disrupts the opt in app or local storage etc. You will see the app installs as one called “Symbol Beta” top make it easy to identify.

It should be safe to use on the same device as the opt in app. We will in a future release, look to replace the opt in app (you will still be able to opt in) but this will be communicated ahead of time and be an update like any other app you use.

The APK can be downloaded from here and installed by following the prompts

Apple

True to form, Apple have some strict approval and testing processes, this will mean that the app becomes available in stages:

  • Initially it will be released using a platform called Test Flight, from Apple. This will run as a closed Beta test group and is limited in size to ~20 people, it requires whitelisting Apple IDs and email addresses etc

  • After that closed group (we think likely 2-3 weeks but depends on Apple) we can release it in an open Beta/test mode where everyone can access it.

  • Then finally, similar to Android we will upgrade it on the App Store and it will replace the Opt In app. It will be possible to install the two apps side by side until that point

The closed beta group requires a little bit of co-ordination and a task will be posted on NEM Hub in the coming days for a call to involvement. Due to the limited numbers, we will need to restrict initial access to people who intend to actively test rather than just see what it looks like, unfortunately.

Translations

The apps are currently in English only, the translation work will continue and be added alongside the other issue resolutions. It was decided, given recent events, it would be better to not have them and ensure they are triple checked

Faucet & Explorer

A reminder that:

If you replace XYZ with your address, it will auto-populate

GitHub Repositories

Right now the code is all on private repositories. Over the next few days it will be moved to the public ones and made open as part of the post release tidy up work

Release Note

Because the repos are private, I can’t link a release note so have pasted it below, some of it is a progression from the earlier ones so may not make full sense but shows the latest features that were added:

[0.4][v0.4] - 20-Nov-2020

Account management

  • Import account by mnemonic
  • Import account by MnemonicQR
  • Create new account
  • Back up account with paper wallets
  • Back up account by mnemonic verification
  • Display account details on dashboard
  • Add child accounts by index
  • Add accounts to profile by private key or scanning QR
  • Edit account names
  • Remove accounts
  • Display account details
  • See account in explorer link
  • Add testnet mosaics with the faucet

Send / Receive

  • Send transfer transactions with multiple mosaics
  • Send transfer transactions with message encrypted
  • Prepare transfer transactions QRs
  • Notifications of incoming and outgoing transactions. (Confirmed and unconfirmed)

Transaction list

  • Display full transaction history
  • All transactions have now a link to explorer
  • Filter transactions by send and received
  • See transactions from your multisig accounts
  • Decrypt transaction messages
  • Sign aggregate bonded transactions from your account or from your multisig accounts

Harvesting

  • Delegated Harvesting part 2
  • Show delegated harvesting info (blocks collected and fees earned)
  • Allow start delegated harvesting form simple account
  • Allow stop delegated harvesting form simple account
  • Allow change node of delegated harvesting form simple account

Mosaics

  • See mosaics information through mosaic list
  • Transfer mosaics with simple transfer transactions
  • See expired mosaics

News

  • See the latest news from NEM with the RSS channel

Settings

  • Switch between locales. (Currently only English supported)
  • Turn on / off passcode and touch / face ID
  • Logout and remove all data from your account
  • Bad network connection / node down alert

NOTE: We have known issues which have been logged and will look for fixing them in the next release.

8 Likes

Update to the mobile wallets post above:

Mobile Wallet Github Repo

Please add all issues, feedback etc to the issues board on this repo

NEM HUB Task for iOS Testing

3 Likes