NEM Framework - Community Proposal

[bcn]

NEM Framework - Community Proposal

Making NEM Blockchain Developer Friendly

UPDATE:

Here you can find the full documentation for the project: https://docs.google.com/document/d/1xORV9EEiR_2mR1qLI9lj5SzyNlE-TTJ_ofH1EH6BSVU/edit#

Furthermore we would like to invite everyone to an Ask Me Anything session on Friday 9th at 12h GMT to answer any question that may arise from the document.

Introducing NEM Framework

After about a year building NEM/Mijin based applications, we would like to propose a long term development project with a needed budget of 15 million XEM from the Community Fund.

In order for NEM to become the leading platform for building enterprise-ready blockchain applications, we need to lower the entry barrier and allow any developer to obtain the platform’s benefits without requiring an extensive knowledge of its core architecture.

It is because of this that our team has been working on the definition of a Development Framework. It will essentially be a set of tools, libraries and views that will enable any developer to create powerful and robust products on top of NEM technology.

The main components of the system would be:

• Libraries that will allow and facilitate that any system integrates with NEM/MIJIN in an agile and secure manner.
• DSL, a Concept Abstraction layer through a traditional Object Oriented system.
• Proxy Layer allowing complex business logic on servers while keeping the private keys on the client.
• Connection Pool that would ensure constant connectivity.
• TestRPC component enabling Blockchain TDD
• Visual components set that will standardize and secure user input.

Some of the benefits that such a Framework would offer are:

• Faster training
• Faster development times
• Safer products
• Better products
• New Business models
• Stronger Dev Community

With the objective of developing a consistent ecosystem, the NEM Framework will be designed following the good development practices of the NEM core team and the vision of the NEM Foundation. Furthermore, we will exclusively use designs and technologies that help the scalability of the systems and the health of the network.

More information about the project may be found here*

About the Team

This project is proposed by Atraura Blockchain and is the result of the experience acquired by working on multiple projects during the last 4 years, the last of them being focused on building NEM/Mijin products.

Some of the NEM projects Atraura Blockchain has worked on are the following:

NEM Authenticator: Open source mobile application built from scratch using Ionic 2, that provides a software security layer to NEM Blockchain for non experienced users to secure their assets. This project is being used as a demonstration of how the Framework would look like. More information can be found here.

NEM Pay: Open source mobile wallet based on QM’s nanowallet and built on ionic that can be easily customized for any organization and mosaic. It is currently accounting for at least 3 different customizations. NEM Pay source code can be found here

NEM Alias System: Open source nanowallet module that enables users to easily transact with each other without using a 40-character long string of alphanumeric characters but a simple alias instead. The project has been discontinued since it will be native to NEM in the near future. More information about the project can be found here

NEM Voting Center: Open source nanowallet module that empowers NEM/Mijin with Governance capabilities. The project will be released in the short term and will enable everyone to create polls, proposals and even setup a Governance system for their own projects.

Landstead: Citizen and Land Registry application that was built as a Proof of Concept for Dubai’s Blockchain Govhack with great reviews. The site of the project can be found here

More information about Atraura Blockchain can be found here. Atraura Blockchain is part of the Atraura group.

Active members

Although the team required for the development of the project may increase significatively throughout the process, the current team members are the following:

Albert Castellana - albert@atraura.com / @kstellana
Product Owner

Aleix Morgadas - aleix@atraura.com / @deleted_user_1
Architect

Guillem Sole - guillem@atraura.com / @guillemsc
Developer

David García - david@atraura.com / @david360
Developer

Andreu Rodriguez - andreu@atraura.com / @anrodon
Developer

Alvaro Llobet - alvaro@atraura.com / @Alvarollobet
Marketing

Gerard Bernal - gerard@atraura.com / @GerardBe
Marketing

Technical reference

An extensive technical reference document has been written for the proposal and will be shared with the core developers for discussion and improvement.

Questions regarding this document can be addressed to any team member through Telegram or at the channel @nemframework.

Amount Requested

We would like to propose a long term development project with a needed budget of 15 million XEM from the Community Fund.

The amount requested will be used to build the Framework, the necessary tools for fast product/PoC development and to give assistance to the developer community.

Stakes in the Game

Our team is fully involved in NEM as community members, investors and product developers. This is because our mission is to help NEM become the main platform for enterprise ready applications and we intend to accomplish it building the necessary tools and products to create a thriving economy.

Milestones

Milestone #1:

• Basic Framework PoC (NEM Authenticator)
• Framework basic Website
• Technical Reference Draft
  Budget percentage: 10%

Milestone #2:

• NEM Library (ECMA6 and TypeScript)
• Continuous Integration System
• Ensure Mijin Integration
• Technical Documentation & Code of Conduct
• External Security Audit
  Budget percentage: 10%

Milestone #3:

• NEM DSL (Domain Specific Language)
• Support for WebSocket protocol
  Budget percentage: 10%

Milestone #4:

• NEM Proxy
• NEM Health Report System
• Open Data Service for NEM network
  Budget percentage: 15%

Milestone #5:

• Visual Components in Angular 4
• Visual Components in Ionic 2
• Visual Components in React: Optional
• Visual Components in ReactNative: Optional
• NEM Proposal & Voting center
  Budget percentage: 15%

Milestone #6:

• TestRPC
• TestRPC Documentation
• TestRPC Docker
  Budget percentage: 15%

Milestone #7:

• Extend the NEM Framework to use new Catapult’s features
• Framework Website
• Technical Documentation
• External Security Audit
  Budget percentage: 25%

Relevant information

Project Details and Summary: Discussed above, but more details will be given on the forum or to the core team for review upon vote.

Company: The company behind the project is Atraura Blockchain.

Real Identity: Albert and AleixMP as Product Owner and Development Team Leader will be revealing their identity during the project applications process to the Core Team reviewing the proposal.

[Paul]

Continuation of what we started on Telegram:

First off, great initiative. I just want to make sure that important questions are answered so that the community can make a better decision:

• Are you asking for 15 million XEM in full upfront?
• What are the minor milestones for each milestone? (more details)
• What are the deadlines for each minor milestone?
• What happens if you don’t meet the deadline of a milestone?
• Who is going to do the acceptance of the delivered product?
• Who is specifying the acceptance criteria for each deliverable?

[deleted_user_1]

Forwarding the telegram question, John Galt says:

are you going to report about work 1 time per month for example?

Every two weeks report

We will be creating, every two weeks, a keeps status report of our work on the project because we believe is vitally important to the success of the project to keep all the community informed.
In this reports we will focus on the following:

• Milestone start and completion dates
• Milestones reached
• Any accomplishments worth mentioning
• Important meetings attended
• Any threats or potential risks to the projected timeline
• Description of any problems we’ve encountered and resolved

Ask Me Anything

We would dedicate a live video with the community to answer any type of questions or problems about the NEM Framework.

[gimre]

M2. What exactly do you mean by Code of Conduct
M3. Why do you need DSL and how exactly would it look like?
M4. What is NEM Health Report System and how it is supposed to work, what it is suuposed to do, where it would run?
M6. What exactly is TestRPC

[bcn]

Hi Paul,

Thanks for taking the time for reading the proposal and writing the questions in the forum.

Let’s go one by one:

Are you asking for 15 million XEM in full upfront?

No, the Community Fund Guidelines estipulate that there shall not be any upfront payment and thus we have prepared a clear roadmap with the expected payments.

However, we need to prepare the necessary infrastructure and build a bigger team and as you know this is not a simple task. This is why we need a first deadline that is doable in the short term and gives us room to advance for long enough without losing focus.

What are the minor milestones for each milestone? (more details)

We will try to expand on that. At the moment we have an extensive Technical Reference that will be shared with the core developers throughout the day for their validation.

What are the deadlines for each minor milestone?

Due to the nature of the project we believe on an agile approach, therefore milestones are not time bounded. Instead, we define that an Iteration is timeboxed in two weeks. In each iteration we do the analysis, design, coding, testing, and so on.

At the end of each Iteration, the NEM Framework will have a potentially shipable state, a status report will be shared and will be released by the means described below. This will give the community an opportunity to download and integrate it in their own projects.

What happens if you don’t meet the deadline of a milestone?

The NEM Framework will have very critical features that involve other people’s money, so the development includes specific tools and strict methodology to ensure the quality.

It is because of this that we will never compromise quality and security due to time constraints.

Who is going to do the acceptance of the delivered product?

Please see below.

Who is specifying the acceptance criteria for each deliverable?

At the beggining of each milestone both the developer team and the core team will decide the acceptance criteria.

Our team will setup a Continuous Integration system that will help everyone validate the code that has been produced and therefore we will get the following benefits:

• Reduce the amount of reparative processes to build the software
• Reduce the risks of late defect discovery, low-quality software, lack of visibility, and lack of releasable software.
• Practices and techniques to ensure that we work effectively the CI system

We will focus on automating almost everything that can be by:

• Runing unit & integration tests
• Verifying code coverage & code quality
• Packaging the NEM Framework
• Publishing the NEM Framework on main package repositories

On the other hand, the development team will follow the following development practices and values:

• Test-Driven Development
• Extreme Programming

The NEM Framework documentation is part of the development process and it includes:

• Code documentation
• Setup documentation
• Usage documentation
• Samples

Because NEM Framework is open source, the issue tracking is done at GitHub page. In case that the project grows faster than expected, alternative tools will be considered.

[patmast3r]

Sounds like a great initiative but there are lots of questions to be asked and answered. You fellas are asking for a lot of money

What does the long term plan look like ? Say this frame work is produced - who will maintain it, how will that maintenance be funded and for how long ?

How much coordination has happened with tech bureau ? I feel like you’re selling a product built for something you know little about. Do you guys have inside information on catapult ?
It sounds like there could also be an overlap with catapult actually. I say that without knowing much about catapult, but if catapult introduces a third tier that is something along the lines of API servers then that strikes me as something that is potentially at least partially overlapping with this initiative. Especially if there’s talk about proxy servers here. It sounds like part of what you’re proposing would end up being a 4th layer (depicted APP SERVER) which seems incredibly inefficient. You’d have External Service -> App Server -> Api Server -> Blockchain-Core-Nodes.

What exaclty is DSL supposed to be and why would we need it. Do you mean something like solidity for ethereum ? Or more in the sense of R for statistics ? In either case, what’s the rational ?

What’s this connection pool thing about ?

With all due respect, are you aware that you’re asking for 300.000 USD for a PoC, a website and the draft of a reference ? In general the milestones strike me as a little unbalanced.

Why 2 security audits ?

Why does the framework need it’s own marketing. Shouldn’t marketing efforts be concentrated on nem as a whole ? Would you like to sell access to this framework ? I’m a little unclear about this.

Imho there can be no vote until the tech reference has been released to the community or until the core devs have confirmed that it’s sane.

Props for revealing real names to the community. Shows you mean it

[NEMergizer]

With all respect to Albert and Atraura Organization, I am totally and completely against funding any third party organization using NEM funds. NEM funds should only get spent to expand the NEM dev team internally “NOT EXTERNALLY”. Any third party organization can create their own crowd funding away from NEM dev funds. Moreover, the requested 15 Million XEM seems too high compared to the proposed solution.

[suky321]

I presume the funding will be allocated from the Community fund allocated to third parties: https://docs.google.com/document/d/1v12DKjHLShbacq4nGyQBm183ADoVgSEQNqbwUYWQBTM/edit

At the moment, I think there are too many questions unanswered, and not enough information provided to make an informed decision. The plans are very high level with not much detail and what the scope will be. The budget percentages appear to be arbitrary at first glance rather than something a lot of thought has gone into.

[gevs]

Hello !

I really like the initiative of doing such project, and doing it openly/publicly. Many developers have joined the NEM ecosystem and I believe publishing about the development of such SDK / Frameworks can be a very big benefit to companies with developers aspiring to use NEM.

I would love to see a peer review process implemented at the core of this Idea. I believe if we want this project to really help in the NEM blockchain adoption, a peer review from the NEM community is mandatory. Maybe I should wait until we can read the Technical Reference because that’s where peer review will be needed too. And also, as you state the project will be available at Github and peer review can be done there, I’m referring here to more Specific Details about what would be implemented in the first place.

Another topic I would like addressed is the Language you plan to use for this Framework? I believe that publishing a single Javascript framework would not fit the bill not only because QM started a nem SDK already but because there should be more possible ways to use the Framework than just Javascript. I believe this Project would need to include more than just one Port of a Framework. It should probably also be available in other languages such as Java, maybe a C++ wrapper, etc. The idea of the Framework should not limit the entry points of developers into the NEM space if you will.

In that sense I still wish to show support to this initiative because I believe it can be more appealing to Developers when they have out-of-the-box full support of the NEM blockchain in SDKs or Frameworks.

[Rafa_Arevalo]

What are the advantages of this initiative vs what we currently have? Thank you in advance!

[deleted_user_1]

Hi @gimre

M2. What exactly do you mean by Code of Conduct

The Code of Conduct is an iniciative to allow any community member to cooperate with the project. This is very popular in other communities/projects and is used to establish good practices and clear guidelines to participate.

This ensures a healthy project and shared responsibility throughout the community.

M3. Why do you need DSL and how exactly would it look like?

NEM Blockchain comes with a lot of concepts that can be hard to grasp by newcomers, a Domain-Specific Language describes the concepts though a traditional object oriented paradigm hidding the underlying complexity and exposing an abstraction layer.

DSL is type embedding, we define domain-specific types and implement them in terms of the types and operations for which they are intended.

Our goal is to lower the entry barrier for new developers as much as possible and maintain standardized codebases.

M4. What is NEM Health Report System and how it is supposed to work, what it is suuposed to do, where it would run?

What NEM Health Report System does is catch the success/error http codes, the network latency and report them to the System to know the actual network status.

Enabling the NEM Health Report is completely optional and all data recorded will always be published openly for anyone to access freely.

M6. What exactly is TestRPC

The TestRPC works as a stub for NEM Blockchain, it allows developers to test their products without the need of accessing the testnet. It is not intended to replace the testnet but to facilitate testing procedures and, ultimately, projects must ensure their correctnes though the testnet.

We are about to send you a Technical Reference where all above is explained in detail.

Thanks

[GodTanu]

Please tell us the development schedule.
How long time are you going to do?

[bcn]

Hi @patmast3r,

Thanks for your kind considerations, we will always try to release as much information as possible.

Please allow me to respond inline:

What does the long term plan look like ? Say this frame work is produced - who will maintain it, how will that maintenance be funded and for how long ?

Our desire is to build a platform that will allow any developer to easily get up to speed when building NEM/Mijin based applications without compromising any critical aspects.

NEM is aiming to become the go-to platform for enterprise level Blockchain implantation. However at the moment any company working on it such as Atraura, Dragonfly, Tech Bureau or any other needs to build the necessary tools to work effectively and it is indeed a very costly endeavor.

Thanks to having a common development framework we will all be able to work closer, continue extending the common tools and thus ensuring long term sustainability. A key aspect for this is the Proxy component, which will allow for new business models and therefore motivate companies to keep evolving it in the future.

How much coordination has happened with tech bureau ? I feel like you’re selling a product built for something you know little about. Do you guys have inside information on catapult ?

As you know Catapult is still going under heavy development and therefore there is little to be known which won’t be changed in the upcoming months. It is clear to us that we will need to work closely with the devs in order to accomodate the new features that Catapult will offer, however there is a lot that will not be affected by it and that is where we will begin. Of course we will still consensuate with the core developers so that no effort is done in vane.

It sounds like there could also be an overlap with catapult actually. I say that without knowing much about catapult, but if catapult introduces a third tier that is something along the lines of API servers then that strikes me as something that is potentially at least partially overlapping with this initiative. Especially if there’s talk about proxy servers here. It sounds like part of what you’re proposing would end up being a 4th layer (depicted APP SERVER) which seems incredibly inefficient. You’d have External Service -> App Server -> Api Server -> Blockchain-Core-Nodes.

The Framework is a higher abstraction layer to NEM Core and therefore, if any feature did overlap (with Catapult or any future release), we could simply use core’s implementation and any application built using the Framework would stay unaffected.

Regarding NEM Proxy, the name is simply to make it understandable, it’s goal is completely different from the API’s. The NEM Proxy component aims to enable tricky business logics that cannot be performed at the moment.

A couple examples would be:

• Delayed transactions
  NEM Proxy would allow transactions to be signed on the client (which keeps the private key) and then securely held (untransmitted) on the server’s proxy layer until a condition is met. This would be a huge breakthrough in terms of the possible business models

• Account Monitoring.
  It allows backend applications to connect to the NEM API only when an event actually happened, providing a more healthy network because it will not require all applications to be connected via Web socket and therefore overloading the network.

What exaclty is DSL supposed to be and why would we need it. Do you mean something like solidity for ethereum ? Or more in the sense of R for statistics ? In either case, what’s the rational ?

As Aleix wrote above, the DSL is not a new language like solidity but simply an object oriented layer that aims to describe domain models through a typed language such as Typescript.

This would be a possible implementation example:

let account = Account.generateWithPrivateKey("privateKey");
let transaction = new Transaction(20, "recipientAddress");
let signedTransaction = account.signTransaction(transaction);

What’s this connection pool thing about ?

The connection pool is a very important feature. The nem-library will know a set of NEM nodes (and their properties) to automatically provide the user with better nem-connectivity. In case that a NEM Node is down, the library will re-try the call using another NEM Node and therefore will distribute load throughout the network ensuring maintained connectivity, higher speeds and higher availability.

With all due respect, are you aware that you’re asking for 300.000 USD for a PoC, a website and the draft of a reference ? In general the milestones strike me as a little unbalanced.

As you can see, most of the milestone rewards are set towards the end of the development effort. However we need to build the necessary structure and maintain it throughout the project, pay for salaries, taxes, legal council and more and, although it may strike as arbitrary, due to NEM’s unknown volatility we need to avoid the project getting jeopardized. (We started working on this proposal 2 months ago)

Why 2 security audits ?

Due to the length of the project we would like to split it in two phases so that any developer can use the framework after the end of the first phase and we can get their feedback. The security audits will ensure the security levels and indicate that we are moving towards the right direction and it is because of this that it should be performed right after the Library. (The library is the one dealing with private keys), and at the end of the project.

[quote=“patmast3r, post:6, topic:5174”]
Why does the framework need it’s own marketing. Shouldn’t marketing efforts be concentrated on nem as a whole ?[/quote]

NEM is a very powerful engine but in order for the enterprises to start using it we need more applications, tools and other companies using it. We intend to build a very professional solution that will allow for fast prototyping and hopefully lead to many companies joining the ecosystem.

Azure introduces a new blockchain proof of concept framework for developers |...

In our opinion this will definitely boost NEM’s visibility as a whole.

Would you like to sell access to this framework ? I’m a little unclear about this.

The Framework will be released under Apache 2.0 licence.

https://tldrlegal.com/license/apache-license-2.0-(apache-2.0)

Imho there can be no vote until the tech reference has been released to the community or until the core devs have confirmed that it’s sane.

Absolutely, we intend to be fully transparent and share everything however we need to discuss the approach with the developers first. Thanks for your understanding.

Props for revealing real names to the community. Shows you mean it

Nothing to hide here, we want to keep a clear and open communication channel throughout the project. We will be doing video AMA sessions plus regular deliveries and reports. We sincerely look forward to hearing your feedback in order to improve the product.

Thank you!

Albert

[bcn]

Please tell us the development schedule.
How long time are you going to do?

We are following agile methodologies ad we will always prioritize security and features over time but you can find an estimation here:

NEM Framework - Overview

We will be able to do better estimations as we move forward.

[patmast3r]

Thank you very much for your detailed response. It sounds like you really thought this through.
I do think that coordination with the core devs will be very crucial though. Some of the feature you mentioned would imho be better placed on-chain (likey delayed transactions, other chains actually already have that) and I was kinda hoping we’d get to see them with catapult

Good luck with this !

[bcn]

Indeed, we would like to see that too.

However, even if we do, they would most likely be used for different situations and therefore bring different benefits.

Thanks for your support

[GetCOINtoday]

Hi, there are some points from btt forum also

1. The possible benefits to NEM. Could you plese explain simple users what benefits nem could get, for example, it could increase time of development in 2-3 times and so on. Would like to see benefits from all sides: dev and business.

2. Could you pelase decomposite budget for each milestone for clear understanding?

3. Is it possible to send payment once Milestone is reached?

Thanks in advance.

[deleted_user_1]

Answering inline

I would love to see a peer review process implemented at the core of this Idea. I believe if we want this project to really help in the NEM blockchain adoption, a peer review from the NEM community is mandatory. Maybe I should wait until we can read the Technical Reference because that’s where peer review will be needed too. And also, as you state the project will be available at Github and peer review can be done there, I’m referring here to more Specific Details about what would be implemented in the first place.

We plan to deliver early and often, a peer review/support with NEM Community will be the best cooperation that we could have, for sure. A goal with this approach is be aware of the NEM needs and provide the higher value as possible, and it has to be accomplished with good practices, great values and with the best cooperation of the all community.

Another topic I would like addressed is the Language you plan to use for this Framework? I believe that publishing a single Javascript framework would not fit the bill not only because QM started a nem SDK already but because there should be more possible ways to use the Framework than just Javascript. I believe this Project would need to include more than just one Port of a Framework. It should probably also be available in other languages such as Java, maybe a C++ wrapper, etc. The idea of the Framework should not limit the entry points of developers into the NEM space if you will.

The first implementation will be done with TypeScript language. TypeScript provides a good support for ECMA6 libraries, and we want to take advantage of the existence libraries to boost the development starting point without re-do all the crypto stuff. So, writing the first implementation with TypeScript, we provide support for TypeScript and ECMAScript6, and it allow us to write the Visual Components for Angular4/Ionic2 and React/ReactNative.

We are not planning to support another language until the project reaches a release candidate state. We are aware that we cannot maintain two code bases at same time. So, when the project reaches a release candidate state, we will publish a list of languages that we want/suggest to port the framework where the community will be able to vote which prefer to be ported.

Thanks for your support!

[tehfiend]

IMO, an advanced development framework with a low entry barrier would be extremely valuable to the NEM platform. On the surface this sounds like a project I would support but I have some concerns on the timing.

There is little information available on “catapult” and how exactly it’s going to effect the NEM platform other than the general assumption that everything is going to be better, faster, stronger. I would think that this project would create the most value after the release and adoption of the new catapult architecture. Since you are proposing the implementation of this framework before the release of catapult, how to you plan to address the migration to the new architecture?

[willowfoot]

There is little information available on “catapult” and how exactly it’s going to effect the NEM platform other than the general assumption that everything is going to be better, faster, stronger. I would think that this project would create the most value after the release and adoption of the new catapult architecture. Since you are proposing the implementation of this framework before the release of catapult, how to you plan to address the migration to the new architecture?

I agree with the above.

Also, are there ‘Development Frameworks’ already done for other coins/platforms that you would like to model?

For example, does BTC, ETH, Ripple, etc… have Development Frameworks out in public similar to what you want to accomplish?