Hello devs,
Regarding sync method. How does it work actually? Do the server sync with a single point or from best node available or all nodes… etc?
If it sync from all nodes, is it possible to include a setting to limit the upload kbps speed in future release? 
The set of all nodes that your node knows about is partitioned into subsets. One subset is the set of active nodes, i.e. nodes that your node can successfully connect with, which are honest in the sense that they authenticated correctly and which can deliver data in a reasonable time. When synchronizing your node chooses a node from this subset and asks that node for blocks.
If it sync from all nodes, is it possible to include a setting to limit the upload kbps speed in future release?
Can you elaborate?
The set of all nodes that your node knows about is partitioned into subsets. One subset is the set of active nodes, i.e. nodes that your node can successfully connect with, which are honest in the sense that they authenticated correctly and which can deliver data in a reasonable time. When synchronizing your node chooses a node from this subset and asks that node for blocks.If it sync from all nodes, is it possible to include a setting to limit the upload kbps speed in future release?
Can you elaborate?
I think he means he want's to be able to limit the upload in case a node is syncing from his node.
Did I understand you corrently biscodyl ?
I'd refrain from doing so unless a high limitis hardcoded. Otherwise people will throttle syncing even further.
I think it is just fair to give the user the chance to limit the upload traffic.
1st off… sorry if it's hard to understand what i'm trying to say… english is not my 1st language…
The set of all nodes that your node knows about is partitioned into subsets. One subset is the set of active nodes, i.e. nodes that your node can successfully connect with, which are honest in the sense that they authenticated correctly and which can deliver data in a reasonable time. When synchronizing your node chooses a node from this subset and asks that node for blocks.
from what i understand… client will search a group of best node available… similar to torrent?
I think he means he want's to be able to limit the upload in case a node is syncing from his node.
Did I understand you corrently biscodyl ?
I'd refrain from doing so unless a high limitis hardcoded. Otherwise people will throttle syncing even further.
Yes, like bittorrent software, client can set the upload speed limit for seeding. however, for NIS maybe limit the minimum setting to e.g 5kbps / 10kbps, but does not allow to disable sync.
some of the reasons …
- if upload speed take place at maximum speed, it can interfere with online activities. e.g browsing speed, watch streaming like youtube, disturb other users in lan… wlan… etc…
- some people have limited upload speed, bandwidth per month… if there is 1000 new users sign up per hours, the sync will never stop, uploading at max speed continuously may consume all bandwidth…
- like bittorrent throttle… some isp may throttle speed if someone use max download / upload speed all the time.
maybe implement a builtin schedule, where it control speed periodically, e.g like utorrent… user can set to only download/upload at night… minimum speed during weekdays 9 to 5 etc…
and/or a function where only node with xx kbps speed can activate harvest… so people have to sync at reasonable speed to be able to earn = quality nodes. all low quality nodes can sit aside and weep… jk
Yes, like bittorrent software, client can set the upload speed limit for seeding. however, for NIS maybe limit the minimum setting to e.g 5kbps / 10kbps, but does not allow to disable sync.
some of the reasons ..
- if upload speed take place at maximum speed, it can interfere with online activities. e.g browsing speed, watch streaming like youtube, disturb other users in lan.. wlan.. etc..
- some people have limited upload speed, bandwidth per month.. if there is 1000 new users sign up per hours, the sync will never stop, uploading at max speed continuously may consume all bandwidth..
- like bittorrent throttle.. some isp may throttle speed if someone use max download / upload speed all the time.
maybe implement a builtin schedule, where it control speed periodically, e.g like utorrent.. user can set to only download/upload at night.. minimum speed during weekdays 9 to 5 etc..
and/or a function where only node with xx kbps speed can activate harvest... so people have to sync at reasonable speed to be able to earn = quality nodes. all low quality nodes can sit aside and weep... jk :D
You cannot compare NIS with bittorrent they are completely different beasts.
Bittorent is used to copy files that not change over time.
Time is not a key factor for Bittorent.
NIS must synchronize a blockchain whose content is generated over time by different subjects.
Time is a key factor for blockchains because if nodes communicate slowly between them the network starts foking.
If people start limiting outgoing bandwidth probably the network will never synchronize.
Fortunately most of the time there are very few blocks that need to propagate so used bandwitdh is small.
If you are really worried about bandwidth used by your node, just avoid other nodes to connect to it and allow only outgoing connections.
i was suggesting bittorrent protocol as discussion, not comparing NIS vs bittorrent in any of my previous sentences. your suggestion is worse than limiting upload speed.
if you look in another thread, i asked how NEM team look at the data size growth over time, and suggest if it is possible to split into smaller chunks, for faster sync, easier backup etc… i think, for a developer it always a possibility.
this thread merely to get clear understanding how NEM/NIS works and discuss development possibility, and most importantly i want to see the project longevity. not to see who is right or wrong…
As long as a node downloads all the needed data from only one (!) other node, it is problematic to limit the upload. Because then a node with limited upload would really slow down that process.
But if the needed data is downloaded from several nodes simultanously, it's no problem if there is a slow node.
The algorithm could even check the downstream of a node and adjust the part to download then. So from a slow node it would be less to download and from faster nodes there would be more.
Frankly speaking, nodes that want to be part of the network must fulfill certain qualities. Syncing with a node that only allows 100KBits/second is a pain. It usually results in a timeout when the blocks contain lots of transactions. The way to limit things is to run the stanalone version and put the nis.maxTransactions parameter to a low value like 500. That way you are limiting the data size that is returned from your node. On the other hand it also means that if you are requesting blocks from remote nodes you will not get many blocks and thus syncing need longer.
People who do not want or are not able to provide bandwidth should not run NIS but simply stay passive and connect NCC to an existing NIS.
As long as a node downloads all the needed data from only one (!) other node, it is problematic to limit the upload. Because then a node with limited upload would really slow down that process.
But if the needed data is downloaded from several nodes simultanously, it's no problem if there is a slow node.
The algorithm could even check the downstream of a node and adjust the part to download then. So from a slow node it would be less to download and from faster nodes there would be more.
The data is not downloaded from one, node. One batch of 100-400 blocks is downloaded from one node, the next time probably another node is picked. A blockchain is not a torrent, there is no official hash of the chain because the chain changes within minutes.
Well there are situations where it makes sense to limit the upstream.
But there could be a reasonable limit for the upload limit. For example you can't set it to a lower value than 400 kBit/s.
(The usage of this limit could even affect the PoI…)
While a node is synchronized it won't download a lot from other nodes, I know that. This whole issue is more likely relevant to new nodes which download the complete blockchain. Will they also only load 100-400 blocks from one node and then search another source?
And: I understand if this is not a high priority issue now, but in general I really don't know why a simultanouse download should not be realized sometime in the future. People could limit their upstream (in certain limits) and still nodes could download with full speed.
The data is not downloaded from one, node. One batch of 100-400 blocks is downloaded from one node, the next time probably another node is picked. A blockchain is not a torrent, there is no official hash of the chain because the chain changes within minutes.
thanks for taking your time...
if i understand this correctly, since the hash is changing, downloading have to be in 'first in first out' (fifo)?
so.. if devs develop the NIS to store data in split 50mb blocks it will be possible for a 'completed block' to have a permanent hash, and that particular block will be able to transmit bits randomly like bittorent? exclude last block...
and simultaneous sync + minimum kb upload per nis seems achievable..
i understand it is not so easy, just as discussion...
People who do not want or are not able to provide bandwidth should not run NIS but simply stay passive and connect NCC to an existing NIS.
perhaps, later devs should consider to develop a custom NCC for phone or desktop that automatically connect to a public NIS, which charge small fees to fund the infrastructure... :P
Internet bandwidth is not constant. I have v = 10 Mbps nominally, but periodically:
v = 0, the Internet at random times can be switched off for 0 … 12 hours duration. Disabling it can happen at the time of data transmission.
v < 0,1 Mbps or because the provider (technical work on the equipment sometimes) or due to other programs on the computer, which currently requires a lot of resources. On one computer to operate only NIS is unrealistic to be parallel to other programs.
Ideal nodes will not. Consequently, NEM software should work intellectually the bad situations, don't hang up, don't stray.
I understand what you guys want but for now we will not change the way we sync. There is a lot of other stuff to do before launch.
Thats totally understandable. But it would be interesting to know what you and the other devs think about the suggestion.
It looks like ethereum have been looking into similar possibility.
[url=http://goo.gl/5ycjZa]http://goo.gl/5ycjZa
BT : [url=http://goo.gl/ihFG9v]http://goo.gl/ihFG9v
Probably only matter of time to see someone came up with new sync method…
If 7890 port is closed, will whether the node used by other nodes for synchronization to download the blocks? If not, then the node will not create a problem even if it has a low bandwidth.
In other words, if a node has a low bandwidth, 7890 port should be closed?
If port 7890 is not open for incoming calls, the node will not answer to synchronization requests. But not opening the port also means that the node does not help distribute new blocks/transactions and that is bad. A better way for low bandwidth nodes that run the standlone version is to set nis.maxTransactions in config.properties to a low value like 500. That means if another node synchronizes then the data transferred is not large.
500 means 500 kbps or something other?
The upload should be measured and the "nix.maxTransactions" (which sounds for me as the actual number of transactions) set to a reasonable value automatically.