NIS Synchronization Problems

Looks like we are everybody DOSing each other :slight_smile:

We can lower timeouts in /etc/sysctl.conf :
<br />net.ipv4.tcp_fin_timeout = 20<br />net.ipv4.tcp_keepalive_time = 600<br />net.ipv4.tcp_keepalive_probes = 4<br />net.ipv4.tcp_keepalive_intvl = 15<br />net.ipv4.tcp_synack_retries = 2<br />net.ipv4.tcp_syn_retries = 3<br />net.ipv4.tcp_max_orphans = 16384<br />net.ipv4.tcp_max_tw_buckets = 16384<br />net.ipv4.tcp_retries2 = 10<br />net.ipv4.tcp_tw_reuse = 1<br />net.ipv4.ip_local_port_range = 16384 65535<br />
and apply changes with:
<br />sysctl -p<br />

I also limit concurrent connections from same IP with iptables connlimit module and connection rate from same IP with iptables hashlimit module


Looks like we are everybody DOSing each other :)

We can lower timeouts in /etc/sysctl.conf :
[code]
net.ipv4.tcp_fin_timeout = 20
net.ipv4.tcp_keepalive_time = 600
net.ipv4.tcp_keepalive_probes = 4
net.ipv4.tcp_keepalive_intvl = 15
net.ipv4.tcp_synack_retries = 2
net.ipv4.tcp_syn_retries = 3
net.ipv4.tcp_max_orphans = 16384
net.ipv4.tcp_max_tw_buckets = 16384
net.ipv4.tcp_retries2 = 10
net.ipv4.tcp_tw_reuse = 1
net.ipv4.ip_local_port_range = 16384 65535
[/code]
and apply changes with:
[code]
sysctl -p
[/code]

I also limit concurrent connections from same IP with iptables connlimit module and connection rate from same IP with iptables hashlimit module


Will this still work if I have my node set to ipv6?

Also, any get this to work without trouble on digitalocean?

Looks like we are everybody DOSing each other :)

We can lower timeouts in /etc/sysctl.conf :
[code]
net.ipv4.tcp_fin_timeout = 20
net.ipv4.tcp_keepalive_time = 600
net.ipv4.tcp_keepalive_probes = 4
net.ipv4.tcp_keepalive_intvl = 15
net.ipv4.tcp_synack_retries = 2
net.ipv4.tcp_syn_retries = 3
net.ipv4.tcp_max_orphans = 16384
net.ipv4.tcp_max_tw_buckets = 16384
net.ipv4.tcp_retries2 = 10
net.ipv4.tcp_tw_reuse = 1
net.ipv4.ip_local_port_range = 16384 65535
[/code]
and apply changes with:
[code]
sysctl -p
[/code]

I also limit concurrent connections from same IP with iptables connlimit module and connection rate from same IP with iptables hashlimit module
I was looking at my router log yesterday and noticed a ton of incoming connections on port 7890 and NEM wasn't even running (but it was running earlier in the day). I had to close the port to stop the connections. Other then that I am using lubuntu and no problems.


Looks like we are everybody DOSing each other :)

We can lower timeouts in /etc/sysctl.conf :
[code]
net.ipv4.tcp_fin_timeout = 20
net.ipv4.tcp_keepalive_time = 600
net.ipv4.tcp_keepalive_probes = 4
net.ipv4.tcp_keepalive_intvl = 15
net.ipv4.tcp_synack_retries = 2
net.ipv4.tcp_syn_retries = 3
net.ipv4.tcp_max_orphans = 16384
net.ipv4.tcp_max_tw_buckets = 16384
net.ipv4.tcp_retries2 = 10
net.ipv4.tcp_tw_reuse = 1
net.ipv4.ip_local_port_range = 16384 65535
[/code]
and apply changes with:
[code]
sysctl -p
[/code]

I also limit concurrent connections from same IP with iptables connlimit module and connection rate from same IP with iptables hashlimit module
I was looking at my router log yesterday and noticed a ton of incoming connections on port 7890 and NEM wasn't even running (but it was running earlier in the day). I had to close the port to stop the connections. Other then that I am using lubuntu and no problems.


Let's hope there is not a retentive NIS table of IPs in the NIS nodes in the network. This should not be the case. Perhaps devs should take note of this.


Looks like we are everybody DOSing each other :)

We can lower timeouts in /etc/sysctl.conf :
[code]
net.ipv4.tcp_fin_timeout = 20
net.ipv4.tcp_keepalive_time = 600
net.ipv4.tcp_keepalive_probes = 4
net.ipv4.tcp_keepalive_intvl = 15
net.ipv4.tcp_synack_retries = 2
net.ipv4.tcp_syn_retries = 3
net.ipv4.tcp_max_orphans = 16384
net.ipv4.tcp_max_tw_buckets = 16384
net.ipv4.tcp_retries2 = 10
net.ipv4.tcp_tw_reuse = 1
net.ipv4.ip_local_port_range = 16384 65535
[/code]
and apply changes with:
[code]
sysctl -p
[/code]

I also limit concurrent connections from same IP with iptables connlimit module and connection rate from same IP with iptables hashlimit module


Will this still work if I have my node set to ipv6?


These settings have no effect on IPv6.

But I think you are mostly using IPv4 even if IPv6 is enabled.

Looks like we are everybody DOSing each other :)


Can you explain in more detail?
(NIS got DDos protection)

It looks likes that even after shutting down NEM (both NIS and NCC) that the other nodes are still pinging you. I ran the client for a bit, checked my incoming router log and saw multiple connections on port 7890. Shutdown the client and after 15 mins those same nodes are still trying to connect.

I'm testing other cryptos and I don't see this activity with them.

Shouldn't the clients be sending out acknowledgement that there are online instead of being constantly pinged to see if they are online?


It looks likes that even after shutting down NEM (both NIS and NCC) that the other nodes are still pinging you. I ran the client for a bit, checked my incoming router log and saw multiple connections on port 7890. Shutdown the client and after 15 mins those same nodes are still trying to connect.

I'm testing other cryptos and I don't see this activity with them.

Shouldn't the clients be sending out acknowledgement that there are online instead of being constantly pinged to see if they are online?


Hi averagejoe,

nodes are constantly checking out what other nodes are online. That way they discover new nodes and remove unreachable nodes from their collection. I don't see a big problem in that. Relying on the nodes sending the info that they are online is not enough since they probably can't reach to entire network. We will reduce the number of of nodes which are pinged in a round for the next release.

Bloody Rookie


Hi averagejoe,

nodes are constantly checking out what other nodes are online. That way they discover new nodes and remove unreachable nodes from their collection. I don't see a big problem in that. Relying on the nodes sending the info that they are online is not enough since they probably can't reach to entire network. We will reduce the number of of nodes which are pinged in a round for the next release.

Bloody Rookie
That's good that the number will be reduced because the way it is now seems like a wasteful way to do it. How about having local groups? A local group, which could be as little as 1, those peers then relay information to their local groups if needed (like a new transaction for example). How about having nodes self load monitoring, so if you have no available capacity to harvest, respond to transactions or have blocks downloaded from your node it doesnt respond to requests again until it does?

Just some ideas.


Looks like we are everybody DOSing each other :)


Can you explain in more detail?
(NIS got DDos protection)


I was referringĀ  to the high number of connections experienced.
Is there a limit to the number of outgoing connections NIS does?
Is there a limit to the number of incoming connections NIS accepts?

That's good that the number will be reduced because the way it is now seems like a wasteful way to do it. How about having local groups? A local group, which could be as little as 1, those peers then relay information to their local groups if needed (like a new transaction for example). How about having nodes self load monitoring, so if you have no available capacity to harvest, respond to transactions or have blocks downloaded from your node it doesnt respond to requests again until it does?

Just some ideas.


hi averagejoe,

having an active network management is more complicated than what we have now. Since we have enough things to do before launch, I think we leave it as it is now for the moment.

Bloody Rookie

I was referringĀ  to the high number of connections experienced.
Is there a limit to the number of outgoing connections NIS does?
Is there a limit to the number of incoming connections NIS accepts?


Hi rigel,

we have "nis.nodeLimit = 20" in the nis configuration. This limits the number of outgoing connections for one broadcast. Of course there could be more than one broadcast within a short time (more than one incoming new and valid transaction and block, refreshing nodes, broadcasting experiences).
For incoming connections, there is no explicit limit.
We do have a general limit of 100 connections in the http method client though.

Bloody Rookie

Are there any plans to stop nodes pinging nodes that have been shutdown? I was running my node for awhile then shut it down and 30 mins later i still had 50 nodes pinging me ( i checked my router incoming log and had to close the port).

@averagejoe: each node should only ping you one time to see if your node is still there. Realizing it is down they will not do it a second time.
This behavior will not change in the near future as we have more important things to do.

Today I am stuck at 21966.
Just made a restart and will let it run for while.

ButtCrack: are you running 0.4.17?


ButtCrack: are you running 0.4.17?

Yep.
It looks I found a work around for the problem.
So after I made a restart and waited for some hours again I was still stuck.
Then I felt like trying to start harvesting even knowing that you should wait until synced.
But guess what happened when I hit start harvest ?
Yep my wallet started to sync and within minutes it was finished.
So right now back in business as such.
Only problem is that users that have no coins can't use that trick to get in sync again.

Believe me , ButtCrack, harvesting can't help when it comes to syncing the chain :slight_smile:


Believe me , ButtCrack, harvesting can't help when it comes to syncing the chain :)

I agree, does not make sense to me either, it should have no impact whatsoever.
But it seemed to have helped in my case.
Wallet was stuck, no restart and right when I started to harvest, wallet started to sync.
And I mean exactly when I hit "start", not a split second later.
I will try to replicate if I get stuck again.
Will upload the logs later on, maybe it gives you a clear answer what really happened.

@ButtCrack: After opening the wallet, NIS needs 1 minute to boot. 1 minute can be perceived as a long time if you are waiting for synchronization to start.