Anybody with low power computer (or even single board computer) running NIS/NCC?

mixmaster · September 3, 2015, 1:22pm

Since there is a huge network spamming since 3 or 4 days, my cubie board 3 has problems. NIS crashed 3 times now. Always with this error:
SEVERE I/O reactor terminated abnormally [...] Caused by: java.net.SocketException: Too many open files

I'm not sure what is the reason for the crash. It is assumed that the CPU is just too weak (ARM Cortex-A7 Dual-Core 1GHz).
The cubie has 2 GB RAM and I have no swap configured. NIS and NCC are running on the cubie.

I did some logging and processed it in Excel:
https://www.dropbox.com/sh/dzhaqpn4c4rmxn7/AAC_BS93FlHTD-aBODvcgLHWa?dl=0

Maybe somebody has an idea?

Next thing to try is: limit the RAM usage for NCC. NIS is at default settings (min 512 MB, max 1 GB), maybe I set min 512MB, max 786MB for NCC?

BloodyRookie · September 3, 2015, 1:22pm

You let ncc run on the cubie too? Maybe try with NIS on cubie and NCC elsewhere first.

mixmaster · September 3, 2015, 1:22pm

I know this could help, but I would rather like to find a solution so I can keep running both on the cubie. It never crashed with 0.4.48-BETA before so it actually is pretty stable. Just with a lot of tx/Block for long time there is a problem.

patmast3r · September 3, 2015, 1:22pm

I know this could help, but I would rather like to find a solution so I can keep running both on the cubie. It never crashed with 0.4.48-BETA before so it actually is pretty stable. Just with a lot of tx/Block for long time there is a problem.

As the db grows really small nodes run out of RAM. I don't think you'll find a solution for that in the near future.
Theoretically release should make it work again as we're starting over.

mixmaster · September 3, 2015, 1:22pm

So you think this is a RAM problem? 2 GB is not enough to run the current db size?

Anyway, this would mean I can solve it with a SWAP file, right?

patmast3r · September 3, 2015, 1:22pm

So you think this is a RAM problem? 2 GB is not enough to run the current db size?

Anyway, this would mean I can solve it with a SWAP file, right?

2 GB should be plenty. 1 GB is recommended i think.
I didn't know that little thing has so much ram :)

mixmaster · September 3, 2015, 1:22pm

Yes, its pretty nice for a mini computer

mixmaster · September 3, 2015, 1:22pm

0.5.4-BETA without xfce:

NIS start: 7 minutes, 59 seconds
Block analysis (0 till 4400): 6 minutes, 30 seconds

So this is still pretty slow compared to previous versions. (I know it depends on how many transactions happened - but it is still slow.)

NCC start: 31 seconds

BloodyRookie · September 3, 2015, 1:22pm

15 seconds here for the block analysis on a decent PC. That is a factor 30
I really would like to know what is slowing down the analysis down so much…
Any way to profile on that cubie board?

mixmaster · September 3, 2015, 1:22pm

Hm, I don't know :-/ Maybe solix has an idea?

lcharles123 · September 3, 2015, 1:22pm

I run in vps service, 1GB RAM and 30 of HDD for $ 19 / year
[url=https://core.weloveservers.net/cart.php?a=add&pid=64]https://core.weloveservers.net/cart.php?a=add&pid=64

root@nemcoin:~# wget freevps.us/downloads/bench.sh      -O - -o /dev/null|bash CPU model :  Intel(R) Xeon(R) CPU E3-1240 V2 @ 3.40GHz Number of cores : 4 CPU frequency :  3400.024 MHz Total amount of ram : 1024 MB Total amount of swap : 1280 MB System uptime :  12 days, 19 min, ...

Here the resource monitor:

This server are NEM, NODE (livenet and testnet), NXT and NFD running at the same time.

KingCole · September 3, 2015, 1:22pm

I took delivery of a Raspberry Pi 2 about 10 days ago and have been running it with NIS 0.5.13-BETA. It seems to be able to cope at the moment so might be a solution to a low power NIS server?

KC

mixmaster · September 3, 2015, 1:22pm

Yes, the Rasp Pi 2 is definitely an interesting solution to run a NIS node at home.

@Icharles123: You are using a lot of swap. I am wondering if the performance is still good? $19 is really cheap…

KingCole · September 3, 2015, 1:22pm

I took delivery of a Raspberry Pi 2 about 10 days ago and have been running it with NIS 0.5.13-BETA. It seems to be able to cope at the moment so might be a solution to a low power NIS server?

KC

Might have spoken too soon. The Pi has now locked up twice whilst running NIS requiring a reboot. Not sure if the Java parameters might need tweaking when the device only has 1GB?

KC

mixmaster · September 3, 2015, 1:22pm

Can you provide more details for your problem?

Anway: If you have only 1 GB RAM and no swap, you should change the nix.runNis.sh from
#!/bin/bash cd nis java -Xms512M -Xmx1G -cp ".:./*:../libs/*" org.nem.core.deploy.CommonStarter cd -
to
 #!/bin/bash cd nis java -Xms512M -Xmx768M -cp ".:./*:../libs/*" org.nem.core.deploy.CommonStarter cd -
and then start NIS again.

Otherwise it will happen that NIS takes 1 GB RAM, but your system also needs a little bit, so that can't work.

BloodyRookie · September 3, 2015, 1:22pm

@KingCole: you might even try to limit to 512M if 768M is still not working. I do that for my 1GB Alice 6 node.

KingCole · September 3, 2015, 1:22pm

Can you provide more details for your problem?

Anway: If you have only 1 GB RAM and no swap, you should change the nix.runNis.sh from
[code]#!/bin/bash

cd nis
java -Xms512M -Xmx1G -cp ".:./*:../libs/*" org.nem.core.deploy.CommonStarter
cd -[/code]
to
[code]
#!/bin/bash

cd nis
java -Xms512M -Xmx768M -cp ".:./*:../libs/*" org.nem.core.deploy.CommonStarter
cd -[/code]
and then start NIS again.

Otherwise it will happen that NIS takes 1 GB RAM, but your system also needs a little bit, so that can't work.

Thanks Mixmaster I'll give that a go and see if stability improves, I'm hopeful the Pi2 will be a nice little NIS server.

KC

gimre · September 3, 2015, 1:22pm

I run in vps service, 1GB RAM and 30 of HDD for $ 19 / year :D

19 A YEAR?! - WOW, :o did you noticed any problems with availability? (I mean random reboots and so on)

KingCole · September 3, 2015, 1:22pm

Can you provide more details for your problem?

Anway: If you have only 1 GB RAM and no swap, you should change the nix.runNis.sh from
[code]#!/bin/bash

cd nis
java -Xms512M -Xmx1G -cp ".:./*:../libs/*" org.nem.core.deploy.CommonStarter
cd -[/code]
to
[code]
#!/bin/bash

cd nis
java -Xms512M -Xmx768M -cp ".:./*:../libs/*" org.nem.core.deploy.CommonStarter
cd -[/code]
and then start NIS again.

Otherwise it will happen that NIS takes 1 GB RAM, but your system also needs a little bit, so that can't work.

Thanks Mixmaster I'll give that a go and see if stability improves, I'm hopeful the Pi2 will be a nice little NIS server.

KC

Things are looking better. Top is showing 90% memory utilisation for NIS. Will see what happens overnight, but impression is a more responsive machine.

KC

sheld0r · September 3, 2015, 1:22pm

Can you provide more details for your problem?

Anway: If you have only 1 GB RAM and no swap, you should change the nix.runNis.sh from
[code]#!/bin/bash

cd nis
java -Xms512M -Xmx1G -cp ".:./*:../libs/*" org.nem.core.deploy.CommonStarter
cd -[/code]
to
[code]
#!/bin/bash

cd nis
java -Xms512M -Xmx768M -cp ".:./*:../libs/*" org.nem.core.deploy.CommonStarter
cd -[/code]
and then start NIS again.

Otherwise it will happen that NIS takes 1 GB RAM, but your system also needs a little bit, so that can't work.

Thanks Mixmaster I'll give that a go and see if stability improves, I'm hopeful the Pi2 will be a nice little NIS server.

KC

Things are looking better. Top is showing 90% memory utilisation for NIS. Will see what happens overnight, but impression is a more responsive machine.

KC

I'm running the same setup PI 2 limited to 768 MB and so far so good but i think the memory usage is crawling slowly up.

edit:
Played around with the NIS mem setting and 768 MB is defenitly the max anything above the PI2 will have a hard time to get the db synced especially when a lot of spam is going around ...