@pheromone: can you tell me the supernode’s name?
A few questions :
-
reward for round 3345-3348 went fine yesterday, but reward for round 3341-3344 (correlation with empty reward wallet) is still missing. Was it just postponed, or this reward cycle has been canceled for everyone (I hope not, there’s no reason for it, I believe) ?
-
my node (#810) failed in round 3353 for Height, Chain Part, Computing Power and Version. Which is interesting / weird, as I have lot of capacities for it and this node was “everything in green” for long months before. Could it be some coincidence with upgrading of something in SN environment or something like that? Has anybody else some “failing experiences” with round 3353 as well ?
-
just a quick note about servant - I still have for running of ./startservant.sh script initial values, which means: -Xms256M -Xmx256M . Is it still enough for running this “guard” ? I suppose it is, but I’m just asking, just in case…
Thanks a lot!
hmpf…looks like you are right and some transactions that the database has recorded for payout actually are not in the chain 
Well that means i have to analyze and find all those missing payouts i guess. Will do that on weekend.
Hard to say what went wrong. I know that there were internet problems in central Europe lately, I myself had unstable internet connection in my location the last few days.
Yes, that should be enough, even 128M should be enough.
supernode name is xmen 
actually it has failed since a few weeks ago,
if thread higher than 500, load average will be above 1.00 for 4vcpu, i have tried up to 50 at this time, it gets around 0.25 load average at all time. probably it fail when transaction frequency increase? will try to increase more thread and see what happen.
The node seems to have very unstable computing power test results. This could be due to your provider hosting other vpses on the same hardware that have peeks in their cpu usage. Maybe you should switch to a more reliable provider like vultr?
Yes, probably too many users. Will try to play with the setting for couple more days, see how it goes. But it look like it’s time for upgrade. Thanks!
Hi @BloodyRookie I am still missing payout for successful round 3341 - 3344 for my supernode NEMczech. Please solve the issue. Thank you.
@janikjan: Zdar kolego ;). Jsem na tom stejně, ale BloodyRookie už mi odpovídal (post #2362), bude to manuálně procesovat o víkendu…
1 Like
BR, are you pretty sure, that everything is fine with measurement system (anyway, we discussed it before, but I’m not sure about current state : Do you have some measuring point also in [Central] Europe, Asia, etc., or every points are just in [I guess] North America?) ? Why I’m asking : My node has been failed in last three rounds (#3358, #3359, #3360) for Bandwidth and Computing Power. Which is really weird, as connection of my server is still the same (very strong 
…it’s directly on the gig upstream network), computing power as well, and everything “was green” for several months. Last fail that I observed, was sometime in mid December 2017, and it was just because I had default values for memory in ./nix.runNis.sh script and that time was NEM network very active due to price rising. So that time I just upgraded memory values for higher limits and everything went smooth again…till now.
Next two rounds (#3361, #3362), next two fails. Bandwidth+Computing power, and in round #3362 also Responsiveness failed (628ms [0/10] ). I checked periodically all the items around my connection, everything is really fine, so there must be something wrong between (at least) my node and measuring node(s), or trouble with measuring itself.
Of course I’m feeling a little bit sad, as I miss several rewards, even if it’s not my fault
It failed all servant tests the last few rounds, so maybe the servant has a problem?
The tests for Bandwidth and Ping are delegated to supernodes that have a good ping to your vps, the computing power test is done by the master.
Since other nodes in Europe were not failing, either the special location was the problem or the vps did not respond for some reason. I will check the logs…
@KloNEM: what i see in the logs is that many nodes report that they cannot ping (done by servant component) your node. Also the master reports that it cannot connect to your node (again to the servant compoenent):
2018-04-13 22:35:42.583 WARNING LinuxCZ did not reply to ping, reason: org.nem.core.connect.InactivePeerException: java.net.ConnectException (org.nem.rewards.master.task.PingTask lambda$getPingResults$23)
Did you restart NIS or the servant in the last few days?
Ok, just say me, what should I check within the servant config or so … but it’s also interesting, as I have still the same environment and set up/conf (not just servant) for really very long time. And also, servant is in version 0.0.4, which is quite old, but I suppose there is no newer version yet, is it ? (As there probably was/isn’t a reason for newer version) …
Yes, I did a restart (as NIS, as servant) somewhere on Thursday (2018-04-12) early morning (of GMT), due to updates as on virtual guest (NIS node), as on physical host, that NIS virtual instance is running on (I’m maintainer of both). But I did these updates and restarts more often, let’s say once per month on average (due to kernel updates, Spectre & Meltdown “funny issues” and so) and didn’t observe such kind of trouble any time before.
I’m also investigating, if this issue couldn’t be related with something in kernel updates (I just updated both machines from kernel 4.15.7 to 4.15.15), but didn’t find any relevant post e.g. on bugzilla about that till now. I’ll do more research, of course…
Just one question, just theoretically: Couldn’t it relate with some NIS DB corruption or something like this, so shouldn’t I try to remove DB and build all the blockchain again from scratch ? I guess it’s not so much probably / relevant, so I’m just asking (and thinking about any aspect of this weird issue).
In any case, thanks for your co-operation, BR!
No, the failures were on the servant side, that has nothing to do with NIS DB.
Depending on the downtime during updates this can be the reason of a test failure.
Right now, your node seems to be responsive, so it should pass tests. That will be visible in round 3363 or 3364 (The displayed information is always behind).
Ok, thanks for confirmation.
It can, but it’s also very unprobably - as the first faiI I observed just a day (more than 24hrs) after the restarts, and “full trouble” start just on the weekend; which is several day after updates. So I guess it will probably not related with.
Ok, let’s wait what will happen in the next round(s). In the meantime, I’ll do some further checks and inspection if there isn’t something suspicious or non-standard (but it’s also not so much probably, as I didn’t change nothing relevant really long time ago). Thanks again!
I made an analysis of the payout part of the database due to the failure we saw a few days ago. Here is a list of all payouts that went wrong:
payout_failures.pdf (56.4 KB)
All missing payouts will be made soon.
1 Like
Ok, let’s wait what will happen in the next round(s). In the meantime, I’ll do some further checks and inspection if there isn’t something suspicious or non-standard (but it’s also not so much probably, as I didn’t change nothing relevant really long time ago). Thanks again!
[/quote]
So, rounds 3363-3367 went fine, “everything in green”. I’m still really not sure, what’s happened there (maybe it were really some network outages on the way, somewhere in the middle?), but I hope that this issue is definitively over.
Anyway, missing payouts for “postponed rounds 3341-3344” still didn’t arrive…do you have some ETA, when the payouts will be processed ? 'm just asking 
… thanks!
Will mijin require higher specs than current version?
Does it still require java?