I’m an IT consultant for a private society who has a nem wallet and I’m trying to get some help with a seemingly difficult issue to crack. The project has been up and running for months in catapult, and a couple years on previous public NEM then migrated private. But a script attack on the mongo db has left us unable to sync and bring the wallet up.
We have a private nem wallet, two nodes. The docker environment is installed and the one that contains the MONGO db is our main problem. Apparently the mongo db is open without a username / password and our nodes are being hit with a hack that dumps the db and replaces it with junk and a ransom note. It’s been reported that this is a vulnerability with mongo, 27K databases effected. So our devs who are new to NEM have been trying to secure that DB to no avail. Attempts to do so are difficult to script and we get a wiretiger error which is driving us nuts. I have a document with details but couldn’t post it and it contains info on where we’re at. At this point I need a consult, maybe just an hour of someone’s time to discuss the issue with my devs and see if we can get somewhere with it. We’re willing to pay for the consult. Our project has been down for over a week now and we’ve been close to restoring it for days…just keep hitting different sections of the same wall. Please help, PM me and I’ll fill you in more. The project is a private society wallet and runs on catapult, from my understanding it’s also a private blockchain built on NEM…not a part of the main network. I have a document of the issue and can share, but when I tried on the telegram group I got kicked out.
If anyone can help, please contact me.
here is the wiretiger error:
2020-10-31T00:08:37.877+0000 E STORAGE [thread2] WiredTiger error (2) [XXXXXXXXXX:877505][1:0x000000000000], log-server: /dbdata/journal: directory-list: opendir: No such file or directory
2020-10-31T00:08:37.877+0000 E STORAGE [thread2] WiredTiger error (2) [XXXXXXXXXX:877551][1:0x000000000000], log-server: log pre-alloc server error: No such file or directory
2020-10-31T00:08:37.877+0000 E STORAGE [thread2] WiredTiger error (2) [XXXXXXXXXX:877559][1:0x000000000000], log-server: log server error: No such file or directory
2020-10-31T00:08:37.877+0000 E STORAGE [thread2] WiredTiger error (-31804) [XXXXXXXXXX:877564][1:0x000000000000], log-server: the process must exit and restart: WT_PANIC: WiredTiger library panic
2020-10-31T00:08:37.877+0000 I -[thread2] Fatal Assertion 28558 at src/mongo/db/storage/wiredtiger/wiredtiger_util.cpp 365
We may need to run a DB repair on the Mongo instance to try and get it running again. A quick search suggests this is from the Mongo instance crashing with tasks still running.
So the issue now is to get that error cleared. At this point if we can clear that error I’m hoping the servers will come back up and be reasonably secure. This error is in the db container so it has a kind of domino effect of crashing any other service connected to it.