What does difficulty REALLY mean on NEM

Technical Discussion
#1

So on Bitcoin difficulty is how hard it is to find a hash below a certain target and that difficulty increases or decreases/etc.

Obviously difficulty increases and decreases marginally with NIS (and I haven’t looked into Catapult yet for that).

But what is really happening when blocks are being written? Is a NIS guessing random numbers? What very technical thing is occurring that requires difficulty.

This leads me to my next question, we know from the technical reference that "The harvesting account gets the fees for the transactions in the block. This gives the harvester an incentive to add as
many transactions to the block as possible."

So having more powerful hardware seems to be the solution to garnering the most fees (if you have been chosen to write a block).

Is this the case?

Or is it more about just having a lucky account that gets picked based on importance score and difficulty?

1 Like
#2

I believe its totally just by chance according to how much poi you have from transacting. Hence the incentive for users to transact more to gain poi, kind of like claiming raffle tickets and you lose them over time for not transacting or holding a significant amount (above 10k xem).

#3

But I’m very curious as to how the chain is secure against 51% attack, since that’s what I understood the hashing difficulty to be meant to prevent, or maybe That’s just to maintain a 60 second blocktime in the case of nem…

#4

Difficulty influences expected value of time network should need to create block.
It is calculated from previous blocks so that average block creation time is 1 min even though not all importance is really harvesting at any given time and it changes over time (not every acc is harvesting, node goes ofline …).

Nem tech reference page 18:
Account can create block when target reaches hit.
Hit is random number created from last block hash and public key of account
Hit is importance x time since last bloc / difficulty

1 Like
#5

When in doubt, check with Technical reference, it’s all there

image
image.png673×571 67.5 KB

image
image.png793×439 48.4 KB

3 Likes
#6

what 3 minute blocktime? nem target time is 60s and average block time is close to that value

#7

No, the account’s public key and the previous block generation hash are hashed. That number (and the importance of the account) determines how long a given account has to wait until it is eligible to harvest a block. Thus, an account can have a “lucky” public key that makes it eligible to harvest a block after a short time, or it can be unlucky and the account has to wait for a long time.

No, putting transactions into a harvested block doesn’t need good hardware.

1 Like
#8

Are not you the guy running and charging for Nem courses?
We was all going to ask you.

#9

Oh really? Cool idk where I learned that then, maybe it changed since then. Thanks for correcting that.

#10

(540 blocks average)

image
image.png1073×970 64.7 KB

1 Like
#11

This was the answer I was looking for

#12

lol
Yeah that is us. The technical reference wasn’t completely clear on the actual process. It describes the algorithm that determines difficulty but not exactly “what” that difficulty is used for or how it is used.

Now I know there is no guessing of numbers or other secret algorithms (since NIS1 is not open source and we can’t know for sure) and I now know that difficulty is used in the process of CHOOSING an eligible public key.

When in doubt BloodyRookie always gives the right answer :slight_smile:

1 Like
#13

Yeah I’ve been through this multiple times and makes complete sense, but it is fuzzy on what that difficulty is used for. But now I know that it is simply used in the process of choosing a public key. Since it is more or less a data point in a random public key choosing algorithm I know now that it would not require hardware/etc - it is simply a datapoint.

1 Like
#14

Haha just so I don’t look like an idiot with my question - your response doesn’t actually answer my questions. The semantics are very important in what I wrote.

But I know now that difficulty is just a data point used in the choosing algorithm. Very different from how it is used in Bitcoin/etc. I’m not even sure that difficulty is the best word that could have been used for this property.

Should have called it “odds” or “probability” as it is much more like a lottery than it is difficulty.

#15

I don’t think that’s what @bloodyrookie wanted to say…

Difficulty is same for all harvesters. It defines how difficult it will be to harvest next block, based on previous block(s).

We’re using last 60 blocks to “smooth” difficulties - i.e. you don’t want sudden spikes of difficulty.
i.e. imagine someone is restarting their supernodes, so part of harvesting accounts goes down for some time. Most likely difficulty will slowly start to drop to compensate for that “loss” of harvesters. (so that rest will have bit easier to harvest a block.

To better understand how this is working you can imagine, that all harvesters are having same hit (that ofc does not happen in reality)

1 Like