This reminds me of headwinds in airplanes. You might think that on average, wind cancels out. Sometimes you have a tailwind, sometimes you have a headwind and over time it will all average out. This is not the case.
On average, wind reduces your groundspeed. The short of it is you spend more time in a headwind, so you spend more time experiencing the bad than good.
There's also the fact that even a direct crosswind slows down you ground speed (you lose some speed for correcting your course) but I don't think this compares to block chains or hitchhikers.
An interesting related fact is that with equal distance traveled through headwinds and tailwinds a slower plane will be relatively more affected. From ~12% longer time with a wind speed 1/3rd plane speed down to ~1% with a wind speed 1/10th plane speed.
No, because you spend some of your forward speed flying into the wind to keep the desired ground track.
Imagine an airplane that flies 100mph. You want to fly north, but you have a 90pmh crosswind from the east. You have to point almost directly into the wind just to keep from getting blown off course. You can turn a little to the north, and you'll go very slowly. If you instead have the wind come from the east, you still have the same problem.
Maybe easier analogy: to get directly across a river, would you rather swim across one going fast or slow?
Sure, with a consistent cross wind of course you'll have to spend energy fighting it, but if you don't fight it and halfway it changes, you'll end up at your destination in the same time as you would have otherwise, no?
Sure; if you can predict the future, and the future happens to be that the crosswindws will oppose each other, then you can just set your heading and get there.
However, if the wind doesn't shift you are now taking even longer to get there because you'll have to turn to go straight upwind once your destination is directly upwind from you.
This crazy world you describe applies in a different axis! Vertical movement of air happens on a much smaller scale, so it tends to cancel out as you fly through it. If you climb in the updrafts and descend in the downdrafts, you only make it worse. Better to ride it out.
The US air traffic control system is setup as a series of highways in the sky. You generally can't just fly in whatever direction you want (with the exception of general aviation).
This isn't what the poisson distribution tells you.
You will, on average, wait 10 minutes. If you hitch hiked that road 1000 times, you'll have waited 10 minutes on average.
But that doesn't mean you won't wait longer that 10 minutes on a given wait. In theory, you're waiting time will follow an exponential distribution, so you could wait 1, 10, 50, or even 1000 minutes (although quite unlikely)!
But again... On average, you'll be waiting for 10 minutes from car to car.
> You might think that on average, wind cancels out. Sometimes you have a tailwind, sometimes you have a headwind and over time it will all average out. This is not the case.
Only if you didn't have to do the standard physics problem comparing swimming X meters upstream and back downstream to swimming X meters across and then back across.
I have been thinking about this as well, in another context. When I am at the tennis court, I always seem to be surrounded by players which are far above average.
This seems paradoxical, but you're more likely to meet the players which train most often, and they are also most likely to be the best tennis players.
It's also the reason why most people are less popular than their average friend is, even though they are (on average) as popular as the average person! Since popular people are on everyone's friend list, but unpopular people are only on a few friend lists, your friend list is biased toward popular people.
Mathematical explanation of why Facebook makes you depressed.
As someone that has more friends('popular'), its all a matter of effort.
I go out every weekend, I talk to everyone, I ask everyone for follow up contact info.
Yeah I get turned down probably 50% of the time, but its a numbers game. Sometimes you meet incredible people, but its taking that first step that matters.
Anyway, if you want to meet people, always go to events you are invited to, meet people. No one remembers how a conversation starts, people that are weird about communication are forgettable.
Bitcoin mining = "Guess a random number between 1 and 5"
If the network guesses too fast due to more cumulative guesses per second, the target number increases.
Interesting historical antidote, in early November of 2017 there was an event where it became more profitable to mine Bitcoin Cash. During that time about 60% of the miners left BTC Core in unison to focus on BTC Cash. Transactions for BTC Core started taking upwards two to three times as long as there wasn't enough hash power to guess the magic number in time.
Eventually the miners returned to mine BTC core and transaction times returned to normal, other than the backlog in the memory pool that built up during that period.
This is of course not surprising - in the long run all difficulties (for similarly hashed coins) should trend towards equalizing profitability lest they create an arbitrage opportunity. The driving factor in the grandparent's case was the short-term spike in BCH after it was listed on Coinbase.
Not entirely. There was a flaw/feature in the difficulty algo for BTC Cash that allowed miners to generate a significant quantity of blocks very rapidly.
The excessive mempool size caused a lot of trouble for us when attempting to read it from bitcoind. Since the mempool was so large, JSON-RPC requests were taking upwards of 15-20 seconds. I’m surprised such a critical endpoint is so sub-optimised.
I never understood why the people behind Bitcoin Cash didn't "manually" lower the difficulty at the forking block. This would've been a strong incentive for miners to mine the fork and could've been good publicity. Instead most people didn't bother to mine the least profitable fork which had the same difficulty as the "main" chain. That resulted in a few chaotic days where almost no blocks were mined before the difficulty automatically and slowly adjusted.
"Assume a hashrate and difficulty corresponding to 1 block per 10 minutes. If I uniformly randomly pick a point in time, what is the expected time between the previous block and the next block?"
This is like a common core math problem. Poorly worded to the point of obscuring any real math talent that could be applied in finding a “correct” result, and instead we get to debate what the question means.
A more interesting question, I think, is given a POW algorithm which adjusts difficultly to target a 10 minute block rate, and assuming network hash rate is constant and propagation is instant, what percentage of blocks take 20 minutes to solve? 60 minutes? And what’s the odds of solving two blocks in <= 60 seconds?
I don't think the question is poorly stated, and all of your questions miss the interesting bit of this one, which is that you are more likely to randomly select a long interval rather than a short one.
* What is the average time it takes to mine a block?
* If I choose a time uniformly what is the expected time it will take to mine that particular block.
The difference between them being that blocks that take a long time occupy more space on the timeline than shorter blocks and are therefore more likely to be chosen.
>What is the average time it takes to mine a block?
How do you get this question from CyrusL's original question? CyrusL explicitly specifies that it is 10 minutes per block [0], and that he is uniformly picking a point in time.
[0] Admittedly, he does not specify that it is a Poisson distribution.
This is why I like block times of 2.5 minutes that Litecoin does. Rarely the block takes over 4 minutes. So even if it takes a couple of blocks, it's still within 10 minutes. Even 5 minutes a block would be better than bitcoin's 10. With bitcoin, if you put a reasonable fee, but your transaction doesn't make the next block, it can take more than an hour, which makes impractical certain transaction types like online shopping.
Most simple criticisms of why Bitcoin does things the way it does and then suggestions for simple ‘improvements’ such as yours (5 minute interval) tend to be naive and wrong. The 10 minute interval for Bitcoin transactions was chosen as the best tradeoff to balance pros and cons for making it shorter or longer.
10 minutes was Satoshi's very rough guess at what a reasonable compromise might be. He didn't have any empirical data and probably didn't give it much thought. Since then we have some real data about propagation times, e.g. [1], and most other blockchains have chosen much faster block times, like ~12s for Ethereum.
According to figure 2 in your second link, a blockchain with 1.5 rounds per block (or one block every ~19 seconds, based on their round length estimate) can withstand at least 40% hash power attacks. That seems like a reasonable tradeoff to me.
If you think 40% is not good enough, okay, let's target 48%. That's still just ~6.5 rounds per block (~82 seconds per block). Bitcoin's parameter of ~47 rounds per block is clearly overkill; the authors' graphs don't even go that far out (see figure 3).
Interestingly, the first paper you linked mentions that although Ethereum was intended to use single-level GHOST scoring, uncle blocks aren't actually counted toward chain difficulty, so it really has the same security model as Bitcoin. This was also confirmed by an Ethereum dev [1].
Say you have 10,000 miners, once you find a hash you can start all your miners mining the next block. Can you use the time between blocks to your advantage?
If you find the hash after say 6 minutes, you could delay broadcasting the block to the network for another minute or two, and give yourself an advantage.
Yes there’s a risk another miner would find the same hash, but statistically you could estimate the optimal delay for risk vs reward.
There is zero benefit in waiting. Supose you had 1/3 of the hashing power and got it in the first second. You have a 1/3 chance of getting the extra block before someone else gets the first block and published risking your block.
However, if you publish the first block you still have a 1/3 chance of getting the next block first and ending up with 2 blocks. Thus waiting provides risk but zero gain.
In theory this changes if you can do a 51% attack, but that erodes trust in the block chain.
There is more than one solution to a block, so if someone does find it before you it's probably not the same solution. Making your head start worthless.
Given that two different miners will be setting the mining fee address differently, I'd say that it is guaranteed that they will have different solutions.
A related statistical "mystery" is the miner cooperation without communication property of Proof-of-Work I describe here: https://grisha.org/blog/2018/01/23/explaining-proof-of-work/ It too is rooted in that the mining problem is progress-free.
Contemplating adding the "20 minute paradox" as a section since I already explain the foundational principles in the write-up.
Note that the dependency of the expected value on sampling method also show up in other places, like Benford's law. The expected value of the first digit of a number drawn from a uniform distribution depends on how the upper bound of that distribution is chosen.
The hitchhiker's paradox is correct, taking a point and looking backward or forward will correctly give an average event 10 minutes away, but combining the events to give an average of 20 minutes is false. Two events have been chosen resulting in a conditional probability.
Put more clearly, an event happens on average 10 minutes in the past, but using the same starting point for the event in the future links the events with a dependency. We could also arbitrarily start looking forward at the instance of the event in the past, or another random point in time.
This is simply incorrect, and the (hypothetical) burden is on you to disprove the theoretical and empirical results that have been shown to prove the solution.
The conditional probability is equal to the unconditional probability, specifically because the Poisson model (which Bitcoin hashing approximates) is a model of fully independent events.
> Correct, that is exactly what I am saying. If you pick a random point in time, you expect 20 minutes between the previous block and the next block on average.
I thought this sounded funny, and I did a little simulation to see if it was correct. Given his assumptions (poisson with lambda 10), you do not get that answer. I got right around 10, which is what I would expect.
You've made the exact mistake the article is talking about. You're weighting all blocks equally, but at any moment the current block is more likely to be one that was the current block for a long time.
You are using rpois as if it returned the time between blocks, but that's wrong. It returns a count of blocks mined in a certain amount of time.
Here is a formula to calculate your cumulative_times array correctly for a 10,000 minute period (which is expected to generate 1,000 blocks but may vary of course):
Does this mean that the second block from a random point in time is, on average, 20 minutes away?
Anyone know what the longest wait time on a block in recent history was?
Final question -- do large mining operations network their miners so that they don't overlap their hashes. I'm thinking the overhead in doing that would probably be counter-productive given the massive problem space. But, if they were networked that then the probability would eventually go down for a long enough wait between blocks (assuming there exists some massive mining operation).
> Does this mean that the second block from a random point in time is, on average, 20 minutes away?
No, under the assumptions in this article the next block is on average 10 minutes away.
The paradox is that at any random point in time the previous block will also have been, on average, 10 minutes ago. Meaning that if you pick a random point in time and wait for the next block then on average you'll wait for 10 minutes, but the block will have taken 20 minutes in total to mine. Despite the fact that blocks are only supposed to take 10 minutes on average.
> do large mining operations network their miners so that they don't overlap their hashes
Interestingly, it's hypothesized that Satoshi did this:
> He suggested that Satoshi had access to 58 machines for mining, so to avoid checking the same nonce twice he gave each machine a different id, which was stamped in the LSB of the nonce.
> do large mining operations network their miners so that they don't overlap their hashes
The short answer is "no" (there is no need).
The SHA-256 space (2^256) is larger than the number of atoms in the observable universe, if you pick the (extra)nonces at random, there is practically no chance you can ever pick the same number as anyone else.
> Final question -- do large mining operations network their miners so that they don't overlap their hashes.
There is no need to do it -- if each miner has a source of true randomness, then it's enough that each miner randoms large number (how large, depends on the number of miners, see the birthday paradox), and slap it on to "salt" their block. If the number space is large enough, it is unlikely that two different miners will get the same salt.
With bitcoin (not bitcoin cash) the difficulty is adjusted every 2016 blocks. If too many blocks were mined in this period the difficulty will decrease and vice versa. This means that many blocks could be mined faster or slower in this period. Until of course the difficulty adjustment occurs again.
You use what is called the nonce in the block header. Each time you do a hash of the block you increment this nonce. If your hash is under or equal to the difficulty the block is valid. The difficulty essentially means how many zeroes are in front of your hash.
What big mining operations do is to slice this nonce into appropriate ranges for each miner. So no miner is hashing with the same nonce. So miner 1 starts at nonce=0 and miner 2 on nonce=2000. The nonce range depends on how many hashes each miner can do in a ten minutes span. By doing this each miner is not doing hashing with the same nonce. That would be wasting hashing operations.
Now bitcoin cash changed the difficulty adjustment algorithm (DAA). Instead of adjusting the difficulty every 2016 blocks this is done after each block has been found. This was done to stabilize the difficulty. So miners stay mining bitcoin cash instead of switching between the most profitable chain (bitcoin or bitcoin cash). This was a problem before the new DAA was implemented for bitcoin cash.
There has been times in recent history were no new block was found for 20 minutes on bitcoin.
For bitcoin cash around 2 hours. This was in November 2017. It has been 10 minutes stable ever since.
You can check this yourself on bitinfocharts.com for each blockchain under Block Time.
With bitcoin (not bitcoin cash) the difficulty is adjusted every 2016 blocks. If too many blocks were mined in this period the difficulty will decrease and vice versa. This means that many blocks could be mined faster or slower in this period. Until of course the difficulty adjustment occurs again.
You use what is called the nonce in the block header. Each time you do a hash of the block you increment this nonce. If your hash is under or equal to the difficulty the block is valid. The difficulty essentially means how many zeroes are in front of your hash.
What big mining operations do is to slice this nonce into appropriate ranges for each miner. So no miner is hashing with the same nonce. So miner 1 starts at nonce=0 and miner 2 on nonce=2000. The nonce range depends on how many hashes each miner can do in a ten minutes span. By doing this each miner is not doing hashing with the same nonce. That would be wasting hashing operations.
Now bitcoin cash changed the difficulty adjustment algorithm (DAA). Instead of adjusting the difficulty every 2016 blocks this is done after each block has been found. This was done to stabilize the difficulty. So miners stay mining bitcoin cash instead of switching between the most profitable chain (bitcoin or bitcoin cash). This was a problem before the new DAA was implemented for bitcoin cash.
There has been times in recent history were no new block was found for 20 minutes on bitcoin. For bitcoin cash around 2 hours. This was in November 2017. It has been 10 minutes stable ever since.
You can check this yourself on bitinfocharts.com for each blockchain under Block Time.
This article is completely independent of difficultly adjustment. The 20 minute average gap is a consequence of bitcoin mining being a Poisson process and the lack of memory property that follows.
Hmm if this is similar to the hitchhiker’s paradoxon, then one would expect to wait 5 min for the next block by intuition, which is wrong and we have to wait 10 minutes. But i do not get why the intuition should be wrong by a factor of 4x (20 mins) instead by a factor of 2x (10 mins).
TLDR: Since blocks that take a long time to mine fill up more wall-clock time, if you pick an arbitrary instant of wall clock time, you're more likely to be in a "slow" block than a fast block. Specifically, you expect to wait 2x the average.
Thanks. Why can't all math teachers talk like that? I mean... I know you're clever and all, but if you were really clever you'd explain it in a way that's easy to understand.
Still, kudos to the OP. That was a great attempt to be precise.
That simple explanation is exactly wrong though! You don't expect to wait 2x the average. You expect to wait 1x the average, but you also expect that when you start waiting the previous block happened 1x the average time ago, and 1+1=2.
But you are waiting 2x the average, because the average waiting time should be half of the period.
Consider buses which arrive every ten minutes, exactly. You arrive at the bus stop at a random time. How long should you expect to wait? Not 10 min but 5 min, on average. You are equally likely to arrive at any point during the 10 minutes wait.
Now change the buses to a Poisson process with mean rate 1 bus every 10 minutes. Now you arrive during an interval of average length 20 min, but wait on average 10 min.
> The hitchhiker's paradox is correct, taking a point and looking backward or forward will correctly give an average event 10 minutes away, but combining the events to give an average of 20 minutes is false.
> I thought this sounded funny, and I did a little simulation to see if it was correct. Given his assumptions (poisson with lambda 10), you do not get that answer. I got right around 10, which is what I would expect.
Yes, it is true. I'm not sure what the first commenter is trying to say exactly. The second commenter's code was wrong, and correcting it gives the expected 20 minute average period.
That collection is a series of lectures from Feynman, and his characteristic clarity, on physics across the breadth of all major topics. It's quite amazing to realize how little you, in general, truly understand even really basic concepts like the conservation laws until you read the writings of a person who did genuinely understand what he was talking about and could share that insight in such a uniquely effective way.
This is the exact same reason that it always seems like you are in “the slow lane” in congested traffic. You spend considerably more time going slow than going fast, and while your lane is slow you have to watch all the other sporadically fast lanes go by.
> if you do half your trip and 10mph and the other at 20mph your average isn't 15mph. You spent more time at 10mph!
If you're measuring "half your trip" in terms of distance covered, sure. If "half your trip" refers to half the total time, then your average is exactly 15 mph.
> And why do 3 buses come at once? Because the first one takes all the passengers, so stops more and gets slowed down. The other 2 catch up.
This is not related to the other phenomena you mention; as described here, the buses are actually being driven together as opposed to varying randomly.
That's not correct. That's the opposite of the correct reason.
Consider a simple (not-exactly-Poisson) set of blocks: 20x1minute intervals, plus 1x10minute interval. The average interval size is (20x1min + 1x10min)/21 = ~1.4min
A random moment in time is twice as likely to be in a fast blocks as in the slow blocks.
But average wait time (given a uniformly random start times) is (20min0.5min + 10min 10min)/30min = 110min^2/30min = ~3.6min.
That per-start-instant average wait time is longer NOT because you are more likely to be in a slow block, but because being in a slow block has a much longer average wait-time than being in any of the fast blocks.
First of all, when the GP says "you are more likely to be in a slow block", it means "you are more likely to be in a slow block, relative to how many blocks there are".
In your example, if you pick a block at random, you have 1/21 chance of being in a slow block. If you pick a block by choosing a moment in time at random, you have 1/3 chance.
It is obviously not true that with any distribution you would be absolutely more likely to be a slower block.
Secondly, your last sentence seems to give a reason which doesn't fully make sense, and certainly isn't 'the opposite' of the given reason. Slow blocks of course have longer average wait-times than fast blocks. But this affects both the 'time-weighted' average and the 'block-weighted' average.
If you think that slow blocks have a longer wait-time than fast blocks, but that slow blocks are less likely than fast blocks (in the time-weighted average), shouldn't that make the time-weighted average LOWER?
Splitting a long block into two shorter (but still long) blocks doesn't make you less likely to be in the covered interval, but does reduce the average weight time.
Actually, this is also slightly incorrect, conflating "expected wait time until next block" with "average current block duration". These are two different expectancies: You always expect to wait exactly the average wait time of 10min until next block; and you expect your block to be of double average duration (20min, with the instant of sampling lying somewhere in the middle).
The concept of memorylessness is not very intuitive, and thus from the true statements "average block time 10min" together with "the current instant is expected to be somewhere in the middle between two blocks" is drawn the wrong conclusion (disregarding memorylessness): "average wait time is 10/2 = 5min". So, new attempt at a tl;dr:
"you expect to wait 2x the intuitively (wrongly) expected time (of 5min)"
If a bunch of miners decide to flee Bitcoin, it might take an hour or more to mine a block.
I've been thinking about the game theory aspect of that, and it's a bit hard to work out whether that will cause even more miners to flee. If so, that could very well be a death spiral.
It's hard because transaction fees might make up the difference. Though it's hard to imagine everyone being ok with >$100 transaction fees.
If a bunch of miners flee, the expected profit for the remaining miners actually increases (marginally). Assume instant network propagation, the probability that a given hash will result in a block is constant. Since miners fleeing does not decrease the hash power of the miners that choice to stay, they have the same hashrate, so the rate at which they mine blocks is constant, and their expected profit is unaffected.
If we acknowledge the propagation delay, their expected profit increases slightly because of the reduction in the probability that their blocks would be orphaned.
Additionally, the increased congestion would likely increase the transaction fees, further increasing the per-block profits.
Once the network readjusts and the difficulty decreases, the remaining miners would see a windfall profit.
The potential death-spiral would only come if the temporary increase in delay causes the value of bitcoin to fall, in which case more miners will likely flee. If this happens fast enough, the blockchain may never reach its next adjustment.
OT: From my perspective, HN is the reference regarding tech content. The upvoted stories, the comments, the discussions are of such a high quality. It's just the best resource to stay informed and educated in tech.
Surprisingly this is not true with Blockchain topics. Upvoted Blockchain stories feel ok but not that relevant or often random. Comments, discussions are sometimes constuctive (like this one) but often anti-blockchain or of low quality.
This is precisely because of: "The upvoted stories, the comments, the discussion are of such a high quality. It's just the best resource to stay informed and educated in tech."
If the best, the brightest, the most educated are anti-blockchain, there are definitely reasons for them to be.
---
BTW. Is this true what the article is saying: it takes 10 minutes for a transaction to clear in bitcoin?
Yes, it takes 10 minutes on average for a transaction to be included in a block. Potentially much longer if you are unlucky or didn't pay a high enough fee. On top of that most places won't accept just one block. The standard is 6 blocks, so 1 hour on average. That's the price you pay for a truly trustless distributed system. Still (much) faster than ACH or wire transfers or securities trade settlement.
SEPA instant will settle transactions 10 seconds. Drawbacks: banks can opt to participate, initially limited to 15k EUR, plus not obvious what the cost is. Banks are charged 1 EUR per transaction, so it'll be at least 1 EUR charged to customers.
The price of a transaction is as important as the speed, IMO.
Private transfers: instant transactions between individuals, no transaction fees. (Though I suspect internally they do the usual reconciliation between banks at the end of the day).
Business: instant transactions to your account from your customers. Yearly fee: 500 SEK ($61), price per transaction: 2 SEK ($0.24), price per reimbursed transaction: 2 SEK ($0.24)
There are also limits on how much money you can send in one go, but it's usually more than enough for individual transactions and for small businesses.
.. but slower than contactless or other card transactions, and (usually) slower than Faster Payments.
(Comparing to securities settlement is interesting, because cryptocurrency exchanges use off-blockchain "settlement" within themselves. Trades execute near-instantly because they're just updating a database. However, settling your money out of an exchange can take a lot longer)
> settling your money out of an exchange can take a lot longer
Usually only if there's something wrong, like a technical or KYC issue. In many cases it is possible to execute a trade and withdraw as fast as the blockchain (or your bank) will allow.
> Surprisingly this is not true with Blockchain topics. Upvoted Blockchain stories feel ok but not that relevant or often random. Comments, discussions are sometimes constuctive (like this one) but often anti-blockchain or of low quality.
This is due to most blockchain projects being overhyped garbage.
The tech itself is also of questionable practical value.
