Lecture 2 — How Bitcoin Achieves Decentralization

https://www.youtube.com/embed/q5GWwTgRIT4

hello and welcome to the second lecture which is going to be all about decentralization in Bitcoin now in the first lecture you saw a lot of the crypto basics that underlie Bitcoin and we ended with a simple currency that we called Scrooge coin that seems to achieve a lot of what we want in a ledger based cryptocurrency except for one big glaring problem which is that it relies upon decentralized Authority called Scrooge and we ended with the question of how do we discourage if I this currency how do we go to a decentralized version of this that's what we're going to see today what I find cool about this is that the way in which Bitcoin achieves decentralization is not purely technical but it's a combination of technical and clever incentive engineering and at the end of this lecture you should have a really good appreciation for how this happens and a lot of the the magic or the mystery of Bitcoin should become clear to you and how it works and why it's secure and what makes it tick so it turns out that decentralization is an important concept not just for Bitcoin but in fact this notion of competing paradigms of centralization versus decentralization plays out in a variety of different digital technologies to understand bitcoins decentralization I want to start with the caveat that decentralization almost always is not all or nothing almost no system is purely decentralized or purely centralized and a good example of this is email which is a decentralized system fundamentally I would say it's based on a standard space to a protocol SMTP but what has happened especially in the last decade or so is that we see a dominance of a few different webmail providers which are sort of centralized service providers and this might be a good model for understanding what might be happening to Bitcoin so with that let's delve into some of the technical aspects of bitcoins decentralization and I would break this down into at least five different questions questions like who maintains this ledger of transactions who has authority over which transactions are valid who creates new bitcoins and in fact other questions like who determines how the rules of the system change and how do bitcoins acquire exchange value so these are all components of decentralization of the Bitcoin protocol more or less and the first three of these are going to be the questions that we will consider in this lecture and when I say how is Bitcoin decentralized what I mean encompasses the first three of these meetings and I want to emphasize that there are aspects to decentralization beyond the protocol that includes things like Bitcoin exchanges where you can convert Bitcoin into other currencies it includes things like wallet software and a variety of other service providers and so even though the underlying protocol is decentralized these services that develop on top of it may be centralized or decentralized to varying degrees and just to drive home this point let me show you three different aspects of Bitcoin and where they fall on the centralization decentralization spectrum first there is the peer-to-peer network in this aspect of Bitcoin I would say is the closest thing to purely decentralized why is that because anybody can run a Bitcoin node and there's a fairly low barrier to entry you can go online you can download a Bitcoin client to yourself it requires a lot of disk consumption on your computer but basically you can run that on your laptop or your PC yourself and currently there are several thousand Bitcoin nodes and so this really resembles a peer-to-peer decentralized system but that's not the only component of Bitcoin there's also Bitcoin mining which will study later in this lecture and Bitcoin mining is technically also open to anyone but it turns out that it requires a very high capital cost it just it's a consequence of how the system happens to have evolved and because of this there has been a high centralization or a concentration of power and the Bitcoin mining ecosystem and the community frequently sees this as quite undesirable so this aspect of Bitcoin is not quite as decentralized as one might want it to be and here's a third aspect updates to the software and this really gets to how and when the rules of the system change and once again here one can conceptually imagine that everybody running a Bitcoin node will look at the Bitcoin specification and maybe even create their own software and again you have a purely decentralized system but of course that's not how it works in practice the core developers are really trusted by the community and they have a lot of power when it comes to determining what Bitcoin software each these notes will run on their computer so we've talked in a generic manner about centralization and decentralization let's now talk at a bit more technical level about Bitcoin and decentralization and a key word that's going to come up again and again here is consensus specifically distributed consensus so what am I talking about here at a technical level the key challenge that you have to solve to build a distributed ecash system is called distributed consensus and this is a class of protocols that's been studied for decades in the in the computer science literature so but intuitively you can think of it as our goal being to decentralize cruge coin which is the hypothetical currency that we saw in the first lecture so as I said there's decades of research in computer science on these consensus protocols and the traditional motivating application for this is reliability and distributed systems what do I mean by that imagine you're in charge of the backend for a company like Google or Facebook these companies typically have thousands or even millions of servers which form a massive distributed database that records all of the actions that happen on the system like users comments and likes and posts and so on so when a new comment let's say comes in the way it'll be recorded is that there might be ten or fifteen different nodes in that massive back-end that might contain copies of this action now what the server needs to make sure is that that comment either gets recorded in all copies of that database or none of them if for some reason because some of these nodes might be faulty the action gets recorded in none of the databases it's ok you can go back to the user and say there was a problem saving your post would you please try again on the other hand if some of the copies of the database saved it and others didn't then you'd be in a lot of trouble because you'd have an inconsistent database so this is the key problem that motivated the traditional research on distributed consensus and you can sort of see the similarities to Bitcoin here but we're going to talk in a bit more detail about the similarities and differences so that was the traditional motivating application but we can also imagine that if we achieved a distributed consensus protocol and we were able to use that to build a massive full-scale distributed key-value store that maps arbitrary keys or names to arbitrary values then that will enable a lot of applications for example a distributed domain name system which is simply a mapping between human understandable domain names to IP addresses or a public key directory which is a mapping between user email addresses let's say to their public keys or even things like stock trades because this distributed database instead of keeping track of who's paid whom how much money would keep track of who's transferred what units of which stock to whom and the cool thing about this is that now that Bitcoin has solved the distributed consensus problem in a certain sense that we'll try to understand in this lecture we can also go ahead and try to think about solutions to all of these other related problems and in fact there are many altcoins in altcoins we'll have several more lectures about altcoins but very briefly altcoins are systems built on Bitcoin like principles to achieve perhaps slightly different goals sometimes currency systems sometimes not currency systems such as one of these applications and so given that we can solve distributed consensus now and given that we can build a global distributed key-value store it enables a lot of these other cool applications okay let's go to a technical definition now the technical definition of distributed consensus is really quite simple imagine that there is a fixed number n of nodes or processes and each of these nodes has some input value and then a consensus protocol happens and the two requirements on this consensus protocol are that the protocol should terminate and all correct nodes should decide on some value the consensus value right and I say correct nodes because some of the nodes might be faulty or even outright malicious and the second requirement is that this value that they agree upon cannot be an arbitrary value but it should be a value that was proposed as input by at least one of these correct nodes so it's really that simple but let's try to see what this might mean in the context of Bitcoin so to understand how distributed consensus could work in Bitcoin let's start with a reminder that Bitcoin is a peer-to-peer system right so what I mean when I say that bitcoin is a peer-to-peer system is that when Alice wants to pay Bob what she does is she's going to broadcast a transaction to all of the Bitcoin nodes that comprise the peer-to-peer network and you can see here the structure of the transaction this is similar to goofy coin that we saw in the first lecture and what a transaction is going to have is it's going to have Alice's signature which the other nodes need in order to know that it really in fact came from Alice it's going to have Bob's public key which also acts as his address at which he wants to receive bitcoins and further it contains a hash what is this hash recall this notion of hash pointers that we saw in the first lecture so this hash is a way for Alice to link together this transaction or this coin to her receipt of this coin from someone else previously right so those are the things that are contained in this data structure that we call a transaction and she's going to broadcast that to all of the Bitcoin peer-to-peer nodes and notice something funny here Bob's computer is nowhere in this picture now Bob if he wants to be notified that this transaction did in fact happen and that he got paid he might want to run a Bitcoin node that's one of these peer-to-peer nodes in order to listen in on the network and be sure that he's received that transaction but his listening is not in fact necessary for him to receive the funds the bitcoins will be his whether or not he's running a node on the network so given this peer-to-peer system what is it exactly that the nodes might want to reach consensus on well given that a variety of users are broadcasting these transactions to the network what everybody wants to reach consensus on is exactly which transactions were broadcasted and the order in which these transactions happened right so what does that mean specifically how consensus could work in Bitcoin is that at any given time all the nodes in the peer-to-peer network would have a sequence of blocks of transactions that they've reached consensus on so recall that in Scrooge coin for optimization purposes for efficiency we put transactions into blocks and we link these blocks together on a blockchain so we're utilizing a similar principle here we could do consensus on transactions one by one that would be okay it would just be an efficient so instead we do consensus on a block by block basis so at any given point all these nodes in the peer-to-peer network would have the sequence of blocks that they've agreed upon already and each node would then have a set of outstanding transactions that it has heard about so recall that for these transactions consensus has not yet happened and so almost by definition each node might have a slightly different version of the outstanding transactions that it's heard about the peer-to-peer network is not perfect so some node may have heard about a transaction but not other nodes so given that we have the set up what could happen is that you have the sequence of blocks that everybody has agreed upon a block is just a series of transactions and now there are these let's say these three nodes in the system each of whom proposes each of whom has an input the set of outstanding transactions that it's heard about and they execute together some consensus protocol and for the consensus protocol to succeed you can select any valid block even if it's a block that was proposed by only one node and for a block to be valid all of these transactions have to have the right crypto signatures and so on so you could select any of these valid blocks and the consensus protocol would still be okay if some transactions somehow didn't make it into this particular block that gets chosen as the result of a consensus protocol it could just wait and get into the next block so maybe this green block get selected it now it gets added to the consensus blockchain and then the protocol proceeds and routes so if you took the traditional theory of distributed consensus and applied that to Bitcoin this is the sort of system that you might end up with now this has some similarities to how Bitcoin works but it's not exactly how Bitcoin works and why is that and the reason for this is simple doing things this way is a really hard technical problem for a variety of reasons there are some obvious ones nodes might crash and nodes might outright be malicious but also because the network is highly imperfect it's a peer-to-peer system not all pairs of nodes are connected to each other there could be faults in the network because of poor internet connectivity and so on and finally there's going to be a lot of latency in the system because all of these things happen over the internet they're not even within a single data center or something like that and one particular consequence of this high latency is that there is no notion of global time what does this mean and why is this important it means that not all nodes can agree to a common ordering of events simply based on observing timestamps it just doesn't work like that so you can't possibly design your protocol by saying things like take the node that sent the first message in step one and have that node do something in step two you just can't work like that because not all nodes will agree on which message was sent first in the first step of the protocol so this really puts serious constraints on what sorts of algorithms you can really put into your consensus protocols and in fact because of these constraints a lot of the literature on distributed consensus is somewhat pessimistic and many impossibility results have been proved I'm just going to name a couple of these in case you want to look them up but I won't go into too much detail one impossibility result that's very well known and pretty simple to understand is called the Byzantine generals problem and a much more subtle one known for the names of the authors who first proved it is called the Fisher Lynch Patterson and possibility results under some conditions which include the nodes acting in a deterministic manner what they proved is that consensus is impossible even with a single faulty process so despite these impossibility results there are a few well-known protocols and Paxos is probably one of the better known and what Paxos does is it makes certain compromises what it gives you is that it never produces an inconsistent result which would be really bad but it accepts the trade-off that under certain conditions albeit rare ones the protocol can get stuck and failed to make any progress but here's the interesting thing these impossibility results were you know they were proved in a different model they were intended to study distributed databases and this model doesn't carry over that well - this is a setting that Bitcoin operates under so what these results really tell us more about them model then about the problem in fact and what bitcoin does is that it violates a lot of the assumptions that go into these models and because of that consensus in Bitcoin ironically works better in practice than in theory and what this really means is that the theory that was developed for a different set of problems needs to catch up in order to be able to say really interesting things about Bitcoin but nevertheless that theory is quite important because for example it can help us predict unforeseen attacks and really be able to come to strong guarantees on the nature of consensus and security in Bitcoin so what are these different assumptions what are some things that Bitcoin does differently well first of all it introduces the idea of incentives and this is very different from any previous system for distributed consensus and this is only possible in Bitcoin because it is a currency and you can use that currency to give incentives to the participants for acting honestly and so Bitcoin doesn't quite solve the distributed consensus problem in a general sense but it solves it in the context of the currency system the other thing that it does differently is that it really embraces the notion of randomness and what I mean by that is one of the things it does is it does away with the notion of a specific starting point and ending point for consensus instead consensus happens over a long period of time about an hour in the practical system and but even at the end of that time you're not a hundred percent sure that a transaction or a block that you're interested in has made it into the consensus blockchain instead as time goes on your probability goes up higher and higher and the probability that you're wrong in making an assumption about a transaction goes down exponentially so that's the kind of inherently probabilistic guarantee that Bitcoin gives you and that's why it's able to completely get around these traditionally in traditional impossibility results on distributed consensus protocols you so let's now dig into the technical details of bitcoins consensus algorithm and while we're looking at that we should keep in mind that Bitcoin does all of this without nodes having any persistent long-term identities and this is yet again a difference from how traditional distribute consensus algorithms operated and if notes did have identities it would make things a lot easier for a couple of reasons one is a pragmatic reason it would allow you to put into your protocol things like now the node with the lowest numerical ID should take some step or something like that so that's a simple pragmatic reason which already if noes are completely anonymous becomes harder to do but a much more serious reason for nodes to have identities as for security because if nodes were identified and it weren't trivial to create new nodes ident node identities then we could make assumptions like let's say that less than 50% of the nodes are malicious and we could derive security properties out of that so for both of these reasons the consensus protocol in Bitcoin is a bit harder but why is it exactly the Bitcoin nodes don't have identities well it's for a couple of reasons one is that if you're in a decentralized model in a peer-to-peer system there is no central authority to give identities to nodes and verify that they're not creating new nodes at will and in fact the technical term for this is a Sybil attack Sybil's are just copies of nodes that a malicious adversary can create to look like there are a lot of different participants when in fact all those pseudo participants are really controlled by the same adversary the other reason is that pseudonymity is inherently a goal of Bitcoin even if it were possible or easy to establish identities for all nodes or all participants we wouldn't necessarily want to do that so bitcoin doesn't give you strong anonymity guarantees out of the box and that's the different transactions that you make can probably be linked together but at the same time nobody is forcing you to put your real life identity like your name or IP address or anything like that in order to participate in the peer-to-peer network and in the blockchain and that's an important property so what we can do instead is we can make a weaker assumption and I kind of want you to take a leap of faith with me here that this weaker assumption is something that it's going to be feasible and I'm going to make this assumption here and later show you how this is actually accomplished and what this weaker assumption is is that we're going to assume that there is some ability somehow to pick a random node in the system and a good motivating analogy for this it's a lottery or a raffle or any number of real-life systems we're tracking and verifying people and giving them identities and verifying those identities is pretty hard and so what we do in those contexts is we might give them tokens or tickets or something of that sort and that then enables us to later pick a random token ID and call upon that person so we're going to do something similar with respect to these Bitcoin notes and further assume for the moment that this token generation and distribution algorithm has enough smarts so that if the adversary is going to try to create a lot of Sybil nodes together all of those symbols just get one token so the adversary is not able to multiply his power that way so let's make this assumption for now and let's see what becomes possible if we make this assumption here's the key idea what becomes possible under this assumption of random node selection is something called implicit consensus so what is implicit consensus in each round and there are going to be multiple rounds each run corresponding to a different Block in the blockchain in each round a random node is somehow selected magically for the moment and this node gets to propose the next block in the chain there is no consensus algorithm there is no voting this node simply unilaterally proposes what the next block and the blockchain is going to be but what if that node is malicious well there is a process for this but it is an implicit one other nodes will implicitly accept or reject that block and how will they do that if they accept that block they will signal signal that acceptance by extending the blockchain starting from that block or if they reject that block they will extend the chain by ignoring that block and starting from whatever was the previous latest block in the blockchain and technically how is that implemented recall that each block contains a hash of the block that it and this is the technical mechanism that allows needs to signal which block it is that they're extending so given this this is what the overall consensus algorithm in Bitcoin is going to look like now this is a little bit simplified and the reason it's simplified is again that I'm assuming sort of this magic random node selection process but except for that simplification this is pretty close to how Bitcoin actually works so whenever Alice wants to pay Bob she will create a transaction and she will broadcast it to all of the nodes and any one of these nodes is constantly listening to the network and collecting a list of outstanding transactions that have not yet made it into the blockchain at some point one of these nodes is going to be randomly called upon to propose the next block it's going to round up all of the outstanding transactions that it's heard about and propose that block now presumably that node was honest but it could also be a malicious node or a faulty node and propose a block that contains some invalid transactions invalid transactions are those that don't have the right crypto signature or where the transaction is already spent in other words an attempt to double spend so if that happens other nodes are going to signal their acceptance or rejection of the block as we saw on the last slide by either including the hash of this latest block in their next block or ignoring this block and including the hash of whatever was the previous block that they consider it to be valid all right so now let's try to understand why this consensus algorithm works and the way I like to understand this is instead of asking why this works let's try to ask how can a malicious adversary try to subvert this process so let's look at that for a second so here we have a couple of blocks in the blockchain assume that this extends to the left a long way back all the way to what is called the Genesis block but here I'm only showing you a couple of blocks in the blockchain and that pointer that you see over there is a block referring to what is the previous block that it extends by including a hash of that previous block within its own contents so let's Tamil Isha's attacker let's call her Alice what might she try to do can she simply steal bitcoins belonging to another user at a different address that she doesn't control now even if it is now Alice's turn to propose the next block in this chain she cannot steal other users bitcoins why because she cannot Forge their signatures so as long as the underlying crypto is solid she's not able to simply steal bitcoins another thing she might try to do is if she really really hates some other user Bob then she can look at Bob's address and she can decide that any transactions originating from Bob's address she will simply not include them in any block that she proposes to get onto the blockchain in other words she's denying service to Bob so this is a valid attack that she can try to mount but luckily it's nothing more than a little annoyance because if Bob's block doesn't make it into the next block that alice proposes he will just wait another block until an honest node gets the chance to propose a block and then his transaction will get into that block so that's not really a good attack either so the only one that were really left with for what a malicious node can try to do here is called a double spending attack so how might a double spending attack work to understand that let's assume that alice is a customer of some online merchant or a website run by bob who provides some online service in exchange for payment in bitcoins let's say he allows us to download of some software so here's how a double spending attack might work Alice goes to Bob's website and decides to buy this item pays for it with bitcoins and what that means in technical terms is that she is going to create a Bitcoin transaction from her address - Bob's address she broadcasts it to the network and let's say that some honest node creates the next block listens to this transaction and includes it in that block so what is going on here so there is this block that was created by an honest node that contains a transaction that represents a payment from Alice to the merchant Bob by C subscript a I mean a coin belonging to Alice and that is now being sent to Bob's address let's zoom into this and a little bit more technical Tail a transaction as we saw earlier as a data structure that contains Alice's signature here and an instruction to pay to Bob's public key and also a hash what is this hash this hash represents a pointer to the transaction where Alice in fact received that coin from somebody else and that must be a pointer to a transaction that was included in some previous block in the consensus chain so visually it's going to look something like this let's pause for a second here because there is something subtle going on there are at least two different types of pointers in this diagram that I've showed you there is in fact a third one corresponding to Merkel trees but we're not going to look at that at the present moment but these two types of pointers that I refer to are blocks that include a hash of the previous block that they're extending and transactions that include a pointer to whatever the previous transaction that where the coin came from right so this is the situation and this block was now generated by an honest node and now let's assume that the next time a random note is called that node is a malicious node controlled by Alice right so this is the blockchain as it stands right now bob has already looked at this blockchain decided that alice has paid him and has allowed Alice to download the software or whatever it is that she was buying on his website right so as far as Bob is concerned he's satisfied the transaction is completed Alice has now received her goods in exchange for the payment now what might happen is if Alice now gets to propose the next block she could propose a block that looks like this ignores altogether this valid block over here and instead contains a pointer to the previous block and furthermore it's going to contain a transaction that contains a transfer of coins of Alice's coins to another address a prime that's also controlled by Alice so this is a classic double spend pattern what is going on here is Alice now creates a new transaction that transfers that coin instead of to Bob's address to another address by her and visually it's going to look like this this is a completely different transaction also with the hash pointer going back to the same transaction referred to earlier right so this is what an attempt at a double spend look like and how do we know if this double spend attempt is going to succeed or not well that depends on whether this green transaction here or this red transaction is going to ultimately end up in the long term consensus chain so what determines that that is determined by the fact that honest nodes are always following the policy of extending the longest valid branch so now which of these is the longest valid branch you might look at this and say AHA the first one is the longest valid branch not the second one because it's a double spent attempt but here's a very subtle point that I want you to appreciate from sort of a moral point of view this transaction in green in the transaction red might look very different because based on the explanation that I've given you the first one is an attempt by Alice to pay Bob whereas the second one is an attempt by Alice to defraud Bob and pay coins back to herself but from a technological point of view these two transactions are completely identical the nodes that are looking at this really have no way to tell which one is the legitimate transaction and putting legitimate in air quotes because it's a moral judgement that we apply to it it's not a technical distinction versus which one is the attempt to double spend it could easily be the other way around now nodes often follow a heuristic of extending the block that they first heard about on the peer-to-peer network but it's not a solid rule and in any case because of network latency that could easily be the other way around so now there is at least some chance that the next node that gets to propose a block will extend this block instead of this one or it could be that even if it's an honest node Alice could try to bribe that node or try to subvert the process in a variety of ways so for whatever reason without going too much into the details let's say that the next node extends the block with the red transaction instead of the green one what this means is that at this point the next honest node is much more likely to extend this block instead of this one because now this has become the long valo chain so let's say that after one more block the situation looks like this now it's starting to look pretty likely that this double spin has succeeded in fact what might happen is that this ends up the long term consensus chain and this block gets completely ignored by the network and this is now called an orphan block and this is an example of a successful double spend so now let's look at this whole situation from Bob the merchants point of view and understanding how Bob can protect himself from this double spending attack it's really going to be a key part of understanding Bitcoin security so let's look at what happened here again we have a couple of blocks in the blockchain and at this point Alice broadcasts a transaction that represents her payment to Bob and so Bob is going to hear about it on the peer-to-peer network right here even before the next block gets created and so Bob can do something even more foolhardy that what he did in the previous light which is that as soon as he hears about the transaction on the peer-to-peer network he can complete the transaction on the website and allow Alice to download whatever she is downloading that's called a zero confirmation transaction or he could wait until the transaction gets one confirmation in the blockchain which means that at least some node has created a block and has proposed this transaction and that has gone into the blockchain but as we saw earlier even after one confirmation there could be an attempt at a double spend so let's say that this actually happens if as in the previous slide the double spend attempt succeeds what Bob should do is to realize that the block that he thought represented Alice paying him has now been orphaned and she's and so he should abandon the transaction instead if it so happens that despite this double spent attempt the next block that's generated turns out to extend the block that he's interested in now he sees that his transaction has two confirmations in the blockchain now he gets a little bit more confidence that his transaction is going to end up on the long-term consensus chain so let's say there's one more and now there are three confirmations in general the more confirmations your transaction gets the higher the probability that it is going to end up on the long-term consensus chain because if you recall the honest nodes behavior that they will always extend the longest valid branch that they see the chance that this one is going to catch up to this longer branch is now very minuscule especially if only a minority of the nodes are malicious because it typically the only reason that this double span attempt block would be extended at this point is if the next node to be picked randomly was a malicious node and then you'd need another malicious node and then another for the shorter branch to then become the longer branch in general the double span probability decreases exponentially with the number of confirmations so if the transaction you're interested in has received K confirmations then the probability that this other transaction is going to end up on the long term consensus chain goes down exponentially as a function of K in the most common heuristic that's used in the Bitcoin ecosystem is that you wait for six confirmations there is nothing really special about the number six it's just a good trade-off between the amount of time you have to wait and your guarantee that the transaction you're interested in ends up on the consensus watching so let's recap what we saw here protection against invalid transactions that is protection against a malicious node simply making up a transaction to steal someone's bitcoins is entirely cryptographic but it is enforced by consensus which means that if a node does attempt that then the only reason that that transaction won't end up in the long term consensus chain is because a majority of the nodes are honest and will treat that transaction as invalid on the other hand protection against double spending is purely about consensus cryptography has nothing to say about this and true transactions that represent a double spending attempt kind of look identical from the perspective of signatures and so on but it's the consensus that determines which one will end up on the long term consensus chain and finally you're never 100% sure that a transaction you're interested in is on the consensus branch but this exponential probability guarantee is pretty good after about six transaction there's virtually no chance that that you're going to go wrong so in the previous section we got a basic look at bitcoins consensus algorithm and a good intuition for why we believe that it's secure but recall that at the beginning of the lecture I told you that bitcoins decentralization is partly a technical mechanism and partly clever incentive engineering so far we've mostly looked at the technical mechanism now let's talk about the incentive engineering that happens in Bitcoin I asked you to take a leap of faith with me earlier in assuming that we're able to pick a random node and perhaps more problematically that at least 50% of the time this process will pick an honest node but of course this assumption of honesty is quite problematic especially if there are financial incentives for participants to subvert this then why would we expect any node to be honest really so what we want to ask is can we give nodes an incentive for behaving honestly let's look at this with respect to the picture we've been looking at this is the long-term consensus chain and this block contains an attempt to double spend we can ask can we penalize somehow the node that created this block but this is problematic for a number of reasons including the fact that nodes don't have identities and so there's no way to go after them to penalize them so instead let's flip the question around and ask can we reward the nodes that created all these blocks that did end up on the long term consensus chain well again sort of the same problem we don't have no identities so we can't mail them cash to their home addresses if only there were some sort of digital currency than we can use to incentivize them a decentralized one perhaps you probably see where I'm getting at in other words we're going to use bitcoins in order to incentivize the nodes that created these blocks so how are we going to do that well so far everything that I've said is just an abstract algorithm for achieving distributed consensus now we're going to break out of that model what I'm going to say now is specific to the fact that what we're achieving through this distributed consensus process is in fact a currency and we're going to incentivize these nodes by paying them in units of this currency so how do we do that there are in fact two separate incentive mechanisms in Bitcoin and the first one is called the blocker ward so what is the block reward it's just this according to the rules of Bitcoin the node that creates each block gets to include a special transaction in that block and that special transaction is a coin creation transaction and this node can also choose the recipient address of this transaction so of course that node will typically choose an address belonging to itself as the recipient of this coin creation transaction thereby paying itself it's sort of you can think of it as a payment in exchange for the service of creating that block to go on to the Consensus chain in fact the value of this coin creation transaction has an interesting property it's critics at 25 bitcoins but it actually has every four years we're now in the second period for the first four years of bitcoins existence it was 50 bitcoins now it's 25 and it's going to keep having this has some interesting consequences we'll come back to that in the next slide but let me ask you this it appears based on what I've said here that this node gets the block reward regardless of whether it proposes a block with only valid transactions or it behaves maliciously so how are we actually providing any incentives for honest behavior via this block reward but I thought think about this well how will the snowed sort of get to collect its reward that will only happen if this block ends up on the long term consensus branch because that's the only case in which this coin creation transaction will be considered valid because the coin creation transaction is not special it's just like every other transaction it's only valid if it ends up on the consensus chain so that's the incentive mechanism here it's very subtle but it's a very neat trick and so it incentivizes nodes to behave honestly or at the very minimum it incentivizes nodes to behave in a way that they think other nodes are going to agree with in creating the next blocks of the blockchain so that's the first instead of mechanism let's come back to this point now this weird sort of having phenomenon that we see here and this can be best illustrated graphically here I'm going to show you graph of time on the x-axis versus the total number of bitcoins in circulation and this over here was the first period where each block resulted in 50 new bitcoins being created and roughly at the end of last year that block reward halves from 50 to 25 and you can see that every four years extending well into the future the slope of this curve is going to keep having and this is a geometric series and you might know that it means that there is a finite sum and in fact there is a total finite supply of bitcoins and if you add up all these numbers it works out to 21 million based on the rate of new block creation which I'm going to get to in a second also worth noting is that this is the only way in which new bitcoins are created there is no other coin generation mechanism and that's why this is a final and total number as a rule Sandow at least for how many bitcoins there can ever be and this new block creation reward is actually going to run out in 2040 as things stand now so that sounds a bit weird does that mean that the system will stop working in 2040 and become insecure because nodes no longer have the incentive to behave honestly well not quite because this is only the first of two incentive mechanisms there is quite another incentive mechanism called the transaction fee and what is a transaction fee so the creator of any transaction not the creator of a block with the creator of a transaction when Alice is paying Bob what she can do is she can choose to make the output value of that coin less than the input value and the way that all the nodes interpret this difference according to the rules of Bitcoin is that it's a transaction fee and whoever creates the block that first puts that transaction into the blockchain gets to collect that transaction fee so if you're a node that's creating a block that contains say 200 transactions then the sum of all those 200 transaction fees accrues to you and to the address that you put into that block of course this transaction fee is purely voluntary like a tip but it we expect based on our understanding of the system that as the block reward starts to run out it will become more and more important most mandatory four nodes to put a transaction fee into their transactions in order to get a reasonable quality of service and to a certain degree this is already starting to happen now but precisely how the system will evolve it really depends on a lot of game theory which hasn't been fully worked out yet so that's an interesting area of open research in Bitcoin so now we've acquired an understanding of how the nodes that create these blocks are incentivized to act honestly or follow the protocol and so if we address a few more of these remaining problems we'll be all set to have a really good understanding of how Bitcoin achieves decentralization what are these remaining problems well one of them the first major one is the leap of faith that I asked you to take which is that somehow we can pick a random node and the second is that we've created a new problem by giving nodes these block rewards and incentives which is that you could easily get into a free-for-all where everybody wants to run a Bitcoin node in the hope of capturing some of these rewards and a third one is an even trickier version of this problem which is that an adversary might create a whole different number of civil nodes in order to really try to subvert this consensus process so number three is sort of a trickier version of number two it turns out that all of these problems are related and all of them have the same solution and that solution is called proof of work so what is proof of work here's the key idea instead of picking a random node we do something a little bit different which is we approximate selecting a random node by instead selecting nodes and proportion to a resource that we hope that nobody can monopolize what does that mean well if that resource that we're talking about is computing power then it's a proof of work system where we somehow select nodes in proportion to their competing power alternately it could be in proportion to ownership of the currency and this is a legitimate alternate model it's not used in Bitcoin but it's been proposed and it's used in a lot of alternatives to Bitcoin and that's called proof of stake which we'll see in a later lecture but let's come back to proof of work let's try to get a better idea of what this means selecting nodes in proportion to their computing power another way to understand this is that we're allowing nodes to compete with each other by using their computing power and that will result in nodes automatically being picked in that proportion so those are two equivalent ways to view proof-of-work you can also think of a third way which is that we're making it moderately hard that through proof-of-work to create new identities so it's a sort of attacks on identity creation and on the Sybil attack this may all appear a bit vague so let me actually go ahead and show you what is the exact proof-of-work system that's used in Bitcoin and that's going to make things a lot clearer so here it is it's called hash puzzles and what this means is that in order to create a block the node that proposes that block is required to find a number a nonce such that when you put together in the block the nonce the previous hash and the list of transactions that comprise that block and take the hash of this whole long string then that hash output should be a number that is very small that falls into this small target space here in relation to this very large space that is the output space of that hash function let's look at it one more time as we looked at it earlier normally a block contains a series of transactions that you're proposing in addition a block also contains a pointer to the previous block as we saw and a pointer is just a string in this context but in addition here we're requiring that a block also contain a nonce and why is this and the idea is that we want to make it moderately difficult to in fact find nonce that satisfies this required property which is that hashing the whole block together including that nonce is going to result in a particular type of output and so we believe that if the hash function is secure then the only way to succeed in solving this hash puzzle is to just try enough nonsense one by one until you get lucky so specifically if this target space were just 1% of the overall output space you would have to try about a hundred nonces before you got lucky and if this hash function were to behave essentially random the only one in a hundred nonces will result in an output that falls within this target space in fact the size of this targets phase is not nearly as high as one percent of the output space it's much much smaller than that which we'll get to in a second but fundamentally this is the computational problem that a node is required to solve in order to produce the block now this notion of hash puzzles in proof of work completely does away with the requirement for somebody somehow to pick a random node instead nodes are simply all the time independently competing to solve these hash puzzles and once in a while one of them will get lucky and will find a random nonce that satisfies this property and that node then gets to propose the next block that's how it's completely decentralized there is nobody deciding which node it is that gets to propose the next block so let's look at it in a little bit more detail now there are three properties that I want to show you essential properties of this proof-of-work function of this hash puzzle and the first is that it needs to be quite difficult to compute I said moderately difficult but what I mean by moderately difficult as of today and you'll see why it varies with time it's about ten to the twenty hashes for a block that you need to compute so the size of the target space is only one over ten to the twenty of the size of the output space of this hash function all right so it's in if you look at it in terms of the amount of computing that your laptop needs to do for example this is simply a humongous and infeasible number and because of this only some nodes even bother to compete in this block creation process and this is what is known as Bitcoin mining basically the process of repeatedly trying and solving these hash puzzles and we call these nodes miners and because of how capital incentive this process is this goes back to what I said at the beginning that even though technically anybody can be a miner there's been a lot of concentration of power or concentration of participation in the mining ecosystem so that's the first property of these proof-of-work puzzles the second property is that we want this cost to be parameterize Abul it's not a cost that that is fixed over all time and the way that that's accomplished is that all the nodes in the Bitcoin peer-to-peer network will automatically recalculate the target that is the size of the target space as a fraction of the output space every two weeks and they do it in such a way that they maintain this invariant which is that the average time between any two successive walks produce globally in the overall Bitcoin network is about ten minutes so let's think about what this means what this means is that if you're a miner and you've invested a certain fixed amount of hardware into Bitcoin mining but the overall mining ecosystem is growing more miners are coming in where they're deploying faster and faster hardware that means that over a two-week period slightly more blocks are going to be found and expected and so nodes will automatically readjust the target and so the amount of work that you have to do to be able to find a block is going to increase so if you put in a fixed amount of hardware investments the rate at which you find blocks is actually dependent upon what other miners are doing so there's a very nice formula to capture this which is that the probability that any given miner alice is going to win the next block is equivalent to the fraction of global hash power that she controls which means that if she has mining Hardware that's about 0.1 percent of total hash power she will compute roughly one in every thousand blocks so why does this readjustment happen why do we want this want to maintain this ten-minute invariant well the reason is simple if walks were to come very close together then there would be a lot of inefficiency and we would lose the optimization benefits of being able to put a lot of transactions as it currently stands several hundred transactions in a single block if you went down from ten minutes to five minutes it would probably be okay and there are a lot of discussions about if we're doing an altcoin now what is the block latency that we should have but everybody agrees that the block latency should be a fixed amount it cannot be allowed to go down without and that's why you have this automatic target recalculation property now because of the way that this cost function and proof-of-work is set up it allows us to reformulate our security assumption here's where we finally depart from the leap of faith that I asked you to take several slides ago instead of saying that somehow the majority of nodes are honest in a context where nodes don't even have identities and not being clear about what that means we can now state crisply that a lot of attacks on Bitcoin are infeasible if the majority of miners waited by hash power are following the protocol or are honest and the reason for that is if a majority of miners waited by hash power or honest because of this competition for proposing the next block this will automatically ensure that there is at least a 50% chance that the next block to be proposed at any point is coming from an honest node instead of a malicious one let's now look at the consequences of the fact that solving hash puzzles is probabilistic why is it probabilistic because nobody can predict which nons is going to result in solving the hash puzzle the only way to do it is to try nonsense one by one and hope that one succeeds right and so this process is called mathematically Bernoulli trials I want to I won't go into detail on that but you can look it up but typically nodes try so many nonces that a discrete probability process called Bernoulli trials can be well approximated by a continuous probability process called a Poisson process and the end result of all of that is that the distribution the probability density function of the time to find the next block by any node in the network globally looks something like this it's called an exponential distribution but really the upshot is that there is some small probability that if a block has been found now the next block is going to be found very soon or within a few seconds or within a minute and there's also some small probability that it will take a long time maybe an hour to find the next block but overall the network automatically adjust the difficulty so that the inter blog time is maintained at an average long-term of 10 minutes right so this is a graph that shows how frequently blocks are going to be created by the entire network not caring about which minor this is coming from if you're a minor specifically interested in how quickly you're finding blocks what does this probability density function look like well it's going to have the same shape but it's just going to have a different scale on the x-axis again it can be represented by a nice equation for a specific minor the mean time to find a block given that you've just found a block is going to be ten minutes divided by the fraction of hash power that you control so again if you have 0.1% of the total network hash power you're going to find blocks once every 10,000 minutes which is several days and so not only is your mean time between blocks going to be very high the variance of the time between blocks found by you is also going to be very high and this has some important consequences that we're going to be looking at in later lectures so now let's turn to the third important probability of this proof-of-work function which is that it's actually a trivial to verify that in a node has computed proof-of-work correctly what does that mean even if it takes a node on average 10 to the 20 tries to find a nonce that succeeds in finding the right property of the hash function that nonce must be published as part of the block so it's trivial for any other node to look at the block contents hash them all together and verify that the output is less than the target so this is an important property because once again it allows you to get rid of centralization you don't need any centralized Authority verifying that miners are doing their job correctly any node or any miner can instantly verify that a block found by another miner satisfies this proof of work property and thereby they can be sure that this miner put in a lot of computing power into finding that block let's now look at mining economics because we've said that it's quite expensive for miners to be an operation because finding a single block takes computing about ten to the twenty hashes at the same time we've also seen that the block reward is about 25 bitcoins which is quite a lot of money so it really boils down to an economics question of whether or not it's profitable for a miner to mine but we can write down a simple equation that represents what does what the inputs into this decision are fundamentally the mining reward that the miner gets is in terms of the block reward and transaction fees the miner asks himself whether that's bigger than or less than their total expenditure which is the hardware and electricity cost in fact Bitcoin mining is so expensive in terms of electricity that that becomes a significant portion of the cost and not just the upfront cost of the hardware and if the rewards are greater than the costs and the Minor Prophets if not the miner incurs loss but there are some complications to this simple equation the first is that as you may have noticed the hardware cost is a fixed cost it's an upfront cost whereas the electricity cost is a variable cost that is incurred over time another complication is that remember that a reward that the miner gets depends upon the rate at which they find blocks which depends on not just the power of their hardware but in fact more accurately as the ratio of the power of their hardware as a fraction of the total global hash rate so that makes it more complicated as well another complication is note that the costs that the miner incurs are in terms of dollars or whatever currency that they're using whereas they're rewarded in terms of bitcoins that are created or bitcoins that are transaction fees so this equation is really going to depend on what the exchange rate of Bitcoin is doing at any given time and finally so far we've assumed that the miner is interested in honestly following the protocol but it could be the case that the miner could deploy some other mining strategy instead of always finding the next block that extends the longest valid branch and so this equation doesn't capture all the nuances of the different strategies of the miner employ so even though we can write down the simple equation actually analyzing what it makes sense for miners to do as a complicated game theory problem and we don't have simple answers to that okay so now we've obtained a pretty good understanding of how Bitcoin achieves decentralization let's put it all together now and do a little bit of a recap and understand some high-level points in order to get an even better understanding so what I'm going to do is I'm going to do a very quick recap of several of the major aspects of Bitcoin that we've learned so far let's start from identities as we've learned there are no real-world identities required to participate in the Bitcoin protocol any user can create a pseudonymous key pair at any moment any number of them and when Alice and Bob want to make a transaction when Alice for example wants to know what address Bob wants to get paid at that's not part of the Bitcoin protocol that needs to be through some other process for example on Bob the merchants website so given these pseudonymous key pairs as identities transactions are basically messages that are broadcast to the Bitcoin peer-to-peer network that are instructions to transfer a coin from one address to another and the coin really is just a tree and chain of transactions to the extent that we can call anything in Bitcoin an actual coin and this is something we'll see in much more detail in future lectures and so this peer-to-peer network that we have looked at its goal is to propagate all new transactions to all the Bitcoin peer nodes as well as new blocks to the Bitcoin peer nodes but it's just going to do sort of the best effort that it can the real security of the system doesn't come from the perfection of the peer-to-peer network in fact the underlying assumption that the network is in fact quite unreliable but instead where the security comes from is from the blockchain and the consensus protocol that we spent a lot of time looking at so what it means for your transaction to be in the blockchain is that it achieves a lot of confirmations it's not a fixed number six is a commonly used heuristic but the more confirmations your transaction is received the more blocks found that extend the block that contained your transaction the more certain you can be that your transaction was part of the consensus change and now often there are going to be a variety of orphan blocks these are blocks that don't make it into the consensus chain this could represent an invalid transaction or a double spent attempt it could simply represent the fact that there is latency in the network and two miners competing to solve this proof-of-work puzzle simply ended up finding new blocks within just a few seconds of each other and so both of these blocks were broadcast nearly simultaneously onto the network so another subtle point here is that if Alice and Bob are two different miners and Alice has hundred times as much computing power is Bob what that means is not that Alice will always win the race against Bob to find the next block but instead Alice and Bob have a ratio a probability ratio of finding the next block and the proportion hundred to one so in the long term Bob will find an average one percent of the blocks that Alice does so those are some of the basics of block shade and consensus and where the security of the system really comes from and finally we looked at hash puzzles in mining miners are special type of nodes that bother to compete in this game of creating new blocks and they're rewarded for their efforts in terms of bitcoins and we expect that miners are going to be typically somewhere near the economic equilibrium of the expenditure that they incur in terms of hardware and electricity being somewhere equal to the rewards that they obtain in terms of the new block creation reward and the transaction fee based rewards so that's a broad recap of the system let me show you an appointed way how deeply this notion of distributed consensus permeates Bitcoin now in a traditional currency there is consensus does come into play in a certain limited extent which is we have a consensus process around what is the exchange rate of the currency you can make a rough analogy to consensus and distributed systems and that is certainly true in Bitcoin as well we need consensus around the value of the exchange rate of Bitcoin but consensus goes much deeper and bit coin than another say fiat currencies in fact you need consensus around state which is what the blockchain accomplishes that is a record of which transactions are valid or which transactions have even happened so even the idea of how many bitcoins you own is subject to consensus what it means when I say I own a certain amount or number of bitcoins is that I mean that the Bitcoin peer-to-peer network as recorded in the blockchain considers me the sum total of all my addresses to own a certain number of bitcoins that is sort of the ultimate nature of truth in Bitcoin so ownership of bitcoins is nothing more than other nodes thinking that I own a certain number of bitcoins and finally we need consensus about the rules of the system because occasionally the rules of the system have to change there are things called hot soft Forks and hard Forks and we're going to see a little bit more detail of this in later lectures now I want to show you another subtle idea which is very tricky and it's this very neat idea of bootstrapping that I really found intriguing the first time I encountered it and so I want to share this with you so what do I mean by bootstrapping I mean the tricky interplay between three things in Bitcoin and what are these three things let's start from the notion of security of the blockchain so obviously we want the blockchain to be secure for Bitcoin to have to be a viable currency but what is necessary for for the blockchain to be secure what this means is that an adversary shouldn't be able to overwhelm the consensus process shouldn't be able to create a lot of nodes and take over 50% or more of the new block creation but when will that be true what is it prerequisite for that a prerequisite for that is having a healthy mining ecosystem made up of largely honest protocol following notes so that's a prerequisite for security of the blockchain but what's a prerequisite for that when can we be sure that a lot of miners will put a lot of computing power into participating in this hash puzzle-solving competition well they're only going to do that if the exchange rate of Bitcoin is pretty high why is that because they receive rewards denominated in bitcoins whereas their expenditures and dollars so the more the value of the currency goes up the more incentivize these miners are going to be but what ensures a high and stable value of the currency that can only happen if users in general people who want to buy bitcoins have trust in the security of the blockchain because if they believe that the network could be overwhelmed at any moment by an attacker then bitcoin is not going to have a lot of values occurrence II so you have this interlocking interdependence between these three things all right so the existence of each of these is predicated on the existence of another so one might flip that around and imagine at the beginning at during bitcoins creation when none of these three things existed when there were no miners other than what we believe to be Nakamoto himself or whoever the Creator was running the mining software and when Bitcoin didn't have a lot of value as a currency and when the blockchain was in fact insecure because there was not a lot of mining going on and so anybody could easily overwhelm this process how do you go from there not having any of these three properties to going all three to having all three of them and that is what I mean by bootstrapping and it's this very tricky process of how all of these three characteristics were acquired by the Bitcoin system in an interdependent manner with each other and this was of course fueled by a lot of media attention as well because the more people hear about Bitcoin the more they're going to get interested in mining and the more they get interested in mining the more confidence people will have in the security of the blockchain because there's now more mining activity going on and so on and so forth and so Bitcoin went from having none of these properties to now having in some large measure all three of these properties and that's the interesting bootstrapping feature of Bitcoin and every new alts coin that wants to succeed also has to somehow solve this problem of pulling itself up by its bootstraps okay let me leave you now with one final thing which is that in order to understand our consensus and what its responsible for and what it's not responsible for a way to do that is to ask ourselves what would happen if consensus failed and there were in fact a 51% attacker somehow who controls 51% or more of the mining power in the Bitcoin network so let's see what happens in that case and let's list a whole variety of things possible bad things that we think might happen and let's ask ourselves which of these are possible for a 51% attacker first of all can this attacker steal coins from an existing address well you might guess that the answer is no because stealing from an existing address is not possible unless you subvert the cryptography it's not enough to subvert the consensus process this is a bit tricky let's follow through this line of argument let's say that this 51% attacker creates an invalid block that contains an invalid transaction that represents stealing bitcoins from an existing address that the attacker doesn't control and transferring them to his own address now this attacker can pretend that that's a valid transaction and pretend that that's a valid block and keep building upon this block and even succeed in making that the the longest branch but the other honest nodes are simply not going to accept this invalid block and are going to keep mining based on the last valid block that they found in the network so what will happen is that there will be what we call a fork in the chain now imagine this from the point of view of the attacker trying to spend these invalid coins and send them to some merchants Bob and buy something in exchange now Bob will presumably be running a Bitcoin node himself and that will be an honest node and that node is going to say oh this might be the longest branch but it's not a valid branch because it contains an invalid transaction because the crypto the signatures doesn't check out and so it's going to simply ignore this longest branch because it's an invalid branch and because of that subverting consensus is not enough you have to subvert cryptography in to steal coins from an existing address so we conclude that this attack is not possible for a 51% attacker by the way in saying all of this I should note that this is all somewhat hypothetical somewhat a thought experiment because if there were in fact actual signs of a 51% attack it will probably happen is that the developers will notice this and will try to react to it and we'll update the Bitcoin software and we might expect that the rules of the system of the p2p network might change in some form to make this attack more difficult to launch but we can't quite predict that so we're working in a simplified model where 51% attack happens but other than that there are no changes or tweaks to the rules of the system okay let's move on can the attackers suppress some transactions let's say there is some user say Carol whom the attacker really doesn't like and the attacker knows some of Carol's addresses and wants to make sure that no coins belonging to any of those addresses can possibly be spent is that possible well let's think about this the attacker since he controls the consensus process of the blockchain can simply refuse to create any new blocks that contain transactions from that from one of Carol's addresses and can in fact also refuse to build upon blocks that contain such transactions and the attacker will be successful at that however the attacker cannot prevent these transactions from even broadcast to the peer-to-peer network because the peer-to-peer network doesn't depend on the blockchain doesn't depend on consensus and we're assuming that the attacker doesn't fully control the network so the transactions are still going to find a way to reach the majority of notes so even if the attacker tries this attack it will be very clear that that attack is happening because the peer-to-peer network will still receive these transactions okay what about this one can the attacker change the block reward can the attacker start pretending that the block reward is instead of 25 bitcoins 100 bitcoins or something like that well this sort of corresponds to changing the rules of the system and because of a reasoning similar to what we applied for stealing bitcoins from an existing address this is also not possible because the attacker doesn't control the copies of the Bitcoin software that all of the honest nodes are running so that's also not possible finally let's ask ourselves if the attacker can somehow destroy confidence in Bitcoin well let's imagine what would happen if there were variety of double spending temps and not at behavior of not extending the longest valid branch and others had she attempted attacks then people are going to look at this and decide that bitcoin is no longer acting as a decentralized ledger that they can trust and so people will simply lose confidence in the currency and we might expect that the exchange rate of Bitcoin is going to plummet in fact if there is a 51% attacker and this is known even if the attacker is not necessarily trying to launch any attacks it's possible that this might happen so this we can classify as not only possible but in fact likely that a 51% attack of any sort will simply destroy confidence in the currency and this last one is in fact the main practical threat if a 51% attack were ever to materialize none of these others really considering the amount of expenditure that the adversary would have to put into attacking Bitcoin and achieving a 51% majority really makes sense from a financial point of view to try any of these other attacks great so now hopefully you've obtained a really good understanding of how decentralization is achieved in Bitcoin and I'm just stood identities understood transactions understood the peer-to-peer network understood the blockchain and consensus understand hash puzzles and mining so you should be in a really good point now and a good launching point for understanding a lot more of the subtle details and nuances of Bitcoin which we're going to start seeing in the next few lectures so the next lecture is going to be by Joe Bono where he will address a lot of questions that take off from the point where we have left off in this lecture the first is how do we get from consensus to currency so this is an assumption that I've already made in this lecture not only are we solving a distributed consensus problem but also we're treating the result of distributed consensus as the currency now in order to incentivize participants but a lot of details are are missing what exactly does it mean to be paid in Bitcoin how does that happen how are those transactions represented and so on we're going to look at that we're going to look at what else can we do with consensus I hinted at this a little bit but it turns out that Bitcoin offers a lot in addition to just doing consensus it has a whole scripting language so there are a lot of interesting things to see there and so Joe is going to take it from there in the next lecture thank you you

As found on YouTube

Crypto is here to stay...

https://cryptogoes.com/ethereum/lecture-2-how-bitcoin-achieves-decentralization/