[bitcoin-dev] Reasons to add sync flags to Bitcoin
I posted this to /r/bitcoin yesterday but it got minimal comments. One uses suggested I try the mailing list so here it is: The idea presented here could have the following benefits: 1. Improve mining decentralization 2. Reduce variance in mining profitability 3. Reduce or eliminate SPV mined blocks 4. Reduce or eliminate empty blocks, smoothing out resource usage 5. Reduce or eliminate the latency bottleneck on throughput 6. Make transaction stuffing by miners be either obvious or costly 7. Gives miners something to do while they wait for attractive transactions to appear 8. Can be easily done with a soft fork #Basic idea: Ideally, all miners would begin hashing the next block at exactly the same time. Miners with a head start are more profitable, and the techniques that help miners receive and validate blocks quickly create centralization pressure. What if there was something that acted like the starting flag at a race, which could suddenly wave and cause all of the miners to simultaneously begin hashing the next block? #Implementation: Let a sync flag be a message consisting of: 1. Hash of the previous block. 2. Bitcoin address 3. Nonce This tiny message could propagate through the network at maximum speed. If miners had to include the hash of this flag in the next block, then all miners wait for this flag, and when it suddenly spread through the network, all miners could simultaneously begin hashing the next block. The sync flag should not be produced too quickly. You want to give everyone enough time to be ready to hash the next block. Let's say that the hash of the sync flag is a proof of work that is targeted for 2 minutes. To fund this proof of work, the protocol is modified to demand that the block produced 10 blocks after the sync flag must allocate 25% of the block reward to the address published by the sync flag. In this way, sync flags are produced in 2 minutes, and blocks are produced in 8 minutes, with 10 minutes total. Illustration 1: https://s32.postimg.org/wzg0hs8lx/sync_flag.png) Illustration 2: https://s32.postimg.org/vc5y9yz4l/sync_flag2.png #Explanation of reasons: **Improve mining decentralization** One factor driving centralization is the imperative miners have to achieve low latency in receiving and validating blocks. To achieve low latency, it helps a lot if you have expensive low-latency internet connections, curated network topologies, and large pools that have a plausible chance of finding consecutive blocks. If miners are expected (or forced) to validate a block prior to mining on top of it, the rational end game would be to outsource the validation step to a trusted third party specialist who can choose optimal locations on the globe to serve their (multiple?) mining pool clients. These are all less decentralized than the mining situation Satoshi and others imagined. **Reduce variance in mining revenue** Currently, there are about 144 opportunities per day for a pool or solo miner to see any revenue at all. With sync flags, that number doubles to 288. Sync flags are only worth 25% of what a block is worth, but this still represents a significant reduction in variance. This variance is one factor causing solo miners to group into pools, and large pools to be more attractive than small pools. **Reduce or eliminate SPV mined blocks** One way miners have sought to make full-block-transmission-and-validation-latency irrelevant has been through "SPV" mining or "Head-first" mining. There is some evidence that these techniques may be widely used, and that badgering the miners about it is an ineffective strategy to stop them. In SPV mining, a miner would simply accept any block header that shows the correct proof of work. All other validation is entrusted to other miners. This practice is quite dangerous as the SPV miners can wander off on some invalid chain, taking SPV nodes with them. If this occurs during a soft fork, these blind miners can also fool unupgraded fully validating nodes into following them. "Head-first" mining means that miners start hashing as soon as they receive the block header with the correct POW, but they simultaneously validate the block, and abandon it if is not valid. I consider this to be pretty safe, as it strictly limits the length of an invalid chain that can result from mining without validating. However, "Head-first" mining can plausibly generate 2 or 3 confirmations of an invalid block. It would be nice if such confirmations did not happen. The sync flag technique is similar to head-first mining, but rather than hashing the next block while they wait for transmission and validation of the prior block, they hash the sync flag. Nodes can differentiate between sync flags and blocks, and can ignore sync flags when counting confirmations. **Reduce or eliminate empty blocks, smoothing out resource usage** Empty blocks are another consequence of SPV or Headfirst mining, because the miner cannot safely include any transactions in th
Re: [bitcoin-dev] Reasons to add sync flags to Bitcoin
> #Basic idea: > > Ideally, all miners would begin hashing the next block at exactly the same > time. Miners with a head start are more profitable, and the techniques that > help miners receive and validate blocks quickly create centralization > pressure. > > What if there was something that acted like the starting flag at a race, > which could suddenly wave and cause all of the miners to simultaneously > begin hashing the next block? > > #Implementation: > > Let a sync flag be a message consisting of: > > 1. Hash of the previous block. > 2. Bitcoin address > 3. Nonce > > This tiny message could propagate through the network at maximum speed. If > miners had to include the hash of this flag in the next block, then all > miners wait for this flag, and when it suddenly spread through the network, > all miners could simultaneously begin hashing the next block. What you describe in this part of your message can be done with no forks whatsoever and I think that this is enough. Don't really see the reason for any change in funding. The idea of sending out a block header is essentially what I called "optimistic mining" and has been described in more detail in my blog here; http://zander.github.io/posts/Innovation%20-%20OnlineScaling/ The video explains with graphics too... You may find this interesting :) ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] Reasons to add sync flags to Bitcoin
- Flags will be mined selfishly, and not published until the advantage gained from withholding is less than the mining reward. This effect may kill the decentralization features, since big miners will be the only ones that can selfish-mine flags. Indeed, collusion would be encouraged... just ship the flag to the miners you do business with, and no one else. At the expense of loss of flag revenue, your in-group would gain a massive advantage in main-chain mining. On Tue, Jul 26, 2016 at 9:51 AM, Tom via bitcoin-dev < bitcoin-dev@lists.linuxfoundation.org> wrote: > > #Basic idea: > > > > Ideally, all miners would begin hashing the next block at exactly the > same > > time. Miners with a head start are more profitable, and the techniques > that > > help miners receive and validate blocks quickly create centralization > > pressure. > > > > What if there was something that acted like the starting flag at a race, > > which could suddenly wave and cause all of the miners to simultaneously > > begin hashing the next block? > > > > #Implementation: > > > > Let a sync flag be a message consisting of: > > > > 1. Hash of the previous block. > > 2. Bitcoin address > > 3. Nonce > > > > This tiny message could propagate through the network at maximum speed. > If > > miners had to include the hash of this flag in the next block, then all > > miners wait for this flag, and when it suddenly spread through the > network, > > all miners could simultaneously begin hashing the next block. > > What you describe in this part of your message can be done with no forks > whatsoever and I think that this is enough. Don't really see the reason for > any change in funding. > > The idea of sending out a block header is essentially what I called > "optimistic mining" and has been described in more detail in my blog here; > http://zander.github.io/posts/Innovation%20-%20OnlineScaling/ > > The video explains with graphics too... > > You may find this interesting :) > ___ > bitcoin-dev mailing list > bitcoin-dev@lists.linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
[bitcoin-dev] BIP proposal: derived mnemonics
(not sure so sent again after subscribing (one use case added)) Dear Bitcoin developers, Below is provided a draft BIP proposal for a master mnemonic sentence from which other mnemonics sentences can be derived in a deterministic non-reversible way (on an offline computer). This would make it much easier to split funds into smaller fractions and use those in a HD-wallet when appropriate (just by inserting 12 or more words), without ever putting the master mnemonic at risk on an online computer. But there are many more use cases. A reference implementation, specifically for use with a Trezor, has been generated and can be found at: http://thebitcoinecosystem.info/DerivedMnemonics.html I'm not a professional programmer or cryptographer, so the idea and reference implementation will probably need a lot of reviewing but I do think Bitcoin needs this extension and the corresponding ease of use and improved security model. In the hope you like the idea, Regards, sumBTC BIP: ??? Title: Derived mnemonics from a master mnemonic. Author: sumBTC Status: For Discussion Type: Created: 2016-07-24 ==Abstract== This BIP??? uses a master mnemonic sentence, as described in BIP39, for the deterministic generation of derived mnemonic sentences. The derived mnemonics are of the same format as the master mnemonic but can consist of a higher or lower number of words. Binary seeds can then be generated for derived mnemonics (and master mnemonic) as described in BIP39. Each of these seeds can be used to generate deterministic wallets using BIP-0032 or similar methods. ==Motivation== A mnemonic code or sentence is superior for human interaction as described in BIP39 and can, for example, be written on paper or even memorized. However, once a mnemonic has been used online, even through the use of a hardware wallet, the mnemonic could be compromised. This should be considered a bad practice from a security standpoint. We therefore propose the generation of a master mnemonic offline and from this generate (also offline) multiple derived mnemonics in a deterministic way for online use. The master mnemonic is never used online and the master mnemonic cannot be obtained from the derived mnemonics. Examples of use cases are described below. ==Generating the master mnemonic== The master mnemonic is first derived as a standard mnemonic as described in BIP39. ==From master mnemonic to derived mnemonics== From the master mnemonic a new string is created: string = MasterMnemonic + " " + Count + " " + Strength; Here, MasterMnemonic are the space separated words of the master mnemonic. Count = 0, 1, 2 denotes the different derived mnemonics of a given strength and Strength = numWords / 3 * 32, where numWords is the number of words desired for the derived mnemonic and only integer arithmetic is used in the calculation (e.g. for numWords = 14, Strength = 128). Both Count and Strength are converted to strings. This string is then hashed using sha512: hash = sha512(string); and turned into a byte array: for (var i=0; i>> ((i%4)*8)) & 0b; } This byte array is then used to generate a new mnemonic as shown in the reference implementation using the method described in BIP39. The core of the new code in the reference manual can be found by jumping to "start: new code" in the reference software. A passphrase for the master mnemonic has the same effect on the derived mnemoncis (so must be included). ==Reference Implementation== The reference implementation generates addresses based on BIP44 for a 24 word master mnemonic and is available from http://thebitcoinecosystem.info/DerivedMnemonics.html or github (not yet) ==Checking the derived mnemonics using Electrum== The displayed addresses in each of the reference implementations can be easily checked using Electrum in the following manner: move the directory ~/.electrum to a backup directory. start Electrum and choose: Restore a wallet or import keys Hardware wallet Restore Electum wallet from device seed words TREZOR wallet Insert one of the mnemonics and check that the same addresses are generated by Electrum Check the private keys: move the directory ~/.electrum to a backup directory. start Electrum and choose: Restore a wallet or import keys Standard wallet Import one of the private keys and check that the correct address has been generated. Some checks should include a passphrase. ==Examples of Use Cases== A person with 25 bitcoin splits funds using 5 derived mnemonics and sends 5 bitcoins to the first address of each derived mnemonic. He can then use a (hardware) HD-wallet and simply insert one of the derived mnemonics to put only 5 bitcoins online and at risk at once. All funds can be recovered from the master mnemonic. A person wants to give 10 bitcoin to each of his family members, giving each participant a derived mnemonic and sending bitcoin to each of them. The donating perso
Re: [bitcoin-dev] BIP proposal: derived mnemonics
Hi > ==Generating the master mnemonic== > > The master mnemonic is first derived as a standard mnemonic as described > in BIP39. > ==From master mnemonic to derived mnemonics== > > From the master mnemonic a new string is created: > > string = MasterMnemonic + " " + Count + " " + Strength; > > Here, MasterMnemonic are the space separated words of the master > mnemonic. Count = 0, 1, 2 denotes the different derived mnemonics of a > given strength and Strength = numWords / 3 * 32, where numWords is the > number of words desired for the derived mnemonic and only integer > arithmetic is used in the calculation (e.g. for numWords = 14, Strength > = 128). Both Count and Strength are converted to strings. > > This string is then hashed using sha512: > > hash = sha512(string); 1) My humble cryptographic understanding tells me that you should probably use sha512_hmac where you add an passphrase and a salt. 2) Side-note: Bip39 does still use PBKDF2 with 2048 iterations which I personally consider "not enough" to protect a serious amount of funds. Also the checksum based on the predetermined wordlist has some security downsides over using a plain 32byte entropy (64hex chars) or a base58check encoded extended private master key. 3) Another idea: What would speak against deriving a child key after bip32, lets say at m/88'/0'/n' and use the derived 256bits to encode your mnemonic? This would at least require your master mnemonic passphrase to derive a valid "child mnemonic". 4) I'm still not convinced if we should encourage users to "only store and backup" the bip39 mnemonic. Reconstructing funds from a seed can be difficult especially if you don't have access to a trusted TX-indexed full node (~150GB of data required). Novice users might also underestimate the risk of losing metadata coupled with their transactions when they only store the wallet seed. signature.asc Description: OpenPGP digital signature ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
[bitcoin-dev] Reasons to add sync flags to Bitcoin
- Flags will be mined selfishly, and not published until the advantage gained from withholding is less than the mining reward. This effect may kill the decentralization features, since big miners will be the only ones that can selfish-mine flags. Indeed, collusion would be encouraged... just ship the flag to the miners you do business with, and no one else. At the expense of loss of flag revenue, your in-group would gain a massive advantage in main-chain mining. --- Is there a reason miners would be more likely to engage in selfish mining of sync flags than they are now with ordinary blocks? ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] Reasons to add sync flags to Bitcoin
On Tue, Jul 26, 2016 at 9:58 PM, Martijn Meijering via bitcoin-dev < bitcoin-dev@lists.linuxfoundation.org> wrote: > Is there a reason miners would be more likely to engage in selfish > mining of sync flags than they are now with ordinary blocks? > This proposal has the same effect as adding mandatory empty blocks. POW targeted at 2 minutes means that the POW for the flag is 25% of the block POW. That gives a flag every 2 minutes and a block every 8 minutes. It has the feature that the conversion rate from hashing power to reward is the same for the flags and the blocks. A flag get 25% of the reward for 25% of the effort. A soft fork to add this rule would have a disadvantage relative to a competing chain. It would divert 20% of its hashing power to the flag blocks, which would be ignored by legacy nodes. The soft fork would need 55% of the hashing power to win the race. This isn't that big a deal if a 75% activation threshold is used. It might be worth bumping it up to 80% in that case. This rule would mean that headers first clients would have to download more information to verify the longest chain. If they only download the headers, they are missing 20% of the POW. ___ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Re: [bitcoin-dev] Reasons to add sync flags to Bitcoin
Moral, Mining the sync flag isn't compatible with the payout structure of non hot-wallet pools like Eligius or decentralized pools like p2pool. Those need the ability to split a reward among multiple parties. Instead of giving an address to send the funds to, you could include the hash of the transaction allowed to spend the sync flag output. You'd have to zero the previous outpoint of the transaction before hashing, since you don't know what the hash of the coinbase ten blocks from now will be. On Tue, Jul 26, 2016 at 6:47 AM, Moral Agent via bitcoin-dev wrote: > I posted this to /r/bitcoin yesterday but it got minimal comments. One uses > suggested I try the mailing list so here it is: > > The idea presented here could have the following benefits: > > 1. Improve mining decentralization > 2. Reduce variance in mining profitability > 3. Reduce or eliminate SPV mined blocks > 4. Reduce or eliminate empty blocks, smoothing out resource usage > 5. Reduce or eliminate the latency bottleneck on throughput > 6. Make transaction stuffing by miners be either obvious or costly > 7. Gives miners something to do while they wait for attractive transactions > to appear > 8. Can be easily done with a soft fork > > #Basic idea: > > Ideally, all miners would begin hashing the next block at exactly the same > time. Miners with a head start are more profitable, and the techniques that > help miners receive and validate blocks quickly create centralization > pressure. > > What if there was something that acted like the starting flag at a race, > which could suddenly wave and cause all of the miners to simultaneously > begin hashing the next block? > > #Implementation: > > Let a sync flag be a message consisting of: > > 1. Hash of the previous block. > 2. Bitcoin address > 3. Nonce > > This tiny message could propagate through the network at maximum speed. If > miners had to include the hash of this flag in the next block, then all > miners wait for this flag, and when it suddenly spread through the network, > all miners could simultaneously begin hashing the next block. > > The sync flag should not be produced too quickly. You want to give everyone > enough time to be ready to hash the next block. Let's say that the hash of > the sync flag is a proof of work that is targeted for 2 minutes. > > To fund this proof of work, the protocol is modified to demand that the > block produced 10 blocks after the sync flag must allocate 25% of the block > reward to the address published by the sync flag. In this way, sync flags > are produced in 2 minutes, and blocks are produced in 8 minutes, with 10 > minutes total. > > > Illustration 1: https://s32.postimg.org/wzg0hs8lx/sync_flag.png) > > Illustration 2: https://s32.postimg.org/vc5y9yz4l/sync_flag2.png > > > #Explanation of reasons: > > **Improve mining decentralization** > > One factor driving centralization is the imperative miners have to achieve > low latency in receiving and validating blocks. To achieve low latency, it > helps a lot if you have expensive low-latency internet connections, curated > network topologies, and large pools that have a plausible chance of finding > consecutive blocks. If miners are expected (or forced) to validate a block > prior to mining on top of it, the rational end game would be to outsource > the validation step to a trusted third party specialist who can choose > optimal locations on the globe to serve their (multiple?) mining pool > clients. These are all less decentralized than the mining situation Satoshi > and others imagined. > > **Reduce variance in mining revenue** > > Currently, there are about 144 opportunities per day for a pool or solo > miner to see any revenue at all. With sync flags, that number doubles to > 288. Sync flags are only worth 25% of what a block is worth, but this still > represents a significant reduction in variance. This variance is one factor > causing solo miners to group into pools, and large pools to be more > attractive than small pools. > > **Reduce or eliminate SPV mined blocks** > > One way miners have sought to make > full-block-transmission-and-validation-latency irrelevant has been through > "SPV" mining or "Head-first" mining. There is some evidence that these > techniques may be widely used, and that badgering the miners about it is an > ineffective strategy to stop them. > > In SPV mining, a miner would simply accept any block header that shows the > correct proof of work. All other validation is entrusted to other miners. > This practice is quite dangerous as the SPV miners can wander off on some > invalid chain, taking SPV nodes with them. If this occurs during a soft > fork, these blind miners can also fool unupgraded fully validating nodes > into following them. > > "Head-first" mining means that miners start hashing as soon as they receive > the block header with the correct POW, but they simultaneously validate the > block, and abandon it if is not valid. I consider this to be pretty safe, as
