On 8/18/20 2:11 PM, Eric Voskuil wrote:
- snip -
Still, I think we're talking pedantics here, and not in a useful way.
Not to be pedantic, but I don’t know what that means.
It means that part of the discussion is not useful, and not worth bothering to go back and figure out what was shipped
before the version increase and what wasn't, lets talk about what makes sense for the future :).
Ultimately we need some kind of negotiation which is flexible in allowing
different software to negotiate different features without a global lock-step
version number increase.
I have shown below how that already works.
Or, to put it another way, if a feature is fully optional, why should there be
a version number increase for it
For the reasons previously given.
- the negotiation of it is independent and a version number only increases confusion over
which change "owns" a given version number.
Presumably this is why we have a standards process. Any new message implies
ownership. Deconflicting that is required, which implies it can easily be
version associated (as it has been).
I think the point is, this doesn't work today, bumping the protocol version requires everyone agreeing on which features
make sense, and as we can see from this email thread alone, that isn't a common result in this community. People happily
ignore BIPs that make no sense, of which there are a lot, and they should totally be able to do that!
You can say that the current world works, but there's a reason over time we've shifted away from the original "shove
another bit on the end of the version message, and everyone agrees on the order of those bits for new feature
negotiation." Version bumping is an extension of that, really.
I presume you'd support a single message that lists the set of features which a
node (optionally) wishes to support on the connection. This proposal is fully
equivalent to that, instead opting to list them as individual messages instead
of one message, which is a bit nicer in that they can be handled more
independently or by different subsystems including even the message hashing.
This presumes an implementation. As part of the handshake, collection of an
arbitrary set of messages is a significant and unnecessary complication *of the
protocol*. Extension of the verack is not. It is the simplest change possible
to implement the desired behavior. Each peer simply supplies the matrix of
sub-protocols it supports and only those supported by both are allowed. There
is no reason for the protocol to split that matrix up into multiple messages,
requiring termination. Independent messages exist because of timing or ordering
requirements. Implementing dependent messages as if they were independent is
wasteful and complicating.
Some things may need further negotiation. eg compact blocks sends multiple redundant messages with different versions
and then deduces the correct version based on the message ordering and version set supported. Doing this via verack
locks you into a very specific possible negotiation protocols. You could extend it further and suggest a verack K-V list
which allows for more flexible negotiation, but I'm not sure that it isn't more complicated than just shoving more
messages on the wire.
I’m well aware of the inefficiency produced by version linearity in the face of
optional sub-protocols. The protocol must negotiate to the version where it can
then negotiate support, which has been done. I support creating a simpler
system, eliminating these extra messages. The existing numeric version can be
reserved exclusively for “must” implement, and can be used to signal an
extension to the verack. The verack can then carry a list of “may” or “should”
sub-protocols for final negotiation.
I think we agree here - the current method of protocol version bumping isn't scalable and something more flexible is
definitely a better world.
The format of the matrix is arbitrary, but the requirement is to list a set of
optional sub-protocols. This implies a namespace. This implies “ownership“ of
names. In other words, that coordination requirement is not eliminated.
This is true, there is some ownership requirement, we could switch to hashes or something of the like, but
human-readable names have shown to be relatively non-colliding in past Bitcoin protocol changes.
e
Matt
On 8/18/20 12:54 PM, Eric Voskuil wrote:
“Bitcoin protocol has always expected clients to ignore unknown messages”
This is not true. Bitcoin has long implemented version negotiation, which is
the opposite expectation. Libbitcoin’s p2p protocol implementation immediately
drops a peer that sends an invalid message according to the negotiated version.
The fact that a given client does not validate the protocol does not make it an
expectation that the protocol not be validated.
Features can clearly be optional within an actual protocol. There have been
post-handshake negotiations implemented for optional messages which are valid
at the negotiated version. The protocol may be flexible while remaining
validateable. There is no reason to force a client to accept unknown message
traffic.
A generalized versioning change can be implemented in or after the handshake.
The latter is already done on an ad-hoc basis. The former is possible as long
as the peer’s version is sufficient to be aware of the behavior. This does not
imply any need to send invalid messages. The verack itself can simply be
extended with a matrix of feature support. There is no reason to complicate
negotiation with an additional message(s).
FWIW, bip37 did this poorly, adding a feature field to the version message,
resulting in bip60. Due to this design, older protocol-validating clients were
broken. In this case it was message length that was presumed to not be
validated.
e
On Aug 18, 2020, at 07:59, Matt Corallo via bitcoin-dev
<bitcoin-dev@lists.linuxfoundation.org> wrote:
This sounds like a great idea!
Bitcoin is no longer a homogeneous network of one client - it is many, with
different features implemented in each. The Bitcoin protocol hasn't (fully)
evolved to capture that reality. Initially the Bitcoin protocol had a simple
numerical version field, but that is wholly impractical for any diverse network
- some clients may not wish to implement every possible new relay mechanic, and
why should they have to in order to use other new features?
Bitcoin protocol changes have, many times in recent history, been made via new dummy
"negotiation" messages, which take advantage of the fact that the Bitcoin
protocol has always expected clients to ignore unknown messages. Given that pattern, it
makes sense to have an explicit negotiation phase - after version and before verack, just
send the list of features that you support to negotiate what the connection will be
capable of. The exact way we do that doesn't matter much, and sending it as a stream of
messages which each indicate support for a given protocol feature perfectly captures the
pattern that has been used in several recent network upgrades, keeping consistency.
Matt
On 8/14/20 3:28 PM, Suhas Daftuar via bitcoin-dev wrote:
Hi,
Back in February I posted a proposal for WTXID-based transaction relay[1] (now
known as BIP 339), which included a proposal for feature negotiation to take
place prior to the VERACK message being received by each side. In my email to
this list, I had asked for feedback as to whether that proposal was
problematic, and didn't receive any responses.
Since then, the implementation of BIP 339 has been merged into Bitcoin Core,
though it has not yet been released.
In thinking about the mechanism used there, I thought it would be helpful to
codify in a BIP the idea that Bitcoin network clients should ignore unknown
messages received before a VERACK. A draft of my proposal is available here[2].
I presume that software upgrading past protocol version 70016 was already
planning to either implement BIP 339, or ignore the wtxidrelay message proposed
in BIP 339 (if not, then this would create network split concerns in the future
-- so I hope that someone would speak up if this were a problem). When we
propose future protocol upgrades that would benefit from feature negotiation at
the time of connection, I think it would be nice to be able to use the same
method as proposed in BIP 339, without even needing to bump the protocol
version. So having an understanding that this is the standard of how other
network clients operate would be helpful.
If, on the other hand, this is problematic for some reason, I look forward to
hearing that as well, so that we can be careful about how we deploy future p2p
changes to avoid disruption.
Thanks,
Suhas Daftuar
[1] https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2020-February/017648.html
<https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2020-February/017648.html>
[2]
https://github.com/sdaftuar/bips/blob/2020-08-generalized-feature-negotiation/bip-p2p-feature-negotiation.mediawiki
<https://github.com/sdaftuar/bips/blob/2020-08-generalized-feature-negotiation/bip-p2p-feature-negotiation.mediawiki>
_______________________________________________
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
_______________________________________________
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
_______________________________________________
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
