> On May 14, 2020, at 5:23 PM, Wilfredo Sánchez Vega > <wsanc...@wsanchez.net(mailto:wsanc...@wsanchez.net)> wrote: > On May 7, 2020, at 12:48 AM, Glyph > <gl...@twistedmatrix.com(mailto:gl...@twistedmatrix.com)> wrote: > > > > If you want to include it in Twisted itself, your best bet is to actually > > develop it within twisted, as a series of small contributions, rather than > > as one gigantic one-shot one. Contributions over, say, 400 lines, take > > exponentially longer to review. > > > > Developing it within Twisted will make things go slower; you'll need to get > > everything code reviewed, you'll need to support multiple versions of > > Python (no py2 any more, but py3.5 is still pretty old), you'll have to > > have full test coverage from the get-go. But doing these things from the > > start is much easier than trying to retrofit them. > > > > I actually think that this would be a pretty good fit for Twisted, in the > > same way that it's been a benefit to have Conch maintained alongside the > > rest of Twisted. I can see you're developing things very much in line with > > Twisted's architecture (using cred for authentication, a realm interface, > > etc) and you've voiced this interest, so it would be great to have you > > along! > I think it's great to get an SMB implementation in the Twisted org, but why > would we even consider adding something like this to the main Twisted > project?
Just practically speaking, we've tried both approaches, and (very broadly generalizing) this one works and the other one doesn't. Separate projects that have been adopted from outside after they gained some traction are doing okay (i.e. Treq and Klein, some of the most popular applications for Twisted). But others (txacme) are suffering from a lack of maintenance and still others (ldaptor) just have a largely non-overlapping community and hardly benefit at all from org membership. And as you've noted, even the ones doing their best often want for reviews for far longer than Twisted's own latency. > The main repo is large, and including Twisted in your project brings along > far more functionality than you are likely to use, and packages are a fine > way to get the functionality you want, so what's the logic for including a > new thing in Twisted proper? Abstractly you make a plausible case. Concretely we tried to start splitting things up on the axes of protocol support and it just made a giant mess and created lots of overhead for the project. Modern infrastructure makes this slightly easier (last time we tried to start this process we didn't have a Github or a Travis), but not much easier. And even modern, better infrastructure leads to an avalanche of tedium as we copy CI configuration back and forth endlessly, replicating lessons that we learn about travis, and azure, and codecov, and (ad nauseam) Even in the abstract case though there’s a question of what twisted “is” and on what axes it should be split up. I think splitting up by protocol was largely a mistake, but splitting some of the really low level core stuff out (towncrier, automat, and hopefully filepath and deferred) has been pretty successful and worthwhile. Particularly in a post-asyncio world, the core event loop isn’t super interesting and the broad protocol support is the appeal. So even in the abstract case, there’s a case to be made that twisted is a bunch of implementations and interfaces for wire protocols that involve navigating hierarchies of things (DNS, HTTP, IMAP, SFTP) and SMB fits right in there. > You cite Conch above, where I'd actually argue that Conch should be moved to > a separate repo. Am I crazy? Not crazy, there's a valid case to be made, but Conch has received a tremendous amount of routine maintenance keeping it up to date with Twisted and Python API evolution by members of the project who would almost certainly otherwise not bother, just to keep the test suite going and the project as a whole integrated. This allows it to remain viable for the much smaller number of SSH-specific contributions that come along. > That said, a counter argument is that projects in the Twisted org suffer > greatly from poor participation and strict development rules, resulting in > glacial progress. It's not unusual for me to have a PR take 6 months to a > year to get a review (I have two right now that have been waiting since > November), and perhaps such PRs would get some attention in the main repo. > But I think that's an unfortunate way to address that problem. This is just a natural consequence of affordances provided by the tooling we presently have. Personally I look at the global queue of pull requests across all these projects and it's kind of exhausting. I don't think many other people do. Case in point: how many ldaptor or txacme reviews have you done since November? This is not to throw any shade; I’m not saying you should have, but it illustrates that right now, separate repos engender a really unfortunate level of community fragmentation. I agree that this is not really an ideal situation, but addressing it requires building a degree of tooling that we just don’t have the resources to do. Our previous attempt to do so was … bad; it kind of illustrated that we don’t have the relevant expertise to build these kinds of tools and mores either, and we just got kind of lucky with the somewhat random pile of jank we have landed on with the current twistedmatrix.com process. Anyway if someone wanted to do the work to separate out the concerns of version control, pip installability, issue tracking, and integrated regression testing for a whole suite of related python packages in an org, I would be extremely excited, but the benefits of just having Twisted be, say, 5% bigger overall to foster development of a whole new popular and intimidating protocol seem clear given the state of the world today. If those tools do show up later to make this tractable, we can easily split it out later! Along with conch, and mail, and names, for that matter… -glyph
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
