Distribution is pretty straightforward, the process I follow after 'build' is complete:
## Release Cassandra CQLSH You must have the maintainer privilege of `https://pypi.org/project/cqlsh/` 1. verify package locally with pip pip install -e . 2. Test upload on testpypi twine upload --repository testpypi dist/* pip install --index-url https://test.pypi.org/simple/ cqlsh 3. prod upload twine upload --repository pypi dist/* Regards, Brad On Thu, Aug 10, 2023 at 3:27 PM Patrick McFadin <pmcfa...@gmail.com> wrote: > Dinesh raises some good points. > > If we do adopt this, there will be non-zero overhead of the release > process. This is fine but we need volunteers to run this process. My > understanding is that they need to be ideally PMC or at least Committers > on the project to go through all the steps to successfully release a new > artifact for our users. > > Which was addressed in the proposed changes part of the CEP: > > > - A document detailing procedures for releasing to PyPI.org. This > document should include details on: > > > 1. How release to PyPI can be integrated into the build process. Can > this be done with automation? > 2. How will credentials, permissions and ownership of packages on PyPI > be managed? > > > - > > My first thought was automation and integration into the build release. > > Can you briefly outline the steps that need to be followed for a PyPI > release, Brad? > > Patrick > > > On Wed, Aug 9, 2023 at 2:54 PM Abe Ratnofsky <a...@aber.io> wrote: > >> I think it would be good for the project to have an official PyPI >> distribution, and the signal from users (40K downloads per month) is a >> clear indication that this is useful. Timely releases would help us get >> future improvements to cqlsh out faster, and moving this to an official >> distribution would protect users against any changes in this volunteer >> effort in case something happens in the future. >> >> +1 (nb) >> >> -- >> Abe >> >> On Aug 9, 2023, at 1:33 PM, Brad <bscho...@gmail.com> wrote: >> >> HI Dinesh, >> >> You are correct that the scope of this CEP is practical, narrow and >> limited to having an official distribution of CQLSH on the official Python >> package repository. Cassandra end-users, who use the CQLSH command line, >> would benefit in several direct ways: >> >> - A timely distribution of new CQLSH versions on the official Python >> package repository aligned with Apache Cassandra releases >> - A trusted distribution overseen by Apache Cassandra instead of >> third party maintainers. Today, there is only trust-based faith that the >> PyPI distribution of CQLSH matches the Apache Open Source one. >> - A lightweight distribution of CQLSH clocking in at 110KB vs >> downloading a 50MB tarball. >> >> Perhaps those are modest goals, but I would suggest they are big wins for >> the Cassandra user community. If you haven't tried it yet, please run '*pip >> install cqlsh*' on your desktop and see how nicely it works. Indeed, the >> return-on-investment of effort here should be really high, as the work is >> mostly already done, it's just run from a private repo at >> https://github.com/jeffwidman/cqlsh and has been maintained continually >> since 2013. >> >> Other initiatives such as subdividing the project(s) or re-writing the >> REPL in another language would be out-of-scope. It would be entirely >> appropriate to have a separate discussion on those two topics, if you wish >> to start that discussion. >> >> The process and degree of overhead required to publish to PyPI will >> require some discovery and discussion. Ideally, it would be possible to >> automate it. That is definitely a topic we need further input from the >> engineers involved in the build-release process. >> >> A pre-CEP discussion of this proposal was started by Jeff on the mailing >> list back in early July, see >> https://lists.apache.org/thread/sy3p2b2tncg1bk6x3r0r60y10dm6l18d. >> >> Regards, >> >> Brad >> >> On Wed, Aug 9, 2023 at 3:31 PM Dinesh Joshi <djo...@apache.org> wrote: >> >>> Brad, >>> >>> Thanks for starting this discussion. My understanding is that we're >>> simply adding pip support for cqlsh and Apache Cassandra project will >>> officially publish a cqlsh pip package. This is a good goal but other >>> than having an official pip package, what is it that we're gaining? >>> Please don't interpret this as push back on your proposal but I am >>> unclear on what we're trying to solve by making this official >>> distribution. There are several distribution channels and it is >>> untenable to officially support all of them. >>> >>> If we do adopt this, there will be non-zero overhead of the release >>> process. This is fine but we need volunteers to run this process. My >>> understanding is that they need to be ideally PMC or at least Committers >>> on the project to go through all the steps to successfully release a new >>> artifact for our users. >>> >>> I would have liked this CEP to go a bit further than just packaging >>> cqlsh in pip. IMHO we should have cqlsh as a separate sub-project. It >>> doesn't need to live in the cassandra repo. Extracting cqlsh into it's >>> separate repo would allow us to truly decouple cqlsh from the server. >>> This is already true for the most part as we rely on the Python driver >>> which is compatible with several cassandra releases. As it stands today >>> it is not possible for us to update cqlsh without making a Cassandra >>> release. >>> >>> If you truly want to go a bit further, we should consider rewriting >>> cqlsh in Java so we can easily share code from the server. We can then >>> potentially use Java Native Image[1] to produce a truly platform >>> independent binary like golang. Python has its strengths but it does get >>> hairy as it expects certain runtime components on the target. Java With >>> Native Image we make things very simple from a user's perspective very >>> similar to how golang produces statically linked binaries. This might be >>> a very far out thought but it is worth exploring. I believe GraalVM's >>> license might allow us to produce binaries that we can incorporate in >>> our release but IANAL so maybe we can ask ASF legal on their opinion. >>> >>> Giving cqlsh it's own identity as a sub-project might help us build a >>> roadmap and evolve it along these lines. >>> >>> I would like other folks to chime in with their opinions. >>> >>> Dinesh >>> >>> On 8/9/23 09:18, Brad wrote: >>> > >>> > As per the CEP process guidelines, I'm starting a formal DISCUSS thread >>> > to resume the conversation started here[1]. >>> > >>> > The developers who maintain the Python CQLSH client on the official >>> > Python PYPI repository would like to integrate and donate their open >>> > source work to the Apache Cassandra project so it can be more tightly >>> > and seamlessly integrated. >>> > >>> > The Apache Cassandra project pre-dates the adoption in Python 3.4 of >>> > PyPI as the default package manager. As a result, an unofficial >>> > distribution has been provided by a group of developers who have >>> > maintained the repository there since October 2013. >>> > >>> > The installable version of CQLSH on PyPI.org allows end users to >>> install >>> > a cqlsh client with PIP - no tarball or path setup required. I.e., >>> > >>> > $ pip install cqlsh >>> > >>> > This popular package has 50K downloads per month and is today >>> maintained >>> > by Jeff Wideman and Brad Schoening. The PYPI package is updated upon >>> > every major release by simply repackaging the CQLSH that ships with >>> > every Cassandra release. >>> > >>> > CQLSH PyPI Repository: https://pypi.org/project/cqlsh/ >>> > <https://pypi.org/project/cqlsh/> >>> > >>> > >>> > This CEP Proposal suggests incorporating PYPI as a regular part of the >>> > Cassandra release process and making the CQLSH project on PYPI an >>> > official distribution point. >>> > >>> > The full CEP can be reviewed at: >>> > >>> > Wiki: CEP-35: Add PIP support for CQLSH >>> > < >>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=263425995 >>> >. >>> > >>> > Jira: CASSANDRA-18654 >>> > <https://issues.apache.org/jira/browse/CASSANDRA-18654> >>> > >>> > >>> > But in brief, the proposal will: >>> > >>> > * Add PyPI.org as an official distribution point for CQLSH >>> > * Allow end users to install CQLSH with simply 'pip install cqlsh' on >>> > MacOS, Windows and Linux platforms. >>> > * Donate the modest amount of existing configuration files by the >>> > authors to Apache Cassandra >>> > * This only involves the Python CQLSH client, no changes to >>> > distribution of Java server side code and tools are involved. >>> > >>> > We welcome further discussion and suggestions regarding this proposal >>> on >>> > the mailing list here. >>> > >>> > Regards, >>> > >>> > Jeff Widman & >>> > Brad Schoening >>> > >>> > [1] https://lists.apache.org/thread/sy3p2b2tncg1bk6x3r0r60y10dm6l18d >>> > < >>> https://lists.apache.org/thread.html/ra7caa1dd42ccaa04bcabfbc33233995c125c655f9a3cdb2c7bd8e9f7%40%3Cdev.cassandra.apache.org%3E >>> > >>> >>> >>
