Thanks for taking a look at this, Henrik! If I understand your proposal, it is: 1. Do only the source release in dist.apache.org. 2. Once that's approved, use that source release to generate PyPI artifacts and publish them to PyPi.
If this is the correct understanding, then I am +1 to it - it is the pragmatic thing to do now. We can revisit what we include in releases after upgrading poetry (or switching to uv). There is a good chance this problem will just go away. On Wed, May 14, 2025 at 4:56 PM Henrik Ingo <hen...@nyrkio.com> wrote: > TLDR: We should release only a single file, the source distribution, and > the file should be named apache-otava-0.6.0.tar.gz > > > Hi all, back to my regular desk and timezone.,.,. > > This discussion raised several deeper questions once I started looking into > it. Related reading: https://realpython.com/python-wheels/ > > Now that I tried to come up with some logical argument for how these files > should be named, the more fundamental question is, why are we releasing 3 > different archives of files anyway? Currently we don't have any C libraries > (.so files) in Otava, so the difference between the .whl and .tar.gz files > is cosmetic. > > Also, since this is python, these are all source code files. In particular > the apache_otava-0.6.0+incubating.tar.gz is literally called "source > distribution". (I believe this is comparable to how a project using > autotools would not ship a raw copy of their git repository either, but > would prepare a source distribution with automake and autoconf, and then > ship that.) > > So, especially since this seems to be the point in the process involving > manual checking and discussions, should we rather minimize the amount of > files here and just release what poetry calls the "source distribution"? > > Now, if we do that, then the naming is also mostly decided for us, we have > no choice but to follow the PyPI naming and versioning scheme.... except, > everything I can google contradicts what is said in this thread: > > https://packaging.python.org/en/latest/specifications/name-normalization/ > > > https://docs.aws.amazon.com/codecatalyst/latest/userguide/python-name-normalization.html#:~:text=For%20Python%20packages%2C%20when%20performing,used%20by%20pip%20and%20twine > . > > The name should be lowercased with all runs of the characters ., -, or _ > replaced with a single - character. This can be implemented in Python with > the re module: > > import re > > def normalize(name): > return re.sub(r"[-_.]+", "-", name).lower() > > This means that the following names are all equivalent: > > friendly-bard (normalized form) > Friendly-Bard > FRIENDLY-BARD > friendly.bard > friendly_bard > > > Why our poetry apparently does something exactly opposite I don't know. One > guess is it's a 5 year old version of poetry running on python 3.8? > > > > As for the version string, it seems the + sign is not intended to be used > by the upstream project (us) and would not be allowed at PyPI even if we > try: > > <quote> > Local version identifiers SHOULD NOT be used when publishing upstream > projects to a public index server, but MAY be used to identify private > builds created directly from the project source. Local version identifiers > SHOULD be used by downstream projects when releasing a version that is API > compatible with the version of the upstream project identified by the > public version identifier, but contains additional changes (such as bug > fixes). As the Python Package Index is intended solely for indexing and > hosting upstream projects, it MUST NOT allow the use of local version > identifiers. > </quote> > > https://packaging.python.org/en/latest/specifications/version-specifiers/ > > Assuming that we don't want to add "-incubating" to the actual package > name, the specification doesn't leave much alternatives. One idea would be > to not publish "final" releases during the incubation, rather always use > .rc1 as the last component. However, that could be confusing to users, and > also it seems various installers might be reluctant to download and install > a dev or rc release - users would have to explicitly define the specific > version to get it installed. > > There is one exception in the document where putting information into the > "local version" part is allowed: build systems that put a git hash into the > version string. e.g. 0.6.0dev+123abc. But note that this would still not be > publishable on PyPI, both because +localsegment is not allowed and because > development versions aren't allowed. > > In short, I don't see a way to include the label "incubating" anywhere in > the filename, short of considering it part of the project name. > Interpreting the intention of this specification, I believe the preferred > way is to add such a label to the package meta-data instead: > - DVCS based label can be stored in the project metadata > - the corresponding Olson database version could be recorded in the > project metadata. > > > Conclusion: We should release only a single file, the source distribution, > and the file should be named apache-otava-0.6.0.tar.gz > > henrik > > > > > On Sun, May 11, 2025 at 5:37 AM Alexander Sorokoumov < > aleksandr.sorokou...@gmail.com> wrote: > > > Hi Dave! > > > > I wish we could use "-" instead of "_"". Unfortunately, Python release > > names must be normalized. As part of this normalization, "-" symbols are > > automatically replaced with "_". > > Case in point - I did use "apache-otava" [1] and poetry automatically > > replaced it with "apache_otava". > > > > As for the "+", it is used in Python releases for "local version labels" > > [2]. This label will be present in the source release and absent in PyPI, > > which is AFAIU what we want [3]. > > > > Taking into account the feedback discussed in [4], I propose the > following > > structure for https://dist.apache.org/repos/dist/dev/incubator/otava/: > > <version>-incubating-rc<n>/ (folder containing each release) > > > > - apache-otava-<version>-incubating-sources.tar.gz (the source > release) > > - pypi/ (pypi-specific artifacts) > > - apache-otava-<version>+incubating-py3-none-any.whl > > - apache_otava-<version>+incubating.tar.gz > > - (sha and asc files are omitted for brevity) > > > > WDYT? > > > > 1. > > > > > https://github.com/apache/otava/blob/f1d21a84b79a4bb411b404437b163250d30edecb/pyproject.toml#L19 > > 2. > > > > > https://packaging.python.org/en/latest/specifications/version-specifiers/#local-version-identifiers > > 3. > > > > > https://cwiki.apache.org/confluence/display/INCUBATOR/Incubator+Release+Checklist > > , > > point 2. > > 4. https://lists.apache.org/thread/hbrgs803r8pn0z2kjlqsr4mcsx36gfqp > > > > Best, > > Alex > > > > On Sat, May 10, 2025 at 11:34 AM Dave Fisher <w...@apache.org> wrote: > > > > > FYI - the usual practice is to use “-“ and not “_” or “+” in these > > > filenames. > > > > > > I recommend: apache-otava-0.6.0-incubating-py3-none-any.whl > > > apache-otava-0.6.0-incubating.tar.gz > > > > > > Best, > > > Dave > > > > > > > On May 9, 2025, at 7:09 PM, Gerrrr (via GitHub) <g...@apache.org> > > wrote: > > > > > > > > > > > > Gerrrr commented on PR #58: > > > > URL: https://github.com/apache/otava/pull/58#issuecomment-2868224768 > > > > > > > > Yes! > > > > > > > > ``` > > > > $ ls dist > > > > apache_otava-0.6.0+incubating-py3-none-any.whl > > > apache_otava-0.6.0+incubating.tar.gz > > > > ``` > > > > > > > > > > > > -- > > > > This is an automated message from the Apache Git Service. > > > > To respond to the message, please log on to GitHub and use the > > > > URL above to go to the specific comment. > > > > > > > > To unsubscribe, e-mail: commits-unsubscr...@otava.apache.org > > > > > > > > For queries about this service, please contact Infrastructure at: > > > > us...@infra.apache.org > > > > > > > > > > > > >
