On 2020-09-13 5:19 p.m., Jarek Potiuk wrote:
Can you please make an inline comment in the document? the Cwiki allows
inline comments, just select a paragraph and comment it there. This is the
easiest way to keep it focused in the document. I am not sure if
understand the Open-Office specific things, i'd love to understand that
though (I used Open-Office for years) :)
Done, and I expanded on this point.
I think that any release of ASF software must have corresponding sources
that can be use to generate those from. Even if there are some binary
files, those too should be generated from some kind of sources or
"officially released" binaries that come from some sources. I'd love to get
some more concrete examples of where it is not possible.
Sure, this is totally possible. I'm just saying that the amount of
source is extreme in the case where you're talking about a desktop app
that runs in Java or Electron (Chrome as a desktop app), as two examples.
-Joan
J.
On Sun, Sep 13, 2020 at 11:09 PM Joan Touzet <woh...@apache.org> wrote:
HI Jarek,
Can you comment on one specific thing? In Proposal 1 you still leave the
text "...MUST only add binary/bytecode files". This is not possible for
convenience packages in many situations - for instance OpenOffice or
other languages - where providing a full release of a product requires a
language runtime. It has always bothered me that this text effectively
prevents redistribution of binary assets in the packages that are not
strictly speaking derived from the source code.
As you go far beyond this with the container packaging in Proposal 2, I
believe Proposal 1 needs to be modified to match. In my opinion a
suitable replacement would be something like:
"In all such cases...version number as the source release, as MUST
include only the binary/bytecode files that are necessary, via the
compiling and packaging of that source code release and its
dependencies, to produce a functional deliverable. All instructions..."
-Joan
On 2020-09-13 4:40 p.m., Jarek Potiuk wrote:
Just for your information - after a discussion in the ComDev mailing
list.
I created a proposal for Apache Software Foundation to introduce changes
to
the "ASF release policies", to make it clear and straightforward to
release
"convenience packages" in the form of "software packaging" (such as Helm
Charts and Container Images) rather than "compiled packages" as
recognised
so far by the ASF policies.
The proposal is here:
https://cwiki.apache.org/confluence/display/COMDEV/Updates+of+policies+for+the+convenience+packages
The discussion in the ComDev ASF mailing list is here:
https://lists.apache.org/thread.html/r49c3ef0a8423664c564c0c2719056662021f03b5678ef5b249892c10%40%3Cdev.community.apache.org%3E
We are going to discuss it and propose to the ASF board to vote on the
changes.
I look forward to all comments and I hope it can pave the way for the ASF
to provide a coherent approach for releasing Container Images, Helm
Charts
for all ASF projects.
On Mon, Aug 31, 2020 at 9:23 PM Jarek Potiuk <jarek.pot...@polidea.com>
wrote:
Just to revive this thread and let you know what we've done in Airflow.
We merged changes to our repository that allow our users to rebuild all
images if they need to -using official sources. It's not very involved
and
not a lot of code to maintain:
https://github.com/apache/airflow/pull/9650/
Next time when we release Airflow Sources including the Helm Chart, any
of
our users will be able to rebuild all the images used in charts from the
ASF-released source package.
The whole discussion ended up to be not about the Licence, but about the
content of the official ASF source package release.
I personally think this is the only way to fulfill this chapter from ASF
release policy:
http://www.apache.org/legal/release-policy.html#what-must-every-release-contain
Every ASF release must contain a source package, which must be
sufficient
for a user to build and test the release provided they have access to
the
appropriate platform and tools.
I would love to hear other thoughts about it.
J.
On Tue, Jun 23, 2020 at 11:42 PM Roman Shaposhnik <ro...@shaposhnik.org
wrote:
On Tue, Jun 23, 2020 at 2:26 AM Jarek Potiuk <jarek.pot...@polidea.com
wrote:
My understanding the bigger problem is the license of the dependency
(and
their dependencies) rather than the official/unofficial status. For
Apache
Yetus' test-patch functionality, we defaulted all of our plugins to
off
because we couldn't depend upon GPL'd binaries being available or
giving
the impression that they were required. By doing so, it put the onus
on
the user to specifically enable features that depends upon GPL'd
functionality. It also pretty much nukes any idea of being user
friendly.
:(
Indeed - Licensing is important, especially for source code
redistribution.
We used to have some GPL-install-on-your-own-if-you-want in the past
but
those dependencies are gone already.
2) If it's not - how do we determine which images are "officially
maintained".
Keep in mind that Docker themselves brand their images as
'official' when they actually come from Docker instead of the
organizations
that own that particular piece of software. It just adds to the
complexity.
Not really. We actually plan to make our own Apache Airflow Docker
image as
official one. Docker has very clear guidelines on how to make images
"official" and it https://docs.docker.com/docker-hub/official_images/
and
there is quite a long iist of those:
https://github.com/docker-library/official-images/tree/master/library
-
most of them maintained by the "authirs" of the image. Docker has a
dedicated team that reviews, checks those images and they encourage
that
the "authors" maintain them. Quote from Docker's docs: "While it is
preferable to have upstream software authors maintaining their
corresponding Official Images, this is not a strict requirement."
3) If yes - how do we put the boundary - when image is acceptable?
Are
there any criteria we can use or/ constraints we can put on the
licences/organizations releasing the images we want to make
dependencies
for released code of ours?
License means everything.
For software distribution - true. It is the "blocker". But I think my
question goes a bit beyond that - i.e. whether it's ok to
encourage/depend
on the work maintained by other organizations than Apache if they are
not
"official". My take is that it's likely OK to depend on that providing
that
there is a kind of statement from those organizations that they
maintain it.
An example risk I see:
Airflow users depend heavily on helm chart to install Airflow - what
happens if the community agrees to implement something that the
organization does not want to implement (for whatever reason).
FWIW: every corporation I ever worked at would commission a
BlackDuck/Palamida
report of a total software scans for its products. There was some
amount of: "this
ASF project pulls a dependency FOO (non ASF) that is declared to be
licensed under
the license X but it actually isn't -- here's why..."
We trust our upstream dependencies, but I don't think we can verify
them as a foundation.
Hence we keep relying on that feedback coming from corporate sides.
Thanks,
Roman.
4) If some images are not acceptable, shoud we bring them in and
release
them in a community-managed registry?
For the Apache Yetus docker image, we're including
everything
that
the project supports. *shrugs*
Yeah. That's perfectly OK in many cases. Our docker image is also
self-containing. However, Airflow is a bit special here. Airflow is an
orchestrator which means that it can talk to many different services.
We
have 58(!) "providers" - basically external services we can talk to.
And
many of those services require many dependencies - for example
Cassandra
(for production installation) requires cython-compiled driver (for
performance) and it takes 10 minutes to build it. The smaller the
images -
the better - therefore the images we release contain the most
"popular"
providers rather than all of them, but the user can build their own
image
from the sources if they want and add those extra dependencies they
need.
Another problem is - helm chart uses - by definition - a collection of
images - so we will always have some images that helm chart depends on
(pgbouncer is a good example). So it cannot be really self-contained.
We
need to have dependencies, but the question is about "who controls
them
:)"
J.
--
Jarek Potiuk
Polidea <https://www.polidea.com/> | Principal Software Engineer
M: +48 660 796 129 <+48660796129>
[image: Polidea] <https://www.polidea.com/>
--
Jarek Potiuk
Polidea <https://www.polidea.com/> | Principal Software Engineer
M: +48 660 796 129 <+48660796129>
[image: Polidea] <https://www.polidea.com/>
