Re: Github Actions

[sebb]

I think the pre-verified code could run on a separate system with
restricted access.
That's how self-service works for creating mailing lists, for example.

In this case, there would need to be a separate host with read access
to Jenkins.
It could accept publish requests from Jenkins, and route them accordingly.

This would require a bit of effort to set up, but could be used for
multiple projects.

On Wed, 28 Aug 2019 at 03:26, Greg Stein  wrote:
>
> Yeah. FIgured as much, hoped that I was missing something :)
>
> (note: we have the same issue with buildbot and jenkins: we simply trust
> the communities to not exfil that data)
>
> On Tue, Aug 27, 2019 at 9:16 PM Matt Sicker  wrote:
>
> > How to avoid leaking secrets: only way to do that is via pre-verified code
> > that executes something with that secret. Otherwise, there’s literally
> > infinite ways to leak it being a Turing machine and all. This applies to
> > all CICD tools.
> >
> > On Tue, Aug 27, 2019 at 20:32, Greg Stein  wrote:
> >
> > > Hi Francis,
> > >
> > > Is the token needed to push from calcite to calcite-site? Is that an
> > oauth
> > > token or something? And are you able to use the repository settings to
> > add
> > > secrets, but you don't have the right token? Or you cannot add secrets at
> > > all? (I can't tell since I have superpowers)
> > >
> > > I've added GSTEIN_TEST_SECRET to Calcite. See if you can extract/print
> > that
> > > into your build/action log. If so, then we can try to figure out the
> > > security here (ie. how do we avoid Actions exfiltrating the token?)
> > >
> > > Thanks,
> > > -g
> > >
> > > On Tue, Aug 27, 2019 at 5:19 AM Francis Chuang  > >
> > > wrote:
> > >
> > > > I have implemented the ability to generate the website and javadoc for
> > > > Calcite using Github Actions. See:
> > > > https://github.com/apache/calcite/tree/test-site/.github/workflows
> > > >
> > > > The missing piece is that we need the token to publish to our
> > > > calcite-site repository to be added as a secret in Github Actions and
> > > > there is currently no clear process as to whether this is allowed or
> > how
> > > > to get this done.
> > > >
> > > > See:
> > > > https://issues.apache.org/jira/browse/INFRA-18874
> > > > https://issues.apache.org/jira/browse/INFRA-18875
> > > >
> > > > Francis
> > > >
> > > > On 27/08/2019 7:52 pm, Greg Stein wrote:
> > > > > Have you had an opportunity to make progress on this, to share with
> > us?
> > > > >
> > > > > Anybody else with news?
> > > > >
> > > > > Thanks!
> > > > > -g
> > > > > InfraAdmin, ASF
> > > > >
> > > > >
> > > > > On Tue, Aug 13, 2019 at 3:59 PM Karl Heinz Marbaise <
> > khmarba...@gmx.de
> > > >
> > > > > wrote:
> > > > >
> > > > >> Hi,
> > > > >>
> > > > >> I've made a simple PoC for the Apache Maven Dependency Plugin on a
> > > > >> separate branch.
> > > > >>
> > > > >> I will try within the next days more features for example Mac OS
> > > builds
> > > > >> etc.
> > > > >>
> > > > >>
> > > > >> Currently I simply push my changes via gitbox ..
> > > > >>
> > > > >> maven-dependency-plugin (GITHUB_ACTIONS)$ git remote -v
> > > > >> origin
> > > https://gitbox.apache.org/repos/asf/maven-dependency-plugin.git
> > > > >> (fetch)
> > > > >> origin
> > > https://gitbox.apache.org/repos/asf/maven-dependency-plugin.git
> > > > >> (push)
> > > > >>
> > > > >>
> > > > >> Also I'm interested to use SonarCloud related with GitHub Actions..?
> > > > >>
> > > > >>
> > > > >> Kind regards
> > > > >> Karl Heinz Marbaise
> > > > >> Apache Maven PMC
> > > > >>
> > > > >> [1]:
> > https://github.com/apache/maven-dependency-plugin/runs/192633340
> > > > >> [2]:
> > > > >>
> > > > >>
> > > >
> > >
> > https://github.com/apache/maven-dependency-plugin/blob/66435b225e7885f44b25207e025469f6d5237107/.github/workflows/maven.yml
> > > > >>
> > > > >> On 12.08.19 00:31, Greg Stein wrote:
> > > > >>> On Sun, Aug 11, 2019 at 5:15 PM Francis Chuang <
> > > > francischu...@apache.org
> > > > >>>
> > > > >>> wrote:
> > > >  ...
> > > > >>>
> > > >  I think there are quite a few ASF projects using gitbox and Github
> > > and
> > > >  this would be a very good complement or replacement for Travis,
> > > > appvoyer
> > > >  and other CI/CD platforms currently in use.
> > > > 
> > > >  Is there any interest from the ASF to enable this for all Gitbox
> > > >  projects when it becomes fully public?
> > > > 
> > > > >>>
> > > > >>> Absolutely. The Infrastructure team would love to see groups try
> > this
> > > > >> out,
> > > > >>> and share the experiences here.
> > > > >>>
> > > > >>> If there are any hurdles, then share them and we'll try to knock
> > them
> > > > >> down.
> > > > >>>
> > > > >>> I am also interested in being able to push to our website
> > > automatically
> > > >  using Github Actions. If the git token that can push to a
> > particular
> > > >  website repository is added as a secret [2] to Github Actions,
> > this
> > > >  would be pretty easy 

Re: Github Actions

[Francis Chuang]

Hi Greg,

Yes, the token is needed to push to calcite-site as we are running the 
job in the calcite repository. In terms of the token, if we are pushing 
to the calcite-site using Gitbox, then the token would be the one that 
is currently used in the git-websites node on Jekins. If we want to push 
directly to the Github calcite-site repository, then the token would 
probably be a Personal Access Token for the Apache org on Github (not 
100% sure on this, but I think this is the case to the best of my 
understanding).


We currently don't have ability to add secrets to Github Actions at all. 
However, this isn't a huge problem for us at the moment. We currently 
just need the Gitbox token on git-websites (which we do not have direct 
access to) or a Github Personal Access Token added to our repositories.


I have added an action to print GSTEIN_TEST_SECRET. See this run here: 
https://github.com/apache/calcite/runs/205292835


I don't know the original value of the secret, but it prints as "***".

Francis

On 28/08/2019 11:32 am, Greg Stein wrote:

Hi Francis,

Is the token needed to push from calcite to calcite-site? Is that an 
oauth token or something? And are you able to use the repository 
settings to add secrets, but you don't have the right token? Or you 
cannot add secrets at all? (I can't tell since I have superpowers)


I've added GSTEIN_TEST_SECRET to Calcite. See if you can extract/print 
that into your build/action log. If so, then we can try to figure out 
the security here (ie. how do we avoid Actions exfiltrating the token?)


Thanks,
-g

On Tue, Aug 27, 2019 at 5:19 AM Francis Chuang > wrote:


I have implemented the ability to generate the website and javadoc for
Calcite using Github Actions. See:
https://github.com/apache/calcite/tree/test-site/.github/workflows

The missing piece is that we need the token to publish to our
calcite-site repository to be added as a secret in Github Actions and
there is currently no clear process as to whether this is allowed or
how
to get this done.

See:
https://issues.apache.org/jira/browse/INFRA-18874
https://issues.apache.org/jira/browse/INFRA-18875

Francis

On 27/08/2019 7:52 pm, Greg Stein wrote:
 > Have you had an opportunity to make progress on this, to share
with us?
 >
 > Anybody else with news?
 >
 > Thanks!
 > -g
 > InfraAdmin, ASF
 >
 >
 > On Tue, Aug 13, 2019 at 3:59 PM Karl Heinz Marbaise
mailto:khmarba...@gmx.de>>
 > wrote:
 >
 >> Hi,
 >>
 >> I've made a simple PoC for the Apache Maven Dependency Plugin on a
 >> separate branch.
 >>
 >> I will try within the next days more features for example Mac OS
builds
 >> etc.
 >>
 >>
 >> Currently I simply push my changes via gitbox ..
 >>
 >> maven-dependency-plugin (GITHUB_ACTIONS)$ git remote -v
 >> origin
https://gitbox.apache.org/repos/asf/maven-dependency-plugin.git
 >> (fetch)
 >> origin
https://gitbox.apache.org/repos/asf/maven-dependency-plugin.git
 >> (push)
 >>
 >>
 >> Also I'm interested to use SonarCloud related with GitHub Actions..?
 >>
 >>
 >> Kind regards
 >> Karl Heinz Marbaise
 >> Apache Maven PMC
 >>
 >> [1]:
https://github.com/apache/maven-dependency-plugin/runs/192633340
 >> [2]:
 >>
 >>

https://github.com/apache/maven-dependency-plugin/blob/66435b225e7885f44b25207e025469f6d5237107/.github/workflows/maven.yml
 >>
 >> On 12.08.19 00:31, Greg Stein wrote:
 >>> On Sun, Aug 11, 2019 at 5:15 PM Francis Chuang
mailto:francischu...@apache.org>
 >>>
 >>> wrote:
  ...
 >>>
  I think there are quite a few ASF projects using gitbox and
Github and
  this would be a very good complement or replacement for
Travis, appvoyer
  and other CI/CD platforms currently in use.
 
  Is there any interest from the ASF to enable this for all Gitbox
  projects when it becomes fully public?
 
 >>>
 >>> Absolutely. The Infrastructure team would love to see groups
try this
 >> out,
 >>> and share the experiences here.
 >>>
 >>> If there are any hurdles, then share them and we'll try to
knock them
 >> down.
 >>>
 >>> I am also interested in being able to push to our website
automatically
  using Github Actions. If the git token that can push to a
particular
  website repository is added as a secret [2] to Github Actions,
this
  would be pretty easy to use for projects to automate the
building of
  their websites.
 
 >>>
 >>> Should be possible. Again, comes back to groups trying this and
reporting
 >>> back how well it went.
 >>>
 >>> Cheers,
 >>> Greg Stein
 >>> Infrastructure Administrator, ASF
 >>>

Re: Github Actions

[Matt Sicker]

Secrets masking is another thing that only works for basic cases. Don’t try
encoding it and printing it, for example. Just a different character set
can throw off some tools let alone base 64.

On Wed, Aug 28, 2019 at 04:23, Francis Chuang 
wrote:

> Hi Greg,
>
> Yes, the token is needed to push to calcite-site as we are running the
> job in the calcite repository. In terms of the token, if we are pushing
> to the calcite-site using Gitbox, then the token would be the one that
> is currently used in the git-websites node on Jekins. If we want to push
> directly to the Github calcite-site repository, then the token would
> probably be a Personal Access Token for the Apache org on Github (not
> 100% sure on this, but I think this is the case to the best of my
> understanding).
>
> We currently don't have ability to add secrets to Github Actions at all.
> However, this isn't a huge problem for us at the moment. We currently
> just need the Gitbox token on git-websites (which we do not have direct
> access to) or a Github Personal Access Token added to our repositories.
>
> I have added an action to print GSTEIN_TEST_SECRET. See this run here:
> https://github.com/apache/calcite/runs/205292835
>
> I don't know the original value of the secret, but it prints as "***".
>
> Francis
>
> On 28/08/2019 11:32 am, Greg Stein wrote:
> > Hi Francis,
> >
> > Is the token needed to push from calcite to calcite-site? Is that an
> > oauth token or something? And are you able to use the repository
> > settings to add secrets, but you don't have the right token? Or you
> > cannot add secrets at all? (I can't tell since I have superpowers)
> >
> > I've added GSTEIN_TEST_SECRET to Calcite. See if you can extract/print
> > that into your build/action log. If so, then we can try to figure out
> > the security here (ie. how do we avoid Actions exfiltrating the token?)
> >
> > Thanks,
> > -g
> >
> > On Tue, Aug 27, 2019 at 5:19 AM Francis Chuang  > > wrote:
> >
> > I have implemented the ability to generate the website and javadoc
> for
> > Calcite using Github Actions. See:
> > https://github.com/apache/calcite/tree/test-site/.github/workflows
> >
> > The missing piece is that we need the token to publish to our
> > calcite-site repository to be added as a secret in Github Actions and
> > there is currently no clear process as to whether this is allowed or
> > how
> > to get this done.
> >
> > See:
> > https://issues.apache.org/jira/browse/INFRA-18874
> > https://issues.apache.org/jira/browse/INFRA-18875
> >
> > Francis
> >
> > On 27/08/2019 7:52 pm, Greg Stein wrote:
> >  > Have you had an opportunity to make progress on this, to share
> > with us?
> >  >
> >  > Anybody else with news?
> >  >
> >  > Thanks!
> >  > -g
> >  > InfraAdmin, ASF
> >  >
> >  >
> >  > On Tue, Aug 13, 2019 at 3:59 PM Karl Heinz Marbaise
> > mailto:khmarba...@gmx.de>>
> >  > wrote:
> >  >
> >  >> Hi,
> >  >>
> >  >> I've made a simple PoC for the Apache Maven Dependency Plugin on
> a
> >  >> separate branch.
> >  >>
> >  >> I will try within the next days more features for example Mac OS
> > builds
> >  >> etc.
> >  >>
> >  >>
> >  >> Currently I simply push my changes via gitbox ..
> >  >>
> >  >> maven-dependency-plugin (GITHUB_ACTIONS)$ git remote -v
> >  >> origin
> > https://gitbox.apache.org/repos/asf/maven-dependency-plugin.git
> >  >> (fetch)
> >  >> origin
> > https://gitbox.apache.org/repos/asf/maven-dependency-plugin.git
> >  >> (push)
> >  >>
> >  >>
> >  >> Also I'm interested to use SonarCloud related with GitHub
> Actions..?
> >  >>
> >  >>
> >  >> Kind regards
> >  >> Karl Heinz Marbaise
> >  >> Apache Maven PMC
> >  >>
> >  >> [1]:
> > https://github.com/apache/maven-dependency-plugin/runs/192633340
> >  >> [2]:
> >  >>
> >  >>
> >
> https://github.com/apache/maven-dependency-plugin/blob/66435b225e7885f44b25207e025469f6d5237107/.github/workflows/maven.yml
> >  >>
> >  >> On 12.08.19 00:31, Greg Stein wrote:
> >  >>> On Sun, Aug 11, 2019 at 5:15 PM Francis Chuang
> > mailto:francischu...@apache.org>
> >  >>>
> >  >>> wrote:
> >   ...
> >  >>>
> >   I think there are quite a few ASF projects using gitbox and
> > Github and
> >   this would be a very good complement or replacement for
> > Travis, appvoyer
> >   and other CI/CD platforms currently in use.
> >  
> >   Is there any interest from the ASF to enable this for all
> Gitbox
> >   projects when it becomes fully public?
> >  
> >  >>>
> >  >>> Absolutely. The Infrastructure team would love to see groups
> > try this
> >  >> out,
> >  >>> and share the experiences here.
> >  >>>
> >  >>> If the

Re: Github Actions

[Joan Touzet]

Continuing the top-post trend...

I'd rather see full audit logs kept ~forever for any use of credentials,
including the code that was executed.

If we can't stop the leak, we can at least keep the paper trail.

Right now, with our aggressive build cleanup steps, I don't think this
is happening. Archiving that data somewhere else for legal purposes
might be a good idea.

-Joan "just an idea" Touzet

On 2019-08-28 5:02, sebb wrote:
> I think the pre-verified code could run on a separate system with
> restricted access.
> That's how self-service works for creating mailing lists, for example.
> 
> In this case, there would need to be a separate host with read access
> to Jenkins.
> It could accept publish requests from Jenkins, and route them accordingly.
> 
> This would require a bit of effort to set up, but could be used for
> multiple projects.
> 
> On Wed, 28 Aug 2019 at 03:26, Greg Stein  wrote:
>>
>> Yeah. FIgured as much, hoped that I was missing something :)
>>
>> (note: we have the same issue with buildbot and jenkins: we simply trust
>> the communities to not exfil that data)
>>
>> On Tue, Aug 27, 2019 at 9:16 PM Matt Sicker  wrote:
>>
>>> How to avoid leaking secrets: only way to do that is via pre-verified code
>>> that executes something with that secret. Otherwise, there’s literally
>>> infinite ways to leak it being a Turing machine and all. This applies to
>>> all CICD tools.
>>>
>>> On Tue, Aug 27, 2019 at 20:32, Greg Stein  wrote:
>>>
 Hi Francis,

 Is the token needed to push from calcite to calcite-site? Is that an
>>> oauth
 token or something? And are you able to use the repository settings to
>>> add
 secrets, but you don't have the right token? Or you cannot add secrets at
 all? (I can't tell since I have superpowers)

 I've added GSTEIN_TEST_SECRET to Calcite. See if you can extract/print
>>> that
 into your build/action log. If so, then we can try to figure out the
 security here (ie. how do we avoid Actions exfiltrating the token?)

 Thanks,
 -g

 On Tue, Aug 27, 2019 at 5:19 AM Francis Chuang >>>
 wrote:

> I have implemented the ability to generate the website and javadoc for
> Calcite using Github Actions. See:
> https://github.com/apache/calcite/tree/test-site/.github/workflows
>
> The missing piece is that we need the token to publish to our
> calcite-site repository to be added as a secret in Github Actions and
> there is currently no clear process as to whether this is allowed or
>>> how
> to get this done.
>
> See:
> https://issues.apache.org/jira/browse/INFRA-18874
> https://issues.apache.org/jira/browse/INFRA-18875
>
> Francis
>
> On 27/08/2019 7:52 pm, Greg Stein wrote:
>> Have you had an opportunity to make progress on this, to share with
>>> us?
>>
>> Anybody else with news?
>>
>> Thanks!
>> -g
>> InfraAdmin, ASF
>>
>>
>> On Tue, Aug 13, 2019 at 3:59 PM Karl Heinz Marbaise <
>>> khmarba...@gmx.de
>
>> wrote:
>>
>>> Hi,
>>>
>>> I've made a simple PoC for the Apache Maven Dependency Plugin on a
>>> separate branch.
>>>
>>> I will try within the next days more features for example Mac OS
 builds
>>> etc.
>>>
>>>
>>> Currently I simply push my changes via gitbox ..
>>>
>>> maven-dependency-plugin (GITHUB_ACTIONS)$ git remote -v
>>> origin
 https://gitbox.apache.org/repos/asf/maven-dependency-plugin.git
>>> (fetch)
>>> origin
 https://gitbox.apache.org/repos/asf/maven-dependency-plugin.git
>>> (push)
>>>
>>>
>>> Also I'm interested to use SonarCloud related with GitHub Actions..?
>>>
>>>
>>> Kind regards
>>> Karl Heinz Marbaise
>>> Apache Maven PMC
>>>
>>> [1]:
>>> https://github.com/apache/maven-dependency-plugin/runs/192633340
>>> [2]:
>>>
>>>
>

>>> https://github.com/apache/maven-dependency-plugin/blob/66435b225e7885f44b25207e025469f6d5237107/.github/workflows/maven.yml
>>>
>>> On 12.08.19 00:31, Greg Stein wrote:
 On Sun, Aug 11, 2019 at 5:15 PM Francis Chuang <
> francischu...@apache.org

 wrote:
> ...

> I think there are quite a few ASF projects using gitbox and Github
 and
> this would be a very good complement or replacement for Travis,
> appvoyer
> and other CI/CD platforms currently in use.
>
> Is there any interest from the ASF to enable this for all Gitbox
> projects when it becomes fully public?
>

 Absolutely. The Infrastructure team would love to see groups try
>>> this
>>> out,
 and share the experiences here.

 If there are any hurdles, then share them and we'll try to knock
>>> them
>>> down.

 I am also interested in being able to push to our website
 automati