[Cloud] Re: WMCS team work transparency (was: New servers don't show up on grafana-labs.wikimedia.org)

2022-10-07 Thread Taavi Väänänen 

On 10/6/22 23:16, Andrew Bogott wrote:
I'm definitely interested in talking and thinking about this more. I 
think it is true that the cloud services staff have started coordinating 
more frequently in video calls, so your comment is a useful reminder 
that we need to redouble our efforts on post-call documentation.


I'll start with the disclaimer that I'm very much involved the 
infrastructure side of WMCS. Others in a different position, for example 
those using our "products" may have different views, and I'd be curious 
to hear them.


I'd also like to make it clear that I'm not angry about a single 
decision or person. Most of this has been in my mind for a while now, 
and Lucas wondering about the current status of the grid engine made me 
realize that I should probably voice these concerns so that we can do 
something about it. I'm happy to see that others care about these points 
too.


Are there other topics, decisions, or work areas that have recently 
vanished behind the curtain? And, if so, do you have thoughts about how 
we can be better?


I feel like that for quite a few projects, the actual technical work is 
tracked publicly in Phabricator, but the planning and roadmapping 
process is happening behind the scenes. The grid engine deprecation and 
build pack stuff are both good examples of this.


The work on Magnum (k8s-as-a-service) also falls in this category I 
think. I know that there's work going on to make Magnum available to 
Cloud VPS users, but I don't know if that's intended to be used by 
non-WMCS managed projects or if there are plans to move PAWS or 
Toolforge to use it. (I'm initially very skeptical to moving Toolforge 
off the current kubeadm setup for various reasons, which I'm happy to 
talk about separately.)


I'm also going to use this opportunity to note that there is WMCS work 
going on which isn't problematic in this sense. For example, the very 
recent work to replace the cloudnet hardware is very easy to follow. For 
example comments like https://phabricator.wikimedia.org/T319300#8285959 
are very helpful.


As a team we definitely aspire to do essentially all of our work in 
public view, but lately I've been struggling a bit with what exactly 
that should mean. Communication channels proliferate and everyone seems 
to only get a 30% view of what's happening depending on which feeds they 
follow. A good example is Arturo's blog posts about Toolforge futures[0] 
which are quite effective as /potential/ communication but may not have 
actually reached the eyes most in need of an update.


[0] https://techblog.wikimedia.org/2022/03/14/toolforge-and-grid-engine/


This is a good question. I'm *not* advocating for a model where there 
are no private meetings or Slack groups or stuff like that. I would like 
to be aware that meetings are happening so that I can provide context on 
matters I'm familiar with or voice my opinions when I have those for 
certain approaches. I also would like to be aware of major decisions, 
especially if they affect projects I'm working on.


As a final note, I've been referring to the #wikimedia-cloud-admin IRC 
channel and the cloud-admin mailing list as public venues. While 
technically true (the IRC channel and the mailing list archives are 
public), I don't think those are mentioned anywhere on Wikitech and 
cloud-admin subscription is moderated for non-staff (and I have a vague 
memory of my subscription being rejected before I had Toolforge admin 
access). I think there's some work to be done here to make it easier for 
people to get involved.


Taavi



OpenPGP_0xEF242F709F912FBE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature
___
Cloud mailing list -- cloud@lists.wikimedia.org
List information: 
https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/

[Cloud] Re: WMCS team work transparency (was: New servers don't show up on grafana-labs.wikimedia.org)

2022-10-07 Thread Vivian Rook 
> The work on Magnum (k8s-as-a-service) also falls in this category I
> think. I know that there's work going on to make Magnum available to
> Cloud VPS users, but I don't know if that's intended to be used by
> non-WMCS managed projects or if there are plans to move PAWS or
> Toolforge to use it. (I'm initially very skeptical to moving Toolforge
> off the current kubeadm setup for various reasons, which I'm happy to
> talk about separately.)

In terms of PAWS, and Quarry, there are plans to move them.
https://phabricator.wikimedia.org/T308873
https://phabricator.wikimedia.org/T301469
have some notes, and associated branches for both. I am aware of no plans
of moving toolforge, thus, assume there are no tickets or similar
discussing it. In terms of the above plans for Quarry and PAWS, how can it
be better communicated what is happening with them?



On Fri, Oct 7, 2022 at 3:14 AM Taavi Väänänen  wrote:

> On 10/6/22 23:16, Andrew Bogott wrote:
> > I'm definitely interested in talking and thinking about this more. I
> > think it is true that the cloud services staff have started coordinating
> > more frequently in video calls, so your comment is a useful reminder
> > that we need to redouble our efforts on post-call documentation.
>
> I'll start with the disclaimer that I'm very much involved the
> infrastructure side of WMCS. Others in a different position, for example
> those using our "products" may have different views, and I'd be curious
> to hear them.
>
> I'd also like to make it clear that I'm not angry about a single
> decision or person. Most of this has been in my mind for a while now,
> and Lucas wondering about the current status of the grid engine made me
> realize that I should probably voice these concerns so that we can do
> something about it. I'm happy to see that others care about these points
> too.
>
> > Are there other topics, decisions, or work areas that have recently
> > vanished behind the curtain? And, if so, do you have thoughts about how
> > we can be better?
>
> I feel like that for quite a few projects, the actual technical work is
> tracked publicly in Phabricator, but the planning and roadmapping
> process is happening behind the scenes. The grid engine deprecation and
> build pack stuff are both good examples of this.
>
> The work on Magnum (k8s-as-a-service) also falls in this category I
> think. I know that there's work going on to make Magnum available to
> Cloud VPS users, but I don't know if that's intended to be used by
> non-WMCS managed projects or if there are plans to move PAWS or
> Toolforge to use it. (I'm initially very skeptical to moving Toolforge
> off the current kubeadm setup for various reasons, which I'm happy to
> talk about separately.)
>
> I'm also going to use this opportunity to note that there is WMCS work
> going on which isn't problematic in this sense. For example, the very
> recent work to replace the cloudnet hardware is very easy to follow. For
> example comments like https://phabricator.wikimedia.org/T319300#8285959
> are very helpful.
>
> > As a team we definitely aspire to do essentially all of our work in
> > public view, but lately I've been struggling a bit with what exactly
> > that should mean. Communication channels proliferate and everyone seems
> > to only get a 30% view of what's happening depending on which feeds they
> > follow. A good example is Arturo's blog posts about Toolforge futures[0]
> > which are quite effective as /potential/ communication but may not have
> > actually reached the eyes most in need of an update.
> >
> > [0] https://techblog.wikimedia.org/2022/03/14/toolforge-and-grid-engine/
>
> This is a good question. I'm *not* advocating for a model where there
> are no private meetings or Slack groups or stuff like that. I would like
> to be aware that meetings are happening so that I can provide context on
> matters I'm familiar with or voice my opinions when I have those for
> certain approaches. I also would like to be aware of major decisions,
> especially if they affect projects I'm working on.
>
> As a final note, I've been referring to the #wikimedia-cloud-admin IRC
> channel and the cloud-admin mailing list as public venues. While
> technically true (the IRC channel and the mailing list archives are
> public), I don't think those are mentioned anywhere on Wikitech and
> cloud-admin subscription is moderated for non-staff (and I have a vague
> memory of my subscription being rejected before I had Toolforge admin
> access). I think there's some work to be done here to make it easier for
> people to get involved.
>
> Taavi
>
> ___
> Cloud mailing list -- cloud@lists.wikimedia.org
> List information:
> https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/
>


-- 

*Vivian Rook (They/Them)*
Site Reliability Engineer
Wikimedia Foundation 
___
Cloud mailing list -- cloud@list

[Cloud] Re: WMCS team work transparency (was: New servers don't show up on grafana-labs.wikimedia.org)

2022-10-07 Thread Bryan Davis 
On Fri, Oct 7, 2022 at 1:14 AM Taavi Väänänen  wrote:
>
> As a final note, I've been referring to the #wikimedia-cloud-admin IRC
> channel and the cloud-admin mailing list as public venues. While
> technically true (the IRC channel and the mailing list archives are
> public), I don't think those are mentioned anywhere on Wikitech and
> cloud-admin subscription is moderated for non-staff (and I have a vague
> memory of my subscription being rejected before I had Toolforge admin
> access). I think there's some work to be done here to make it easier for
> people to get involved.

The cloud-ad...@lists.wikimedia.org mailing list has always had public
archives (which are much more useful now that we have mailman3) but
closed membership. The intent of this list was never to be a forum for
community discussion, but we did consciously choose to make the
archives open so that interested parties could see the discussions
that were held here.

Bryan
-- 
Bryan Davis  Technical Engagement  Wikimedia Foundation
Principal Software Engineer   Boise, ID USA
[[m:User:BDavis_(WMF)]]  irc: bd808
___
Cloud mailing list -- cloud@lists.wikimedia.org
List information: 
https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/

[Cloud] Re: [Small wiki toolkits] Upcoming "How to interact with Wikidata via Pywikibot" workshop on Friday, October 7th, 16:00 UTC

2022-10-07 Thread Srishti Sethi 
Hello everyone, This is a reminder that the workshop will start in 30
minutes. Thank you!

On Thu, Sep 29, 2022 at 9:07 PM Srishti Sethi  wrote:

> Hello everyone,
>
> The eighth workshop on the topic of "How to interact with Wikidata via
> Pywikibot" is coming up - it will take place on Friday, October 7th at
> 16:00 UTC. You can find more details on the workshop and a link to join
> here: <
> https://meta.wikimedia.org/wiki/Small_wiki_toolkits/Workshops#How_to_interact_with_Wikidata_via_Pywikibot>
> [1].
>
> This workshop will summarise how to access and save information to
> Wikidata, including through page generators and queries*, using Pywikibot.
> It will also cover the Wikidata conventions about the bot editing,
> including the bot approvals process (*queries not included, please bring
> your own!). This session will be run by Mike Peel.
>
> To participate in this workshop, you would need basic familiarity with
> Wikidata and Pywikibot installation.
>
> We look forward to your participation!
>
> Best,
> Srishti
>
> On behalf of the SWT Workshops Organization team
>
> [1]
> https://meta.wikimedia.org/wiki/Small_wiki_toolkits/Workshops#How_to_interact_with_Wikidata_via_Pywikibot
>
> *Srishti Sethi*
> Senior Developer Advocate
> Wikimedia Foundation 
>
-- 
*Srishti Sethi*
Senior Developer Advocate
Wikimedia Foundation 
___
Cloud mailing list -- cloud@lists.wikimedia.org
List information: 
https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/

[Cloud] Re: WMCS team work transparency (was: New servers don't show up on grafana-labs.wikimedia.org)

2022-10-07 Thread Andrew Otto 
FWIW, the WMF Data Engineering team has had some discussions about this in
the past, and we decided that all decisions and real work doc has to be
either in Phabricator or on a Wiki.  We use Slack or IRC to discuss things,
but if we come to a conclusion, we try to  summarize and document the
conclusion in Phab.  I think we do this well with most things, but not as
well as we should with annual and quarterly planning docs (I think these
are in google docs and better works).





On Fri, Oct 7, 2022 at 10:54 AM Bryan Davis  wrote:

> On Fri, Oct 7, 2022 at 1:14 AM Taavi Väänänen  wrote:
> >
> > As a final note, I've been referring to the #wikimedia-cloud-admin IRC
> > channel and the cloud-admin mailing list as public venues. While
> > technically true (the IRC channel and the mailing list archives are
> > public), I don't think those are mentioned anywhere on Wikitech and
> > cloud-admin subscription is moderated for non-staff (and I have a vague
> > memory of my subscription being rejected before I had Toolforge admin
> > access). I think there's some work to be done here to make it easier for
> > people to get involved.
>
> The cloud-ad...@lists.wikimedia.org mailing list has always had public
> archives (which are much more useful now that we have mailman3) but
> closed membership. The intent of this list was never to be a forum for
> community discussion, but we did consciously choose to make the
> archives open so that interested parties could see the discussions
> that were held here.
>
> Bryan
> --
> Bryan Davis  Technical Engagement  Wikimedia Foundation
> Principal Software Engineer   Boise, ID USA
> [[m:User:BDavis_(WMF)]]  irc: bd808
> ___
> Cloud mailing list -- cloud@lists.wikimedia.org
> List information:
> https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/
>
___
Cloud mailing list -- cloud@lists.wikimedia.org
List information: 
https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/

[Cloud] Re: WMCS team work transparency (was: New servers don't show up on grafana-labs.wikimedia.org)

2022-10-07 Thread Amir Sarabadani 
Andrew,
I think one general issue is the lack of transparency (and community
involvement) in the decision making discussions. Not the documentation of
the decisions once done. That's good to have but if we have transparency as
a guiding principle, we should include that in the discussions too.

My 2c

Andrew Otto  schrieb am Fr., 7. Okt. 2022, 18:40:

> FWIW, the WMF Data Engineering team has had some discussions about this in
> the past, and we decided that all decisions and real work doc has to be
> either in Phabricator or on a Wiki.  We use Slack or IRC to discuss things,
> but if we come to a conclusion, we try to  summarize and document the
> conclusion in Phab.  I think we do this well with most things, but not as
> well as we should with annual and quarterly planning docs (I think these
> are in google docs and better works).
>
>
>
>
>
> On Fri, Oct 7, 2022 at 10:54 AM Bryan Davis  wrote:
>
>> On Fri, Oct 7, 2022 at 1:14 AM Taavi Väänänen  wrote:
>> >
>> > As a final note, I've been referring to the #wikimedia-cloud-admin IRC
>> > channel and the cloud-admin mailing list as public venues. While
>> > technically true (the IRC channel and the mailing list archives are
>> > public), I don't think those are mentioned anywhere on Wikitech and
>> > cloud-admin subscription is moderated for non-staff (and I have a vague
>> > memory of my subscription being rejected before I had Toolforge admin
>> > access). I think there's some work to be done here to make it easier for
>> > people to get involved.
>>
>> The cloud-ad...@lists.wikimedia.org mailing list has always had public
>> archives (which are much more useful now that we have mailman3) but
>> closed membership. The intent of this list was never to be a forum for
>> community discussion, but we did consciously choose to make the
>> archives open so that interested parties could see the discussions
>> that were held here.
>>
>> Bryan
>> --
>> Bryan Davis  Technical Engagement  Wikimedia Foundation
>> Principal Software Engineer   Boise, ID USA
>> [[m:User:BDavis_(WMF)]]  irc: bd808
>> ___
>> Cloud mailing list -- cloud@lists.wikimedia.org
>> List information:
>> https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/
>>
> ___
> Cloud mailing list -- cloud@lists.wikimedia.org
> List information:
> https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/
>
___
Cloud mailing list -- cloud@lists.wikimedia.org
List information: 
https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/

[Cloud] Re: WMCS team work transparency (was: New servers don't show up on grafana-labs.wikimedia.org)

2022-10-07 Thread Andrew Otto 
+1

On Fri, Oct 7, 2022 at 2:04 PM Amir Sarabadani  wrote:

> Andrew,
> I think one general issue is the lack of transparency (and community
> involvement) in the decision making discussions. Not the documentation of
> the decisions once done. That's good to have but if we have transparency as
> a guiding principle, we should include that in the discussions too.
>
> My 2c
>
> Andrew Otto  schrieb am Fr., 7. Okt. 2022, 18:40:
>
>> FWIW, the WMF Data Engineering team has had some discussions about this
>> in the past, and we decided that all decisions and real work doc has to be
>> either in Phabricator or on a Wiki.  We use Slack or IRC to discuss things,
>> but if we come to a conclusion, we try to  summarize and document the
>> conclusion in Phab.  I think we do this well with most things, but not as
>> well as we should with annual and quarterly planning docs (I think these
>> are in google docs and better works).
>>
>>
>>
>>
>>
>> On Fri, Oct 7, 2022 at 10:54 AM Bryan Davis  wrote:
>>
>>> On Fri, Oct 7, 2022 at 1:14 AM Taavi Väänänen  wrote:
>>> >
>>> > As a final note, I've been referring to the #wikimedia-cloud-admin IRC
>>> > channel and the cloud-admin mailing list as public venues. While
>>> > technically true (the IRC channel and the mailing list archives are
>>> > public), I don't think those are mentioned anywhere on Wikitech and
>>> > cloud-admin subscription is moderated for non-staff (and I have a vague
>>> > memory of my subscription being rejected before I had Toolforge admin
>>> > access). I think there's some work to be done here to make it easier
>>> for
>>> > people to get involved.
>>>
>>> The cloud-ad...@lists.wikimedia.org mailing list has always had public
>>> archives (which are much more useful now that we have mailman3) but
>>> closed membership. The intent of this list was never to be a forum for
>>> community discussion, but we did consciously choose to make the
>>> archives open so that interested parties could see the discussions
>>> that were held here.
>>>
>>> Bryan
>>> --
>>> Bryan Davis  Technical Engagement  Wikimedia Foundation
>>> Principal Software Engineer   Boise, ID USA
>>> [[m:User:BDavis_(WMF)]]  irc: bd808
>>> ___
>>> Cloud mailing list -- cloud@lists.wikimedia.org
>>> List information:
>>> https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/
>>>
>> ___
>> Cloud mailing list -- cloud@lists.wikimedia.org
>> List information:
>> https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/
>>
> ___
> Cloud mailing list -- cloud@lists.wikimedia.org
> List information:
> https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/
>
___
Cloud mailing list -- cloud@lists.wikimedia.org
List information: 
https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/