date:20240624

[Cloud] Re: Tunnel to Trove instance?

2024-06-24 Thread David Caro


If you have a small snippet of code doing the connection that has the issue
that can be looked at it would help a lot debugging.

Something that comes to mind might be that it's trying to use a unix socket (as
it's connecting to `localhost`), but that might depend on your configuration
and the options passed to the connection.

Connectivity-wise, as you checked with the mysql cli there should be no
difference :/


On 06/21 12:08, Travis Briggs wrote:
> I also have just now tried the command line mysql client through the tunnel
> and things seem to work, so maybe it's some Python setting that's causing
> the connection to drop?
> 
> -Travis
> 
> On Fri, Jun 21, 2024 at 12:06 PM Travis Briggs  wrote:
> 
> > When I try tunnelling through bastion.wmcloud.org or
> > primary.bastion.wmcloud.org I just get:
> >
> > pymysql.err.OperationalError: (2003, "Can't connect to MySQL server on
> > 'localhost' ([Errno 61] Connection refused)")
> >
> > I can resolve and connect to the trove instance from toolforge, and as I
> > mentioned in my original email, I tried connecting through
> > mwcurator.mwoffliner.eqiad1.wikimedia.cloud which is our VPS box.
> >
> > Thanks,
> > -Travis
> >
> > On Fri, Jun 21, 2024 at 10:21 AM Daniel Zahn  wrote:
> >
> >> I noticed you say it's a VPS as in Cloud VPS project but you attempt to
> >> tunnel via login.toolforge.org.
> >>
> >> Maybe try tunneling via bastion.wmcloud.org or
> >> primary.bastion.wmcloud.org
> >>
> >> as the bastion hosts for the cloud VPS projects.
> >>
> >>
> >> https://wikitech.wikimedia.org/w/index.php?go=Go&search=bastion.wmcloud.org&title=Special:Search
> >>
> >> On Fri, Jun 21, 2024 at 9:36 AM Travis Briggs 
> >> wrote:
> >>
> >>> Hello,
> >>>
> >>> I'm trying to tunnel to my VPS Trove instance, so that I can test some
> >>> code against the production database.
> >>>
> >>> I'm using the following:
> >>>
> >>> ssh -N -L 4177:tdlqt33y3nt.svc.trove.eqiad1.wikimedia.cloud:3306
> >>> login.toolforge.org
> >>>
> >>> I've also tried:
> >>>
> >>> ssh -N -L 4177:tdlqt33y3nt.svc.trove.eqiad1.wikimedia.cloud:3306
> >>> mwcurator.mwoffliner.eqiad1.wikimedia.cloud
> >>>
> >>> The initial connection seems to work, but in my application I'm getting
> >>> dropped connections. I see the following in the terminal where I set up 
> >>> the
> >>> tunnel:
> >>>
> >>> channel 2: open failed: connect failed: Connection refused
> >>> channel 2: open failed: connect failed: Connection refused
> >>>
> >>> I imagine this isn't a fully supported workflow but I'm wondering if
> >>> there's some way to get it to work?
> >>>
> >>> Thanks,
> >>> -Travis
> >>> ___
> >>> Cloud mailing list -- cloud@lists.wikimedia.org
> >>> List information:
> >>> https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/
> >>>
> >>
> >>
> >> --
> >> Daniel Zahn 
> >> Site Reliability Engineer
> >> ___
> >> Cloud mailing list -- cloud@lists.wikimedia.org
> >> List information:
> >> https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/
> >>
> >

> ___
> Cloud mailing list -- cloud@lists.wikimedia.org
> List information: 
> https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/


-- 
David Caro
SRE - Cloud Services
Wikimedia Foundation 
PGP Signature: 7180 83A2 AC8B 314F B4CE  1171 4071 C7E1 D262 69C3

"Imagine a world in which every single human being can freely share in the
sum of all knowledge. That's our commitment."


signature.asc
Description: PGP signature
___
Cloud mailing list -- cloud@lists.wikimedia.org
List information: 
https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/

[Cloud] Re: Tunnel to Trove instance?

2024-06-24 Thread Francesco Negri

This could be the same issue as T367464 [0], due to Trove databases
having a default wait_timeout=120, which means that idle connections
get terminated after 2 minutes. Interactive connections (the ones
created when using the "mysql" CLI) have a different
`interactive_timeout` so are not affected.

You can customize the value of wait_timeout for your Trove instance by
creating a "configuration group". [1]

HTH,
Francesco

[0] https://phabricator.wikimedia.org/T367464
[1] 
https://wikitech.wikimedia.org/wiki/Help:Trove_database_user_guide#Modify_database_configuration

On Mon, Jun 24, 2024 at 10:47 AM David Caro  wrote:
>
>
> If you have a small snippet of code doing the connection that has the issue
> that can be looked at it would help a lot debugging.
>
> Something that comes to mind might be that it's trying to use a unix socket 
> (as
> it's connecting to `localhost`), but that might depend on your configuration
> and the options passed to the connection.
>
> Connectivity-wise, as you checked with the mysql cli there should be no
> difference :/
>
>
> On 06/21 12:08, Travis Briggs wrote:
> > I also have just now tried the command line mysql client through the tunnel
> > and things seem to work, so maybe it's some Python setting that's causing
> > the connection to drop?
> >
> > -Travis
> >
> > On Fri, Jun 21, 2024 at 12:06 PM Travis Briggs  wrote:
> >
> > > When I try tunnelling through bastion.wmcloud.org or
> > > primary.bastion.wmcloud.org I just get:
> > >
> > > pymysql.err.OperationalError: (2003, "Can't connect to MySQL server on
> > > 'localhost' ([Errno 61] Connection refused)")
> > >
> > > I can resolve and connect to the trove instance from toolforge, and as I
> > > mentioned in my original email, I tried connecting through
> > > mwcurator.mwoffliner.eqiad1.wikimedia.cloud which is our VPS box.
> > >
> > > Thanks,
> > > -Travis
> > >
> > > On Fri, Jun 21, 2024 at 10:21 AM Daniel Zahn  wrote:
> > >
> > >> I noticed you say it's a VPS as in Cloud VPS project but you attempt to
> > >> tunnel via login.toolforge.org.
> > >>
> > >> Maybe try tunneling via bastion.wmcloud.org or
> > >> primary.bastion.wmcloud.org
> > >>
> > >> as the bastion hosts for the cloud VPS projects.
> > >>
> > >>
> > >> https://wikitech.wikimedia.org/w/index.php?go=Go&search=bastion.wmcloud.org&title=Special:Search
> > >>
> > >> On Fri, Jun 21, 2024 at 9:36 AM Travis Briggs 
> > >> wrote:
> > >>
> > >>> Hello,
> > >>>
> > >>> I'm trying to tunnel to my VPS Trove instance, so that I can test some
> > >>> code against the production database.
> > >>>
> > >>> I'm using the following:
> > >>>
> > >>> ssh -N -L 4177:tdlqt33y3nt.svc.trove.eqiad1.wikimedia.cloud:3306
> > >>> login.toolforge.org
> > >>>
> > >>> I've also tried:
> > >>>
> > >>> ssh -N -L 4177:tdlqt33y3nt.svc.trove.eqiad1.wikimedia.cloud:3306
> > >>> mwcurator.mwoffliner.eqiad1.wikimedia.cloud
> > >>>
> > >>> The initial connection seems to work, but in my application I'm getting
> > >>> dropped connections. I see the following in the terminal where I set up 
> > >>> the
> > >>> tunnel:
> > >>>
> > >>> channel 2: open failed: connect failed: Connection refused
> > >>> channel 2: open failed: connect failed: Connection refused
> > >>>
> > >>> I imagine this isn't a fully supported workflow but I'm wondering if
> > >>> there's some way to get it to work?
> > >>>
> > >>> Thanks,
> > >>> -Travis
> > >>> ___
> > >>> Cloud mailing list -- cloud@lists.wikimedia.org
> > >>> List information:
> > >>> https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/
> > >>>
> > >>
> > >>
> > >> --
> > >> Daniel Zahn 
> > >> Site Reliability Engineer
> > >> ___
> > >> Cloud mailing list -- cloud@lists.wikimedia.org
> > >> List information:
> > >> https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/
> > >>
> > >
>
> > ___
> > Cloud mailing list -- cloud@lists.wikimedia.org
> > List information: 
> > https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/
>
>
> --
> David Caro
> SRE - Cloud Services
> Wikimedia Foundation 
> PGP Signature: 7180 83A2 AC8B 314F B4CE  1171 4071 C7E1 D262 69C3
>
> "Imagine a world in which every single human being can freely share in the
> sum of all knowledge. That's our commitment."
> ___
> Cloud mailing list -- cloud@lists.wikimedia.org
> List information: 
> https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/



-- 
Francesco Negri (he/him) -- IRC: dhinus
Site Reliability Engineer, Cloud Services team
Wikimedia Foundation
___
Cloud mailing list -- cloud@lists.wikimedia.org
List information: 
https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/

[Cloud] Changes to the GitLab migration

2024-06-24 Thread Tyler Cipriani

We’re going to keep both Gerrit and GitLab.

Many of the repositories now on Gerrit require features that are lacking in
GitLab:

   - Cross-repository dependent merge requests
   - Project gating and deterministic, atomic merges
   - Stacked patchsets
   - Multiple reviewers

Likewise, GitLab has features that our developers have come to depend on:

   - Familiar fork, branch, and merge workflow
   - Self-service repository creation
   - Jupyter Notebook rendering
   - GitLab's self-service CI/CD pipeline
   - Bring your own runners
   - Artifact and packages registries

As a result, we'll be running both systems for at least the next two years.

More information is available on MediaWiki.org[0], summarized below.
___

*Details*

These repositories have requirements that mean they must remain on Gerrit:

   - MediaWiki core
   - The subset of extensions and skins which track MediaWiki core’s
   mainline branch (including all Wikimedia production-deployed extensions,
   skins, and MediaWiki vendor)
   - SRE’s Puppet repository along with dependencies and dependent
   repositories.

Other repositories on GitLab may return to Gerrit to lessen the mental
burden of working with two systems.

Likewise, some repositories on Gerrit may wish to continue to migrate to
GitLab.

We'll use the Phabricator tag "GitLab (Project Migration)"[1] to track
requests from project stewards to migrate between systems. We’ll be
monitoring that workboard closely through the end of this calendar year to
assist developers with their migrations.
___


*What now*

   - We have no intention of shutting down either system for the next two
   years.
   - We've posted a longer explanation on MediaWiki.org[0]. Please use the
   talk page there (rather than this mailing list) for discussion.
   - We've gathered a list of questions we anticipate some folks may have
   on MediaWiki.org[2].
   - Specific requests from project stewards to migrate repositories
   between either of our systems should use the Phabricator tag "GitLab
   (Project Migration)"[1].
   - We'll be hosting office hours to answer any questions. More details
   about these sessions will come later.

The decision to keep both systems was challenging. Having two code forges
adds to the fragmentation of our systems, the mental overhead for our
developers, and the maintenance burden of stewards. But each system is
well-suited to a subset of our needs, keeping both systems safeguards the
productivity of our developers and the stability of our systems.

For more details, I encourage you to review MediaWiki.org[0] and engage on
the talk page.

Tyler Cipriani (he/him)
Engineering Manager, Release Engineering
Wikimedia Foundation

[0]: 
[1]: 
[2]: 
___
Cloud mailing list -- cloud@lists.wikimedia.org
List information: 
https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/

[Cloud] Re: Tunnel to Trove instance?

2024-06-24 Thread Travis Briggs

Hi Francesco,

That actually sounds very plausible! I think I'm creating the tunnel, then
not starting the script for a few minutes. Or there could be several
minutes of idle time as the script is running.

I've done configuration groups before, so I'll try that and report back.

Thanks!
-Travis

On Mon, Jun 24, 2024 at 6:11 AM Francesco Negri 
wrote:

> This could be the same issue as T367464 [0], due to Trove databases
> having a default wait_timeout=120, which means that idle connections
> get terminated after 2 minutes. Interactive connections (the ones
> created when using the "mysql" CLI) have a different
> `interactive_timeout` so are not affected.
>
> You can customize the value of wait_timeout for your Trove instance by
> creating a "configuration group". [1]
>
> HTH,
> Francesco
>
> [0] https://phabricator.wikimedia.org/T367464
> [1]
> https://wikitech.wikimedia.org/wiki/Help:Trove_database_user_guide#Modify_database_configuration
>
> On Mon, Jun 24, 2024 at 10:47 AM David Caro  wrote:
> >
> >
> > If you have a small snippet of code doing the connection that has the
> issue
> > that can be looked at it would help a lot debugging.
> >
> > Something that comes to mind might be that it's trying to use a unix
> socket (as
> > it's connecting to `localhost`), but that might depend on your
> configuration
> > and the options passed to the connection.
> >
> > Connectivity-wise, as you checked with the mysql cli there should be no
> > difference :/
> >
> >
> > On 06/21 12:08, Travis Briggs wrote:
> > > I also have just now tried the command line mysql client through the
> tunnel
> > > and things seem to work, so maybe it's some Python setting that's
> causing
> > > the connection to drop?
> > >
> > > -Travis
> > >
> > > On Fri, Jun 21, 2024 at 12:06 PM Travis Briggs 
> wrote:
> > >
> > > > When I try tunnelling through bastion.wmcloud.org or
> > > > primary.bastion.wmcloud.org I just get:
> > > >
> > > > pymysql.err.OperationalError: (2003, "Can't connect to MySQL server
> on
> > > > 'localhost' ([Errno 61] Connection refused)")
> > > >
> > > > I can resolve and connect to the trove instance from toolforge, and
> as I
> > > > mentioned in my original email, I tried connecting through
> > > > mwcurator.mwoffliner.eqiad1.wikimedia.cloud which is our VPS box.
> > > >
> > > > Thanks,
> > > > -Travis
> > > >
> > > > On Fri, Jun 21, 2024 at 10:21 AM Daniel Zahn 
> wrote:
> > > >
> > > >> I noticed you say it's a VPS as in Cloud VPS project but you
> attempt to
> > > >> tunnel via login.toolforge.org.
> > > >>
> > > >> Maybe try tunneling via bastion.wmcloud.org or
> > > >> primary.bastion.wmcloud.org
> > > >>
> > > >> as the bastion hosts for the cloud VPS projects.
> > > >>
> > > >>
> > > >>
> https://wikitech.wikimedia.org/w/index.php?go=Go&search=bastion.wmcloud.org&title=Special:Search
> > > >>
> > > >> On Fri, Jun 21, 2024 at 9:36 AM Travis Briggs 
> > > >> wrote:
> > > >>
> > > >>> Hello,
> > > >>>
> > > >>> I'm trying to tunnel to my VPS Trove instance, so that I can test
> some
> > > >>> code against the production database.
> > > >>>
> > > >>> I'm using the following:
> > > >>>
> > > >>> ssh -N -L 4177:tdlqt33y3nt.svc.trove.eqiad1.wikimedia.cloud:3306
> > > >>> login.toolforge.org
> > > >>>
> > > >>> I've also tried:
> > > >>>
> > > >>> ssh -N -L 4177:tdlqt33y3nt.svc.trove.eqiad1.wikimedia.cloud:3306
> > > >>> mwcurator.mwoffliner.eqiad1.wikimedia.cloud
> > > >>>
> > > >>> The initial connection seems to work, but in my application I'm
> getting
> > > >>> dropped connections. I see the following in the terminal where I
> set up the
> > > >>> tunnel:
> > > >>>
> > > >>> channel 2: open failed: connect failed: Connection refused
> > > >>> channel 2: open failed: connect failed: Connection refused
> > > >>>
> > > >>> I imagine this isn't a fully supported workflow but I'm wondering
> if
> > > >>> there's some way to get it to work?
> > > >>>
> > > >>> Thanks,
> > > >>> -Travis
> > > >>> ___
> > > >>> Cloud mailing list -- cloud@lists.wikimedia.org
> > > >>> List information:
> > > >>>
> https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/
> > > >>>
> > > >>
> > > >>
> > > >> --
> > > >> Daniel Zahn 
> > > >> Site Reliability Engineer
> > > >> ___
> > > >> Cloud mailing list -- cloud@lists.wikimedia.org
> > > >> List information:
> > > >>
> https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/
> > > >>
> > > >
> >
> > > ___
> > > Cloud mailing list -- cloud@lists.wikimedia.org
> > > List information:
> https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/
> >
> >
> > --
> > David Caro
> > SRE - Cloud Services
> > Wikimedia Foundation 
> > PGP Signature: 7180 83A2 AC8B 314F B4CE  1171 4071 C7E1 D262 69C3
> >
> > "Imagine a world in which every single human being can freely share in
> the
> >

[Cloud] [Cloud-announce] Buster VM Deprecation Notice

2024-06-24 Thread Seyram Komla Sapaty

Hello!

You may have been notified about Buster VM deprecation on projects where
you are not an admin.
This was in error.
Kindly ignore.

Thanks!
-- 
Seyram Komla Sapaty
Developer Advocate
Wikimedia Cloud Services
___
Cloud-announce mailing list -- cloud-annou...@lists.wikimedia.org
List information: 
https://lists.wikimedia.org/postorius/lists/cloud-announce.lists.wikimedia.org/
___
Cloud mailing list -- cloud@lists.wikimedia.org
List information: 
https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/

[Cloud] Re: Tunnel to Trove instance?

[Cloud] Re: Tunnel to Trove instance?

[Cloud] Changes to the GitLab migration

[Cloud] Re: Tunnel to Trove instance?

[Cloud] [Cloud-announce] Buster VM Deprecation Notice

5 matches

Site Navigation

Mail list logo

Footer information