Re: Sysadmin Load Reduction: Subversion Infrastructure
Ben Cooksley ha scritto: > Hi all, > > When KDE committed to performing a migration to Git back in 2010, one > of the things that was agreed at the time was that translators could > remain on Subversion to avoid disrupting their workflows. > > This however has led to a certain amount of additional infrastructure > which Sysadmin needs to continue to maintain. > > In recognition of the fact that with few exceptions, everything has > now migrated from Subversion aside from Translations, i'd like to > reduce the level of infrastructure supporting our Subversion > repository to the bare minimum necessary. > > This would include the shutdown of WebSVN in particular, which when > coupled with the shutdown of our two CGit instances would also allow > for us to eliminate an entire virtual machine from our systems. As websvn has proven to be useful, would it help the sysadmin (pending agreement with aacid and the other people with root access on rosetta) to move websvn to rosetta and under localization team's responsibility? Ciao -- Luigi
Re: Sysadmin Load Reduction: Code Related Services
Ben Cooksley ha scritto: > Hi all, > > In the category of code related services, Sysadmin currently supports > a wide-ranging number of services (which makes sense given the nature > of the community). Some of these however may no longer be in use or > will be duplicative of other services following the transition to > Gitlab. > > In the category of duplicative, we have our two CGit instances at > cgit.kde.org and packaging.neon.kde.org. Prior to Gitlab, these were > justifiable as there was no other way of browsing scratch/clone > repositories over the web. > > With the migration to Gitlab however all repositories will become > browsable through it, meaning it no longer makes sense to offer two > browsers that provide the exact same information (with Gitlab having > greater capabilities). I'd therefore like to shut both of these down > once Code Hosting has transitioned to Gitlab. Luckily we tried to use commits.kde.org as generic redirect. Will the generic redirect commits.kde.org be updated to point to the proper repository? It may be complicated if the new structure contains the namespaces for each repository, but we need to keep it working. > > In the category of no longer in use, we have the compatibility > generator for the kde_projects.xml file. This was introduced when we > shutdown Redmine/Chiliproject and migrated to Phabricator, as a way of > keeping services that needed to discover a list of KDE Projects > functional. > > As we've since migrated to using YAML files within the > sysadmin/repo-metadata repository for both the CI System and > kdesrc-build (and with LXR using kdesrc-build to do it's code > checkouts) there shouldn't to my knowledge be anything still relying > on this (aside from perhaps scripty).> > I'd therefore like to shut this generator down as well, along with the > compaibility redirector running at projects.kde.org (given that it has > been some time since we were using that site, and many projects have > moved around in the virtual structure since then, making the redirects > it is able to offer useless) Two points: - scripty still uses the XML file and we may need some time to migrate. - we may have a few links around which still points to projects.kde.org (for example, the "Consistency" goal listed a few of them), so we may need to go through all of them before doing that. Do we have measures that shows how much projects.kde.org is hit by services which are not scripty? -- Luigi
Re: General Infrastructure Maintainability: API and EBN
On 09/11/19 01:33, Ben Cooksley wrote: > Hi all, Hi Ben > > Over the past number of years one of the tasks the Sysadmin team has > worked on has been improving the overall maintainability of our > systems, with a significant number of specialised cronjobs, exceptions > and hidden linkages being eliminated. > > That is with one great exception: api.kde.org and ebn.kde.org. > > Both of these are suffering from an extreme amount of digital bitrot > and special casing and in general are now in a condition where I > cannot say for certain whether it would be possible to replicate the > setup on a new system without us experiencing some degree of breakage > (some of which we may not discover until weeks/months afterwards). > > In addition, the current setup relies on an old-fashioned overnight > reprocessing of all repositories, which is inefficient and resource > expensive. A more modern approach would have the various projects api > documentation generated on a delayed cycle from relevant branches as > part of something like a CI job (but not part of the actual CI > workflow itself). > > For this one, i'm not certain on the best path forward at this stage, > however the current state of affairs cannot continue. We have tried > over the past few years to find people to work on a replacement for > the tooling involved, but alas we've yet to have success here. > > Thoughts anyone? I'd say api.kde.org is too important to let it go. The EBN is less important but still useful, I still see people pushing EBN-based fixes once in a while. Have we ever tried to use a GSoC project to modernize either of them? If not, would it make sense to try next year? > > Regards, > Ben Cheers, Elvis
Re: Sysadmin Load Reduction: Code Related Services
On Sun, Nov 10, 2019 at 12:29 AM Luigi Toscano wrote: > > Ben Cooksley ha scritto: > > Hi all, > > > > In the category of code related services, Sysadmin currently supports > > a wide-ranging number of services (which makes sense given the nature > > of the community). Some of these however may no longer be in use or > > will be duplicative of other services following the transition to > > Gitlab. > > > > In the category of duplicative, we have our two CGit instances at > > cgit.kde.org and packaging.neon.kde.org. Prior to Gitlab, these were > > justifiable as there was no other way of browsing scratch/clone > > repositories over the web. > > > > With the migration to Gitlab however all repositories will become > > browsable through it, meaning it no longer makes sense to offer two > > browsers that provide the exact same information (with Gitlab having > > greater capabilities). I'd therefore like to shut both of these down > > once Code Hosting has transitioned to Gitlab. > > Luckily we tried to use commits.kde.org as generic redirect. Will the generic > redirect commits.kde.org be updated to point to the proper repository? It may > be complicated if the new structure contains the namespaces for each > repository, but we need to keep it working. The commits.kde.org redirector is intended to provide permanent links, so yes it will be updated to redirect people to Gitlab instead once the switch to Gitlab is completed. > > > > > In the category of no longer in use, we have the compatibility > > generator for the kde_projects.xml file. This was introduced when we > > shutdown Redmine/Chiliproject and migrated to Phabricator, as a way of > > keeping services that needed to discover a list of KDE Projects > > functional. > > > > As we've since migrated to using YAML files within the > > sysadmin/repo-metadata repository for both the CI System and > > kdesrc-build (and with LXR using kdesrc-build to do it's code > > checkouts) there shouldn't to my knowledge be anything still relying > > on this (aside from perhaps scripty).> > > I'd therefore like to shut this generator down as well, along with the > > compaibility redirector running at projects.kde.org (given that it has > > been some time since we were using that site, and many projects have > > moved around in the virtual structure since then, making the redirects > > it is able to offer useless) > > Two points: > - scripty still uses the XML file and we may need some time to migrate. I feared this may have been the case. What sort of timeline are we looking at in terms of switching scripty over? > - we may have a few links around which still points to projects.kde.org (for > example, the "Consistency" goal listed a few of them), so we may need to go > through all of them before doing that. > Do we have measures that shows how much projects.kde.org is hit by services > which are not scripty? I've reviewed the logs for the past 24 hours and it would appear the majority of the legitimate hits now are for the RSS/ATOM feeds that Chiliproject used to offer and which have no corresponding replacement (amazingly, these are still configured in people's feed readers). After excluding bots and spiders, and people running automated tools against old URL sets, very little in the way of hits remained. The remainder are a very small proportion of hits from places such as the Archlinux wiki, which we can likely request to be updated (referencing Bluedevil in particular it would seem). > > -- > Luigi Cheers, Ben
Re: Sysadmin Load Reduction: Subversion Infrastructure
On Sun, Nov 10, 2019 at 12:20 AM Luigi Toscano wrote: > > Ben Cooksley ha scritto: > > Hi all, > > > > When KDE committed to performing a migration to Git back in 2010, one > > of the things that was agreed at the time was that translators could > > remain on Subversion to avoid disrupting their workflows. > > > > This however has led to a certain amount of additional infrastructure > > which Sysadmin needs to continue to maintain. > > > > In recognition of the fact that with few exceptions, everything has > > now migrated from Subversion aside from Translations, i'd like to > > reduce the level of infrastructure supporting our Subversion > > repository to the bare minimum necessary. > > > > This would include the shutdown of WebSVN in particular, which when > > coupled with the shutdown of our two CGit instances would also allow > > for us to eliminate an entire virtual machine from our systems. > > As websvn has proven to be useful, would it help the sysadmin (pending > agreement with aacid and the other people with root access on rosetta) to move > websvn to rosetta and under localization team's responsibility? Unfortunately Rosetta does not have the necessary free disk space, as the Subversion repository is approximately 80GB these days in size, and operating WebSVN requires a full copy of the Subversion repository locally in order to operate. > > Ciao > -- > Luigi Cheers, Ben
Re: Looking for Subtitle Composer sponsor
One thing I forgot before we set up the git repo: you will need a developer account. Please apply for one at https://community.kde.org/Infrastructure/Get_a_Developer_Account Nate On 11/7/19 12:58 PM, Nate Graham wrote: On 11/7/19 12:55 PM, Mladen Milinkovic wrote: Hello Nate, On 11/7/19 3:31 PM, Nate Graham wrote: I would be happy to be your sponsor! Thank you! Per https://community.kde.org/Incubator#Candidate, we need you to provide the following information: - A list of the people regularly committing to the project That would be only me I believe. Right now I'm the only one with write permissions to repository. There are occasional pull requests, but nothing regular. Most pull requests are translations which since recently are pulled from crowdin.com. This should be the list of contributors since Nov 2011: https://github.com/maxrd2/SubtitleComposer/graphs/contributors - A plan to be in compliance with the KDE manifesto (https://manifesto.kde.org/index.html). In particular, you'll need to be okay moving your source code repo and other infrastructure away from GitHub and into KDE's GitLab instance at https://invent.kde.org. Have already created account on https://invent.kde.org/users/milinkovic. Should I create a personal repository there for the project? Sysadmin will get one set up for you. Sysadmin, can we have a repo to host https://github.com/maxrd2/SubtitleComposer on KDE's infrastructure? Nate
Re: Sysadmin Load Reduction: Subversion Infrastructure
сб, 9 нояб. 2019 г. в 03:20, Ben Cooksley : > This would include the shutdown of WebSVN in particular, which when > coupled with the shutdown of our two CGit instances would also allow > for us to eliminate an entire virtual machine from our systems. Will there be any web interface for SVN after shutdown of WebSVN? Can we assume https://phabricator.kde.org/source/svn/ remains available during the next 10 years? I'm not sure what use case Luigi has in mind, but among Russian l10n team we often use WebSVN to refer to specific SVN revisions, for example: https://websvn.kde.org/?view=revision&revision=1555342 -- Alexander Potashev
Re: Sysadmin Load Reduction: Subversion Infrastructure
On Sun, Nov 10, 2019 at 7:18 AM Alexander Potashev wrote: > > сб, 9 нояб. 2019 г. в 03:20, Ben Cooksley : > > This would include the shutdown of WebSVN in particular, which when > > coupled with the shutdown of our two CGit instances would also allow > > for us to eliminate an entire virtual machine from our systems. > > Will there be any web interface for SVN after shutdown of WebSVN? > > Can we assume https://phabricator.kde.org/source/svn/ remains > available during the next 10 years? > Phabricator's browser will be retired as part of the shutdown of Phabricator, which will take place once Gitlab has assumed responsibility for code hosting and review, and the tasks have been migrated from Phabricator. Should WebSVN be shutdown as well, then there would be no web interface to our SVN repository. > > I'm not sure what use case Luigi has in mind, but among Russian l10n > team we often use WebSVN to refer to specific SVN revisions, for > example: https://websvn.kde.org/?view=revision&revision=1555342 > > -- > Alexander Potashev Regards, Ben
Re: General Infrastructure Maintainability: API and EBN
On Sun, Nov 10, 2019 at 4:28 AM Elvis Angelaccio wrote: > > > > On 09/11/19 01:33, Ben Cooksley wrote: > > Hi all, > > Hi Ben Hi Elvis, > > > > > Over the past number of years one of the tasks the Sysadmin team has > > worked on has been improving the overall maintainability of our > > systems, with a significant number of specialised cronjobs, exceptions > > and hidden linkages being eliminated. > > > > That is with one great exception: api.kde.org and ebn.kde.org. > > > > Both of these are suffering from an extreme amount of digital bitrot > > and special casing and in general are now in a condition where I > > cannot say for certain whether it would be possible to replicate the > > setup on a new system without us experiencing some degree of breakage > > (some of which we may not discover until weeks/months afterwards). > > > > In addition, the current setup relies on an old-fashioned overnight > > reprocessing of all repositories, which is inefficient and resource > > expensive. A more modern approach would have the various projects api > > documentation generated on a delayed cycle from relevant branches as > > part of something like a CI job (but not part of the actual CI > > workflow itself). > > > > For this one, i'm not certain on the best path forward at this stage, > > however the current state of affairs cannot continue. We have tried > > over the past few years to find people to work on a replacement for > > the tooling involved, but alas we've yet to have success here. > > > > Thoughts anyone? > > I'd say api.kde.org is too important to let it go. The EBN is less > important but still useful, I still see people pushing EBN-based fixes > once in a while. > > Have we ever tried to use a GSoC project to modernize either of them? If > not, would it make sense to try next year? > My only concern there would be whether the project is large enough in scope to justify the amount of time commitment a GSoC project normally spans. >From my understanding of the problem in question it quite probably does not meet those requirements. > > > > Regards, > > Ben > > Cheers, > Elvis > Cheers, Ben
Re: Sysadmin Load Reduction: Subversion Infrastructure
Ben Cooksley ha scritto: > On Sun, Nov 10, 2019 at 12:20 AM Luigi Toscano > wrote: >> >> Ben Cooksley ha scritto: >>> Hi all, >>> >>> When KDE committed to performing a migration to Git back in 2010, one >>> of the things that was agreed at the time was that translators could >>> remain on Subversion to avoid disrupting their workflows. >>> >>> This however has led to a certain amount of additional infrastructure >>> which Sysadmin needs to continue to maintain. >>> >>> In recognition of the fact that with few exceptions, everything has >>> now migrated from Subversion aside from Translations, i'd like to >>> reduce the level of infrastructure supporting our Subversion >>> repository to the bare minimum necessary. >>> >>> This would include the shutdown of WebSVN in particular, which when >>> coupled with the shutdown of our two CGit instances would also allow >>> for us to eliminate an entire virtual machine from our systems. >> >> As websvn has proven to be useful, would it help the sysadmin (pending >> agreement with aacid and the other people with root access on rosetta) to >> move >> websvn to rosetta and under localization team's responsibility? > > Unfortunately Rosetta does not have the necessary free disk space, as > the Subversion repository is approximately 80GB these days in size, > and operating WebSVN requires a full copy of the Subversion repository > locally in order to operate. Can we extend rosetta's disk then, or add an external volume? In case it's not clear, I'd like to do everything possible to have websvn still up. -- Luigi
Re: Sysadmin Load Reduction: Subversion Infrastructure
El dissabte, 9 de novembre de 2019, a les 19:27:07 CET, Ben Cooksley va escriure: > On Sun, Nov 10, 2019 at 7:18 AM Alexander Potashev > wrote: > > > > сб, 9 нояб. 2019 г. в 03:20, Ben Cooksley : > > > This would include the shutdown of WebSVN in particular, which when > > > coupled with the shutdown of our two CGit instances would also allow > > > for us to eliminate an entire virtual machine from our systems. > > > > Will there be any web interface for SVN after shutdown of WebSVN? > > > > Can we assume https://phabricator.kde.org/source/svn/ remains > > available during the next 10 years? > > > > Phabricator's browser will be retired as part of the shutdown of > Phabricator, which will take place once Gitlab has assumed > responsibility for code hosting and review, and the tasks have been > migrated from Phabricator. > > Should WebSVN be shutdown as well, then there would be no web > interface to our SVN repository. That's not acceptable. Cheers, Albert > > > > > I'm not sure what use case Luigi has in mind, but among Russian l10n > > team we often use WebSVN to refer to specific SVN revisions, for > > example: https://websvn.kde.org/?view=revision&revision=1555342 > > > > -- > > Alexander Potashev > > Regards, > Ben >
Re: Sysadmin Load Reduction: Code Related Services
El dissabte, 9 de novembre de 2019, a les 18:47:57 CET, Ben Cooksley va escriure: > On Sun, Nov 10, 2019 at 12:29 AM Luigi Toscano > wrote: > > > > Ben Cooksley ha scritto: > > > Hi all, > > > > > > In the category of code related services, Sysadmin currently supports > > > a wide-ranging number of services (which makes sense given the nature > > > of the community). Some of these however may no longer be in use or > > > will be duplicative of other services following the transition to > > > Gitlab. > > > > > > In the category of duplicative, we have our two CGit instances at > > > cgit.kde.org and packaging.neon.kde.org. Prior to Gitlab, these were > > > justifiable as there was no other way of browsing scratch/clone > > > repositories over the web. > > > > > > With the migration to Gitlab however all repositories will become > > > browsable through it, meaning it no longer makes sense to offer two > > > browsers that provide the exact same information (with Gitlab having > > > greater capabilities). I'd therefore like to shut both of these down > > > once Code Hosting has transitioned to Gitlab. > > > > Luckily we tried to use commits.kde.org as generic redirect. Will the > > generic > > redirect commits.kde.org be updated to point to the proper repository? It > > may > > be complicated if the new structure contains the namespaces for each > > repository, but we need to keep it working. > > The commits.kde.org redirector is intended to provide permanent links, > so yes it will be updated to redirect people to Gitlab instead once > the switch to Gitlab is completed. > > > > > > > > > In the category of no longer in use, we have the compatibility > > > generator for the kde_projects.xml file. This was introduced when we > > > shutdown Redmine/Chiliproject and migrated to Phabricator, as a way of > > > keeping services that needed to discover a list of KDE Projects > > > functional. > > > > > > As we've since migrated to using YAML files within the > > > sysadmin/repo-metadata repository for both the CI System and > > > kdesrc-build (and with LXR using kdesrc-build to do it's code > > > checkouts) there shouldn't to my knowledge be anything still relying > > > on this (aside from perhaps scripty).> > > > I'd therefore like to shut this generator down as well, along with the > > > compaibility redirector running at projects.kde.org (given that it has > > > been some time since we were using that site, and many projects have > > > moved around in the virtual structure since then, making the redirects > > > it is able to offer useless) > > > > Two points: > > - scripty still uses the XML file and we may need some time to migrate. > > I feared this may have been the case. What sort of timeline are we > looking at in terms of switching scripty over? Over to what? Cheers, Albert
Re: Sysadmin Load Reduction: Subversion Infrastructure
On Sun, Nov 10, 2019 at 8:37 AM Albert Astals Cid wrote: > > El dissabte, 9 de novembre de 2019, a les 19:27:07 CET, Ben Cooksley va > escriure: > > On Sun, Nov 10, 2019 at 7:18 AM Alexander Potashev > > wrote: > > > > > > сб, 9 нояб. 2019 г. в 03:20, Ben Cooksley : > > > > This would include the shutdown of WebSVN in particular, which when > > > > coupled with the shutdown of our two CGit instances would also allow > > > > for us to eliminate an entire virtual machine from our systems. > > > > > > Will there be any web interface for SVN after shutdown of WebSVN? > > > > > > Can we assume https://phabricator.kde.org/source/svn/ remains > > > available during the next 10 years? > > > > > > > Phabricator's browser will be retired as part of the shutdown of > > Phabricator, which will take place once Gitlab has assumed > > responsibility for code hosting and review, and the tasks have been > > migrated from Phabricator. > > > > Should WebSVN be shutdown as well, then there would be no web > > interface to our SVN repository. > > That's not acceptable. Mind explaining why? Bear in mind that there is a cost both in terms of infrastructure, and people time to maintain a service such as WebSVN. This includes having to maintain a mirror of the repository on the machine that runs WebSVN, along with the associated infrastructure for allowing that mirroring to happen on the master server. > > Cheers, > Albert Regards, Ben > > > > > > > > > I'm not sure what use case Luigi has in mind, but among Russian l10n > > > team we often use WebSVN to refer to specific SVN revisions, for > > > example: https://websvn.kde.org/?view=revision&revision=1555342 > > > > > > -- > > > Alexander Potashev > > > > Regards, > > Ben > > > > > >
Re: General Infrastructure Maintainability: API and EBN
Some of what ebn is doing in terms of linting c/c++ code would be better moved to clazy/clang-tidy plugins. The current scan for functions such open false positives for strings and comments. This is just open example of the limits imposed by not having a compilers eye view of the code. I personally don't care for having an extra place to check for errors. Clazy can run automatically on each build so warnings show up in your ide not just on some website. On Sat, Nov 9, 2019, 10:28 AM Elvis Angelaccio wrote: > > > On 09/11/19 01:33, Ben Cooksley wrote: > > Hi all, > > Hi Ben > > > > > Over the past number of years one of the tasks the Sysadmin team has > > worked on has been improving the overall maintainability of our > > systems, with a significant number of specialised cronjobs, exceptions > > and hidden linkages being eliminated. > > > > That is with one great exception: api.kde.org and ebn.kde.org. > > > > Both of these are suffering from an extreme amount of digital bitrot > > and special casing and in general are now in a condition where I > > cannot say for certain whether it would be possible to replicate the > > setup on a new system without us experiencing some degree of breakage > > (some of which we may not discover until weeks/months afterwards). > > > > In addition, the current setup relies on an old-fashioned overnight > > reprocessing of all repositories, which is inefficient and resource > > expensive. A more modern approach would have the various projects api > > documentation generated on a delayed cycle from relevant branches as > > part of something like a CI job (but not part of the actual CI > > workflow itself). > > > > For this one, i'm not certain on the best path forward at this stage, > > however the current state of affairs cannot continue. We have tried > > over the past few years to find people to work on a replacement for > > the tooling involved, but alas we've yet to have success here. > > > > Thoughts anyone? > > I'd say api.kde.org is too important to let it go. The EBN is less > important but still useful, I still see people pushing EBN-based fixes > once in a while. > > Have we ever tried to use a GSoC project to modernize either of them? If > not, would it make sense to try next year? > > > > > Regards, > > Ben > > Cheers, > Elvis > >
Re: Sysadmin Load Reduction: Subversion Infrastructure
On Sat, Nov 9, 2019 at 6:53 PM Ben Cooksley wrote: > > Unfortunately Rosetta does not have the necessary free disk space, as > the Subversion repository is approximately 80GB these days in size, > and operating WebSVN requires a full copy of the Subversion repository > locally in order to operate. Is this also the case if you run WebSVN next to the SVN server itself (ie: on the same machine)? If a single repo copy can be shared by WebSVN and the actual SVN server, that would reduce the storage needs. Albert
Re: Sysadmin Load Reduction: Code Related Services
On Sun, Nov 10, 2019 at 8:43 AM Albert Astals Cid wrote: > > El dissabte, 9 de novembre de 2019, a les 18:47:57 CET, Ben Cooksley va > escriure: > > On Sun, Nov 10, 2019 at 12:29 AM Luigi Toscano > > wrote: > > > > > > Ben Cooksley ha scritto: > > > > Hi all, > > > > > > > > In the category of code related services, Sysadmin currently supports > > > > a wide-ranging number of services (which makes sense given the nature > > > > of the community). Some of these however may no longer be in use or > > > > will be duplicative of other services following the transition to > > > > Gitlab. > > > > > > > > In the category of duplicative, we have our two CGit instances at > > > > cgit.kde.org and packaging.neon.kde.org. Prior to Gitlab, these were > > > > justifiable as there was no other way of browsing scratch/clone > > > > repositories over the web. > > > > > > > > With the migration to Gitlab however all repositories will become > > > > browsable through it, meaning it no longer makes sense to offer two > > > > browsers that provide the exact same information (with Gitlab having > > > > greater capabilities). I'd therefore like to shut both of these down > > > > once Code Hosting has transitioned to Gitlab. > > > > > > Luckily we tried to use commits.kde.org as generic redirect. Will the > > > generic > > > redirect commits.kde.org be updated to point to the proper repository? It > > > may > > > be complicated if the new structure contains the namespaces for each > > > repository, but we need to keep it working. > > > > The commits.kde.org redirector is intended to provide permanent links, > > so yes it will be updated to redirect people to Gitlab instead once > > the switch to Gitlab is completed. > > > > > > > > > > > > > In the category of no longer in use, we have the compatibility > > > > generator for the kde_projects.xml file. This was introduced when we > > > > shutdown Redmine/Chiliproject and migrated to Phabricator, as a way of > > > > keeping services that needed to discover a list of KDE Projects > > > > functional. > > > > > > > > As we've since migrated to using YAML files within the > > > > sysadmin/repo-metadata repository for both the CI System and > > > > kdesrc-build (and with LXR using kdesrc-build to do it's code > > > > checkouts) there shouldn't to my knowledge be anything still relying > > > > on this (aside from perhaps scripty).> > > > > I'd therefore like to shut this generator down as well, along with the > > > > compaibility redirector running at projects.kde.org (given that it has > > > > been some time since we were using that site, and many projects have > > > > moved around in the virtual structure since then, making the redirects > > > > it is able to offer useless) > > > > > > Two points: > > > - scripty still uses the XML file and we may need some time to migrate. > > > > I feared this may have been the case. What sort of timeline are we > > looking at in terms of switching scripty over? > > Over to what? To using the YAML files within sysadmin/repo-metadata, which is what the legacy kde_projects.xml file is generated from currently. (Originally the kde_projects.xml file was generated from the Redmine/Chiliproject database - those YAML files just replaced it) > > Cheers, > Albert > > Thanks, Ben
Re: Sysadmin Load Reduction: Subversion Infrastructure
On Sun, Nov 10, 2019 at 9:32 AM Albert Vaca Cintora wrote: > > On Sat, Nov 9, 2019 at 6:53 PM Ben Cooksley wrote: > > > > Unfortunately Rosetta does not have the necessary free disk space, as > > the Subversion repository is approximately 80GB these days in size, > > and operating WebSVN requires a full copy of the Subversion repository > > locally in order to operate. > > Is this also the case if you run WebSVN next to the SVN server itself > (ie: on the same machine)? If a single repo copy can be shared by > WebSVN and the actual SVN server, that would reduce the storage needs. Running WebSVN on the same machine is not safe for us to do, because certain requests cause WebSVN to generate a substantial amount of system load, and have caused out of memory events on systems running it in the past. This poses an unacceptable risk to the canonical copy of our repositories, as well as to Gitlab (which will be hosted on the same machine as the master Subversion repository) > > Albert Regards, Ben
Re: Sysadmin Load Reduction: Subversion Infrastructure
El dissabte, 9 de novembre de 2019, a les 21:22:16 CET, Ben Cooksley va escriure: > On Sun, Nov 10, 2019 at 8:37 AM Albert Astals Cid wrote: > > > > El dissabte, 9 de novembre de 2019, a les 19:27:07 CET, Ben Cooksley va > > escriure: > > > On Sun, Nov 10, 2019 at 7:18 AM Alexander Potashev > > > wrote: > > > > > > > > сб, 9 нояб. 2019 г. в 03:20, Ben Cooksley : > > > > > This would include the shutdown of WebSVN in particular, which when > > > > > coupled with the shutdown of our two CGit instances would also allow > > > > > for us to eliminate an entire virtual machine from our systems. > > > > > > > > Will there be any web interface for SVN after shutdown of WebSVN? > > > > > > > > Can we assume https://phabricator.kde.org/source/svn/ remains > > > > available during the next 10 years? > > > > > > > > > > Phabricator's browser will be retired as part of the shutdown of > > > Phabricator, which will take place once Gitlab has assumed > > > responsibility for code hosting and review, and the tasks have been > > > migrated from Phabricator. > > > > > > Should WebSVN be shutdown as well, then there would be no web > > > interface to our SVN repository. > > > > That's not acceptable. > > Mind explaining why? Because we use it in l10n.kde.org to link to po files. > Bear in mind that there is a cost both in terms of infrastructure, and > people time to maintain a service such as WebSVN. We have money, we don't have to shut down things we use because there is a cost. Cheers, Albert > This includes having to maintain a mirror of the repository on the > machine that runs WebSVN, along with the associated infrastructure for > allowing that mirroring to happen on the master server. > > > > > Cheers, > > Albert > > Regards, > Ben > > > > > > > > > > > > > > I'm not sure what use case Luigi has in mind, but among Russian l10n > > > > team we often use WebSVN to refer to specific SVN revisions, for > > > > example: https://websvn.kde.org/?view=revision&revision=1555342 > > > > > > > > -- > > > > Alexander Potashev > > > > > > Regards, > > > Ben > > > > > > > > > > > >
Re: Sysadmin Load Reduction: Code Related Services
El dissabte, 9 de novembre de 2019, a les 21:52:13 CET, Ben Cooksley va escriure: > On Sun, Nov 10, 2019 at 8:43 AM Albert Astals Cid wrote: > > > > El dissabte, 9 de novembre de 2019, a les 18:47:57 CET, Ben Cooksley va > > escriure: > > > On Sun, Nov 10, 2019 at 12:29 AM Luigi Toscano > > > wrote: > > > > > > > > Ben Cooksley ha scritto: > > > > > Hi all, > > > > > > > > > > In the category of code related services, Sysadmin currently supports > > > > > a wide-ranging number of services (which makes sense given the nature > > > > > of the community). Some of these however may no longer be in use or > > > > > will be duplicative of other services following the transition to > > > > > Gitlab. > > > > > > > > > > In the category of duplicative, we have our two CGit instances at > > > > > cgit.kde.org and packaging.neon.kde.org. Prior to Gitlab, these were > > > > > justifiable as there was no other way of browsing scratch/clone > > > > > repositories over the web. > > > > > > > > > > With the migration to Gitlab however all repositories will become > > > > > browsable through it, meaning it no longer makes sense to offer two > > > > > browsers that provide the exact same information (with Gitlab having > > > > > greater capabilities). I'd therefore like to shut both of these down > > > > > once Code Hosting has transitioned to Gitlab. > > > > > > > > Luckily we tried to use commits.kde.org as generic redirect. Will the > > > > generic > > > > redirect commits.kde.org be updated to point to the proper repository? > > > > It may > > > > be complicated if the new structure contains the namespaces for each > > > > repository, but we need to keep it working. > > > > > > The commits.kde.org redirector is intended to provide permanent links, > > > so yes it will be updated to redirect people to Gitlab instead once > > > the switch to Gitlab is completed. > > > > > > > > > > > > > > > > > In the category of no longer in use, we have the compatibility > > > > > generator for the kde_projects.xml file. This was introduced when we > > > > > shutdown Redmine/Chiliproject and migrated to Phabricator, as a way of > > > > > keeping services that needed to discover a list of KDE Projects > > > > > functional. > > > > > > > > > > As we've since migrated to using YAML files within the > > > > > sysadmin/repo-metadata repository for both the CI System and > > > > > kdesrc-build (and with LXR using kdesrc-build to do it's code > > > > > checkouts) there shouldn't to my knowledge be anything still relying > > > > > on this (aside from perhaps scripty).> > > > > > I'd therefore like to shut this generator down as well, along with the > > > > > compaibility redirector running at projects.kde.org (given that it has > > > > > been some time since we were using that site, and many projects have > > > > > moved around in the virtual structure since then, making the redirects > > > > > it is able to offer useless) > > > > > > > > Two points: > > > > - scripty still uses the XML file and we may need some time to migrate. > > > > > > I feared this may have been the case. What sort of timeline are we > > > looking at in terms of switching scripty over? > > > > Over to what? > > To using the YAML files within sysadmin/repo-metadata, which is what > the legacy kde_projects.xml file is generated from currently. Well, someone has to change the code that parses the xml to code that gets stuff from git and parses yaml, it's not hard, but has to be done. > (Originally the kde_projects.xml file was generated from the > Redmine/Chiliproject database - those YAML files just replaced it) Yes i know :) Cheers, Albert > > > > > Cheers, > > Albert > > > > > > Thanks, > Ben >
Stopping the "KDE Applications" brand
We're planning to stop using the "KDE Applications" brand Phabricator: https://phabricator.kde.org/T11933 Release-team threads: https://mail.kde.org/pipermail/release-team/2019-October/011582.html https://mail.kde.org/pipermail/release-team/2019-September/011512.html Please contribute to the discussion in those channels :) Cheers, Albert
Re: Sysadmin Load Reduction: Code Related Services
сб, 9 нояб. 2019 г. в 02:51, Ben Cooksley : > In the category of no longer in use, we have the compatibility > generator for the kde_projects.xml file. This was introduced when we > shutdown Redmine/Chiliproject and migrated to Phabricator, as a way of > keeping services that needed to discover a list of KDE Projects > functional. > > As we've since migrated to using YAML files within the > sysadmin/repo-metadata repository for both the CI System and > kdesrc-build (and with LXR using kdesrc-build to do it's code > checkouts) there shouldn't to my knowledge be anything still relying > on this (aside from perhaps scripty). > > I'd therefore like to shut this generator down as well, along with the > compaibility redirector running at projects.kde.org (given that it has > been some time since we were using that site, and many projects have > moved around in the virtual structure since then, making the redirects > it is able to offer useless) Hi Ben, I am developing a new version of the "opensrc" plugin for Lokalize [1] and it currently depends on kde_projects.xml. Of course I can add new code to scan the Git repo instead of just fetching kde_projects.xml, however it would be more complicated. Is there any good reason for killing kde_projects.xml? Would that really reduce the workload on Sysadmin team? I can't suspect any "moving parts" in the generator that may require maintenance. [1] https://cgit.kde.org/scratch/aspotashev/lokalize-opensrc-py.git/ -- Alexander Potashev
kde-devel mailing list archive
Hi, How do I access kde-devel mailing list archive? The archive linked at https://mail.kde.org/mailman/listinfo/kde-devel appears empty. -- Alexander Potashev
Re: Sysadmin Load Reduction: Subversion Infrastructure
Am Samstag, 9. November 2019, 21:22:16 CET schrieb Ben Cooksley: > On Sun, Nov 10, 2019 at 8:37 AM Albert Astals Cid wrote: > > El dissabte, 9 de novembre de 2019, a les 19:27:07 CET, Ben Cooksley va escriure: > > > On Sun, Nov 10, 2019 at 7:18 AM Alexander Potashev wrote: > > > > сб, 9 нояб. 2019 г. в 03:20, Ben Cooksley : > > > > > This would include the shutdown of WebSVN in particular, which when > > > > > coupled with the shutdown of our two CGit instances would also allow > > > > > for us to eliminate an entire virtual machine from our systems. > > > > > > > > Will there be any web interface for SVN after shutdown of WebSVN? > > > > > > > > Can we assume https://phabricator.kde.org/source/svn/ remains > > > > available during the next 10 years? > > > > > > Phabricator's browser will be retired as part of the shutdown of > > > Phabricator, which will take place once Gitlab has assumed > > > responsibility for code hosting and review, and the tasks have been > > > migrated from Phabricator. > > > > > > Should WebSVN be shutdown as well, then there would be no web > > > interface to our SVN repository. > > > > That's not acceptable. > > Mind explaining why? FWIW, everytime I had to deal with translations as developer (like checking pot files as well as .po files contents) I found having the web interface and its browsing feature very valuable to quickly find what I was looking for, over having to locally mess around with svn commands and juggling between commandline & file viewers. Including url bookmarks for quick access to browsing certain sets of files. Incidents which I remember right now included: * finding out whether extraction scripts were working as intended * comparing translations seen by users over what they should see Are there any other KDE clients of the svn repos still around, besides translation system? Perhaps the "full clone" needed for WebSVN could be reduced to the translation subtrees, would that improve situation to a degree if possible? (well, you surely thought of this yourself, just in case) For me as developer contributor to projects in KDE spheres, losing the web browsing interface for raw translation files would be a regression in developer experience. Cheers Friedrich
Re: Sysadmin Load Reduction: Subversion Infrastructure
El sáb., 9 de nov. de 2019 a la(s) 19:48, Friedrich W. H. Kossebau (kosse...@kde.org) escribió: > FWIW, everytime I had to deal with translations as developer (like checking > pot files as well as .po files contents) I found having the web interface and > its browsing feature very valuable to quickly find what I was looking for, > over having to locally mess around with svn commands and juggling between > commandline & file viewers. Including url bookmarks for quick access to > browsing certain sets of files. > > Incidents which I remember right now included: > * finding out whether extraction scripts were working as intended > * comparing translations seen by users over what they should see > > Are there any other KDE clients of the svn repos still around, besides > translation system? > Perhaps the "full clone" needed for WebSVN could be reduced to the translation > subtrees, would that improve situation to a degree if possible? (well, you > surely thought of this yourself, just in case) This is unfortunately not possible. WebSVN needs a full copy of the repository with its history, not a svn checkout, and that can't be trimmed to a subtree. Or maybe you *can* extract a subtree, but then you can't keep that in sync with new changes that appear in the master repository. Even in git that's a giant pain. -- Nicolás
Re: kde-devel mailing list archive
El dissabte, 9 de novembre de 2019, a les 23:45:30 CET, Alexander Potashev va escriure: > Hi, > > How do I access kde-devel mailing list archive? https://marc.info/?l=kde-devel&r=1&w=2 Cheers, Albert > > The archive linked at https://mail.kde.org/mailman/listinfo/kde-devel > appears empty. > >
Re: kde-devel mailing list archive
Thanks! вс, 10 нояб. 2019 г. в 02:30, Albert Astals Cid : > > El dissabte, 9 de novembre de 2019, a les 23:45:30 CET, Alexander Potashev va > escriure: > > Hi, > > > > How do I access kde-devel mailing list archive? > > https://marc.info/?l=kde-devel&r=1&w=2 > > Cheers, > Albert > > > > > The archive linked at https://mail.kde.org/mailman/listinfo/kde-devel > > appears empty. > > > > > > > > -- Alexander Potashev
Re: Sysadmin Load Reduction: Code Related Services
On Sun, Nov 10, 2019 at 11:37 AM Alexander Potashev wrote: > > сб, 9 нояб. 2019 г. в 02:51, Ben Cooksley : > > In the category of no longer in use, we have the compatibility > > generator for the kde_projects.xml file. This was introduced when we > > shutdown Redmine/Chiliproject and migrated to Phabricator, as a way of > > keeping services that needed to discover a list of KDE Projects > > functional. > > > > As we've since migrated to using YAML files within the > > sysadmin/repo-metadata repository for both the CI System and > > kdesrc-build (and with LXR using kdesrc-build to do it's code > > checkouts) there shouldn't to my knowledge be anything still relying > > on this (aside from perhaps scripty). > > > > I'd therefore like to shut this generator down as well, along with the > > compaibility redirector running at projects.kde.org (given that it has > > been some time since we were using that site, and many projects have > > moved around in the virtual structure since then, making the redirects > > it is able to offer useless) > > Hi Ben, Hi Alexander, > > I am developing a new version of the "opensrc" plugin for Lokalize [1] > and it currently depends on kde_projects.xml. Of course I can add new > code to scan the Git repo instead of just fetching kde_projects.xml, > however it would be more complicated. > > > Is there any good reason for killing kde_projects.xml? Would that > really reduce the workload on Sysadmin team? > I can't suspect any "moving parts" in the generator that may require > maintenance. > This is mostly a case of another (small) thing that needs to be kept running, and fixed should it break. All of the reductions i've proposed will have a quite small effect, however cumulatively they will have a much larger effect. Given that it is duplicative of what we have in the YAML files (which are substantially easier to maintain) it makes more sense to standardise on the original source of the information and remove the legacy compatibility XML file. > > [1] https://cgit.kde.org/scratch/aspotashev/lokalize-opensrc-py.git/ > > -- > Alexander Potashev Cheers, Ben
Re: Sysadmin Load Reduction: Code Related Services
On Sun, Nov 10, 2019 at 11:22 AM Albert Astals Cid wrote: > > El dissabte, 9 de novembre de 2019, a les 21:52:13 CET, Ben Cooksley va > escriure: > > On Sun, Nov 10, 2019 at 8:43 AM Albert Astals Cid wrote: > > > > > > El dissabte, 9 de novembre de 2019, a les 18:47:57 CET, Ben Cooksley va > > > escriure: > > > > On Sun, Nov 10, 2019 at 12:29 AM Luigi Toscano > > > > wrote: > > > > > > > > > > Ben Cooksley ha scritto: > > > > > > Hi all, > > > > > > > > > > > > In the category of code related services, Sysadmin currently > > > > > > supports > > > > > > a wide-ranging number of services (which makes sense given the > > > > > > nature > > > > > > of the community). Some of these however may no longer be in use or > > > > > > will be duplicative of other services following the transition to > > > > > > Gitlab. > > > > > > > > > > > > In the category of duplicative, we have our two CGit instances at > > > > > > cgit.kde.org and packaging.neon.kde.org. Prior to Gitlab, these were > > > > > > justifiable as there was no other way of browsing scratch/clone > > > > > > repositories over the web. > > > > > > > > > > > > With the migration to Gitlab however all repositories will become > > > > > > browsable through it, meaning it no longer makes sense to offer two > > > > > > browsers that provide the exact same information (with Gitlab having > > > > > > greater capabilities). I'd therefore like to shut both of these down > > > > > > once Code Hosting has transitioned to Gitlab. > > > > > > > > > > Luckily we tried to use commits.kde.org as generic redirect. Will the > > > > > generic > > > > > redirect commits.kde.org be updated to point to the proper > > > > > repository? It may > > > > > be complicated if the new structure contains the namespaces for each > > > > > repository, but we need to keep it working. > > > > > > > > The commits.kde.org redirector is intended to provide permanent links, > > > > so yes it will be updated to redirect people to Gitlab instead once > > > > the switch to Gitlab is completed. > > > > > > > > > > > > > > > > > > > > > In the category of no longer in use, we have the compatibility > > > > > > generator for the kde_projects.xml file. This was introduced when we > > > > > > shutdown Redmine/Chiliproject and migrated to Phabricator, as a way > > > > > > of > > > > > > keeping services that needed to discover a list of KDE Projects > > > > > > functional. > > > > > > > > > > > > As we've since migrated to using YAML files within the > > > > > > sysadmin/repo-metadata repository for both the CI System and > > > > > > kdesrc-build (and with LXR using kdesrc-build to do it's code > > > > > > checkouts) there shouldn't to my knowledge be anything still relying > > > > > > on this (aside from perhaps scripty).> > > > > > > I'd therefore like to shut this generator down as well, along with > > > > > > the > > > > > > compaibility redirector running at projects.kde.org (given that it > > > > > > has > > > > > > been some time since we were using that site, and many projects have > > > > > > moved around in the virtual structure since then, making the > > > > > > redirects > > > > > > it is able to offer useless) > > > > > > > > > > Two points: > > > > > - scripty still uses the XML file and we may need some time to > > > > > migrate. > > > > > > > > I feared this may have been the case. What sort of timeline are we > > > > looking at in terms of switching scripty over? > > > > > > Over to what? > > > > To using the YAML files within sysadmin/repo-metadata, which is what > > the legacy kde_projects.xml file is generated from currently. > > Well, someone has to change the code that parses the xml to code that gets > stuff from git and parses yaml, it's not hard, but has to be done. *nod* I do recall that at one point in our conversations that scripty only used the kde_projects.xml file as a check against it's own internal data. Has that since been changed? > > > (Originally the kde_projects.xml file was generated from the > > Redmine/Chiliproject database - those YAML files just replaced it) > > Yes i know :) > > Cheers, > Albert Cheers, Ben > > > > > > > > > Cheers, > > > Albert > > > > > > > > > > Thanks, > > Ben > > > > > >
Re: Sysadmin Load Reduction: Code Related Services
El diumenge, 10 de novembre de 2019, a les 1:04:04 CET, Ben Cooksley va escriure: > On Sun, Nov 10, 2019 at 11:22 AM Albert Astals Cid wrote: > > > > El dissabte, 9 de novembre de 2019, a les 21:52:13 CET, Ben Cooksley va > > escriure: > > > On Sun, Nov 10, 2019 at 8:43 AM Albert Astals Cid wrote: > > > > > > > > El dissabte, 9 de novembre de 2019, a les 18:47:57 CET, Ben Cooksley va > > > > escriure: > > > > > On Sun, Nov 10, 2019 at 12:29 AM Luigi Toscano > > > > > wrote: > > > > > > > > > > > > Ben Cooksley ha scritto: > > > > > > > Hi all, > > > > > > > > > > > > > > In the category of code related services, Sysadmin currently > > > > > > > supports > > > > > > > a wide-ranging number of services (which makes sense given the > > > > > > > nature > > > > > > > of the community). Some of these however may no longer be in use > > > > > > > or > > > > > > > will be duplicative of other services following the transition to > > > > > > > Gitlab. > > > > > > > > > > > > > > In the category of duplicative, we have our two CGit instances at > > > > > > > cgit.kde.org and packaging.neon.kde.org. Prior to Gitlab, these > > > > > > > were > > > > > > > justifiable as there was no other way of browsing scratch/clone > > > > > > > repositories over the web. > > > > > > > > > > > > > > With the migration to Gitlab however all repositories will become > > > > > > > browsable through it, meaning it no longer makes sense to offer > > > > > > > two > > > > > > > browsers that provide the exact same information (with Gitlab > > > > > > > having > > > > > > > greater capabilities). I'd therefore like to shut both of these > > > > > > > down > > > > > > > once Code Hosting has transitioned to Gitlab. > > > > > > > > > > > > Luckily we tried to use commits.kde.org as generic redirect. Will > > > > > > the generic > > > > > > redirect commits.kde.org be updated to point to the proper > > > > > > repository? It may > > > > > > be complicated if the new structure contains the namespaces for each > > > > > > repository, but we need to keep it working. > > > > > > > > > > The commits.kde.org redirector is intended to provide permanent links, > > > > > so yes it will be updated to redirect people to Gitlab instead once > > > > > the switch to Gitlab is completed. > > > > > > > > > > > > > > > > > > > > > > > > > In the category of no longer in use, we have the compatibility > > > > > > > generator for the kde_projects.xml file. This was introduced when > > > > > > > we > > > > > > > shutdown Redmine/Chiliproject and migrated to Phabricator, as a > > > > > > > way of > > > > > > > keeping services that needed to discover a list of KDE Projects > > > > > > > functional. > > > > > > > > > > > > > > As we've since migrated to using YAML files within the > > > > > > > sysadmin/repo-metadata repository for both the CI System and > > > > > > > kdesrc-build (and with LXR using kdesrc-build to do it's code > > > > > > > checkouts) there shouldn't to my knowledge be anything still > > > > > > > relying > > > > > > > on this (aside from perhaps scripty).> > > > > > > > I'd therefore like to shut this generator down as well, along > > > > > > > with the > > > > > > > compaibility redirector running at projects.kde.org (given that > > > > > > > it has > > > > > > > been some time since we were using that site, and many projects > > > > > > > have > > > > > > > moved around in the virtual structure since then, making the > > > > > > > redirects > > > > > > > it is able to offer useless) > > > > > > > > > > > > Two points: > > > > > > - scripty still uses the XML file and we may need some time to > > > > > > migrate. > > > > > > > > > > I feared this may have been the case. What sort of timeline are we > > > > > looking at in terms of switching scripty over? > > > > > > > > Over to what? > > > > > > To using the YAML files within sysadmin/repo-metadata, which is what > > > the legacy kde_projects.xml file is generated from currently. > > > > Well, someone has to change the code that parses the xml to code that gets > > stuff from git and parses yaml, it's not hard, but has to be done. > > *nod* > > I do recall that at one point in our conversations that scripty only > used the kde_projects.xml file as a check against it's own internal > data. Yes and no, we have our i18n branch data hardcoded, but we don't have a list of repos, so we need the list of repos from somewhere (and we use the branch data from kde_projects.xml to compare it to our hardcoded one to see if there's some mismatch). Cheers, Albert > Has that since been changed? > > > > > > (Originally the kde_projects.xml file was generated from the > > > Redmine/Chiliproject database - those YAML files just replaced it) > > > > Yes i know :) > > > > Cheers, > > Albert > > Cheers, > Ben > > > > > > > > > > > > > > Cheers, > > > > Albert > > > > > > > > > > > > > > Thanks, > > > Ben > > > > > > > > > > > >
19.12 releases branches created [*]
The branch naming has changed to try to accommodate for the stopping of the "KDE Applications" brand, it's now called release/19.12 Make sure you commit anything you want to end up in the 19.12 releases to them We're already past the dependency freeze. The Freeze and Beta is this Thursday 14 of November. More interesting dates November 28, 2019: KDE Applications 19.12 RC (19.11.90) Tagging and Release December 5, 2019: KDE Applications 19.12 Tagging December 12, 2019: KDE Applications 19.12 Release https://community.kde.org/Schedules/Applications/19.08_Release_Schedule Cheers, Albert P.S: Yes, this release unfortunately falls in the middle of the debranding of "KDE Applications" and there's still a few things called "KDE Applications" here and there [*] There's a small issue with kwave we're working on figuring it out
Re: 19.12 releases branches created [*]
El diumenge, 10 de novembre de 2019, a les 1:28:30 CET, Albert Astals Cid va escriure: > The branch naming has changed to try to accommodate for the stopping of the > "KDE Applications" brand, it's now called >release/19.12 > > Make sure you commit anything you want to end up in the 19.12 releases to them > > We're already past the dependency freeze. > > The Freeze and Beta is this Thursday 14 of November. > > More interesting dates > November 28, 2019: KDE Applications 19.12 RC (19.11.90) Tagging and Release > December 5, 2019: KDE Applications 19.12 Tagging > December 12, 2019: KDE Applications 19.12 Release > > https://community.kde.org/Schedules/Applications/19.08_Release_Schedule https://community.kde.org/Schedules/Applications/19.12_Release_Schedule obviously ^_^ Cheers, Albert > > Cheers, > Albert > > P.S: Yes, this release unfortunately falls in the middle of the debranding of > "KDE Applications" and there's still a few things called "KDE Applications" > here and there > > [*] There's a small issue with kwave we're working on figuring it out > > >
Re: Sysadmin Load Reduction: Code Related Services
On Sun, Nov 10, 2019 at 1:24 PM Albert Astals Cid wrote: > > El diumenge, 10 de novembre de 2019, a les 1:04:04 CET, Ben Cooksley va > escriure: > > On Sun, Nov 10, 2019 at 11:22 AM Albert Astals Cid wrote: > > > > > > El dissabte, 9 de novembre de 2019, a les 21:52:13 CET, Ben Cooksley va > > > escriure: > > > > On Sun, Nov 10, 2019 at 8:43 AM Albert Astals Cid wrote: > > > > > > > > > > El dissabte, 9 de novembre de 2019, a les 18:47:57 CET, Ben Cooksley > > > > > va escriure: > > > > > > On Sun, Nov 10, 2019 at 12:29 AM Luigi Toscano > > > > > > wrote: > > > > > > > > > > > > > > Ben Cooksley ha scritto: > > > > > > > > Hi all, > > > > > > > > > > > > > > > > In the category of code related services, Sysadmin currently > > > > > > > > supports > > > > > > > > a wide-ranging number of services (which makes sense given the > > > > > > > > nature > > > > > > > > of the community). Some of these however may no longer be in > > > > > > > > use or > > > > > > > > will be duplicative of other services following the transition > > > > > > > > to > > > > > > > > Gitlab. > > > > > > > > > > > > > > > > In the category of duplicative, we have our two CGit instances > > > > > > > > at > > > > > > > > cgit.kde.org and packaging.neon.kde.org. Prior to Gitlab, these > > > > > > > > were > > > > > > > > justifiable as there was no other way of browsing scratch/clone > > > > > > > > repositories over the web. > > > > > > > > > > > > > > > > With the migration to Gitlab however all repositories will > > > > > > > > become > > > > > > > > browsable through it, meaning it no longer makes sense to offer > > > > > > > > two > > > > > > > > browsers that provide the exact same information (with Gitlab > > > > > > > > having > > > > > > > > greater capabilities). I'd therefore like to shut both of these > > > > > > > > down > > > > > > > > once Code Hosting has transitioned to Gitlab. > > > > > > > > > > > > > > Luckily we tried to use commits.kde.org as generic redirect. Will > > > > > > > the generic > > > > > > > redirect commits.kde.org be updated to point to the proper > > > > > > > repository? It may > > > > > > > be complicated if the new structure contains the namespaces for > > > > > > > each > > > > > > > repository, but we need to keep it working. > > > > > > > > > > > > The commits.kde.org redirector is intended to provide permanent > > > > > > links, > > > > > > so yes it will be updated to redirect people to Gitlab instead once > > > > > > the switch to Gitlab is completed. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > In the category of no longer in use, we have the compatibility > > > > > > > > generator for the kde_projects.xml file. This was introduced > > > > > > > > when we > > > > > > > > shutdown Redmine/Chiliproject and migrated to Phabricator, as a > > > > > > > > way of > > > > > > > > keeping services that needed to discover a list of KDE Projects > > > > > > > > functional. > > > > > > > > > > > > > > > > As we've since migrated to using YAML files within the > > > > > > > > sysadmin/repo-metadata repository for both the CI System and > > > > > > > > kdesrc-build (and with LXR using kdesrc-build to do it's code > > > > > > > > checkouts) there shouldn't to my knowledge be anything still > > > > > > > > relying > > > > > > > > on this (aside from perhaps scripty).> > > > > > > > > I'd therefore like to shut this generator down as well, along > > > > > > > > with the > > > > > > > > compaibility redirector running at projects.kde.org (given that > > > > > > > > it has > > > > > > > > been some time since we were using that site, and many projects > > > > > > > > have > > > > > > > > moved around in the virtual structure since then, making the > > > > > > > > redirects > > > > > > > > it is able to offer useless) > > > > > > > > > > > > > > Two points: > > > > > > > - scripty still uses the XML file and we may need some time to > > > > > > > migrate. > > > > > > > > > > > > I feared this may have been the case. What sort of timeline are we > > > > > > looking at in terms of switching scripty over? > > > > > > > > > > Over to what? > > > > > > > > To using the YAML files within sysadmin/repo-metadata, which is what > > > > the legacy kde_projects.xml file is generated from currently. > > > > > > Well, someone has to change the code that parses the xml to code that > > > gets stuff from git and parses yaml, it's not hard, but has to be done. > > > > *nod* > > > > I do recall that at one point in our conversations that scripty only > > used the kde_projects.xml file as a check against it's own internal > > data. > > Yes and no, we have our i18n branch data hardcoded, but we don't have a list > of repos, so we need the list of repos from somewhere (and we use the branch > data from kde_projects.xml to compare it to our hardcoded one to see if > there's some mismatch). > Thanks for confirming how that is used. > Cheers, > Albert > Cheers, Ben > > Has that s
Re: Sysadmin Load Reduction: Subversion Infrastructure
On Sun, Nov 10, 2019 at 11:47 AM Friedrich W. H. Kossebau wrote: > > Am Samstag, 9. November 2019, 21:22:16 CET schrieb Ben Cooksley: > > On Sun, Nov 10, 2019 at 8:37 AM Albert Astals Cid wrote: > > > El dissabte, 9 de novembre de 2019, a les 19:27:07 CET, Ben Cooksley va > escriure: > > > > On Sun, Nov 10, 2019 at 7:18 AM Alexander Potashev > wrote: > > > > > сб, 9 нояб. 2019 г. в 03:20, Ben Cooksley : > > > > > > This would include the shutdown of WebSVN in particular, which when > > > > > > coupled with the shutdown of our two CGit instances would also allow > > > > > > for us to eliminate an entire virtual machine from our systems. > > > > > > > > > > Will there be any web interface for SVN after shutdown of WebSVN? > > > > > > > > > > Can we assume https://phabricator.kde.org/source/svn/ remains > > > > > available during the next 10 years? > > > > > > > > Phabricator's browser will be retired as part of the shutdown of > > > > Phabricator, which will take place once Gitlab has assumed > > > > responsibility for code hosting and review, and the tasks have been > > > > migrated from Phabricator. > > > > > > > > Should WebSVN be shutdown as well, then there would be no web > > > > interface to our SVN repository. > > > > > > That's not acceptable. > > > > Mind explaining why? > > FWIW, everytime I had to deal with translations as developer (like checking > pot files as well as .po files contents) I found having the web interface and > its browsing feature very valuable to quickly find what I was looking for, > over having to locally mess around with svn commands and juggling between > commandline & file viewers. Including url bookmarks for quick access to > browsing certain sets of files. > > Incidents which I remember right now included: > * finding out whether extraction scripts were working as intended > * comparing translations seen by users over what they should see > > Are there any other KDE clients of the svn repos still around, besides > translation system? The only other clients I am aware of are a small number of websites, which can be found at /trunk/www/ in SVN. The list of ones we serve can be found at https://invent.kde.org/sysadmin/binary-factory-tooling/blob/master/staticweb/standard-jobs.yaml#L69 > Perhaps the "full clone" needed for WebSVN could be reduced to the translation > subtrees, would that improve situation to a degree if possible? (well, you > surely thought of this yourself, just in case) Unfortunately WebSVN needs a full copy of the underlying repository to be able to work. > > For me as developer contributor to projects in KDE spheres, losing the web > browsing interface for raw translation files would be a regression in > developer experience. > > Cheers > Friedrich > > Regards, Ben
Re: Sysadmin Load Reduction: Subversion Infrastructure
On Sun, Nov 10, 2019 at 11:19 AM Albert Astals Cid wrote: > > El dissabte, 9 de novembre de 2019, a les 21:22:16 CET, Ben Cooksley va > escriure: > > On Sun, Nov 10, 2019 at 8:37 AM Albert Astals Cid wrote: > > > > > > El dissabte, 9 de novembre de 2019, a les 19:27:07 CET, Ben Cooksley va > > > escriure: > > > > On Sun, Nov 10, 2019 at 7:18 AM Alexander Potashev > > > > wrote: > > > > > > > > > > сб, 9 нояб. 2019 г. в 03:20, Ben Cooksley : > > > > > > This would include the shutdown of WebSVN in particular, which when > > > > > > coupled with the shutdown of our two CGit instances would also allow > > > > > > for us to eliminate an entire virtual machine from our systems. > > > > > > > > > > Will there be any web interface for SVN after shutdown of WebSVN? > > > > > > > > > > Can we assume https://phabricator.kde.org/source/svn/ remains > > > > > available during the next 10 years? > > > > > > > > > > > > > Phabricator's browser will be retired as part of the shutdown of > > > > Phabricator, which will take place once Gitlab has assumed > > > > responsibility for code hosting and review, and the tasks have been > > > > migrated from Phabricator. > > > > > > > > Should WebSVN be shutdown as well, then there would be no web > > > > interface to our SVN repository. > > > > > > That's not acceptable. > > > > Mind explaining why? > > Because we use it in l10n.kde.org to link to po files. Mind detailing what those links are used for? > > > Bear in mind that there is a cost both in terms of infrastructure, and > > people time to maintain a service such as WebSVN. > > We have money, we don't have to shut down things we use because there is a > cost. I wasn't referring to monetary cost there, I was referring to the flow on effects (such as having to maintain the necessary components on the master server to allow for the Subversion repository to be mirrored). Note also the "people time" component there. > > Cheers, > Albert Thanks, Ben > > > This includes having to maintain a mirror of the repository on the > > machine that runs WebSVN, along with the associated infrastructure for > > allowing that mirroring to happen on the master server. > > > > > > > > Cheers, > > > Albert > > > > Regards, > > Ben > > > > > > > > > > > > > > > > > > > I'm not sure what use case Luigi has in mind, but among Russian l10n > > > > > team we often use WebSVN to refer to specific SVN revisions, for > > > > > example: https://websvn.kde.org/?view=revision&revision=1555342 > > > > > > > > > > -- > > > > > Alexander Potashev > > > > > > > > Regards, > > > > Ben > > > > > > > > > > > > > > > > > > > > > >
KDE Frameworks 5.64.0 released
10th November 2019. KDE today announces the release of KDE Frameworks 5.64.0. KDE Frameworks are 81 addon libraries to Qt which provide a wide variety of commonly needed functionality in mature, peer reviewed and well tested libraries with friendly licensing terms. For an introduction see https://kde.org/products/frameworks/ Attica Add some std::move in setter functions Baloo Make it compile against qt5.15 Use propertymap to store properties in Baloo::Result Add standalone conversion functions for PropertyMap to Json and vice versa [Database] Rework handling environment flags Replace recursion in FilteredDirIterator with loop iteration Breeze Icons Center-align the non-square 64px audio mimetype icons (bug 393550) Delete remnants of nepomuk icon Move colorful 32px help-about icon into actions (bug 396626) Improve draw icons (bug 399665) Delete nepomuk icon Fill middle mouse button area Add folder-recent, extend hand of clock in folder-temp Use a more correct and appropriate visual metaphor for "get hot new stuff" icon (bug 400500) Update elisa icon Use css instead of scss as output format Fix incorrect rendering of 22px edit-opacity icon Add edit-opacity icons (bug 408283) Icons for windy weather (bug 412718) Fix incorrect margins in 16/22px media icons Use the text rather than highlight color for rating/star emblem Add draw-arrow icons (bug 408283) Add draw-highlight action icons (bug 408283) Add PATH/LD_LIBRARY_PATH to qrcAlias invocation Add applications-network icon for renaming Internet category to Network Add edit-line-width icons (bug 408283) Extra CMake Modules Don't set C/C++ standards if already set Use modern way to set the C/CXX standad Raise CMake requirements to 3.5 ECMAddQch: support PREDEFINED_MACROS/BLANK_MACROS with blanks & quotes Framework Integration Add standard icons to support to all entries in QDialogButtonBox (bug 398973) ensure winId() not called on non-native widgets (bug 412675) KActivitiesStats tests: fix macos build failure Windows MSVC compile fix Add a utility accessor to get a QUrl from a ResultSet::Result KArchive Fix memory leak in KXzFilter::init Fix null pointer reference when extraction fails decodeBCJ2: Fix assert with broken files KXzFilter::Private: remove unused props K7Zip: Fix memory use in readAndDecodePackedStreams KCalendarCore # Add libical version too Explicitly define the Journal copy ctor KCMUtils Conditionally show navigation buttons in the header for multi-page KCMs don't use a custom header height (bug 404396) add extra include Fix memory leak of KQuickAddons::ConfigModule objects (bug 412998) [KCModuleLoader] Show error when QML fails to load KConfig kconfig_compiler: Move the KSharedConfig::Ptr when using them Make it compile against qt5.15 without deprecated method Expose isImmutable to introspection (e.g. QML) Add convenience for defaults/dirty states to KCoreConfigSkeleton Make kconfig_compiler generate ctors with the optional parent arg Make preferences() a public function KConfigWidgets Avoid overloading KCModule::changed KContacts Install translations KCoreAddons KProcessInfoList -- add proclist backend for FreeBSD KDeclarative Use compile time checked connect Make the settingChanged() slot protected Get KQuickAddons::ConfigModule to expose if we're in the defaults state Grab the keyboard when KeySequenceItem is recording Add ManagedConfigModule Remove outdated comment about [$e] expansion [ConfigModule] Expose mainUi component status and error string KDELibs 4 Support KLocale api docs: make it easier to find how to port code away from it KDocTools man: use instead of KFileMetaData Fix crash in writer collection and cleanup KHTML Extend KHtmlView::print() to use a predefined QPrinter instance (bug 405011) KI18n Add KLocalizedString::untranslatedText Replace all qWarning and related calls with categorised logging KIconThemes Fix usage of the new deprecation macros for assignIconsToContextMenu Deprecate KIconTheme::assignIconsToContextMenu KIO Const & signature of new introduced SlaveBase::configValue Port to the QSslError variant of KSslInfoDialog Port KSSLD internals from KSslError to QSslError Make non-ignorable SSL errors explicit auto-enable KIO_ASSERT_SLAVE_STATES also for from-git builds Port (most of) KSslInfoDialog from KSslError to QSslError kio_http: avoid double Content-Type and Depth when used by KDAV Port the KSSLD D-Bus interface from KSslError to QSslError Replace usage of SlaveBase::config()->readEntry by SlaveBase::configValue Remove two unused member variables using KSslError Avoid sending KDirNotify::emitFilesAdded when the emptytrashjob finishes Deprecate the KTcpSocket-based variant of SslUi::askIgnoreSslErrors Treat "application/x-ms-dos-executable" as executable on all platforms (bug 412694) Replace usage of
Re: Sysadmin Load Reduction: Subversion Infrastructure
вс, 10 нояб. 2019 г. в 04:10, Ben Cooksley :
>
> On Sun, Nov 10, 2019 at 11:19 AM Albert Astals Cid wrote:
> >
> > El dissabte, 9 de novembre de 2019, a les 21:22:16 CET, Ben Cooksley va
> > escriure:
> > > On Sun, Nov 10, 2019 at 8:37 AM Albert Astals Cid wrote:
> > > >
> > > > El dissabte, 9 de novembre de 2019, a les 19:27:07 CET, Ben Cooksley va
> > > > escriure:
> > > > > On Sun, Nov 10, 2019 at 7:18 AM Alexander Potashev
> > > > > wrote:
> > > > > >
> > > > > > сб, 9 нояб. 2019 г. в 03:20, Ben Cooksley :
> > > > > > > This would include the shutdown of WebSVN in particular, which
> > > > > > > when
> > > > > > > coupled with the shutdown of our two CGit instances would also
> > > > > > > allow
> > > > > > > for us to eliminate an entire virtual machine from our systems.
> > > > > >
> > > > > > Will there be any web interface for SVN after shutdown of WebSVN?
> > > > > >
> > > > > > Can we assume https://phabricator.kde.org/source/svn/ remains
> > > > > > available during the next 10 years?
> > > > > >
> > > > >
> > > > > Phabricator's browser will be retired as part of the shutdown of
> > > > > Phabricator, which will take place once Gitlab has assumed
> > > > > responsibility for code hosting and review, and the tasks have been
> > > > > migrated from Phabricator.
> > > > >
> > > > > Should WebSVN be shutdown as well, then there would be no web
> > > > > interface to our SVN repository.
> > > >
> > > > That's not acceptable.
> > >
> > > Mind explaining why?
> >
> > Because we use it in l10n.kde.org to link to po files.
>
> Mind detailing what those links are used for?
A common workflow suggested for translators without SVN account
(and/or those who find it hard to use SVN) is as folllows:
1. Go to https://l10n.kde.org/stats/gui/trunk-kf5/team/ru/ ("ru" for
Russian) and pick a .po translation file that needs update - e.g.
incomplete translation or one containing mistakes.
2. Click on the .po file at l10n.kde.org to downloaded. WebSVN is
used for this.
3. Edit the downloaded .po file, send for review, etc.
WebSVN is even mentioned in KDE's primary translation how-to:
https://l10n.kde.org/docs/translation-howto/gui-translation.html
--
Alexander Potashev
