[Cloud] Re: LLM services
Huji wrote: > Even free, lightweight LLMs (like LLaMa) could be helpful LLaMa itself is not under a free license. Let's call it an "almost-free license". So, I'm not sure if it would be acceptable to run it, given the requisite that > All code in the Tools project must be published under an OSI approved open source license I think the debate would be whether the models (in this case LLaMa) is "code" or "data". It might be considered both ways. Separatedly, regarding the resources point mentioned by Siddhart: > LLMs might also require significant memory/CPU resources and/or system software not available to tools LLMs are very memory-hungry, but what they would benefit most would be from GPU memory. Of which we probably don't have any in cloud. The ideal setup would probably be a specific host in cloud providing a LLM service and offering that to tools and VMs. (*) It's still possible to run LLMs without GPU and get acceptable results, although the time required and amount of requests that can be fulfilled (as well as number of models loaded) would be much more limited. ___ Cloud mailing list -- cloud@lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/
[Cloud] Re: LLM services
On 1/10/25 05:21, Huji Lee wrote: Hi all, Are there any LLMs available on Cloud services, or are there any plans for them? I think there are many possible use cases. Even free, lightweight LLMs (like LLaMa) could be helpful, e.g. in bots that review edits, categorize pages, etc. Hi Huji, No, there is none at this point in time. But this has been in my radar for some time now, and it would be useful to know particular use cases and needs before we can work on any implementation. This email is already helpful (to know there is at least one person interested). A phabricator ticket would also be useful, to help inform future actions. Would you mind creating one? regards. ___ Cloud mailing list -- cloud@lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/
[Cloud] Re: LLM services
Since it's now possible in Toolforge to expose services at custom ports[0], I think it should already be possible for someone to host an LLM service for other tools to use. I could be wrong though, as LLMs might also require significant memory/CPU resources and/or system software not available to tools. [0]: https://wikitech.wikimedia.org/wiki/Help:Toolforge/Jobs_framework#Configuring_internal_domain_names_for_your_jobs On Fri, 10 Jan 2025 at 16:31, Francesco Negri wrote: > > A phabricator ticket would also be useful, to help inform future > actions. Would > > you mind creating one? > > We have https://phabricator.wikimedia.org/T336905 that seems to cover > the same topic. > > -- > Francesco Negri (he/him) -- IRC: dhinus > Site Reliability Engineer, Cloud Services team > Wikimedia Foundation > ___ > Cloud mailing list -- cloud@lists.wikimedia.org > List information: > https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/ > ___ Cloud mailing list -- cloud@lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/
[Cloud] [Mediawiki-api-announce] [Breaking Change] Fix of 'sub' field format in the OAuth identity endpoint
The OAuth identity endpoint (the Special:OAuth/identify special page for OAuth 1, the oauth2/resource/profile REST API endpoint for OAuth 2) used to return an incorrectly formatted JSON web token, where value of the 'sub' field (the user's CentralAuth central user ID) was an integer, rather than a string as required by the JWT spec. Due to the latest release of the pyJWT library getting more strict about format validation, this started causing errors in various tools recently. As of this week, this behavior has been fixed for Wikimedia sites, and it has been fixed in all maintained versions (MediaWiki 1.39 and upwards) of the OAuth MediaWiki extension which provides this API. Because the old behavior was a spec violation and caused errors, and it's unlikely the correct behavior would break clients, we are making this fix as a breaking change rather than following the usual API deprecation policies. For more details and discussion see https://phabricator.wikimedia.org/T382139 ___ Mediawiki-api-announce mailing list -- mediawiki-api-annou...@lists.wikimedia.org To unsubscribe send an email to mediawiki-api-announce-le...@lists.wikimedia.org ___ Cloud mailing list -- cloud@lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/
[Cloud] [Cloud-announce] Re: mild dns changes in cloud-vps on Wednesday
This didn't happen! Further testing suggests that my original assertion
("now reaching public IPs works as expected") is not as true as I
thought it was.
On 1/6/25 9:26 AM, Andrew Bogott wrote:
Tl;dr:
Please let me know if you encounter changes in network connectivity
between cloud-vps hosts, specifically when routing to floating IPs.
Complicated explanation:
A previous version of our network setup prevented routing between
private and floating IPs within cloud-vps. To work around this we run
an agent called 'labs-ip-aliaser' which insertes corresponding private
IPs for any dns lookup initiating from within cloud-vps that would
otherwise return a floating IP.
So, for instance, if you did something like
$ ping hostname.wmcloud.org
you would actually be pinging vmname.project.eqiad.wikimedia.cloud
instead, because that worked and pinging the wmcloud.org address did not.
We have since refactored the network so that now reaching public IPs
works as expected. So, later in the week I'm going to remove the
labs-ip-aliaser hack so that floating-ip-associated hostnames can
resolve to the actual floating IPs as expected. This change should be
almost entirely invisible to cloud-vps services, but please notify me
if you find an interesting edge-case!
https://phabricator.wikimedia.org/T374129
https://gerrit.wikimedia.org/r/c/operations/puppet/+/1105443/7
___
Cloud-announce mailing list -- cloud-annou...@lists.wikimedia.org
List information:
https://lists.wikimedia.org/postorius/lists/cloud-announce.lists.wikimedia.org/
___
Cloud mailing list -- cloud@lists.wikimedia.org
List information:
https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/
[Cloud] Re: LLM services
> A phabricator ticket would also be useful, to help inform future actions. > Would > you mind creating one? We have https://phabricator.wikimedia.org/T336905 that seems to cover the same topic. -- Francesco Negri (he/him) -- IRC: dhinus Site Reliability Engineer, Cloud Services team Wikimedia Foundation ___ Cloud mailing list -- cloud@lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/
