Re: [Cloud] Parsoid APIs?

[Subramanya Sastry]

See https://www.mediawiki.org/wiki/Parsoid/API now.  -Subbu.

On 9/10/20 10:53 AM, Roy Smith wrote:
Could you update https://www.mediawiki.org/wiki/Parsoid/API to 
indicate that?  Even just a note saying, "This is an internal API. 
 For external use, see https://en.wikipedia.org/api/rest_v1";, or 
something like that.  The way thing are now, it looks like this is the 
API you're supposed to be using, and the routes just don't respond. 
 There's no way for a reader to differentiate between, "The 
documentation is wrong" vs, "I'm doing something wrong".



On Sep 10, 2020, at 11:38 AM, Subramanya Sastry 
mailto:ssas...@wikimedia.org>> wrote:


HI Roy,

Sorry it took a while before we could respond.

On 9/4/20 12:40 PM, Roy Smith wrote:
I know there's been a ton of work done of Parsoid lately.  This is 
great, and the amount of effort that's gone into this functionality 
is really appreciated.  It's clear that Parsoid is the way of the 
future, but the documentation of how you get a Parsoid parse tree 
via an AP call isI kind of confusing.


I found https://www.mediawiki.org/wiki/Parsoid/API, which looks like 
it's long out of date.  The last edit was almost 2 years ago.  As 
far as I can tell, most of what it says is obsolete, and refers to a 
series of /v3 routes which don't actually exist.


That page is not out of date. That refers to Parsoid's API which is 
what you would use if you were querying Parsoid directly. When we 
ported Parsoid from JS to PHP, we ensured that the API routes didn't 
change. What changed was the base url of the Parsoid service (so 
clients could simply switch this URL in the configuration without 
having to change their code).


For example, 
http://en.wikipedia.org/w/rest.php/en.wikipedia.org/v3/page/html// 
works if you curl this url on a production server (if you have 
access). But this Parsoid API is not accessible on the public 
internet. However, wikitech.wikimedia.org 
's Parsoid API is currently accessible 
on the public internet. So, for example: 
https://wikitech.wikimedia.org/w/rest.php/wikitech.wikimedia.org/v3/page/html/Parsoid 
works. So, you can verify that the API routes on the Parsoid wiki 
page will work on wikitech.wikimedia.org .


But, anyway, this is not directly relevant to your usecase unless you 
are directly contacting a Parsoid service somewhere. In production 
wikimedia wikis, as I said, Parsoid's API isn't public (it wasn't 
public with the JS version either). You can only access Parsoid 
content via RESTBase's public API which you reference below.


I also found https://en.wikipedia.org/api/rest_v1/#/Page%20content 
, which seems 
more in line with the current reality.


Yes, this is the API that RESTBase provides. Behind the scenes, it 
accesses Parsoid's API when it needs fresh content.


 But, the call I was most interested in, 
​/page​/data-parsoid​/{title}​/{revision}​/{tid}, doesn't actually 
respond (at least not on en.wikipedia.org ).


Joaquin already responded to this part (thanks Joaquin), so I'll skip 
this here.


I will respond to your other Parsoid HTML questions / comments by 
responding to your other post.


Subbu.



___
Wikimedia Cloud Services mailing list
Cloud@lists.wikimedia.org (formerly lab...@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/cloud

Re: [Cloud] Parsoid APIs?

[Roy Smith]

Thank you.  That would have definitely kept me from getting off on the wrong 
track.


> On Sep 11, 2020, at 1:16 PM, Subramanya Sastry  wrote:
> 
> See https://www.mediawiki.org/wiki/Parsoid/API 
>  now.  -Subbu.
> On 9/10/20 10:53 AM, Roy Smith wrote:
>> Could you update https://www.mediawiki.org/wiki/Parsoid/API 
>>  to indicate that?  Even just a 
>> note saying, "This is an internal API.  For external use, see  
>> https://en.wikipedia.org/api/rest_v1 
>> ", or something like that.  The way 
>> thing are now, it looks like this is the API you're supposed to be using, 
>> and the routes just don't respond.  There's no way for a reader to 
>> differentiate between, "The documentation is wrong" vs, "I'm doing something 
>> wrong".
>> 
>> 
>>> On Sep 10, 2020, at 11:38 AM, Subramanya Sastry >> > wrote:
>>> 
>>> HI Roy,
>>> 
>>> Sorry it took a while before we could respond.
>>> On 9/4/20 12:40 PM, Roy Smith wrote:
 I know there's been a ton of work done of Parsoid lately.  This is great, 
 and the amount of effort that's gone into this functionality is really 
 appreciated.  It's clear that Parsoid is the way of the future, but the 
 documentation of how you get a Parsoid parse tree via an AP call isI kind 
 of confusing.
 
 I found https://www.mediawiki.org/wiki/Parsoid/API 
 , which looks like it's long 
 out of date.  The last edit was almost 2 years ago.  As far as I can tell, 
 most of what it says is obsolete, and refers to a series of /v3 routes 
 which don't actually exist.
>>> That page is not out of date. That refers to Parsoid's API which is what 
>>> you would use if you were querying Parsoid directly. When we ported Parsoid 
>>> from JS to PHP, we ensured that the API routes didn't change. What changed 
>>> was the base url of the Parsoid service (so clients could simply switch 
>>> this URL in the configuration without having to change their code).
>>> For example, 
>>> http://en.wikipedia.org/w/rest.php/en.wikipedia.org/v3/page/html/ 
>>> /
>>>  works if you curl this url on a production server (if you have access). 
>>> But this Parsoid API is not accessible on the public internet. However, 
>>> wikitech.wikimedia.org 's Parsoid API is 
>>> currently accessible on the public internet. So, for 
>>> example:https://wikitech.wikimedia.org/w/rest.php/wikitech.wikimedia.org/v3/page/html/Parsoid
>>>  
>>> 
>>>  works. So, you can verify that the API routes on the Parsoid wiki page 
>>> will work on wikitech.wikimedia.org .
>>> But, anyway, this is not directly relevant to your usecase unless you are 
>>> directly contacting a Parsoid service somewhere. In production wikimedia 
>>> wikis, as I said, Parsoid's API isn't public (it wasn't public with the JS 
>>> version either). You can only access Parsoid content via RESTBase's public 
>>> API which you reference below.
 I also found https://en.wikipedia.org/api/rest_v1/#/Page%20content 
 , which seems more in line with the current reality.
>>> Yes, this is the API that RESTBase provides. Behind the scenes, it accesses 
>>> Parsoid's API when it needs fresh content.
  But, the call I was most interested in, 
 ​/page​/data-parsoid​/{title}​/{revision}​/{tid}, doesn't actually respond 
 (at least not on en.wikipedia.org ).
>>> Joaquin already responded to this part (thanks Joaquin), so I'll skip this 
>>> here.
>>> I will respond to your other Parsoid HTML questions / comments by 
>>> responding to your other post.
>>> 
>>> Subbu.
>> 

___
Wikimedia Cloud Services mailing list
Cloud@lists.wikimedia.org (formerly lab...@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/cloud

[Cloud] [Cloud-announce] Clarification: database service names still use eqiad.wmflabs, not eqiad1.wikimedia.cloud

[Bryan Davis]

Several people have asked on IRC and Phabricator if the deprecation of
*.wmflabs names for Cloud VPS instances means that the service names
used to connect to ToolsDB and the Wiki Replicas are changing. The
answer is no, these names are not changing yet. We will be replacing
these service names eventually, but for now they are staying in the
wmflabs pseudo domain.


In 2017 [0] we established new canonical service names for accessing
the shared database servers for the Cloud Services environment. Those
names are still the same today.

The naming convention for connecting to the Wiki Replica servers is:
".{analytics,web}.db.svc.eqiad.wmflabs". The
*.web.db.svc.eqiad.wmflabs service names are intended for queries that
need a real-time response. The *.analytics.db.svc.eqiad.wmflabs
service names are intended for batch jobs and long running queries.
See the announcement from 2017 for more details [0].

The preferred service name for ToolsDB is tools.db.svc.eqiad.wmflabs.

[0]: 
https://phabricator.wikimedia.org/phame/post/view/70/new_wiki_replica_servers_ready_for_use/

Bryan, on behalf of the Cloud Services team
-- 
Bryan Davis  Technical Engagement  Wikimedia Foundation
Principal Software Engineer   Boise, ID USA
[[m:User:BDavis_(WMF)]]  irc: bd808

___
Wikimedia Cloud Services announce mailing list
cloud-annou...@lists.wikimedia.org (formerly labs-annou...@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/cloud-announce

___
Wikimedia Cloud Services mailing list
Cloud@lists.wikimedia.org (formerly lab...@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/cloud

Re: [Cloud] [Cloud-announce] Fwd: Phasing out the .wmflabs tld on September 8th

[Daniel Zahn]

On Tue, Sep 8, 2020 at 10:07 AM Andrew Bogott  wrote:
> The modern name for that bastion is 'restricted.bastion.wmcloud.org'.

Ah, thanks! Updated my local config.

>  today's change did not affect public domains, only private .eqiad.wmflabs 
> domain names.  We have no plans to eliminate existing .wmflabs.org domain 
> names because we don't want to break existing URLs.

ACK, thanks for not breaking URLs. It was also nice that removing a
proxy in Horizon and adding a new one with wmcloud
meant that old URLs don't break and they automatically redirect
without the end user having to deal with it. Cool!

> And is the distinction between regular bastion and restricted bastion still a 
> thing?
>
> It is, although I'm not 100% convinced that it has value.

Yea, me neither. But that's for another day maybe to bring up with the
wider teams.

-- 
Daniel Zahn 
Operations Engineer

___
Wikimedia Cloud Services mailing list
Cloud@lists.wikimedia.org (formerly lab...@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/cloud