Likely the case. The expense of collating all that data and presenting it to
their site visitors is considerable. They use advertising to cover those
costs. If the data were easily scrapable, scrapers diminish revenue, putting
the resource itself at risk.
Some data provides offer APIs. When
Yes.
put url "https://uk.finance.yahoo.com/quote/SHEL.L/history/";
In the message box on 9.6.11 and 9.6.12 under Windows 11. Both return a
pile of HTML text that is all the formatting and CSS linked stuff to
show a "404" page.
This suggests that put URL is working and it is the Yahoo server
Did you try that in the message box?
Bob S
> On Jun 28, 2024, at 10:50 AM, Paul Dupuis via use-livecode
> wrote:
>
> I get a response from Yahoos that is an html page with a 404 information as
> part of it.
>
> This happens under LC 9.6.12 and 9.6.11
>
> I think this is Yahoo Finance not
I get a response from Yahoos that is an html page with a 404 information
as part of it.
This happens under LC 9.6.12 and 9.6.11
I think this is Yahoo Finance not being able to detect the browser type
and intentionally returning a 404 as a method of deterring screen scraping.
On 6/28/2024 1:
I get the HTML of the page. Are you trying to open the page in a browser?
Bob S
> On Jun 28, 2024, at 10:04 AM, Hugh Senior via use-livecode
> wrote:
>
>
> Platform: Windows 11, LC 9.6.12
> Query: Using URL to access a web page
>
> Problem:
> Enter "https://uk.finance.yahoo.com/quote/SHEL.
Platform: Windows 11, LC 9.6.12
Query: Using URL to access a web page
Problem:
Enter "https://uk.finance.yahoo.com/quote/SHEL.L/history/"; into any web
browser and the page is displayed as expected.
Use LC's URL command to access the same page direct returns a 404
put url "https://uk.finance.ya
Added error checking. Also the payload can now be a string or an array.
command packagePayload @pPayload, pUseEncryption
try
if pPayload is an array then \
put arrayEncode(pPayload) into pPayload
if pUseEncryption then \
put slyEncrypt(pPayload) in
I love using red herring in conversations. Most people don’t know what that is,
so they can’t contradict me! :-)
Bob S
On Jun 28, 2024, at 5:02 AM, Mark Waddingham via use-livecode
wrote:
so thinking this is regex related is a red-herring.
___
use
On 2024-06-28 11:15, Neville Smythe via use-livecode wrote:
In my last epistle I mentioned the repeat loop had only 32 iterations.
Much more relevant of course is the inner loop on the number of lines
of the data variable fff. In this case fff had 1760 lines. So the total
possible number of ite
In my last epistle I mentioned the repeat loop had only 32 iterations. Much
more relevant of course is the inner loop on the number of lines of the data
variable fff. In this case fff had 1760 lines. So the total possible number of
iterations was around 3 to 5, getting up there but still
10 matches
Mail list logo
