I've examined the Perl Finance::Quote ASX.pm module used by GnuCash to
fetch price data from the Australian Securities Exchange (ASX) and can
confirm that this problem is caused by an upgrade to the ASX web site:
"Please note that the ASX beta site is a test website and is still
undergoing final testing before its official release. The sole purpose
of this beta site is to conduct testing and obtain feedback on the
quality, usability, performance and the identification of defects. The
beta site and all content found on it should not be relied on. Please
refer to asx.com.au for the most up to date content and information."
When I reviewed the new web site I discovered a JSON API that provides
the necessary information, and I have adapted the perl ASX.pm module to
use this API instead. I have only tested the changes on Windows 10 with
GnuCash 4.1.
If anyone is interested, I have attached a copy of the modified perl
module to this post, and have done the same with the github issue:
https://github.com/finance-quote/finance-quote/issues/166
I am totally unfamiliar with the process to get changes made to
Finance::Quote and do not know what (if anything) to do next?
To use this module, all you need to do is:
(1) Locate the Finance::Quote directory in your Perl installation, eg
C:\Strawberry\perl\site\lib\Finance\Quote\
(2) Rename the existing ASX.pm file to something like ASX.pm.ORIGINAL
(3) Put the attached ASX.pm.txt file in the same directory
(4) Rename it to ASX.pm
(5) Test it with the GnuCash gnc-fq-dump utility:
C:\Program Files (x86)\gnucash\bin>perl gnc-fq-dump ASX BHP
Finance::Quote fields Gnucash uses:
symbol: BHP <=== required
date: 10/14/2020 <=== recommended
currency: AUD <=== required
last: 36.1 <=\
nav: <=== one of these
price: 36.1 <=/
timezone: <=== optional
Hope this helps.
Regards
Geoff
=====
On 13/10/2020 9:16 pm, dull...@gmail.com wrote:
Okay I see this is reported here:
https://github.com/finance-quote/finance-quote/issues/166
-----Original Message-----
From: gnucash-user <gnucash-user-bounces+dullege=gmail....@gnucash.org> On
Behalf Of wrd2093au
Sent: Tuesday, 13 October 2020 2:44 PM
To: gnucash-user@gnucash.org
Subject: [GNC] ASX quotes not working
I'm in Australia and suddenly all my stock price updates from ASX are now
failing the past couple of days - anyone experienced this? I have two
different PCs ... both showing same problem. The source for the quotes in
the Security Editor is single "Australian Stock Exchange, AU".
This is version 4.1 gnucash, Finance::quote version 1.49 ... under Windows
10.
--
Sent from: http://gnucash.1415818.n4.nabble.com/GnuCash-User-f1415819.html
_______________________________________________
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
If you are using Nabble or Gmane, please see
https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
-----
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.
_______________________________________________
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
If you are using Nabble or Gmane, please see
https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
-----
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.
#!/usr/bin/perl -w
#
# Copyright (C) 1998, Dj Padzensky <djp...@padz.net>
# Copyright (C) 1998, 1999 Linas Vepstas <li...@linas.org>
# Copyright (C) 2000, Yannick LE NY <y-le...@ifrance.com>
# Copyright (C) 2000, Brent Neal <bre...@users.sourceforge.net>
# Copyright (C) 2001, Leigh Wedding <leigh.wedd...@telstra.com>
# Copyright (C) 2000-2004, Paul Fenwick <p...@cpan.org>
# Copyright (C) 2014, Chris Good <chris.good@@ozemail.com.au>
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA
# 02111-1307, USA
#
#
# This code derived from Padzensky's work on package Finance::YahooQuote,
# but extends its capabilites to encompas a greater number of data sources.
#
# This code was developed as part of GnuCash <http://www.gnucash.org/>
require 5.005;
use strict;
use warnings;
package Finance::Quote::ASX;
use HTTP::Request::Common;
use LWP::UserAgent;
use HTML::TableExtract;
use Encode;
use JSON;
use vars qw/$ASX_URL @ASX_SEC_CODES/;
our $VERSION = '1.49'; # VERSION
$ASX_URL = 'https://www.asx.com.au/asx/1/share/';
# These are the ASX codes starting with X that are securities not indexes
# and thus need currency AUD returned
# See http://www.asx.com.au/asx/research/listedCompanies.do
@ASX_SEC_CODES = (qw/XAM XIP XRO XPD XPE XF1 XRF XST XTD XTE XTV/);
sub methods {return (australia => \&asx,asx => \&asx)}
{
my @labels = qw/last net p_change bid offer high low volume
price method exchange/;
sub labels { return (australia => \@labels,
asx => \@labels); }
}
# Australian Stock Exchange (ASX)
# The ASX provides free delayed quotes through their webpage.
#
# Maintainer of this section is Paul Fenwick <p...@cpan.org>
# 5-May-2001 Updated by Leigh Wedding <leigh.wedd...@telstra.com>
# 24-Feb-2014 Updated by Chris Good <chris.good@@ozemail.com.au>
sub asx {
my $quoter = shift;
my @all_stocks = @_;
return unless @all_stocks;
my @stocks;
my %info;
my $ua = $quoter->user_agent;
# Map the Finance::Quote labels (left) to the corresponding ASX labels
(right)
my %label_map = (
'bid' => 'bid_price',
'cap' => 'market_cap',
'close' => 'previous_close_price',
'date' => 'last_trade_date',
'div_yield' => 'annual_dividend_yield',
'eps' => 'eps',
'high' => 'day_high_price',
'last' => 'last_price',
'low' => 'day_low_price',
'net' => 'change_price',
'offer' => 'offer_price',
'open' => 'open_price',
'p_change' => 'change_in_percent',
'pe' => 'pe',
'volume' => 'volume',
'year_range' => 'year_change_price',
);
# ASX webpage only handles 1 quote requests at a time
while (@stocks = splice(@all_stocks, 0, 1)) {
sleep 1 if $#all_stocks > 0; # Don't hammer the ASX web site
and get blocked!
my $response = $ua->request(GET $ASX_URL.join("%20",@stocks));
unless ($response->is_success) {
foreach my $stock (@stocks, @all_stocks) {
$info{$stock,"success"} = 0;
$info{$stock,"errormsg"} = "HTTP session
failed";
}
return wantarray() ? %info : \%info;
}
my $json = $response->content;
my $decoded_json = eval{decode_json( $json )};
if($@) {
foreach my $stock (@stocks, @all_stocks) {
$info{$stock,"success"} = 0;
$info{$stock,"errormsg"} = "Failed to parse
JSON data, error '$@'.";
}
return wantarray() ? %info : \%info;
}
# Pack the resulting data into our structure.
my $stock = $stocks[0];
# Delete spaces and '*' which sometimes appears after
the code.
# Also delete high bit characters.
$stock =~ tr/* \200-\377//d;
# Delete any whitespace characters
$stock =~ s/\s//g;
$info{$stock,'symbol'} = $stock;
foreach my $label (keys %label_map) {
$info{$stock,$label} =
%$decoded_json{$label_map{$label}};
# Again, get rid of nasty high-bit characters.
$info{$stock,$label} =~ tr/ \200-\377//d
unless ($label eq "name");
}
# get rid of trailing whitespace after 'last'
$info{$stock,'last'} =~ s/\s//g;
# If that stock does not exist, it will have a empty
# string for all the fields. The "last" price should
# always be defined (even if zero), if we see an empty
# string here then we know we've found a bogus stock.
if ($info{$stock,'last'} eq '') {
$info{$stock,'success'} = 0;
$info{$stock,'errormsg'}="Stock does not exist
on ASX.";
next;
}
# Drop commas from volume.
$info{$stock,"volume"} =~ tr/,//d;
# The ASX returns zeros for a number of things if there
# has been no trading. This not only looks silly, but
# can break things later. "correct" zero'd data.
foreach my $label (qw/open high low/) {
if ($info{$stock,$label} == 0) {
$info{$stock,$label} =
$info{$stock,"last"};
}
}
# Remove trailing percentage sign from p_change
$info{$stock,"p_change"} =~ tr/%//d;
## This logic is flawed because the hard coded list of "X" companies is
out of date
## # Australian indexes all begin with X, so don't tag them
## # as having currency info.
##
## $info{$stock, "currency"} = "AUD" unless ($stock =~
/^X/);
##
## # There are some companies starting with X, so DO tag
## # them with currency AUD
##
## if ( grep( /^$stock$/, @ASX_SEC_CODES ) ) {
## $info{$stock, "currency"} = "AUD";
## }
$info{$stock, "currency"} = "AUD";
# Convert Date from 'CCYY-MM-DDThh:mm:ss+tztz' to
Finance::Quote's MM/DD/CCYY format
$info{$stock, "date"} = substr($info{$stock,
"date"},5,2) . '/' .
substr($info{$stock, "date"},8,2) . '/' .
substr($info{$stock, "date"},0,4);
$info{$stock, "method"} = "asx";
$info{$stock, "exchange"} = "Australian Stock Exchange";
$info{$stock, "price"} = $info{$stock,"last"};
$info{$stock, "success"} = 1;
}
# All done.
return %info if wantarray;
return \%info;
}
1;
=head1 NAME
Finance::Quote::ASX - Obtain quotes from the Australian Stock Exchange.
=head1 SYNOPSIS
use Finance::Quote;
$q = Finance::Quote->new;
%stockinfo = $q->fetch("asx","BHP"); # Only query ASX.
%stockinfo = $q->fetch("australia","BHP"); # Failover to other sources OK.
=head1 DESCRIPTION
This module obtains information from the Australian Stock Exchange
http://www.asx.com.au/. All Australian stocks and indicies are
available. Indexes start with the letter 'X'. For example, the
All Ordinaries is "XAO".
This module is loaded by default on a Finance::Quote object. It's
also possible to load it explicity by placing "ASX" in the argument
list to Finance::Quote->new().
This module provides both the "asx" and "australia" fetch methods.
Please use the "australia" fetch method if you wish to have failover
with other sources for Australian stocks (such as Yahoo). Using
the "asx" method will guarantee that your information only comes
from the Australian Stock Exchange.
Information returned by this module is governed by the Australian
Stock Exchange's terms and conditions.
=head1 LABELS RETURNED
The following labels may be returned by Finance::Quote::ASX:
bid, offer, open, high, low, last, net, p_change, volume,
and price.
=head1 SEE ALSO
Australian Stock Exchange, http://www.asx.com.au/
Finance::Quote::Yahoo::Australia.
=cut
43d42
< use JSON;
49c48
< $ASX_URL = 'https://www.asx.com.au/asx/1/share/';
---
> $ASX_URL =
> 'http://www.asx.com.au/asx/markets/priceLookup.do?by=asxCodes&asxCodes=';
82,105c81
< # Map the Finance::Quote labels (left) to the corresponding ASX labels
(right)
< my %label_map = (
< 'bid' => 'bid_price',
< 'cap' => 'market_cap',
< 'close' => 'previous_close_price',
< 'date' => 'last_trade_date',
< 'div_yield' => 'annual_dividend_yield',
< 'eps' => 'eps',
< 'high' => 'day_high_price',
< 'last' => 'last_price',
< 'low' => 'day_low_price',
< 'net' => 'change_price',
< 'offer' => 'offer_price',
< 'open' => 'open_price',
< 'p_change' => 'change_in_percent',
< 'pe' => 'pe',
< 'volume' => 'volume',
< 'year_range' => 'year_change_price',
< );
<
< # ASX webpage only handles 1 quote requests at a time
< while (@stocks = splice(@all_stocks, 0, 1)) {
< sleep 1 if $#all_stocks > 0; # Don't hammer the ASX web site
and get blocked!
< my $response = $ua->request(GET $ASX_URL.join("%20",@stocks));
---
> # ASX webpage only handles up to 10 quote requests at once
106a83,84
> while (@stocks = splice(@all_stocks, 0, 10)) {
> my $response = $ua->request(GET $ASX_URL.join("%20",@stocks));
115,117c93,102
< my $json = $response->content;
< my $decoded_json = eval{decode_json( $json )};
< if($@) {
---
> my $te = HTML::TableExtract->new(
> automap => 0,
> headers => ["Code", "Last", '\+/-', "% Chg", "Bid",
> "Offer", "Open", "High", "Low", "Volume"]);
>
> $te->parse(decode('utf-8',$response->content));
>
> # Extract table contents.
> my @rows;
> unless (($te->tables > 0) && ( @rows = $te->rows)) {
120c105
< $info{$stock,"errormsg"} = "Failed to parse
JSON data, error '$@'.";
---
> $info{$stock,"errormsg"} = "Failed to parse
> HTML table.";
126c111,115
< my $stock = $stocks[0];
---
> foreach my $row (@rows) {
> my $stock = shift(@$row);
>
> # Skip any blank lines.
> next unless $stock;
136,138c125,128
<
< foreach my $label (keys %label_map) {
< $info{$stock,$label} =
%$decoded_json{$label_map{$label}};
---
>
> foreach my $label (qw/last net p_change bid offer open
> high low volume/) {
> $info{$stock,$label} = shift(@$row);
175,192c165,175
< ## This logic is flawed because the hard coded list of "X" companies is
out of date
< ## # Australian indexes all begin with X, so don't tag them
< ## # as having currency info.
< ##
< ## $info{$stock, "currency"} = "AUD" unless ($stock =~
/^X/);
< ##
< ## # There are some companies starting with X, so DO tag
< ## # them with currency AUD
< ##
< ## if ( grep( /^$stock$/, @ASX_SEC_CODES ) ) {
< ## $info{$stock, "currency"} = "AUD";
< ## }
< $info{$stock, "currency"} = "AUD";
<
< # Convert Date from 'CCYY-MM-DDThh:mm:ss+tztz' to
Finance::Quote's MM/DD/CCYY format
< $info{$stock, "date"} = substr($info{$stock,
"date"},5,2) . '/' .
<
substr($info{$stock, "date"},8,2) . '/' .
<
substr($info{$stock, "date"},0,4);
---
> # Australian indexes all begin with X, so don't tag them
> # as having currency info.
>
> $info{$stock, "currency"} = "AUD" unless ($stock =~
> /^X/);
>
> # There are some companies starting with X, so DO tag
> # them with currency AUD
>
> if ( grep( /^$stock$/, @ASX_SEC_CODES ) ) {
> $info{$stock, "currency"} = "AUD";
> }
198c181
<
---
> }
_______________________________________________
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
If you are using Nabble or Gmane, please see
https://wiki.gnucash.org/wiki/Mailing_Lists for more information.
-----
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.
