I forgot to mention that the AFM format is correct and the extension will handle it (in this particular case for MDZ005 and for MDZ006) if I force the scanner/parser to ignore the 'Including' line. I'm pretty sure that 'cheat' will fail though on MDZ013 and MDZ014 in that same dataset. Perhaps need to check the line splits to 7 tokens and the first token is a numeric value or a numeric value less than 1300 before passing it to date2ts() and not worry about first token is 'Including'?
Mike/AJ9X On Thursday, October 8, 2020 at 1:49:03 PM UTC-4 Michael Bruski wrote: > > I attached a text file containing one of the latest results from URL > https://forecast.weather.gov/product.php?site=NWS&product=AFM&format=txt&issuedby=LWX > . > > These are the options I have configured in my weewx.conf file: > > # Options for extension 'forecast' > [Forecast] > data_binding = forecast_binding > [[XTide]] > location = "INSERT_LOCATION_HERE (e.g., Boston)" > [[Zambretti]] > hemisphere = NORTH > [[NWS]] > lid = "MDZ005" > foid = "LWX" > url = > http://forecast.weather.gov/product.php?site=NWS&product=AFM&format=txt&issuedby=LWX > [[WU]] > api_key = INSERT_WU_API_KEY_HERE > [[OWM]] > api_key = INSERT_OWM_API_KEY_HERE > [[UKMO]] > api_key = INSERT_UKMO_API_KEY_HERE > location = INSERT_UK_LOCATION_HERE > [[Aeris]] > client_id = INSERT_AERIS_CLIENT_ID_HERE > client_secret = INSERT_AERIS_CLIENT_SECRET_HERE > [[WWO]] > api_key = INSERT_WWO_API_KEY_HERE > [[DS]] > api_key = INSERT_DS_API_KEY_HERE > > I appreciate any effort you put into resolving this. It will be nice to > see the extension handle these instead of silently failing. > > Also, a thank you to the other group member who supplied a link to the > AFM_format.pdf document. I mentioned to Chris at NWS Balt/Wash this > morning that it is difficult to find any specifications for the AFM/PFM on > the NWS web site. Perhaps that will get addressed in the future as well. > > Mike/AJ9X > > On Thursday, October 8, 2020 at 11:24:50 AM UTC-4 jo...@johnkline.com > wrote: > >> This is all very helpful. >> >> Would you please send your NWS configuration for the problem AFM report >> as well as sample output from the URL you specified that the extension >> chokes on? >> >> Have you determined if the output is valid as specified by the AFM format >> doc? >> >> I’ll add proper support for AFM in the coming week or so. >> >> Cheers, >> John >> >> On Oct 8, 2020, at 6:08 AM, Michael Bruski <michael...@gmail.com> wrote: >> >> >> >> I queried the NWS Baltimore/Washington D.D. forecast office about the >> differences between AFM and PFM and received this reply: >> >> Hello, >> >> Thanks for writing. The AFMs and PFMs are set to a national >> standard. So they should be the same across all the forecast offices. >> >> Our Directive that talks about these products, as well as links to >> format decoding, are in this NWS Directive >> <https://www.nws.noaa.gov/directives/sym/pd01005003curr.pdf> between >> pages 10-12. >> >> Hope that helps. >> >> Chris Strong >> NWS Baltimore/Washington >> >> The reference Chris sites contains links to AFM/PFM format PDFs and the >> element specifications PDF for both reports. The only difference between >> the two that I could detect was in lines 2 and 3 of the report (Description >> and Location in the PFM; zone and included cities in the AFM). >> >> On Wednesday, October 7, 2020 at 8:28:01 PM UTC-4 etji...@gmail.com >> wrote: >> >>> Here is the spec on AFMs from the NWS: >>> https://www.weather.gov/media/notification/dir/AFM_Format.pdf >>> >>> >>> On Wed, Oct 7, 2020 at 5:57 PM John Kline <jo...@johnkline.com> wrote: >>> >>>> Absent a spec on area forecast matrices, and absent any problem reports >>>> on point forecast matrices, there isn’t much to go on here. >>>> >>>> On Oct 7, 2020, at 2:35 PM, Michael Bruski <michael...@gmail.com> >>>> wrote: >>>> >>>> >>>> Hi John, >>>> >>>> I've looked at several forecast matrices from my local and surrounding >>>> NWS forecast offices and haven't seen anything format wise that would lead >>>> me to think they would cause a problem for the forecast extension. I have >>>> selected several different AFMs and see they parse fine. But I ran into >>>> this issue with my local AFM and spent several hours trying to track down >>>> the cause. The issue is not with the matrix itself but with text >>>> preceeding the DTG. To test further, I put a cheap hack into the parse >>>> loop so that date2ts() isn't called if the first word in 'line' is >>>> 'Including'. This actually works (in this specific instance) and a valid >>>> forecast is produced. What other lines of text might NWS put in the >>>> header >>>> preceeding the DTG, I just don't know. The URLs mentioned in >>>> forecast.py >>>> are dead links so I couldn't find any specifics on decoding the reports. >>>> >>>> I'm not terribly comfortable programming in python so I'm not sure the >>>> best approach to work this issue but sending bad data to time.mktime() via >>>> date2ts() is probably bad unless there is a way to trap the error. Not >>>> meant to criticize... just my thoughts on how to make it better. >>>> >>>> Mike/AJ9X >>>> >>>> On Wednesday, October 7, 2020 at 4:24:56 PM UTC-4 jo...@johnkline.com >>>> wrote: >>>> >>>>> Perhaps Area Forecast Matrix responses can’t be treated just like >>>>> Point Forecast Matrix responses. AFMs are only supported to the extent >>>>> that they provide an identically formatted response as the PFM response. >>>>> >>>>> What do you know about AFMs and the differences between them and PFMs? >>>>> >>>>> On Oct 7, 2020, at 12:18 PM, Michael Bruski <michael...@gmail.com> >>>>> wrote: >>>>> >>>>> >>>>> >>>>> >>>>> Hi all, >>>>> >>>>> First of all, I am running WeeWX 4.1.1 with forecast extension >>>>> 3.4.0b10. >>>>> >>>>> I had recently managed to use a Area Forecast Matrix for one location >>>>> and that is working fine. Recently I tried to setup a second location >>>>> to >>>>> use the AFM (different LID and FOID) and it fails to get a forecast even >>>>> though I see it in the web browser using the URL that forecast.py uses. >>>>> After careful application of some trace print statements, I see the >>>>> forecast is succesfully download from the FOID and that the forecast for >>>>> the specified LID is also located and extracted but nothing gets parsed. >>>>> Below is a copy of my trace output from the log: >>>>> >>>>> Oct 7 14:36:45 server weewx[3014] INFO user.forecast: NWSThread: NWS: >>>>> forecast downloaded successfully from ' >>>>> http://forecast.weather.gov/product.php?site=NWS&product=AFM&format=txt&issuedby=LWX >>>>> ' >>>>> Oct 7 14:36:45 server weewx[3014] INFO user.forecast: NWSThread: NWS: >>>>> 'MDZ005-072100-' >>>>> Oct 7 14:36:45 server weewx[3014] INFO user.forecast: NWSThread: NWS: >>>>> 'Carroll-' >>>>> Oct 7 14:36:45 server weewx[3014] INFO user.forecast: NWSThread: NWS: >>>>> 'Including the cities of Eldersburg and Westminster' >>>>> Oct 7 14:36:45 server weewx[3014] INFO user.forecast: NWSThread: NWS: >>>>> '137 PM EDT Wed Oct 7 2020' >>>>> Oct 7 14:36:45 server weewx[3014] INFO user.forecast: NWSThread: NWS: >>>>> '' >>>>> Oct 7 14:36:45 server weewx[3014] INFO user.forecast: NWSThread: NWS: >>>>> 'Date Wed 10/07/20 Thu 10/08/20 Fri >>>>> 10/09/20' >>>>> Oct 7 14:36:45 server weewx[3014] INFO user.forecast: NWSThread: NWS: >>>>> 'EDT 3hrly 05 08 11 14 17 20 23 02 05 08 11 14 17 20 23 02 05 08 11 >>>>> 14 >>>>> 17 20' >>>>> Oct 7 14:36:45 server weewx[3014] INFO user.forecast: NWSThread: NWS: >>>>> 'UTC 3hrly 09 12 15 18 21 00 03 06 09 12 15 18 21 00 03 06 09 12 15 >>>>> 18 >>>>> 21 00' >>>>> Oct 7 14:36:45 server weewx[3014] INFO user.forecast: NWSThread: NWS: >>>>> '' >>>>> Oct 7 14:36:45 server weewx[3014] INFO user.forecast: NWSThread: NWS: >>>>> 'Max/Min 75 77 80 47 48 52 65 67 70 42 44 48 >>>>> 68 >>>>> 70 73' >>>>> Oct 7 14:36:45 server weewx[3014] INFO user.forecast: NWSThread: NWS: >>>>> 'Temp 76 76 68 61 56 51 50 60 65 64 55 50 47 45 45 61 >>>>> 68 >>>>> 67 59' >>>>> Oct 7 14:36:45 server weewx[3014] INFO user.forecast: NWSThread: NWS: >>>>> 'Dewpt 52 52 53 50 48 46 45 45 43 42 43 42 41 40 41 45 >>>>> 45 >>>>> 46 47' >>>>> Oct 7 14:36:45 server weewx[3014] INFO user.forecast: NWSThread: NWS: >>>>> 'RH 43 43 59 67 74 83 83 58 45 45 64 74 79 82 86 56 >>>>> 43 >>>>> 47 64' >>>>> Oct 7 14:36:45 server weewx[3014] INFO user.forecast: NWSThread: NWS: >>>>> 'Wind dir W W NW NW NW NW NW NW NW NW NW NW W W W SW >>>>> SW >>>>> SW S' >>>>> Oct 7 14:36:45 server weewx[3014] INFO user.forecast: NWSThread: NWS: >>>>> 'Wind spd 19 17 10 6 5 5 6 11 12 10 5 3 1 2 1 4 >>>>> 6 >>>>> 8 4' >>>>> Oct 7 14:36:45 server weewx[3014] INFO user.forecast: NWSThread: NWS: >>>>> 'Wind gust 33 31' >>>>> Oct 7 14:36:45 server weewx[3014] INFO user.forecast: NWSThread: NWS: >>>>> 'Clouds CL CL SC FW FW CL CL FW CL CL CL CL FW FW FW FW >>>>> FW >>>>> FW SC' >>>>> Oct 7 14:36:45 server weewx[3014] INFO user.forecast: NWSThread: NWS: >>>>> 'PoP 12hr 20 20 0 >>>>> 0 0' >>>>> Oct 7 14:36:45 server weewx[3014] INFO user.forecast: NWSThread: NWS: >>>>> 'QPF 12hr 0 0 0 >>>>> 0 0' >>>>> Oct 7 14:36:45 server weewx[3014] INFO user.forecast: NWSThread: NWS: >>>>> 'Rain shwrs S S' >>>>> Oct 7 14:36:45 server weewx[3014] INFO user.forecast: NWSThread: NWS: >>>>> '' >>>>> Oct 7 14:36:45 server weewx[3014] INFO user.forecast: NWSThread: NWS: >>>>> '' >>>>> Oct 7 14:36:45 server weewx[3014] INFO user.forecast: NWSThread: NWS: >>>>> 'Date Sat 10/10/20 Sun 10/11/20 Mon 10/12/20 Tue 10/13/20' >>>>> Oct 7 14:36:45 server weewx[3014] INFO user.forecast: NWSThread: NWS: >>>>> 'EDT 6hrly 02 08 14 20 02 08 14 20 02 08 14 20 02 08 14 20' >>>>> Oct 7 14:36:45 server weewx[3014] INFO user.forecast: NWSThread: NWS: >>>>> 'UTC 6hrly 06 12 18 00 06 12 18 00 06 12 18 00 06 12 18 00' >>>>> Oct 7 14:36:45 server weewx[3014] INFO user.forecast: NWSThread: NWS: >>>>> '' >>>>> Oct 7 14:36:45 server weewx[3014] INFO user.forecast: NWSThread: NWS: >>>>> 'Min/Max 54 75 56 69 53 66 54 68' >>>>> Oct 7 14:36:45 server weewx[3014] INFO user.forecast: NWSThread: NWS: >>>>> 'Temp 55 56 73 66 61 58 66 60 56 55 63 59 56 56 65 61' >>>>> Oct 7 14:36:45 server weewx[3014] INFO user.forecast: NWSThread: NWS: >>>>> 'Dewpt 50 54 59 60 59 56 55 54 54 53 56 56 56 56 58 57' >>>>> Oct 7 14:36:45 server weewx[3014] INFO user.forecast: NWSThread: NWS: >>>>> 'PWind dir S SW SW SE E E E SE' >>>>> Oct 7 14:36:45 server weewx[3014] INFO user.forecast: NWSThread: NWS: >>>>> 'Wind char LT GN LT LT GN GN LT LT' >>>>> Oct 7 14:36:45 server weewx[3014] INFO user.forecast: NWSThread: NWS: >>>>> 'Avg clouds SC B1 B1 B1 B1 B1 B1 B1 B2 B2 B2 B2 B2 B2 B2 B1' >>>>> Oct 7 14:36:45 server weewx[3014] INFO user.forecast: NWSThread: NWS: >>>>> 'PoP 12hr 5 20 30 30 40 40 30 30' >>>>> Oct 7 14:36:45 server weewx[3014] INFO user.forecast: NWSThread: NWS: >>>>> 'Rain shwrs S C C C C C C C C C C C C' >>>>> Oct 7 14:36:45 server weewx[3014] INFO user.forecast: NWSThread: NWS: >>>>> '' >>>>> Oct 7 14:36:45 server weewx[3014] INFO user.forecast: NWSThread: NWS: >>>>> '$$' >>>>> Oct 7 14:36:45 server weewx[3014] INFO user.forecast: NWSThread: NWS: >>>>> successfully extracted forecast for 'MDZ005' >>>>> >>>>> Examining NWSParseForecast, nothing is parsed until the Date/Time >>>>> Group (a line with 7 tokens) is located. In the above AFM another line >>>>> preceeds the DTG which also happens to have 7 tokens. I think this line >>>>> is >>>>> being passed to date2ts() which in turn fails and causes the parser to >>>>> stop >>>>> looking for further input. I didn't see a way to raise the issue on >>>>> GitHub >>>>> so I thought I would bring this up here. >>>>> >>>>> Mike/AJ9X >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "weewx-user" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to weewx-user+...@googlegroups.com. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/weewx-user/05fa293c-659c-4619-9b06-9ed97a30efa4o%40googlegroups.com >>>>> >>>>> <https://groups.google.com/d/msgid/weewx-user/05fa293c-659c-4619-9b06-9ed97a30efa4o%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "weewx-user" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to weewx-user+...@googlegroups.com. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/weewx-user/3936dc80-0d29-414f-a5b9-5f286bd659b1n%40googlegroups.com >>>> >>>> <https://groups.google.com/d/msgid/weewx-user/3936dc80-0d29-414f-a5b9-5f286bd659b1n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "weewx-user" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to weewx-user+...@googlegroups.com. >>>> >>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/weewx-user/85F7E477-9A8C-4AF3-8713-4A121E5AE3A3%40johnkline.com >>>> >>>> <https://groups.google.com/d/msgid/weewx-user/85F7E477-9A8C-4AF3-8713-4A121E5AE3A3%40johnkline.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "weewx-user" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to weewx-user+...@googlegroups.com. >> >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/weewx-user/05c0fc4d-63d1-4e42-acc5-590605ae924en%40googlegroups.com >> >> <https://groups.google.com/d/msgid/weewx-user/05c0fc4d-63d1-4e42-acc5-590605ae924en%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> >> -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/8e330107-4462-4d0c-ab87-ce12f9760f81n%40googlegroups.com.
