I agree with your conclusions. I learned a long time ago not to waste too much energy chasing WU flaws. There are many, and they come and go. Good way to frustrate yourself...😞
On Mon, Feb 3, 2020 at 4:32 PM Leon Shaner <[email protected]> wrote: > Tom, All, > > On that WU forum, an official rep said the rtupdate and weatherstation > URL's go the same place. > He said it shouldn't matter which one. So we can converge on a single URL. > > This is down to same old same old "WU is a steaming pile" being the reason > for the 404's. > > Regards, > \Leon > -- > Leon Shaner :: Dearborn, Michigan (iPad) > > On Feb 3, 2020, at 3:35 PM, Leon Shaner <[email protected]> wrote: > >  Hey, WeeWx'ers. > > Even my workaround of setting pws_url to the value of the rf_url is still > getting 404's, even if I explicitly set a Host header. :-((((( > > This is what I have tried... I guess WU is just a steaming pile as per > usual. :-/ > > Here is my attempted fix, which is part hack. > The "fix" is adding the header. The "hack" is forcing the code to use > the rtupdate URL (because I couldn't spot why the PWS URL was being used > for RF). > > $ diff restx.py restx.py_getting404 > 100d99 > < from urllib.parse import urlparse > 454,458c453 > < _parseduri = urlparse(url) > < _parsedhost = '{uri.netloc}'.format(uri=_parseduri) > < _request.add_header("Host", "%s" % _parsedhost) > < # log.debug("%s: Added Header: 'Host: %s'" > < # % (self.protocol_name, _parsedhost)) > --- > > _request.add_header("User-Agent", "weewx/%s" % weewx.__version__) > 610,611c605 > < #pws_url = > "https://weatherstation.wunderground.com/weatherstation/updateweatherstation.php"; > <https://weatherstation.wunderground.com/weatherstation/updateweatherstation.php> > < pws_url = > "https://rtupdate.wunderground.com/weatherstation/updateweatherstation.php"; > <https://rtupdate.wunderground.com/weatherstation/updateweatherstation.php> > --- > > pws_url = > "https://weatherstation.wunderground.com/weatherstation/updateweatherstation.php"; > <https://weatherstation.wunderground.com/weatherstation/updateweatherstation.php> > > Or in case your line numbers don't match mine (likely), here it is in > context: > > ... > # Python 2/3 compatiblity shims > import six > from six.moves import http_client > from six.moves import queue > from six.moves import urllib > from urllib.parse import urlparse > > ... > > def get_request(self, url): > """Get a request object. This can be overridden to add any special > headers.""" > _request = urllib.request.Request(url) > _parseduri = urlparse(url) > _parsedhost = '{uri.netloc}'.format(uri=_parseduri) > _request.add_header("Host", "%s" % _parsedhost) > # log.debug("%s: Added Header: 'Host: %s'" > # % (self.protocol_name, _parsedhost)) > _request.add_header("User-Agent", "weewx/%s" % weewx.__version__) > return _request > > > ... > > # the rapidfire URL: > rf_url = > "https://rtupdate.wunderground.com/weatherstation/updateweatherstation.php"; > <https://rtupdate.wunderground.com/weatherstation/updateweatherstation.php> > # the personal weather station URL: > #pws_url = > "https://weatherstation.wunderground.com/weatherstation/updateweatherstation.php"; > <https://weatherstation.wunderground.com/weatherstation/updateweatherstation.php> > pws_url = > "https://rtupdate.wunderground.com/weatherstation/updateweatherstation.php"; > <https://rtupdate.wunderground.com/weatherstation/updateweatherstation.php> > > > > > [email protected] - Dearborn, Michigan > > On 2/3/20 2:43 PM, Leon Shaner wrote: > > Tom, > > Looking at the code, under the do_rapidfire_post, we clearly see it uses > rf_url. In my debug output I referenced protocol_name and it's showing > Wunderground-RF as the protocol, but the hostname is from pws_url. > I have temporarily set pws_url to the rtupdate URL and my 404's have > stopped. > Looks like the rtupdate URL will take regular posts, but the > weatherstation URL won't take rf posts. :-/ > > My debug with pws_url replaced with the rf_url value now shows: > > Feb 3 14:36:30 nixie weewx[8747] DEBUG weewx.restx: Wunderground-RF: > Added Header: 'Host: rtupdate.wunderground.com' > > That's what it should have shown if rf_url was getting used for > do_rapidfire_post. > I'm stumped. :-( > > Regards, > \Leon > > [email protected] - Dearborn, Michigan > > On 2/3/20 2:23 PM, Leon Shaner wrote: > > Brice, > > Thanks! That's interesting... I just added an explicit Host header to > the rtupdate request and put debug output there. > This was unexpected: > > Feb 3 14:16:34 nixie weewx[8584] DEBUG weewx.restx: Wunderground-RF: > Added Header: 'Host: *weatherstation.wunderground.com > <http://weatherstation.wunderground.com>*' > > That isn't the host that should be used for rapidfire, but it's taken from > the URL being used. :-( > We have: > > $ grep wunderground.com restx.py > rf_url = > "https://rtupdate.wunderground.com/weatherstation/updateweatherstation.php"; > <https://rtupdate.wunderground.com/weatherstation/updateweatherstation.php> > pws_url = > "https://weatherstation.wunderground.com/weatherstation/updateweatherstation.php"; > <https://weatherstation.wunderground.com/weatherstation/updateweatherstation.php> > > It will take me some time to figure out why restx.py is using the wrong > URL. > Must have been doing this for a long time, but seems that due to changes > on the WU side, they no longer accept rapidfire updates to the > weatherstation.wunderground.com host. > > Regards, > \Leon > > [email protected] - Dearborn, Michigan > > On 2/3/20 12:59 PM, Brice Ruth wrote: > > header would be something like `Host: rtupdate.wunderground.com` ... > basically, whatever you're using as the hostname in the URL, needs to be > sent as the `Host` header ... that allows load-balancers to route the > request to the correct backend. > > Brice Ruth, FCD > Software Engineer, Madison WI > > > On Mon, Feb 3, 2020 at 11:44 AM Leon Shaner <[email protected]> wrote: > >> Hey, WeeWx'ers, >> >> I am still getting 404 errors from the rtupdate URL site. >> There is a reply over here about it. >> I'm now looking into what the headers need to be. >> Does anyone know? >> >> >> https://apicommunity.wunderground.com/weatherapi/topics/is-the-rtupdate-wunderground-com-address-no-longer-valid >> >> For requests to route properly, the >> host header must be set correctly >> path must be correct >> >> >> Regards, >> \Leon >> -- >> Leon Shaner :: Dearborn, Michigan (iPad) >> >> On Feb 1, 2020, at 12:39 PM, Leon Shaner <[email protected]> wrote: >> >> Hey, WeeWx'ers. >> >> I took a different approach to working around the WU certificate issue -- >> keep SSL but don't verify the CERT. Scroll for that "solution." >> >> But meanwhile, using Wunderground-RF (rapid fire) I am getting an HTTP >> Error 404: Not Found. >> Any ideas on that? Did they change the URL, or is it just down? >> >> I noticed that even their own WunderStation App on iOS/iPadOS is subject >> to the certificate issue and reports all stations as "not reporting" -- >> even ones that are, like mine, which I can verify via the WU website has >> the data and has reported within minutes ago. >> >> From the logs, re: RapidFire HTTP 404: >> >> Feb 1 12:31:51 nixie weewx[13184] DEBUG weewx.restx: Wunderground-RF: >> Failed upload attempt 1: HTTP Error 404: Not Found >> Feb 1 12:31:51 nixie weewx[13184] ERROR weewx.restx: Wunderground-RF: >> Failed to publish record 2020-02-01 12:31:51 EST (1580578311): Failed >> upload after 1 tries >> Feb 1 12:31:56 nixie weewx[13184] DEBUG weewx.restx: Wunderground-RF: >> Failed upload attempt 1: HTTP Error 404: Not Found >> Feb 1 12:31:56 nixie weewx[13184] ERROR weewx.restx: Wunderground-RF: >> Failed to publish record 2020-02-01 12:31:56 EST (1580578316): Failed >> upload after 1 tries >> Feb 1 12:32:04 nixie weewx[13184] DEBUG weewx.restx: Wunderground-RF: >> Failed upload attempt 1: HTTP Error 404: Not Found >> Feb 1 12:32:04 nixie weewx[13184] ERROR weewx.restx: Wunderground-RF: >> Failed to publish record 2020-02-01 12:32:04 EST (1580578324): Failed >> upload after 1 tries >> Feb 1 12:32:05 nixie weewx[13184] DEBUG weewx.restx: Wunderground-RF: >> Failed upload attempt 1: HTTP Error 404: Not Found >> Feb 1 12:32:05 nixie weewx[13184] ERROR weewx.restx: Wunderground-RF: >> Failed to publish record 2020-02-01 12:32:05 EST (1580578325): Failed >> upload after 1 tries >> Feb 1 12:32:06 nixie weewx[13184] DEBUG weewx.restx: Wunderground-RF: >> Failed upload attempt 1: HTTP Error 404: Not Found >> Feb 1 12:32:06 nixie weewx[13184] ERROR weewx.restx: Wunderground-RF: >> Failed to publish record 2020-02-01 12:32:05 EST (1580578325): Failed >> upload after 1 tries >> >> >> Here is the diff for my SSL CERT workaround: >> >> $ diff restx.py restx.py.20200201.1 >> 110,115d109 >> < # SSL certificate hack >> < global WUssl >> < WUssl = ssl.create_default_context(); >> < WUssl.check_hostname=False >> < WUssl.verify_mode=ssl.CERT_NONE >> < >> 454d447 >> < _request.add_header("User-Agent", "weewx/%s" % >> weewx.__version__) >> 543,544c536 >> < # _response = urllib.request.urlopen(request, data=data_bytes, >> timeout=self.timeout) >> < _response = urllib.request.urlopen(request, data=data_bytes, >> timeout=self.timeout, context=WUssl) >> --- >> > _response = urllib.request.urlopen(request, data=data_bytes, >> timeout=self.timeout) >> 1051,1052c1043 >> < # _response = urllib.request.urlopen(request, >> timeout=self.timeout) >> < _response = urllib.request.urlopen(request, >> timeout=self.timeout, context=WUssl) >> --- >> > _response = urllib.request.urlopen(request, >> timeout=self.timeout) >> >> >> Regards, >> \Leon >> -- >> Leon Shaner :: Dearborn, Michigan (iPad) >> >> On Jan 31, 2020, at 4:16 PM, Thomas Keffer <[email protected]> wrote: >> >>  >> Travis: what is the error? If it's a certificate error, Version 4 will do >> a retry after an hour. Unfortunately, Version 3 does not. >> >> On Fri, Jan 31, 2020 at 1:04 PM Travis Bully <[email protected]> wrote: >> >>> Not here yet. It'll work for awhile and then will get the random cert >>> error I sent earlier. A restart of weewx will get it going again. I wish >>> restx would just try again after xx seconds. >>> >>> >>> >>> On Friday, January 31, 2020 at 2:48:50 PM UTC-5, J B wrote: >>>> >>>> Working here too. I had to restart Weewx. >>>> >>>> On Friday, January 31, 2020 at 11:34:19 AM UTC-8, Brice Ruth wrote: >>>>> >>>>> Looking like it's working to me, too. >>>>> >>>>> Brice Ruth, FCD >>>>> Software Engineer, Madison WI >>>>> >>>>> >>>>> On Fri, Jan 31, 2020 at 1:31 PM Travis Bully <[email protected]> wrote: >>>>> >>>>>> Agreed. Seems like I still get one cert error every now and then. >>>>>> Likely still propagating changes through their infrastructure? >>>>>> >>>>>> Jan 31 14:29:25 homeauto03 weewx[3784]: restx: Wunderground-RF: >>>>>> Published record 2020-01-31 14:29:24 EST (1580498964) >>>>>> Jan 31 14:29:26 homeauto03 weewx[3784]: restx: Wunderground-RF: >>>>>> Failed upload attempt 1: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] >>>>>> certificate verify failed (_ssl.c:727)> >>>>>> Jan 31 14:29:31 homeauto03 weewx[3784]: restx: Wunderground-RF: >>>>>> Failed to publish record 2020-01-31 14:29:26 EST (1580498966): Failed >>>>>> upload after 1 tries >>>>>> Jan 31 14:29:31 homeauto03 weewx[3784]: restx: Wunderground-RF: >>>>>> Published record 2020-01-31 14:29:30 EST (1580498970) >>>>>> >>>>>> On Fri, Jan 31, 2020 at 2:30 PM Denny Page <[email protected]> wrote: >>>>>> >>>>>>> It's fixed now. >>>>>>> >>>>>>> >>>>>>> On Friday, January 31, 2020 at 8:54:02 AM UTC-8, Denny Page wrote: >>>>>>>> >>>>>>>> Wunderground just posted a note about the intermediate certificate >>>>>>>> issue. Hopefully they will fix it shortly. >>>>>>>> >>>>>>> -- >>>>>>> You received this message because you are subscribed to a topic in >>>>>>> the Google Groups "weewx-user" group. >>>>>>> To unsubscribe from this topic, visit >>>>>>> https://groups.google.com/d/topic/weewx-user/mYhw6CSKHHg/unsubscribe >>>>>>> . >>>>>>> To unsubscribe from this group and all its topics, send an email to >>>>>>> [email protected]. >>>>>>> To view this discussion on the web visit >>>>>>> https://groups.google.com/d/msgid/weewx-user/f5e5741a-e5f8-419b-9c0d-df1972fe5ced%40googlegroups.com >>>>>>> <https://groups.google.com/d/msgid/weewx-user/f5e5741a-e5f8-419b-9c0d-df1972fe5ced%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>> . >>>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "weewx-user" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> To view this discussion on the web visit >>>>>> https://groups.google.com/d/msgid/weewx-user/CAF_%2Bh57S88%3DZOX1mXK8oZBx-_KEHvDPFP%3Dvtd4u3_1qg7994yg%40mail.gmail.com >>>>>> <https://groups.google.com/d/msgid/weewx-user/CAF_%2Bh57S88%3DZOX1mXK8oZBx-_KEHvDPFP%3Dvtd4u3_1qg7994yg%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>> . >>>>>> >>>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "weewx-user" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/weewx-user/eaa42f60-98fc-4974-8d25-915a4c7e0d80%40googlegroups.com >>> <https://groups.google.com/d/msgid/weewx-user/eaa42f60-98fc-4974-8d25-915a4c7e0d80%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "weewx-user" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/weewx-user/CAPq0zECD1_-w%3D8V5eb5-qYsgMBPcvnNunAOe%2BQbnqpKsOsyGUg%40mail.gmail.com >> <https://groups.google.com/d/msgid/weewx-user/CAPq0zECD1_-w%3D8V5eb5-qYsgMBPcvnNunAOe%2BQbnqpKsOsyGUg%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> >> -- >> You received this message because you are subscribed to the Google Groups >> "weewx-user" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/weewx-user/CE8C91F5-D61F-4618-A317-7E97B9ECE912%40isylum.org >> <https://groups.google.com/d/msgid/weewx-user/CE8C91F5-D61F-4618-A317-7E97B9ECE912%40isylum.org?utm_medium=email&utm_source=footer> >> . >> > -- > You received this message because you are subscribed to the Google Groups > "weewx-user" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/weewx-user/CAFbExW60hmrJRd1yWJAK8f0H%2BXnZsnYe8%3DxMpvKe9n2ws4sq9w%40mail.gmail.com > <https://groups.google.com/d/msgid/weewx-user/CAFbExW60hmrJRd1yWJAK8f0H%2BXnZsnYe8%3DxMpvKe9n2ws4sq9w%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > > > -- > You received this message because you are subscribed to the Google Groups > "weewx-user" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/weewx-user/6438f7e3-b03f-4ea3-e324-d89640c88e09%40isylum.org > <https://groups.google.com/d/msgid/weewx-user/6438f7e3-b03f-4ea3-e324-d89640c88e09%40isylum.org?utm_medium=email&utm_source=footer> > . > > > -- > You received this message because you are subscribed to the Google Groups > "weewx-user" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/weewx-user/c017d408-7abb-695c-8968-9058e22b3389%40isylum.org > <https://groups.google.com/d/msgid/weewx-user/c017d408-7abb-695c-8968-9058e22b3389%40isylum.org?utm_medium=email&utm_source=footer> > . > > > -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/CAPq0zEC1jQc_6ek00S6ojqatzjhP7M%3DD6tEtXRGB%2BmAajAv8PQ%40mail.gmail.com.
