Tom, All, On that WU forum, an official rep said the rtupdate and weatherstation URL's go the same place. He said it shouldn't matter which one. So we can converge on a single URL.
This is down to same old same old "WU is a steaming pile" being the reason for the 404's. Regards, \Leon -- Leon Shaner :: Dearborn, Michigan (iPad) > On Feb 3, 2020, at 3:35 PM, Leon Shaner <[email protected]> wrote: > > Hey, WeeWx'ers. > > Even my workaround of setting pws_url to the value of the rf_url is still > getting 404's, even if I explicitly set a Host header. :-((((( > > This is what I have tried... I guess WU is just a steaming pile as per > usual. :-/ > > Here is my attempted fix, which is part hack. > The "fix" is adding the header. The "hack" is forcing the code to use the > rtupdate URL (because I couldn't spot why the PWS URL was being used for RF). > > $ diff restx.py restx.py_getting404 > 100d99 > < from urllib.parse import urlparse > 454,458c453 > < _parseduri = urlparse(url) > < _parsedhost = '{uri.netloc}'.format(uri=_parseduri) > < _request.add_header("Host", "%s" % _parsedhost) > < # log.debug("%s: Added Header: 'Host: %s'" > < # % (self.protocol_name, _parsedhost)) > --- > > _request.add_header("User-Agent", "weewx/%s" % weewx.__version__) > 610,611c605 > < #pws_url = > "https://weatherstation.wunderground.com/weatherstation/updateweatherstation.php"; > < pws_url = > "https://rtupdate.wunderground.com/weatherstation/updateweatherstation.php"; > --- > > pws_url = > > "https://weatherstation.wunderground.com/weatherstation/updateweatherstation.php"; > > Or in case your line numbers don't match mine (likely), here it is in context: > ... > # Python 2/3 compatiblity shims > import six > from six.moves import http_client > from six.moves import queue > from six.moves import urllib > from urllib.parse import urlparse > > ... > > def get_request(self, url): > """Get a request object. This can be overridden to add any special > headers.""" > _request = urllib.request.Request(url) > _parseduri = urlparse(url) > _parsedhost = '{uri.netloc}'.format(uri=_parseduri) > _request.add_header("Host", "%s" % _parsedhost) > # log.debug("%s: Added Header: 'Host: %s'" > # % (self.protocol_name, _parsedhost)) > _request.add_header("User-Agent", "weewx/%s" % weewx.__version__) > return _request > > > ... > > # the rapidfire URL: > rf_url = > "https://rtupdate.wunderground.com/weatherstation/updateweatherstation.php"; > # the personal weather station URL: > #pws_url = > "https://weatherstation.wunderground.com/weatherstation/updateweatherstation.php"; > pws_url = > "https://rtupdate.wunderground.com/weatherstation/updateweatherstation.php"; > > > > > [email protected] - Dearborn, Michigan >> On 2/3/20 2:43 PM, Leon Shaner wrote: >> Tom, >> >> Looking at the code, under the do_rapidfire_post, we clearly see it uses >> rf_url. In my debug output I referenced protocol_name and it's showing >> Wunderground-RF as the protocol, but the hostname is from pws_url. >> I have temporarily set pws_url to the rtupdate URL and my 404's have stopped. >> Looks like the rtupdate URL will take regular posts, but the weatherstation >> URL won't take rf posts. :-/ >> >> My debug with pws_url replaced with the rf_url value now shows: >> Feb 3 14:36:30 nixie weewx[8747] DEBUG weewx.restx: Wunderground-RF: Added >> Header: 'Host: rtupdate.wunderground.com' >> That's what it should have shown if rf_url was getting used for >> do_rapidfire_post. >> I'm stumped. :-( >> >> Regards, >> \Leon >> [email protected] - Dearborn, Michigan >>> On 2/3/20 2:23 PM, Leon Shaner wrote: >>> Brice, >>> >>> Thanks! That's interesting... I just added an explicit Host header to the >>> rtupdate request and put debug output there. >>> This was unexpected: >>> Feb 3 14:16:34 nixie weewx[8584] DEBUG weewx.restx: Wunderground-RF: Added >>> Header: 'Host: weatherstation.wunderground.com' >>> That isn't the host that should be used for rapidfire, but it's taken from >>> the URL being used. :-( >>> We have: >>> $ grep wunderground.com restx.py >>> rf_url = >>> "https://rtupdate.wunderground.com/weatherstation/updateweatherstation.php"; >>> pws_url = >>> "https://weatherstation.wunderground.com/weatherstation/updateweatherstation.php"; >>> It will take me some time to figure out why restx.py is using the wrong URL. >>> Must have been doing this for a long time, but seems that due to changes on >>> the WU side, they no longer accept rapidfire updates to the >>> weatherstation.wunderground.com host. >>> >>> Regards, >>> \Leon >>> [email protected] - Dearborn, Michigan >>> On 2/3/20 12:59 PM, Brice Ruth wrote: >>>> header would be something like `Host: rtupdate.wunderground.com` ... >>>> basically, whatever you're using as the hostname in the URL, needs to be >>>> sent as the `Host` header ... that allows load-balancers to route the >>>> request to the correct backend. >>>> >>>> Brice Ruth, FCD >>>> Software Engineer, Madison WI >>>> >>>> >>>> On Mon, Feb 3, 2020 at 11:44 AM Leon Shaner <[email protected]> wrote: >>>>> Hey, WeeWx'ers, >>>>> >>>>> I am still getting 404 errors from the rtupdate URL site. >>>>> There is a reply over here about it. >>>>> I'm now looking into what the headers need to be. >>>>> Does anyone know? >>>>> >>>>> https://apicommunity.wunderground.com/weatherapi/topics/is-the-rtupdate-wunderground-com-address-no-longer-valid >>>>> >>>>>> For requests to route properly, the >>>>>> host header must be set correctly >>>>>> path must be correct >>>>> >>>>> Regards, >>>>> \Leon >>>>> -- >>>>> Leon Shaner :: Dearborn, Michigan (iPad) >>>>> >>>>>> On Feb 1, 2020, at 12:39 PM, Leon Shaner <[email protected]> wrote: >>>>>> >>>>>> Hey, WeeWx'ers. >>>>>> >>>>>> I took a different approach to working around the WU certificate issue >>>>>> -- keep SSL but don't verify the CERT. Scroll for that "solution." >>>>>> >>>>>> But meanwhile, using Wunderground-RF (rapid fire) I am getting an HTTP >>>>>> Error 404: Not Found. >>>>>> Any ideas on that? Did they change the URL, or is it just down? >>>>>> >>>>>> I noticed that even their own WunderStation App on iOS/iPadOS is subject >>>>>> to the certificate issue and reports all stations as "not reporting" -- >>>>>> even ones that are, like mine, which I can verify via the WU website has >>>>>> the data and has reported within minutes ago. >>>>>> >>>>>> From the logs, re: RapidFire HTTP 404: >>>>>> >>>>>> Feb 1 12:31:51 nixie weewx[13184] DEBUG weewx.restx: Wunderground-RF: >>>>>> Failed upload attempt 1: HTTP Error 404: Not Found >>>>>> Feb 1 12:31:51 nixie weewx[13184] ERROR weewx.restx: Wunderground-RF: >>>>>> Failed to publish record 2020-02-01 12:31:51 EST (1580578311): Failed >>>>>> upload after 1 tries >>>>>> Feb 1 12:31:56 nixie weewx[13184] DEBUG weewx.restx: Wunderground-RF: >>>>>> Failed upload attempt 1: HTTP Error 404: Not Found >>>>>> Feb 1 12:31:56 nixie weewx[13184] ERROR weewx.restx: Wunderground-RF: >>>>>> Failed to publish record 2020-02-01 12:31:56 EST (1580578316): Failed >>>>>> upload after 1 tries >>>>>> Feb 1 12:32:04 nixie weewx[13184] DEBUG weewx.restx: Wunderground-RF: >>>>>> Failed upload attempt 1: HTTP Error 404: Not Found >>>>>> Feb 1 12:32:04 nixie weewx[13184] ERROR weewx.restx: Wunderground-RF: >>>>>> Failed to publish record 2020-02-01 12:32:04 EST (1580578324): Failed >>>>>> upload after 1 tries >>>>>> Feb 1 12:32:05 nixie weewx[13184] DEBUG weewx.restx: Wunderground-RF: >>>>>> Failed upload attempt 1: HTTP Error 404: Not Found >>>>>> Feb 1 12:32:05 nixie weewx[13184] ERROR weewx.restx: Wunderground-RF: >>>>>> Failed to publish record 2020-02-01 12:32:05 EST (1580578325): Failed >>>>>> upload after 1 tries >>>>>> Feb 1 12:32:06 nixie weewx[13184] DEBUG weewx.restx: Wunderground-RF: >>>>>> Failed upload attempt 1: HTTP Error 404: Not Found >>>>>> Feb 1 12:32:06 nixie weewx[13184] ERROR weewx.restx: Wunderground-RF: >>>>>> Failed to publish record 2020-02-01 12:32:05 EST (1580578325): Failed >>>>>> upload after 1 tries >>>>>> >>>>>> >>>>>> Here is the diff for my SSL CERT workaround: >>>>>> >>>>>> $ diff restx.py restx.py.20200201.1 >>>>>> 110,115d109 >>>>>> < # SSL certificate hack >>>>>> < global WUssl >>>>>> < WUssl = ssl.create_default_context(); >>>>>> < WUssl.check_hostname=False >>>>>> < WUssl.verify_mode=ssl.CERT_NONE >>>>>> < >>>>>> 454d447 >>>>>> < _request.add_header("User-Agent", "weewx/%s" % >>>>>> weewx.__version__) >>>>>> 543,544c536 >>>>>> < # _response = urllib.request.urlopen(request, data=data_bytes, >>>>>> timeout=self.timeout) >>>>>> < _response = urllib.request.urlopen(request, data=data_bytes, >>>>>> timeout=self.timeout, context=WUssl) >>>>>> --- >>>>>> > _response = urllib.request.urlopen(request, data=data_bytes, >>>>>> > timeout=self.timeout) >>>>>> 1051,1052c1043 >>>>>> < # _response = urllib.request.urlopen(request, >>>>>> timeout=self.timeout) >>>>>> < _response = urllib.request.urlopen(request, >>>>>> timeout=self.timeout, context=WUssl) >>>>>> --- >>>>>> > _response = urllib.request.urlopen(request, >>>>>> > timeout=self.timeout) >>>>>> >>>>>> >>>>>> Regards, >>>>>> \Leon >>>>>> -- >>>>>> Leon Shaner :: Dearborn, Michigan (iPad) >>>>>> >>>>>>> On Jan 31, 2020, at 4:16 PM, Thomas Keffer <[email protected]> wrote: >>>>>>> >>>>>>> >>>>>>> Travis: what is the error? If it's a certificate error, Version 4 will >>>>>>> do a retry after an hour. Unfortunately, Version 3 does not. >>>>>>> >>>>>>> On Fri, Jan 31, 2020 at 1:04 PM Travis Bully <[email protected]> wrote: >>>>>>>> Not here yet. It'll work for awhile and then will get the random cert >>>>>>>> error I sent earlier. A restart of weewx will get it going again. I >>>>>>>> wish restx would just try again after xx seconds. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Friday, January 31, 2020 at 2:48:50 PM UTC-5, J B wrote: >>>>>>>>> >>>>>>>>> Working here too. I had to restart Weewx. >>>>>>>>> >>>>>>>>> On Friday, January 31, 2020 at 11:34:19 AM UTC-8, Brice Ruth wrote: >>>>>>>>>> >>>>>>>>>> Looking like it's working to me, too. >>>>>>>>>> >>>>>>>>>> Brice Ruth, FCD >>>>>>>>>> Software Engineer, Madison WI >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Fri, Jan 31, 2020 at 1:31 PM Travis Bully <[email protected]> >>>>>>>>>> wrote: >>>>>>>>>>> Agreed. Seems like I still get one cert error every now and then. >>>>>>>>>>> Likely still propagating changes through their infrastructure? >>>>>>>>>>> >>>>>>>>>>> Jan 31 14:29:25 homeauto03 weewx[3784]: restx: Wunderground-RF: >>>>>>>>>>> Published record 2020-01-31 14:29:24 EST (1580498964) >>>>>>>>>>> Jan 31 14:29:26 homeauto03 weewx[3784]: restx: Wunderground-RF: >>>>>>>>>>> Failed upload attempt 1: <urlopen error [SSL: >>>>>>>>>>> CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:727)> >>>>>>>>>>> Jan 31 14:29:31 homeauto03 weewx[3784]: restx: Wunderground-RF: >>>>>>>>>>> Failed to publish record 2020-01-31 14:29:26 EST (1580498966): >>>>>>>>>>> Failed upload after 1 tries >>>>>>>>>>> Jan 31 14:29:31 homeauto03 weewx[3784]: restx: Wunderground-RF: >>>>>>>>>>> Published record 2020-01-31 14:29:30 EST (1580498970) >>>>>>>>>>> >>>>>>>>>>> On Fri, Jan 31, 2020 at 2:30 PM Denny Page <[email protected]> >>>>>>>>>>> wrote: >>>>>>>>>>>> It's fixed now. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Friday, January 31, 2020 at 8:54:02 AM UTC-8, Denny Page wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> Wunderground just posted a note about the intermediate >>>>>>>>>>>>> certificate issue. Hopefully they will fix it shortly. >>>>>>>>>>>> -- >>>>>>>>>>>> You received this message because you are subscribed to a topic in >>>>>>>>>>>> the Google Groups "weewx-user" group. >>>>>>>>>>>> To unsubscribe from this topic, visit >>>>>>>>>>>> https://groups.google.com/d/topic/weewx-user/mYhw6CSKHHg/unsubscribe. >>>>>>>>>>>> To unsubscribe from this group and all its topics, send an email >>>>>>>>>>>> to [email protected]. >>>>>>>>>>>> To view this discussion on the web visit >>>>>>>>>>>> https://groups.google.com/d/msgid/weewx-user/f5e5741a-e5f8-419b-9c0d-df1972fe5ced%40googlegroups.com. >>>>>>>>>>> -- >>>>>>>>>>> You received this message because you are subscribed to the Google >>>>>>>>>>> Groups "weewx-user" group. >>>>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>>>> send an email to [email protected]. >>>>>>>>>>> To view this discussion on the web visit >>>>>>>>>>> https://groups.google.com/d/msgid/weewx-user/CAF_%2Bh57S88%3DZOX1mXK8oZBx-_KEHvDPFP%3Dvtd4u3_1qg7994yg%40mail.gmail.com. >>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "weewx-user" group. >>>>>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>>>>> an email to [email protected]. >>>>>>>> To view this discussion on the web visit >>>>>>>> https://groups.google.com/d/msgid/weewx-user/eaa42f60-98fc-4974-8d25-915a4c7e0d80%40googlegroups.com. >>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "weewx-user" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>>>> an email to [email protected]. >>>>>>> To view this discussion on the web visit >>>>>>> https://groups.google.com/d/msgid/weewx-user/CAPq0zECD1_-w%3D8V5eb5-qYsgMBPcvnNunAOe%2BQbnqpKsOsyGUg%40mail.gmail.com. >>>>> -- >>>>> You received this message because you are subscribed to the Google Groups >>>>> "weewx-user" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send an >>>>> email to [email protected]. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/weewx-user/CE8C91F5-D61F-4618-A317-7E97B9ECE912%40isylum.org. >>>> -- >>>> You received this message because you are subscribed to the Google Groups >>>> "weewx-user" group. >>>> To unsubscribe from this group and stop receiving emails from it, send an >>>> email to [email protected]. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/weewx-user/CAFbExW60hmrJRd1yWJAK8f0H%2BXnZsnYe8%3DxMpvKe9n2ws4sq9w%40mail.gmail.com. >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "weewx-user" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/weewx-user/6438f7e3-b03f-4ea3-e324-d89640c88e09%40isylum.org. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "weewx-user" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/weewx-user/c017d408-7abb-695c-8968-9058e22b3389%40isylum.org. > -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/D216EF64-2213-48B7-8A70-4D77CBCCC687%40isylum.org.
