FWIW, I haven't seen any 404's since around these times from last night: Feb 4 20:43:52 Wunderground-RF: Failed upload attempt 1: HTTP Error 404: Not Found Feb 4 20:43:52 Wunderground-RF: Failed to publish record 2020-02-04 20:43:53 EST (1580867033): Failed upload after 1 tries Feb 4 20:49:10 Wunderground-RF: Failed upload attempt 1: <urlopen error _ssl.c:1039: The handshake operation timed out> Feb 4 20:49:10 Wunderground-RF: Failed to publish record 2020-02-04 20:48:49 EST (1580867329): Failed upload after 1 tries Feb 4 20:49:45 Wunderground-RF: Failed upload attempt 1: <urlopen error timed out> Feb 4 20:49:45 Wunderground-RF: Failed to publish record 2020-02-04 20:49:15 EST (1580867355): Failed upload after 1 tries Feb 4 20:50:19 Wunderground-RF: Failed upload attempt 1: <urlopen error timed out> Feb 4 20:50:19 Wunderground-RF: Failed to publish record 2020-02-04 20:49:49 EST (1580867389): Failed upload after 1 tries Feb 4 21:04:41 Wunderground-RF: Failed upload attempt 1: <urlopen error [Errno 104] Connection reset by peer> Feb 4 21:04:41 Wunderground-RF: Failed to publish record 2020-02-04 21:04:41 EST (1580868281): Failed upload after 1 tries Feb 4 21:04:44 Wunderground-RF: Failed upload attempt 1: HTTP Error 404: Not Found
Regards, \Leon -- Leon Shaner :: Dearborn, Michigan (iPad) > On Feb 4, 2020, at 9:15 PM, Thomas Keffer <[email protected]> wrote: > > > I agree with your conclusions. I learned a long time ago not to waste too > much energy chasing WU flaws. There are many, and they come and go. Good way > to frustrate yourself...😞 > >> On Mon, Feb 3, 2020 at 4:32 PM Leon Shaner <[email protected]> wrote: >> Tom, All, >> >> On that WU forum, an official rep said the rtupdate and weatherstation URL's >> go the same place. >> He said it shouldn't matter which one. So we can converge on a single URL. >> >> This is down to same old same old "WU is a steaming pile" being the reason >> for the 404's. >> >> Regards, >> \Leon >> -- >> Leon Shaner :: Dearborn, Michigan (iPad) >> >>>> On Feb 3, 2020, at 3:35 PM, Leon Shaner <[email protected]> wrote: >>>> >>> Hey, WeeWx'ers. >>> >>> Even my workaround of setting pws_url to the value of the rf_url is still >>> getting 404's, even if I explicitly set a Host header. :-((((( >>> >>> This is what I have tried... I guess WU is just a steaming pile as per >>> usual. :-/ >>> >>> Here is my attempted fix, which is part hack. >>> The "fix" is adding the header. The "hack" is forcing the code to use the >>> rtupdate URL (because I couldn't spot why the PWS URL was being used for >>> RF). >>> >>> $ diff restx.py restx.py_getting404 >>> 100d99 >>> < from urllib.parse import urlparse >>> 454,458c453 >>> < _parseduri = urlparse(url) >>> < _parsedhost = '{uri.netloc}'.format(uri=_parseduri) >>> < _request.add_header("Host", "%s" % _parsedhost) >>> < # log.debug("%s: Added Header: 'Host: %s'" >>> < # % (self.protocol_name, _parsedhost)) >>> --- >>> > _request.add_header("User-Agent", "weewx/%s" % weewx.__version__) >>> 610,611c605 >>> < #pws_url = >>> "https://weatherstation.wunderground.com/weatherstation/updateweatherstation.php"; >>> < pws_url = >>> "https://rtupdate.wunderground.com/weatherstation/updateweatherstation.php"; >>> --- >>> > pws_url = >>> > "https://weatherstation.wunderground.com/weatherstation/updateweatherstation.php"; >>> >>> Or in case your line numbers don't match mine (likely), here it is in >>> context: >>> ... >>> # Python 2/3 compatiblity shims >>> import six >>> from six.moves import http_client >>> from six.moves import queue >>> from six.moves import urllib >>> from urllib.parse import urlparse >>> >>> ... >>> >>> def get_request(self, url): >>> """Get a request object. This can be overridden to add any special >>> headers.""" >>> _request = urllib.request.Request(url) >>> _parseduri = urlparse(url) >>> _parsedhost = '{uri.netloc}'.format(uri=_parseduri) >>> _request.add_header("Host", "%s" % _parsedhost) >>> # log.debug("%s: Added Header: 'Host: %s'" >>> # % (self.protocol_name, _parsedhost)) >>> _request.add_header("User-Agent", "weewx/%s" % weewx.__version__) >>> return _request >>> >>> >>> ... >>> >>> # the rapidfire URL: >>> rf_url = >>> "https://rtupdate.wunderground.com/weatherstation/updateweatherstation.php"; >>> # the personal weather station URL: >>> #pws_url = >>> "https://weatherstation.wunderground.com/weatherstation/updateweatherstation.php"; >>> pws_url = >>> "https://rtupdate.wunderground.com/weatherstation/updateweatherstation.php"; >>> >>> >>> >>> >>> [email protected] - Dearborn, Michigan >>>> On 2/3/20 2:43 PM, Leon Shaner wrote: >>>> Tom, >>>> >>>> Looking at the code, under the do_rapidfire_post, we clearly see it uses >>>> rf_url. In my debug output I referenced protocol_name and it's showing >>>> Wunderground-RF as the protocol, but the hostname is from pws_url. >>>> I have temporarily set pws_url to the rtupdate URL and my 404's have >>>> stopped. >>>> Looks like the rtupdate URL will take regular posts, but the >>>> weatherstation URL won't take rf posts. :-/ >>>> >>>> My debug with pws_url replaced with the rf_url value now shows: >>>> Feb 3 14:36:30 nixie weewx[8747] DEBUG weewx.restx: Wunderground-RF: >>>> Added Header: 'Host: rtupdate.wunderground.com' >>>> That's what it should have shown if rf_url was getting used for >>>> do_rapidfire_post. >>>> I'm stumped. :-( >>>> >>>> Regards, >>>> \Leon >>>> [email protected] - Dearborn, Michigan >>>>> On 2/3/20 2:23 PM, Leon Shaner wrote: >>>>> Brice, >>>>> >>>>> Thanks! That's interesting... I just added an explicit Host header to >>>>> the rtupdate request and put debug output there. >>>>> This was unexpected: >>>>> Feb 3 14:16:34 nixie weewx[8584] DEBUG weewx.restx: Wunderground-RF: >>>>> Added Header: 'Host: weatherstation.wunderground.com' >>>>> That isn't the host that should be used for rapidfire, but it's taken >>>>> from the URL being used. :-( >>>>> We have: >>>>> $ grep wunderground.com restx.py >>>>> rf_url = >>>>> "https://rtupdate.wunderground.com/weatherstation/updateweatherstation.php"; >>>>> pws_url = >>>>> "https://weatherstation.wunderground.com/weatherstation/updateweatherstation.php"; >>>>> It will take me some time to figure out why restx.py is using the wrong >>>>> URL. >>>>> Must have been doing this for a long time, but seems that due to changes >>>>> on the WU side, they no longer accept rapidfire updates to the >>>>> weatherstation.wunderground.com host. >>>>> >>>>> Regards, >>>>> \Leon >>>>> [email protected] - Dearborn, Michigan >>>>> On 2/3/20 12:59 PM, Brice Ruth wrote: >>>>>> header would be something like `Host: rtupdate.wunderground.com` ... >>>>>> basically, whatever you're using as the hostname in the URL, needs to be >>>>>> sent as the `Host` header ... that allows load-balancers to route the >>>>>> request to the correct backend. >>>>>> >>>>>> Brice Ruth, FCD >>>>>> Software Engineer, Madison WI >>>>>> >>>>>> >>>>>> On Mon, Feb 3, 2020 at 11:44 AM Leon Shaner <[email protected]> wrote: >>>>>>> Hey, WeeWx'ers, >>>>>>> >>>>>>> I am still getting 404 errors from the rtupdate URL site. >>>>>>> There is a reply over here about it. >>>>>>> I'm now looking into what the headers need to be. >>>>>>> Does anyone know? >>>>>>> >>>>>>> https://apicommunity.wunderground.com/weatherapi/topics/is-the-rtupdate-wunderground-com-address-no-longer-valid >>>>>>> >>>>>>>> For requests to route properly, the >>>>>>>> host header must be set correctly >>>>>>>> path must be correct >>>>>>> >>>>>>> Regards, >>>>>>> \Leon >>>>>>> -- >>>>>>> Leon Shaner :: Dearborn, Michigan (iPad) >>>>>>> >>>>>>>> On Feb 1, 2020, at 12:39 PM, Leon Shaner <[email protected]> wrote: >>>>>>>> >>>>>>>> Hey, WeeWx'ers. >>>>>>>> >>>>>>>> I took a different approach to working around the WU certificate issue >>>>>>>> -- keep SSL but don't verify the CERT. Scroll for that "solution." >>>>>>>> >>>>>>>> But meanwhile, using Wunderground-RF (rapid fire) I am getting an HTTP >>>>>>>> Error 404: Not Found. >>>>>>>> Any ideas on that? Did they change the URL, or is it just down? >>>>>>>> >>>>>>>> I noticed that even their own WunderStation App on iOS/iPadOS is >>>>>>>> subject to the certificate issue and reports all stations as "not >>>>>>>> reporting" -- even ones that are, like mine, which I can verify via >>>>>>>> the WU website has the data and has reported within minutes ago. >>>>>>>> >>>>>>>> From the logs, re: RapidFire HTTP 404: >>>>>>>> >>>>>>>> Feb 1 12:31:51 nixie weewx[13184] DEBUG weewx.restx: Wunderground-RF: >>>>>>>> Failed upload attempt 1: HTTP Error 404: Not Found >>>>>>>> Feb 1 12:31:51 nixie weewx[13184] ERROR weewx.restx: Wunderground-RF: >>>>>>>> Failed to publish record 2020-02-01 12:31:51 EST (1580578311): Failed >>>>>>>> upload after 1 tries >>>>>>>> Feb 1 12:31:56 nixie weewx[13184] DEBUG weewx.restx: Wunderground-RF: >>>>>>>> Failed upload attempt 1: HTTP Error 404: Not Found >>>>>>>> Feb 1 12:31:56 nixie weewx[13184] ERROR weewx.restx: Wunderground-RF: >>>>>>>> Failed to publish record 2020-02-01 12:31:56 EST (1580578316): Failed >>>>>>>> upload after 1 tries >>>>>>>> Feb 1 12:32:04 nixie weewx[13184] DEBUG weewx.restx: Wunderground-RF: >>>>>>>> Failed upload attempt 1: HTTP Error 404: Not Found >>>>>>>> Feb 1 12:32:04 nixie weewx[13184] ERROR weewx.restx: Wunderground-RF: >>>>>>>> Failed to publish record 2020-02-01 12:32:04 EST (1580578324): Failed >>>>>>>> upload after 1 tries >>>>>>>> Feb 1 12:32:05 nixie weewx[13184] DEBUG weewx.restx: Wunderground-RF: >>>>>>>> Failed upload attempt 1: HTTP Error 404: Not Found >>>>>>>> Feb 1 12:32:05 nixie weewx[13184] ERROR weewx.restx: Wunderground-RF: >>>>>>>> Failed to publish record 2020-02-01 12:32:05 EST (1580578325): Failed >>>>>>>> upload after 1 tries >>>>>>>> Feb 1 12:32:06 nixie weewx[13184] DEBUG weewx.restx: Wunderground-RF: >>>>>>>> Failed upload attempt 1: HTTP Error 404: Not Found >>>>>>>> Feb 1 12:32:06 nixie weewx[13184] ERROR weewx.restx: Wunderground-RF: >>>>>>>> Failed to publish record 2020-02-01 12:32:05 EST (1580578325): Failed >>>>>>>> upload after 1 tries >>>>>>>> >>>>>>>> >>>>>>>> Here is the diff for my SSL CERT workaround: >>>>>>>> >>>>>>>> $ diff restx.py restx.py.20200201.1 >>>>>>>> 110,115d109 >>>>>>>> < # SSL certificate hack >>>>>>>> < global WUssl >>>>>>>> < WUssl = ssl.create_default_context(); >>>>>>>> < WUssl.check_hostname=False >>>>>>>> < WUssl.verify_mode=ssl.CERT_NONE >>>>>>>> < >>>>>>>> 454d447 >>>>>>>> < _request.add_header("User-Agent", "weewx/%s" % >>>>>>>> weewx.__version__) >>>>>>>> 543,544c536 >>>>>>>> < # _response = urllib.request.urlopen(request, >>>>>>>> data=data_bytes, timeout=self.timeout) >>>>>>>> < _response = urllib.request.urlopen(request, data=data_bytes, >>>>>>>> timeout=self.timeout, context=WUssl) >>>>>>>> --- >>>>>>>> > _response = urllib.request.urlopen(request, data=data_bytes, >>>>>>>> > timeout=self.timeout) >>>>>>>> 1051,1052c1043 >>>>>>>> < # _response = urllib.request.urlopen(request, >>>>>>>> timeout=self.timeout) >>>>>>>> < _response = urllib.request.urlopen(request, >>>>>>>> timeout=self.timeout, context=WUssl) >>>>>>>> --- >>>>>>>> > _response = urllib.request.urlopen(request, >>>>>>>> > timeout=self.timeout) >>>>>>>> >>>>>>>> >>>>>>>> Regards, >>>>>>>> \Leon >>>>>>>> -- >>>>>>>> Leon Shaner :: Dearborn, Michigan (iPad) >>>>>>>> >>>>>>>>> On Jan 31, 2020, at 4:16 PM, Thomas Keffer <[email protected]> wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> Travis: what is the error? If it's a certificate error, Version 4 >>>>>>>>> will do a retry after an hour. Unfortunately, Version 3 does not. >>>>>>>>> >>>>>>>>> On Fri, Jan 31, 2020 at 1:04 PM Travis Bully <[email protected]> wrote: >>>>>>>>>> Not here yet. It'll work for awhile and then will get the random >>>>>>>>>> cert error I sent earlier. A restart of weewx will get it going >>>>>>>>>> again. I wish restx would just try again after xx seconds. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Friday, January 31, 2020 at 2:48:50 PM UTC-5, J B wrote: >>>>>>>>>>> >>>>>>>>>>> Working here too. I had to restart Weewx. >>>>>>>>>>> >>>>>>>>>>> On Friday, January 31, 2020 at 11:34:19 AM UTC-8, Brice Ruth wrote: >>>>>>>>>>>> >>>>>>>>>>>> Looking like it's working to me, too. >>>>>>>>>>>> >>>>>>>>>>>> Brice Ruth, FCD >>>>>>>>>>>> Software Engineer, Madison WI >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Fri, Jan 31, 2020 at 1:31 PM Travis Bully <[email protected]> >>>>>>>>>>>> wrote: >>>>>>>>>>>>> Agreed. Seems like I still get one cert error every now and >>>>>>>>>>>>> then. Likely still propagating changes through their >>>>>>>>>>>>> infrastructure? >>>>>>>>>>>>> >>>>>>>>>>>>> Jan 31 14:29:25 homeauto03 weewx[3784]: restx: Wunderground-RF: >>>>>>>>>>>>> Published record 2020-01-31 14:29:24 EST (1580498964) >>>>>>>>>>>>> Jan 31 14:29:26 homeauto03 weewx[3784]: restx: Wunderground-RF: >>>>>>>>>>>>> Failed upload attempt 1: <urlopen error [SSL: >>>>>>>>>>>>> CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:727)> >>>>>>>>>>>>> Jan 31 14:29:31 homeauto03 weewx[3784]: restx: Wunderground-RF: >>>>>>>>>>>>> Failed to publish record 2020-01-31 14:29:26 EST (1580498966): >>>>>>>>>>>>> Failed upload after 1 tries >>>>>>>>>>>>> Jan 31 14:29:31 homeauto03 weewx[3784]: restx: Wunderground-RF: >>>>>>>>>>>>> Published record 2020-01-31 14:29:30 EST (1580498970) >>>>>>>>>>>>> >>>>>>>>>>>>> On Fri, Jan 31, 2020 at 2:30 PM Denny Page <[email protected]> >>>>>>>>>>>>> wrote: >>>>>>>>>>>>>> It's fixed now. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Friday, January 31, 2020 at 8:54:02 AM UTC-8, Denny Page >>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Wunderground just posted a note about the intermediate >>>>>>>>>>>>>>> certificate issue. Hopefully they will fix it shortly. >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> You received this message because you are subscribed to a topic >>>>>>>>>>>>>> in the Google Groups "weewx-user" group. >>>>>>>>>>>>>> To unsubscribe from this topic, visit >>>>>>>>>>>>>> https://groups.google.com/d/topic/weewx-user/mYhw6CSKHHg/unsubscribe. >>>>>>>>>>>>>> To unsubscribe from this group and all its topics, send an email >>>>>>>>>>>>>> to [email protected]. >>>>>>>>>>>>>> To view this discussion on the web visit >>>>>>>>>>>>>> https://groups.google.com/d/msgid/weewx-user/f5e5741a-e5f8-419b-9c0d-df1972fe5ced%40googlegroups.com. >>>>>>>>>>>>> -- >>>>>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>>>>> Google Groups "weewx-user" group. >>>>>>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>>>>>> send an email to [email protected]. >>>>>>>>>>>>> To view this discussion on the web visit >>>>>>>>>>>>> https://groups.google.com/d/msgid/weewx-user/CAF_%2Bh57S88%3DZOX1mXK8oZBx-_KEHvDPFP%3Dvtd4u3_1qg7994yg%40mail.gmail.com. >>>>>>>>>> -- >>>>>>>>>> You received this message because you are subscribed to the Google >>>>>>>>>> Groups "weewx-user" group. >>>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>>> send an email to [email protected]. >>>>>>>>>> To view this discussion on the web visit >>>>>>>>>> https://groups.google.com/d/msgid/weewx-user/eaa42f60-98fc-4974-8d25-915a4c7e0d80%40googlegroups.com. >>>>>>>>> -- >>>>>>>>> You received this message because you are subscribed to the Google >>>>>>>>> Groups "weewx-user" group. >>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>> send an email to [email protected]. >>>>>>>>> To view this discussion on the web visit >>>>>>>>> https://groups.google.com/d/msgid/weewx-user/CAPq0zECD1_-w%3D8V5eb5-qYsgMBPcvnNunAOe%2BQbnqpKsOsyGUg%40mail.gmail.com. >>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "weewx-user" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>>>> an email to [email protected]. >>>>>>> To view this discussion on the web visit >>>>>>> https://groups.google.com/d/msgid/weewx-user/CE8C91F5-D61F-4618-A317-7E97B9ECE912%40isylum.org. >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "weewx-user" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>>> an email to [email protected]. >>>>>> To view this discussion on the web visit >>>>>> https://groups.google.com/d/msgid/weewx-user/CAFbExW60hmrJRd1yWJAK8f0H%2BXnZsnYe8%3DxMpvKe9n2ws4sq9w%40mail.gmail.com. >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google Groups >>>>> "weewx-user" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send an >>>>> email to [email protected]. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/weewx-user/6438f7e3-b03f-4ea3-e324-d89640c88e09%40isylum.org. >>>> >>>> -- >>>> You received this message because you are subscribed to the Google Groups >>>> "weewx-user" group. >>>> To unsubscribe from this group and stop receiving emails from it, send an >>>> email to [email protected]. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/weewx-user/c017d408-7abb-695c-8968-9058e22b3389%40isylum.org. >>> -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/9C99CEBB-BC85-42A0-8778-4649FF55BC3E%40isylum.org.
