So, I'm a little stumped on why you're getting 404s unless you happen to be hitting a region that's totally horked - where are you physically located? I'm currently using 3.9.2-1 on my Pi 4 and I had temporarily disabled `https`, but now have `https` re-enabled, but with the recent addition of `/usr/bin/env PYTHONHTTPSVERIFY=0` to stop the weewx thread from failing out after awhile. I've used rapidfire from the start and I don't see anywhere that it's using the `weatherstation` URL - do you have `archive_post` enabled as well, maybe? My errors were always indicating failures in talking to `rtupdate...` as well, so I'm pretty sure on this side, it hasn't been talking to anything but the rapidfire URL ... just not sure why your setup seems to be doing that?
Brice Ruth, FCD Software Engineer, Madison WI On Mon, Feb 3, 2020 at 2:35 PM Leon Shaner <[email protected]> wrote: > Hey, WeeWx'ers. > > Even my workaround of setting pws_url to the value of the rf_url is still > getting 404's, even if I explicitly set a Host header. :-((((( > > This is what I have tried... I guess WU is just a steaming pile as per > usual. :-/ > > Here is my attempted fix, which is part hack. > The "fix" is adding the header. The "hack" is forcing the code to use > the rtupdate URL (because I couldn't spot why the PWS URL was being used > for RF). > > $ diff restx.py restx.py_getting404 > 100d99 > < from urllib.parse import urlparse > 454,458c453 > < _parseduri = urlparse(url) > < _parsedhost = '{uri.netloc}'.format(uri=_parseduri) > < _request.add_header("Host", "%s" % _parsedhost) > < # log.debug("%s: Added Header: 'Host: %s'" > < # % (self.protocol_name, _parsedhost)) > --- > > _request.add_header("User-Agent", "weewx/%s" % weewx.__version__) > 610,611c605 > < #pws_url = > "https://weatherstation.wunderground.com/weatherstation/updateweatherstation.php"; > <https://weatherstation.wunderground.com/weatherstation/updateweatherstation.php> > < pws_url = > "https://rtupdate.wunderground.com/weatherstation/updateweatherstation.php"; > <https://rtupdate.wunderground.com/weatherstation/updateweatherstation.php> > --- > > pws_url = > "https://weatherstation.wunderground.com/weatherstation/updateweatherstation.php"; > <https://weatherstation.wunderground.com/weatherstation/updateweatherstation.php> > > Or in case your line numbers don't match mine (likely), here it is in > context: > > ... > # Python 2/3 compatiblity shims > import six > from six.moves import http_client > from six.moves import queue > from six.moves import urllib > from urllib.parse import urlparse > > ... > > def get_request(self, url): > """Get a request object. This can be overridden to add any special > headers.""" > _request = urllib.request.Request(url) > _parseduri = urlparse(url) > _parsedhost = '{uri.netloc}'.format(uri=_parseduri) > _request.add_header("Host", "%s" % _parsedhost) > # log.debug("%s: Added Header: 'Host: %s'" > # % (self.protocol_name, _parsedhost)) > _request.add_header("User-Agent", "weewx/%s" % weewx.__version__) > return _request > > > ... > > # the rapidfire URL: > rf_url = > "https://rtupdate.wunderground.com/weatherstation/updateweatherstation.php"; > <https://rtupdate.wunderground.com/weatherstation/updateweatherstation.php> > # the personal weather station URL: > #pws_url = > "https://weatherstation.wunderground.com/weatherstation/updateweatherstation.php"; > <https://weatherstation.wunderground.com/weatherstation/updateweatherstation.php> > pws_url = > "https://rtupdate.wunderground.com/weatherstation/updateweatherstation.php"; > <https://rtupdate.wunderground.com/weatherstation/updateweatherstation.php> > > > > > [email protected] - Dearborn, Michigan > > On 2/3/20 2:43 PM, Leon Shaner wrote: > > Tom, > > Looking at the code, under the do_rapidfire_post, we clearly see it uses > rf_url. In my debug output I referenced protocol_name and it's showing > Wunderground-RF as the protocol, but the hostname is from pws_url. > I have temporarily set pws_url to the rtupdate URL and my 404's have > stopped. > Looks like the rtupdate URL will take regular posts, but the > weatherstation URL won't take rf posts. :-/ > > My debug with pws_url replaced with the rf_url value now shows: > > Feb 3 14:36:30 nixie weewx[8747] DEBUG weewx.restx: Wunderground-RF: > Added Header: 'Host: rtupdate.wunderground.com' > > That's what it should have shown if rf_url was getting used for > do_rapidfire_post. > I'm stumped. :-( > > Regards, > \Leon > > [email protected] - Dearborn, Michigan > > On 2/3/20 2:23 PM, Leon Shaner wrote: > > Brice, > > Thanks! That's interesting... I just added an explicit Host header to > the rtupdate request and put debug output there. > This was unexpected: > > Feb 3 14:16:34 nixie weewx[8584] DEBUG weewx.restx: Wunderground-RF: > Added Header: 'Host: *weatherstation.wunderground.com > <http://weatherstation.wunderground.com>*' > > That isn't the host that should be used for rapidfire, but it's taken from > the URL being used. :-( > We have: > > $ grep wunderground.com restx.py > rf_url = > "https://rtupdate.wunderground.com/weatherstation/updateweatherstation.php"; > <https://rtupdate.wunderground.com/weatherstation/updateweatherstation.php> > pws_url = > "https://weatherstation.wunderground.com/weatherstation/updateweatherstation.php"; > <https://weatherstation.wunderground.com/weatherstation/updateweatherstation.php> > > It will take me some time to figure out why restx.py is using the wrong > URL. > Must have been doing this for a long time, but seems that due to changes > on the WU side, they no longer accept rapidfire updates to the > weatherstation.wunderground.com host. > > Regards, > \Leon > > [email protected] - Dearborn, Michigan > > On 2/3/20 12:59 PM, Brice Ruth wrote: > > header would be something like `Host: rtupdate.wunderground.com` ... > basically, whatever you're using as the hostname in the URL, needs to be > sent as the `Host` header ... that allows load-balancers to route the > request to the correct backend. > > Brice Ruth, FCD > Software Engineer, Madison WI > > > On Mon, Feb 3, 2020 at 11:44 AM Leon Shaner <[email protected]> wrote: > >> Hey, WeeWx'ers, >> >> I am still getting 404 errors from the rtupdate URL site. >> There is a reply over here about it. >> I'm now looking into what the headers need to be. >> Does anyone know? >> >> >> https://apicommunity.wunderground.com/weatherapi/topics/is-the-rtupdate-wunderground-com-address-no-longer-valid >> >> For requests to route properly, the >> host header must be set correctly >> path must be correct >> >> >> Regards, >> \Leon >> -- >> Leon Shaner :: Dearborn, Michigan (iPad) >> >> On Feb 1, 2020, at 12:39 PM, Leon Shaner <[email protected]> wrote: >> >> Hey, WeeWx'ers. >> >> I took a different approach to working around the WU certificate issue -- >> keep SSL but don't verify the CERT. Scroll for that "solution." >> >> But meanwhile, using Wunderground-RF (rapid fire) I am getting an HTTP >> Error 404: Not Found. >> Any ideas on that? Did they change the URL, or is it just down? >> >> I noticed that even their own WunderStation App on iOS/iPadOS is subject >> to the certificate issue and reports all stations as "not reporting" -- >> even ones that are, like mine, which I can verify via the WU website has >> the data and has reported within minutes ago. >> >> From the logs, re: RapidFire HTTP 404: >> >> Feb 1 12:31:51 nixie weewx[13184] DEBUG weewx.restx: Wunderground-RF: >> Failed upload attempt 1: HTTP Error 404: Not Found >> Feb 1 12:31:51 nixie weewx[13184] ERROR weewx.restx: Wunderground-RF: >> Failed to publish record 2020-02-01 12:31:51 EST (1580578311): Failed >> upload after 1 tries >> Feb 1 12:31:56 nixie weewx[13184] DEBUG weewx.restx: Wunderground-RF: >> Failed upload attempt 1: HTTP Error 404: Not Found >> Feb 1 12:31:56 nixie weewx[13184] ERROR weewx.restx: Wunderground-RF: >> Failed to publish record 2020-02-01 12:31:56 EST (1580578316): Failed >> upload after 1 tries >> Feb 1 12:32:04 nixie weewx[13184] DEBUG weewx.restx: Wunderground-RF: >> Failed upload attempt 1: HTTP Error 404: Not Found >> Feb 1 12:32:04 nixie weewx[13184] ERROR weewx.restx: Wunderground-RF: >> Failed to publish record 2020-02-01 12:32:04 EST (1580578324): Failed >> upload after 1 tries >> Feb 1 12:32:05 nixie weewx[13184] DEBUG weewx.restx: Wunderground-RF: >> Failed upload attempt 1: HTTP Error 404: Not Found >> Feb 1 12:32:05 nixie weewx[13184] ERROR weewx.restx: Wunderground-RF: >> Failed to publish record 2020-02-01 12:32:05 EST (1580578325): Failed >> upload after 1 tries >> Feb 1 12:32:06 nixie weewx[13184] DEBUG weewx.restx: Wunderground-RF: >> Failed upload attempt 1: HTTP Error 404: Not Found >> Feb 1 12:32:06 nixie weewx[13184] ERROR weewx.restx: Wunderground-RF: >> Failed to publish record 2020-02-01 12:32:05 EST (1580578325): Failed >> upload after 1 tries >> >> >> Here is the diff for my SSL CERT workaround: >> >> $ diff restx.py restx.py.20200201.1 >> 110,115d109 >> < # SSL certificate hack >> < global WUssl >> < WUssl = ssl.create_default_context(); >> < WUssl.check_hostname=False >> < WUssl.verify_mode=ssl.CERT_NONE >> < >> 454d447 >> < _request.add_header("User-Agent", "weewx/%s" % >> weewx.__version__) >> 543,544c536 >> < # _response = urllib.request.urlopen(request, data=data_bytes, >> timeout=self.timeout) >> < _response = urllib.request.urlopen(request, data=data_bytes, >> timeout=self.timeout, context=WUssl) >> --- >> > _response = urllib.request.urlopen(request, data=data_bytes, >> timeout=self.timeout) >> 1051,1052c1043 >> < # _response = urllib.request.urlopen(request, >> timeout=self.timeout) >> < _response = urllib.request.urlopen(request, >> timeout=self.timeout, context=WUssl) >> --- >> > _response = urllib.request.urlopen(request, >> timeout=self.timeout) >> >> >> Regards, >> \Leon >> -- >> Leon Shaner :: Dearborn, Michigan (iPad) >> >> On Jan 31, 2020, at 4:16 PM, Thomas Keffer <[email protected]> wrote: >> >> >> Travis: what is the error? If it's a certificate error, Version 4 will do >> a retry after an hour. Unfortunately, Version 3 does not. >> >> On Fri, Jan 31, 2020 at 1:04 PM Travis Bully <[email protected]> wrote: >> >>> Not here yet. It'll work for awhile and then will get the random cert >>> error I sent earlier. A restart of weewx will get it going again. I wish >>> restx would just try again after xx seconds. >>> >>> >>> >>> On Friday, January 31, 2020 at 2:48:50 PM UTC-5, J B wrote: >>>> >>>> Working here too. I had to restart Weewx. >>>> >>>> On Friday, January 31, 2020 at 11:34:19 AM UTC-8, Brice Ruth wrote: >>>>> >>>>> Looking like it's working to me, too. >>>>> >>>>> Brice Ruth, FCD >>>>> Software Engineer, Madison WI >>>>> >>>>> >>>>> On Fri, Jan 31, 2020 at 1:31 PM Travis Bully <[email protected]> wrote: >>>>> >>>>>> Agreed. Seems like I still get one cert error every now and then. >>>>>> Likely still propagating changes through their infrastructure? >>>>>> >>>>>> Jan 31 14:29:25 homeauto03 weewx[3784]: restx: Wunderground-RF: >>>>>> Published record 2020-01-31 14:29:24 EST (1580498964) >>>>>> Jan 31 14:29:26 homeauto03 weewx[3784]: restx: Wunderground-RF: >>>>>> Failed upload attempt 1: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] >>>>>> certificate verify failed (_ssl.c:727)> >>>>>> Jan 31 14:29:31 homeauto03 weewx[3784]: restx: Wunderground-RF: >>>>>> Failed to publish record 2020-01-31 14:29:26 EST (1580498966): Failed >>>>>> upload after 1 tries >>>>>> Jan 31 14:29:31 homeauto03 weewx[3784]: restx: Wunderground-RF: >>>>>> Published record 2020-01-31 14:29:30 EST (1580498970) >>>>>> >>>>>> On Fri, Jan 31, 2020 at 2:30 PM Denny Page <[email protected]> wrote: >>>>>> >>>>>>> It's fixed now. >>>>>>> >>>>>>> >>>>>>> On Friday, January 31, 2020 at 8:54:02 AM UTC-8, Denny Page wrote: >>>>>>>> >>>>>>>> Wunderground just posted a note about the intermediate certificate >>>>>>>> issue. Hopefully they will fix it shortly. >>>>>>>> >>>>>>> -- >>>>>>> You received this message because you are subscribed to a topic in >>>>>>> the Google Groups "weewx-user" group. >>>>>>> To unsubscribe from this topic, visit >>>>>>> https://groups.google.com/d/topic/weewx-user/mYhw6CSKHHg/unsubscribe >>>>>>> . >>>>>>> To unsubscribe from this group and all its topics, send an email to >>>>>>> [email protected]. >>>>>>> To view this discussion on the web visit >>>>>>> https://groups.google.com/d/msgid/weewx-user/f5e5741a-e5f8-419b-9c0d-df1972fe5ced%40googlegroups.com >>>>>>> <https://groups.google.com/d/msgid/weewx-user/f5e5741a-e5f8-419b-9c0d-df1972fe5ced%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>> . >>>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "weewx-user" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> To view this discussion on the web visit >>>>>> https://groups.google.com/d/msgid/weewx-user/CAF_%2Bh57S88%3DZOX1mXK8oZBx-_KEHvDPFP%3Dvtd4u3_1qg7994yg%40mail.gmail.com >>>>>> <https://groups.google.com/d/msgid/weewx-user/CAF_%2Bh57S88%3DZOX1mXK8oZBx-_KEHvDPFP%3Dvtd4u3_1qg7994yg%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>> . >>>>>> >>>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "weewx-user" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/weewx-user/eaa42f60-98fc-4974-8d25-915a4c7e0d80%40googlegroups.com >>> <https://groups.google.com/d/msgid/weewx-user/eaa42f60-98fc-4974-8d25-915a4c7e0d80%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "weewx-user" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/weewx-user/CAPq0zECD1_-w%3D8V5eb5-qYsgMBPcvnNunAOe%2BQbnqpKsOsyGUg%40mail.gmail.com >> <https://groups.google.com/d/msgid/weewx-user/CAPq0zECD1_-w%3D8V5eb5-qYsgMBPcvnNunAOe%2BQbnqpKsOsyGUg%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> >> -- >> You received this message because you are subscribed to the Google Groups >> "weewx-user" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/weewx-user/CE8C91F5-D61F-4618-A317-7E97B9ECE912%40isylum.org >> <https://groups.google.com/d/msgid/weewx-user/CE8C91F5-D61F-4618-A317-7E97B9ECE912%40isylum.org?utm_medium=email&utm_source=footer> >> . >> > -- > You received this message because you are subscribed to the Google Groups > "weewx-user" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/weewx-user/CAFbExW60hmrJRd1yWJAK8f0H%2BXnZsnYe8%3DxMpvKe9n2ws4sq9w%40mail.gmail.com > <https://groups.google.com/d/msgid/weewx-user/CAFbExW60hmrJRd1yWJAK8f0H%2BXnZsnYe8%3DxMpvKe9n2ws4sq9w%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > > > -- > You received this message because you are subscribed to the Google Groups > "weewx-user" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/weewx-user/6438f7e3-b03f-4ea3-e324-d89640c88e09%40isylum.org > <https://groups.google.com/d/msgid/weewx-user/6438f7e3-b03f-4ea3-e324-d89640c88e09%40isylum.org?utm_medium=email&utm_source=footer> > . > > > -- > You received this message because you are subscribed to the Google Groups > "weewx-user" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/weewx-user/c017d408-7abb-695c-8968-9058e22b3389%40isylum.org > <https://groups.google.com/d/msgid/weewx-user/c017d408-7abb-695c-8968-9058e22b3389%40isylum.org?utm_medium=email&utm_source=footer> > . > > > -- > You received this message because you are subscribed to the Google Groups > "weewx-user" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/weewx-user/45f04b3b-ae67-aaeb-9d25-625ea42b111b%40isylum.org > <https://groups.google.com/d/msgid/weewx-user/45f04b3b-ae67-aaeb-9d25-625ea42b111b%40isylum.org?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/CAFbExW6Ef6USxqExNXABs96B_os42mdFOwRiYhfwFzsi_8sdUQ%40mail.gmail.com.
