Hi everyone,I'm new to this list, and have been using gpsd silently for 3-4 years now. I'm using it with my U-Blox ZED-F9T time-focused GNSS receiver. Thanks for all your great work so far! I decided it's time to at least try to contribute back a bit.
While tracking crashes in my gpsd-prometheus-exporter instance, I identified that gpsd (3.27.5) emits invalid JSON data for specific subframes. I activated SFRBX reporting on my ublox receiver a while ago, and since then my exporter reports decode errors about once every (other) day. It crashes-on-error, so it's quite visible to me when this happens.
Looking at my monitoring dashboards & log streams for gpsd and the exporter, I can say that this issue definitely originates from gpsd emitting invalid JSON messages, and it keeps the exporter dysfunctional for around 10-15 minutes. Then the emission of invalid messages vanish until they re-appears again, usually a day or two later. It happened 6 times over the last 7 days.
Here are some JSON messages that were emitted by gpsd that triggered a JSON decoder exception in the exporter:
{"class":"SUBFRAME","device":"/dev/serial/by-id/usb-u-blox_AG_-_www.u-blox.com_u-blox_GNSS_receiver-if00","gnssId":254,"tSV":0,"frame":5,"WN":754977538,"TOW":22000000,"scaled":true,"dataid":106,"ALMANAC":{"ID":0,"Health":59,"e":2.09393066406250e+03,"toa":4729542938196168858,"deltai":nan,"Omegad":3.45845952088873e-323,"sqrtA":0.00000000000000e+00,"Omega0":0.00000000000000e+00,"omega":1.40949850528800e-66,"M0":3.18672341567604e-319,"af0":nan,"af1":3.10535655936610e+07}}
{"class":"SUBFRAME","device":"/dev/serial/by-id/usb-u-blox_AG_-_www.u-blox.com_u-blox_GNSS_receiver-if00","gnssId":252,"tSV":0,"frame":5,"WN":738200322,"TOW":22000000,"scaled":true,"dataid":106,"ALMANAC":{"ID":0,"Health":47,"e":2.09414550781250e+03,"toa":4729543027241852045,"deltai":nan,"Omegad":3.45845952088873e-323,"sqrtA":0.00000000000000e+00,"Omega0":0.00000000000000e+00,"omega":1.40949850528800e-66,"M0":3.18672341567604e-319,"af0":nan,"af1":3.10543700771775e+07}}
{"class":"SUBFRAME","device":"/dev/serial/by-id/usb-u-blox_AG_-_www.u-blox.com_u-blox_GNSS_receiver-if00","gnssId":250,"tSV":0,"frame":5,"WN":738200322,"TOW":22000000,"scaled":true,"dataid":106,"ALMANAC":{"ID":0,"Health":18,"e":2.09445190429688e+03,"toa":4729543116337803563,"deltai":nan,"Omegad":3.45845952088873e-323,"sqrtA":0.00000000000000e+00,"Omega0":0.00000000000000e+00,"omega":1.40949850528800e-66,"M0":3.18672341567604e-319,"af0":nan,"af1":3.10551747304498e+07}}
{"class":"SUBFRAME","device":"/dev/serial/by-id/usb-u-blox_AG_-_www.u-blox.com_u-blox_GNSS_receiver-if00","gnssId":246,"tSV":0,"frame":5,"WN":738200322,"TOW":22000000,"scaled":true,"dataid":106,"ALMANAC":{"ID":0,"Health":29,"e":2.09562377929688e+03,"toa":4729543294685522946,"deltai":nan,"Omegad":3.45845952088873e-323,"sqrtA":0.00000000000000e+00,"Omega0":0.00000000000000e+00,"omega":1.40949850086628e-66,"M0":3.18672341567604e-319,"af0":nan,"af1":3.10567842577047e+07}}
One can see that all messages are frameid 5 dataid 106 almanac data, and both (d_)deltai and (d_)af0 have a value of nan, which raises the decoder error (correctly). I have not found any other frameid/dataid combination yet, all the messages that triggered the exception so far look quite similar to these above.
I took a look at the code, and I think construction of these messages happens in gpsd/gpsd_json.c, on around line 1220-1259, in function json_subframe_dump(). (current master, HEAD commit b057ecf87d23148e2090db8525a8d340b86104d3). I have not yet established a full understanding of why this happens in the first place, i.e. where and why (d_)deltai (d_)af0 exactly became NaN.
json_subframe_dump() seems to construct the subframe json string data in a single str_appendf() call, while e.g. json_subframe_dump_orb() and json_log_dump() do use isfinite()-if-checks quite often, appending only if the values are not NAN/INFINITY. Maybe json_subframe_dump() should also use the same isfinite()-guards? But the main question to me is if it is actually okay/acceptable for deltai/d_deltai & af0/d_af0 to be NaN and this indicates that these values were simply not populated (yet), or if this indicates an issue in the subframe data processing itself.
I have some spare time at hand over the next days/weeks, so I will try to analyze and fix this issue. If anyone has some input & insights on this it would be great, since I'm not very experienced with subframe-data and its processing details yet.
Cheers, Mario
