Thanks David, That indeed flags the error and shows the special character is all messed up. I don't know iconv but if this is the issue and there's no way around it than fixing the raw text input, I can find a code solution. Shame Prometheus doesn't support any other encodings but heigh ho.
Thanks for the useful information. On Thu, 25 Mar 2021 at 12:31, David Leadbeater <[email protected]> wrote: > Can you try: > > curl host/metrics | promtool check metrics > > Which will avoid any issues due to copying and means Prometheus sees > the raw bytes. > > C++ std::string types (which is what prometheus-cpp uses: > > https://jupp0r.github.io/prometheus-cpp/structprometheus_1_1ClientMetric_1_1Label.html > ) > don't have character set information associated with them, so it's > likely you need to convert them yourself as the API doesn't have > enough info to know what they are, e.g. using iconv() from ISO-8859-1 > to UTF-8, which you then can put in a std::string. > > David > > On Thu, 25 Mar 2021 at 11:07, John Dexter <[email protected]> wrote: > > > > It's the latest version as of a few weeks ago. > > Other modules which are identical except their label values contain no > special characters work fine so I am fairly confident this is the reason - > should I be able to find more verbose logs telling me exactly what it > didn't like because I could not find anything so far? > > > > I have wondered if the clipboard is fixing this when I copy-paste so you > can't reproduce! > > > > On Thu, 25 Mar 2021 at 10:52, Julien Pivotto <[email protected]> > wrote: > >> > >> Do you have an up to date version of prometheus? I tryied this and it > >> workd for me. > >> > >> Are you sure that those are the offending metrics? > >> > >> Thanks. > >> > >> On 25 Mar 10:47, John Dexter wrote: > >> > I wouldn't be surprised though it does validate online (unless > copy-paste > >> > in chrome automatically changes the encoding!) > >> > I'm using the prometheus-cpp library and I cannot find any mention of > >> > character support. Does Prometheus not have any way to convert > incoming > >> > data? > >> > The strings I am using as labels come from very old code which is > quite > >> > fragile, so fixing it outside our code would be far preferable - > other code > >> > may rely on it being in whatever encoding it is in. > >> > > >> > On Thu, 25 Mar 2021 at 10:38, Ben Kochie <[email protected]> wrote: > >> > > >> > > Sounds like you may have some data that's encoded in a pre-UTF-8 > format > >> > > like https://en.wikipedia.org/wiki/Windows-1257 or > >> > > https://en.wikipedia.org/wiki/ISO/IEC_8859. > >> > > > >> > > You'll have a character encoding converter in your exporter in > order to > >> > > ingest the data. > >> > > > >> > > On Thu, Mar 25, 2021 at 11:32 AM John Dexter <[email protected] > > > >> > > wrote: > >> > > > >> > >> I understand that metric and label *names* are standard ASCII but > >> > >> *values* support Unicode, at least according to docs. > >> > >> I am working on a Swedish-language system and just realised > several of my > >> > >> targets are reporting this error, however their endpoints work > just fine. > >> > >> > >> > >> I have a metric with: > >> > >> > >> > >> obfuscated_metric_name{location="Törn"} 0.00000000000000000 > >> > >> > >> > >> message_bytes_total{direction="sent",subsystem="ÖTPP"} > 0.00000000000000000 > >> > >> > >> > >> I copy-pasted this from the endpoint apart from hiding the metric > name > >> > >> for privacy reasons so what is wrong? Our code is quite old C++ so > quite > >> > >> how these characters are handled is not obvious to me but we > haven't had > >> > >> this issue before (we use these strings all over the place) and > when I > >> > >> paste this text into a UTF8 validity checked, it reports no > problems. > >> > >> Does Prometheus have any configuration I can tweak otherwise I'm a > bit > >> > >> stuck! > >> > >> > >> > >> Thanks for any help. > >> > >> > >> > >> -- > >> > >> You received this message because you are subscribed to the Google > Groups > >> > >> "Prometheus Users" group. > >> > >> To unsubscribe from this group and stop receiving emails from it, > send an > >> > >> email to [email protected]. > >> > >> To view this discussion on the web visit > >> > >> > https://groups.google.com/d/msgid/prometheus-users/CAGJRanhVcN%3DgsyF_QDPckNopVpUm1_4NY6cDPbxdjjKw4ZFGpA%40mail.gmail.com > >> > >> < > https://groups.google.com/d/msgid/prometheus-users/CAGJRanhVcN%3DgsyF_QDPckNopVpUm1_4NY6cDPbxdjjKw4ZFGpA%40mail.gmail.com?utm_medium=email&utm_source=footer > > > >> > >> . > >> > >> > >> > > > >> > > >> > -- > >> > You received this message because you are subscribed to the Google > Groups "Prometheus Users" group. > >> > To unsubscribe from this group and stop receiving emails from it, > send an email to [email protected]. > >> > To view this discussion on the web visit > https://groups.google.com/d/msgid/prometheus-users/CAGJRania%3DmzAgg%3DvSD0hjfd6%3D-CEswoMKkJPboPy8PK-L1J%3DTg%40mail.gmail.com > . > >> > >> -- > >> Julien Pivotto > >> @roidelapluie > > > > -- > > You received this message because you are subscribed to the Google > Groups "Prometheus Users" group. > > To unsubscribe from this group and stop receiving emails from it, send > an email to [email protected]. > > To view this discussion on the web visit > https://groups.google.com/d/msgid/prometheus-users/CAGJRanjLA7p-VUM8%3DefCUDiDA0DLMKUyZBRVvimA%2BoR484qCSw%40mail.gmail.com > . > -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/CAGJRanguAwkDryrkatDgkifiqCBPRRt%2Bhw3hZegP1j%3DUWRZFdw%40mail.gmail.com.
