Hello What you need is downsampling, which is not supported directly by prometheus (as a matter of fact, storing historical data is not either recommended on bare prometheus), It is time to check external storage solutions for prometheus and choose the one that suits your needs (mimir, cortex, thanos, m3db, etc ...)
On Thu, Jun 15, 2023 at 9:01 AM Boris Behrens <[email protected]> wrote: > Hi Ben, > I've read the link you gave me and I will try to answer the best I can. > > > What problem are you solving for? > I want to keep historical data for longer periods with less disk space > used. > I currently keep around 4 month of data for around 1.3T of disk space. > and I would love to have 1-2 years of data without having to buy 20TB > disks :) > > > Why do you think you need this? > For debugging and monitoring I only need the last couple of days with > a raw resolution. And seeing trends can be downsampled (is this the > correct term?) because it is just historical trend data. > I think it's rather a use case than an actual problem I am working on. > I would like to see in the past how things changed and overlay it with > timestamps from changes. Things begin to looks way different when you > check the past. > > > You say "a lot of endpoints", but what is a lot? 1,000? 10,000? 100,000? > 1,000,000? How many series? > A couple hundred endpoints with around 1.5m of series. But I don't > think that this information is helpful in the current context. > > Thank you for your time and effort to help me with my stupid little > problem. > Cheers > Boris > > Am Do., 15. Juni 2023 um 06:24 Uhr schrieb Ben Kochie <[email protected]>: > > > > My first question is "why?" (https://xyproblem.info/) > > > > What problem are you solving for? Why do you think you need this? > > > > You say "a lot of endpoints", but what is a lot? 1,000? 10,000? 100,000? > 1,000,000? How many series? > > > > On Thu, Jun 15, 2023 at 6:21 AM Boris <[email protected]> wrote: > >> > >> Dear prometheus community, > >> > >> my prometheus scrapes a lot of endpoints and I only need to 10s for a > month. After that I would like to rollup the data to 5m averages and after > 6 months I would like to have only hourly averages. > >> > >> So I can have quite a long time of data and can see changes over a long > time with the same dashboard. > >> > >> I am sorry to ask this question but it seems like I don't have the > correct search term. Downsampling seems to keep the raw data and all > searches lead me to timescaledb. > >> > >> Cheers > >> Boris > >> > >> -- > >> You received this message because you are subscribed to the Google > Groups "Prometheus Users" group. > >> To unsubscribe from this group and stop receiving emails from it, send > an email to [email protected]. > >> To view this discussion on the web visit > https://groups.google.com/d/msgid/prometheus-users/8D01579C-5696-40BB-9841-A1E3E0094DD1%40kervyn.de > . > > > > -- > Die Selbsthilfegruppe "UTF-8-Probleme" trifft sich diesmal abweichend > im groüen Saal. > > -- > You received this message because you are subscribed to the Google Groups > "Prometheus Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/prometheus-users/CAHjiyFTAWUHwrpfT9NJZxwudSrUveSvsKpG7bcpb3nRm_-CrRA%40mail.gmail.com > . > -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/CALnV8Wg8zOXbx4ifhyFr-dfSw%2BqbXYAKzJBMXa%3DDCi3%2BiJVH5Q%40mail.gmail.com.
