I want to store logs in a simple table.
Here my columns:
Primary-key (auto generated)
timestamp
host
service-on-host
loglevel
msg
json (optional)
I am unsure which DB to choose: Postgres, ElasticSearch or ...?
We don't have high traffic. About 200k rows per day.
My heart beats f
Am 19.08.2016 um 09:42 schrieb John R Pierce:
On 8/19/2016 12:32 AM, Thomas Güttler wrote:
What do you think?
I store most of my logs in flat textfiles syslog style, and use grep for adhoc
querying.
200K rows/day, thats 1.4 million/week, 6 million/month, pretty soon you're
talkin
Am 19.08.2016 um 11:21 schrieb Sameer Kumar:
On Fri, Aug 19, 2016 at 4:58 PM Thomas Güttler mailto:guettl...@thomas-guettler.de>> wrote:
Am 19.08.2016 um 09:42 schrieb John R Pierce:
> On 8/19/2016 12:32 AM, Thomas Güttler wrote:
>> What do you think?
>
Am 19.08.2016 um 12:44 schrieb Andreas Kretschmer:
Thomas Güttler wrote:
How will you be using the logs? What kind of queries? What kind of searches?
Correlating events and logs from various sources could be really easy with
joins, count and summary operations.
Wishes raise with
Thank you Chris for looking at my issue in such detail.
Yes, the parallel feature rocks.
Regards,
Thomas Güttler
Am 19.08.2016 um 22:40 schrieb Chris Mair:
On 19/08/16 10:57, Thomas Güttler wrote:
What do you think?
I store most of my logs in flat textfiles syslog style, and use grep
Am 19.08.2016 um 19:59 schrieb Andy Colson:
On 8/19/2016 2:32 AM, Thomas Güttler wrote:
I want to store logs in a simple table.
Here my columns:
Primary-key (auto generated)
timestamp
host
service-on-host
loglevel
msg
json (optional)
I am unsure which DB to choose: Postgres
ipedia.org/wiki/Reliable_Event_Logging_Protocol
Regards,
Thomas Güttler
--
Thomas Guettler http://www.thomas-guettler.de/
--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general
well. It could well be
overkill for your needs, but I don't know
what your environment looks like.
Thank you for this hint. I will look at it.
Regards,
Thomas Güttler
--
Thomas Guettler http://www.thomas-guettler.de/
--
Sent via pgsql-general mailing list (pgsql-general@postgresql.or
too.
What is the current state of the art?
Is it feasible to store file in PostgreSQL?
Are there already projects which use PostgreSQL as storage backend?
I have the hope, that it would be easier to backup only the files which changed.
Regards,
Thomas Güttler
Related question at rsnapshot ma
Am 28.11.2016 um 16:01 schrieb Adrian Klaver:
On 11/28/2016 06:28 AM, Thomas Güttler wrote:
Hi,
PostgreSQL is rock solid and one of the most reliable parts of our
toolchain.
Thank you
Up to now, we don't store files in PostgreSQL.
I was told, that you must not do this But thi
Am 28.11.2016 um 17:43 schrieb Daniel Verite:
Thomas Güttler wrote:
Up to now we use rsync (via rsnapshot) to backup our data.
But it takes longer and longer for rsync to detect
the changes. Rsync checks many files. But daily only
very few files really change. More than 99.9% don&#
Am 29.11.2016 um 01:52 schrieb Mike Sofen:
From: Thomas Güttler Sent: Monday, November 28, 2016 6:28 AM
...I have 2.3TBytes of files. File count is 17M
Since we already store our structured data in postgres, I think about storing
the files in PostgreSQL, too.
Is it feasible to store file
explained.
But I think the docs don't state the timing of normal AFTER triggers.
Or am I blind?
Regards,
Thomas Güttler
--
Thomas Guettler http://www.thomas-guettler.de/
--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
Yes, you are right.
But "after" the statement could mean before commit, too.
Why not add this?
Proposal:
When no CONSTRAINT option is specified, this command creates a normal trigger.
They
get fired at the end of the statement (IMMEDIATE).
Regards,
Thomas Güttler
Am 26.05.201
rg/home/81-up-gws01w4g-memory32g-emmc-boardwo-vesa-plate.html
# Question
Is running linux with postgres on eMMC a bad idea in general? Or is my hardware
broken?
Regards,
Thomas Güttler
output of dmesg:
[18471.780031] sdhci: Timeout waiting for Buffer Read Ready interrupt
during tuning
Am 08.02.2017 um 07:25 schrieb Thomas Güttler:
> Hi PostgreSQL experts,
>
> ...
# Update
After following the hints from [this answer][1], I could sync via owncloud for
hours, and no file system error occurs. This is no big surprise since now only
very few io-operations happen on
Am 10.02.2017 um 09:16 schrieb Mark Morgan Lloyd:
> On 09/02/17 23:00, Christoph Moench-Tegeder wrote:
>> ## Thomas Güttler (guettl...@thomas-guettler.de):
>>
>>> Is running linux with postgres on eMMC a bad idea in general?
>>
>> I'd say that running any
yself, since there a tons of possible race
conditions:
- inserts can happen during syncing.
- Network can break during syncing.
- inserts into the central table can break (e.g. disk full): No loss at the
satellite database must happen.
- ...
How to solve this with PostgreSQL?
Regards,
T
Am 21.02.2017 um 15:12 schrieb Adrian Klaver:
On 02/21/2017 12:53 AM, Thomas Güttler wrote:
I want to move table rows from one database to an central database.
You actually talking about moving from ~100 databases to the central database,
correct?
Both run PostgreSQL.
Are all the
nt of the rows should happen in background.
Regards,
Thomas Güttler
--
Thomas Guettler http://www.thomas-guettler.de/
--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general
Depending on how much data you want to move, and if the tables have the same
structure, you might also want to consider
using
pg_dump -a
OR
multiple instances of
on satellite
COPY { table_name [ ( column_name [, ...] ) ] | ( query ) }
TO { 'filename' | PROGRAM 'command' | STDOUT }
[ [
I misunderstood your original intent, I thought this was a one time process to
move data to the central database. Given
that it is to be a continuous process a FDW may not be the answer, one of the
reasons being the above question. You will
be denied the data in the remote table during the outag
n the satellite after
transfer.
I don't know how to delete the data which was copied, since inserts can happen
during the copy statement.
Regards,
Thomas Güttler
--
Thomas Guettler http://www.thomas-guettler.de/
--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To
Am 22.02.2017 um 16:00 schrieb Adrian Klaver:
On 02/22/2017 04:51 AM, Thomas Güttler wrote:
I have other concerns: atomar transaction. Movement should happen
completely or not all.
I don't think you can do this reliable (atomic transaction) with
"copy table_name".
You can if
Am 23.02.2017 um 10:33 schrieb Francisco Olarte:
Thomas:
On Wed, Feb 22, 2017 at 1:51 PM, Thomas Güttler
wrote:
I want to **move** the data. The data should get deleted on the satellite
after transfer.
I don't know how to delete the data which was copied, since inserts can
happen durin
Am 23.02.2017 um 13:44 schrieb Leknín Řepánek:
Maybe you can do something like
WITH cte AS (
DELETE FROM t1 WHERE cond
RETURNINIG *
)
INSERT into t2
SELECT * FROM cte;
To move rows between tables with combination with fdw_postgres and
foreign table.
... this way you don't need the second t
without a database
adapter like psycopg2.
Regards,
Thomas
Am 23.02.2017 um 17:40 schrieb Francisco Olarte:
Thomas:
On Thu, Feb 23, 2017 at 4:16 PM, Thomas Güttler
wrote:
Am 22.02.2017 um 16:00 schrieb Adrian Klaver:
only written on a successful transfer. To improve the chances of
successful
d the other types of servers.
The benefit is not very obvious on the first sight. I think it will saves you
time, money and energy only in the long run.
What do you think?
Regards,
Thomas Güttler
--
I am looking for feedback for my personal programming guidelines:
https://github.com/guettli/pr
Am 30.04.2017 um 15:39 schrieb Thomas Delrue:
> On April 30, 2017 1:37:02 PM GMT+02:00, "Thomas Güttler"
> wrote:
>> Is is possible that PostgreSQL will replace these building blocks in
>> the future?
>>
>> - redis (Caching)
>> - rabbitmq (amqp)
&g
Am 30.04.2017 um 17:09 schrieb Bill Moran:
> On Sun, 30 Apr 2017 13:37:02 +0200
> Thomas Güttler wrote:
>
>> Is is possible that PostgreSQL will replace these building blocks in the
>> future?
>>
>> - redis (Caching)
>> - rabbitmq (amqp)
>> - s
Am 02.05.2017 um 05:43 schrieb Jeff Janes:
On Sun, Apr 30, 2017 at 4:37 AM, Thomas Güttler mailto:guettl...@thomas-guettler.de>> wrote:
Is is possible that PostgreSQL will replace these building blocks in the
future?
- redis (Caching)
PostgreSQL has its own caching. It mig
and elsing" in my python code (less
conditions, less bugs, more SQL, more performance)
How could an application which gets written from scratch use PostgreSQL to
implement
row based permissions?
Regards,
Thomas Güttler
--
Thomas Guettler http://www.thomas-guettler.de/
--
Sent via p
Am 25.07.2017 um 12:59 schrieb vinny:
On 2017-07-25 11:40, Thomas Güttler wrote:
I would like to reduce the "ifing and elsing" in my python code (less
conditions, less bugs, more SQL, more performance)
Regards,
Thomas Güttler
A quick brainstorm:
You could, probably...
but you
Am 27.07.2017 um 12:05 schrieb vinny:
On 2017-07-27 10:27, Thomas Güttler wrote:
Am 25.07.2017 um 12:59 schrieb vinny:
On 2017-07-25 11:40, Thomas Güttler wrote:
I would like to reduce the "ifing and elsing" in my python code (less
conditions, less bugs, more SQL, more p
an object or not.
This is an argument for permission checking in code.
On the other side I still think perm checking in SQL WHERE has more benefits.
Regards,
Thomas Güttler
--
Thomas Guettler http://www.thomas-guettler.de/
--
Sent via pgsql-general mailing list (pgsql-general@postgresq
.postgresql.org/docs/current/static/app-pg-dumpall.html
How could the wording look like? I am not a native speaker
Or do you have a better/different idea?
Regards,
Thomas Güttler
--
Thomas Guettler http://www.thomas-guettler.de/
I am looking for feedback: https://github.com/guettli/program
Just for the records, I asked here the same question and got some replies:
https://dba.stackexchange.com/questions/186045/how-do-i-prevent-changes-to-my-database
Am 11.09.2017 um 15:25 schrieb Thomas Güttler:
I did a stupid mistake. I run the final pg_dumpall without switching to
single-user
We run a PostgreSQL 9.6 server in a virtual machine.
The virtual machine is managed by the customer.
He does backup the VM.
Is this enough, is this safe?
Regards,
Thomas Güttler
--
Thomas Guettler http://www.thomas-guettler.de/
I am looking for feedback: https://github.com/guettli
Thank you for "quiesced vs non-quiesced". I will ask the company providing the
VM.
Regards,
Thomas Güttler
Am 21.09.2017 um 03:48 schrieb Michael Paquier:
On Wed, Sep 20, 2017 at 5:45 PM, Albe Laurenz wrote:
Thomas Güttler wrote:
We run a PostgreSQL 9.6 server in a virtual mac
: In my case a short down-time is no problem.
Regards,
Thomas
Am 20.09.2017 um 09:59 schrieb Thomas Güttler:
Just for the records, I asked here the same question and got some replies:
https://dba.stackexchange.com/questions/186045/how-do-i-prevent-changes-to-my-database
Am 11.09.2017 um 15:25
40 matches
Mail list logo
