Greg Smith writes:
> Joachim Worringen wrote:
>> my Python application (http://perfbase.tigris.org) repeatedly needs to
>> insert lots of data into an exsting, non-empty, potentially large
>> table. Currently, the bottleneck is with the Python application, so I
>> intend to multi-thread it. Each t
On 06/01/2010 05:45 AM, Greg Smith wrote:
Two thoughts. First, build a test performance case assuming it will fail
to scale upwards, looking for problems. If you get lucky, great, but
don't assume this will work--it's proven more difficult than is obvious
in the past for others.
Second, if you d
Joachim Worringen wrote:
my Python application (http://perfbase.tigris.org) repeatedly needs to
insert lots of data into an exsting, non-empty, potentially large
table. Currently, the bottleneck is with the Python application, so I
intend to multi-thread it. Each thread should work on a part of
On 28/05/2010 9:17 PM, Joachim Worringen wrote:
In the "import data" case, however, I really need concurrent processing
on the CPU in the first place, so you may be right on this one. I'll
check it.
If you run into GIL problems, you can always work around it by spawning
multiple processes and
On 05/28/2010 02:55 PM, Craig Ringer wrote:
On 28/05/10 17:41, Joachim Worringen wrote:
Greetings,
my Python application (http://perfbase.tigris.org) repeatedly needs to
insert lots of data into an exsting, non-empty, potentially large table.
Currently, the bottleneck is with the Python applica
On 28/05/10 17:41, Joachim Worringen wrote:
Greetings,
my Python application (http://perfbase.tigris.org) repeatedly needs to
insert lots of data into an exsting, non-empty, potentially large table.
Currently, the bottleneck is with the Python application, so I intend to
multi-thread it.
That
On 28 May 2010, at 12:14, Martin Gainty wrote:
> Good Afternoon Szymon!
>
> Could you explain what a Python GIL is? and if there is any workaround to
> Python GIL we can implement to achieve better performance..possibly at the
> database level?
See here: http://en.wikipedia.org/wiki/Global_In
nt donné que les email peuvent facilement
être sujets à la manipulation, nous ne pouvons accepter aucune responsabilité
pour le contenu fourni.
Date: Fri, 28 May 2010 11:48:16 +0200
Subject: Re: [GENERAL] INSERTing lots of data
From: mabew...@gmail.com
To: joachim.worrin...@iathh.de
CC: pgsql-g
On 05/28/2010 11:48 AM, Szymon Guz wrote:
Remember about Python's GIL in some Python implementations so those
threads could be serialized at the Python level.
My multi-threaded queries scale nicely with Python 2.6 on Linux, so this
is not an issue here. But the queries do not perform concurren
2010/5/28 Joachim Worringen
> Greetings,
>
> my Python application (http://perfbase.tigris.org) repeatedly needs to
> insert lots of data into an exsting, non-empty, potentially large table.
> Currently, the bottleneck is with the Python application, so I intend to
> multi-thread it. Each thread
Greetings,
my Python application (http://perfbase.tigris.org) repeatedly needs to
insert lots of data into an exsting, non-empty, potentially large table.
Currently, the bottleneck is with the Python application, so I intend to
multi-thread it. Each thread should work on a part of the input fi
11 matches
Mail list logo
