Dennis Lee Bieber wrote:
> The problem though, is that the original poster claimed the Oracle
> data was being emitted in multiple chunks, not as a single output -- and
> they want to avoid collecting the data in a temporary file...
I think he wanted to know whether concurrent reads from Oracle a
Dennis Lee Bieber wrote:
>
> "need"? No... the "copy ... from ..." statement "needs" such, but do
> you really "need" to use "copy ... from ..." to load the data -- or is
> this just a concept that caught your fancy?
In chapter 13 of the PostgreSQL 8.1 documentation ("Performance Tips"),
descendi
On 1/11/07, Dennis Lee Bieber <[EMAIL PROTECTED]> wrote:
> On 11 Jan 2007 04:49:21 -0800, "Sean Davis" <[EMAIL PROTECTED]>
> declaimed the following in comp.lang.python:
>
>
> >
> > The machine running the script is distinct from the Oracle machine
> > which is distinct from the Postgresql machine.
Sean Davis wrote:
> I solved this problem by creating a temporary file as an
> intermediary, but why wait for Oracle to finish dumping data when
> I can potentially be loading into postgres at the same time that
> the data is coming in? So, I am actually
> looking for a solution to this problem
Laurent Pointal wrote:
> Not so sure, there is low CPU in the Python script,
Yes.
> but there may be CPU+disk activity on the database sides [with
> cache management and other optimizations on disk access].
That's it. So data queues up on the database side and you won't get
much value from fake
Sean Davis wrote:
>
> As for the specifics, Oracle data is going to be coming in as a DB-API
> 2 cursor in manageable chunks (and at a relatively slow pace). On the
> postgres loading side, I wanted to use the pscycopg2 copy_from
> function, which expects an open file-like object (that has read an
Sean Davis wrote:
> at the same time that the data is coming in? So, I am actually looking
> for a solution to this problem that doesn't require an intermediate
> file and allows simultaneous reading and writing, with the caveat that
> the data cannot all be read into memory simultaneously, so w
On Jan 11, 3:20 am, Laurent Pointal <[EMAIL PROTECTED]> wrote:
> Bjoern Schliessmann a écrit :
>
> > Sean Davis wrote:
>
> >> The author of one of the python database clients mentioned that
> >> using one thread to retrieve the data from the oracle database and
> >> another to insert the data int
On Jan 10, 9:27 pm, johnf <[EMAIL PROTECTED]> wrote:
> Bjoern Schliessmann wrote:
> > Sean Davis wrote:
>
> >> The author of one of the python database clients mentioned that
> >> using one thread to retrieve the data from the oracle database and
> >> another to insert the data into postgresql wi
Bjoern Schliessmann a écrit :
> Sean Davis wrote:
>
>> The author of one of the python database clients mentioned that
>> using one thread to retrieve the data from the oracle database and
>> another to insert the data into postgresql with something like a
>> pipe between the two threads might mak
Bjoern Schliessmann wrote:
> Sean Davis wrote:
>
>> The author of one of the python database clients mentioned that
>> using one thread to retrieve the data from the oracle database and
>> another to insert the data into postgresql with something like a
>> pipe between the two threads might make
Sean Davis wrote:
> The author of one of the python database clients mentioned that
> using one thread to retrieve the data from the oracle database and
> another to insert the data into postgresql with something like a
> pipe between the two threads might make sense, keeping both IO
> streams bus
I am working on a simple script to read from one database (oracle) and
write to another (postgresql). I retrieve the data from oracle in
chunks and drop the data to postgresql continuously. The author of one
of the python database clients mentioned that using one thread to
retrieve the data from
13 matches
Mail list logo
