Dennis Lee Bieber wrote:
> The problem though, is that the original poster claimed the Oracle
> data was being emitted in multiple chunks, not as a single output -- and
> they want to avoid collecting the data in a temporary file...
I think he wanted to know whether concurrent reads from Oracle a
Dennis Lee Bieber wrote:
>
> "need"? No... the "copy ... from ..." statement "needs" such, but do
> you really "need" to use "copy ... from ..." to load the data -- or is
> this just a concept that caught your fancy?
In chapter 13 of the PostgreSQL 8.1 documentation ("Performance Tips"),
descendi
On 1/11/07, Dennis Lee Bieber <[EMAIL PROTECTED]> wrote:
> On 11 Jan 2007 04:49:21 -0800, "Sean Davis" <[EMAIL PROTECTED]>
> declaimed the following in comp.lang.python:
>
>
> >
> > The machine running the script is distinct from the Oracle machine
> > which is distinct from the Postgresql machine.
Sean Davis wrote:
> I solved this problem by creating a temporary file as an
> intermediary, but why wait for Oracle to finish dumping data when
> I can potentially be loading into postgres at the same time that
> the data is coming in? So, I am actually
> looking for a solution to this problem
Laurent Pointal wrote:
> Not so sure, there is low CPU in the Python script,
Yes.
> but there may be CPU+disk activity on the database sides [with
> cache management and other optimizations on disk access].
That's it. So data queues up on the database side and you won't get
much value from fake
Sean Davis wrote:
>
> As for the specifics, Oracle data is going to be coming in as a DB-API
> 2 cursor in manageable chunks (and at a relatively slow pace). On the
> postgres loading side, I wanted to use the pscycopg2 copy_from
> function, which expects an open file-like object (that has read an
Sean Davis wrote:
> at the same time that the data is coming in? So, I am actually looking
> for a solution to this problem that doesn't require an intermediate
> file and allows simultaneous reading and writing, with the caveat that
> the data cannot all be read into memory simultaneously, so w
On Jan 11, 3:20 am, Laurent Pointal <[EMAIL PROTECTED]> wrote:
> Bjoern Schliessmann a écrit :
>
> > Sean Davis wrote:
>
> >> The author of one of the python database clients mentioned that
> >> using one thread to retrieve the data from the oracle database and
> >> another to insert the data int
On Jan 10, 9:27 pm, johnf <[EMAIL PROTECTED]> wrote:
> Bjoern Schliessmann wrote:
> > Sean Davis wrote:
>
> >> The author of one of the python database clients mentioned that
> >> using one thread to retrieve the data from the oracle database and
> >> another to insert the data into postgresql wi
Bjoern Schliessmann a écrit :
> Sean Davis wrote:
>
>> The author of one of the python database clients mentioned that
>> using one thread to retrieve the data from the oracle database and
>> another to insert the data into postgresql with something like a
>> pipe between the two threads might mak
Bjoern Schliessmann wrote:
> Sean Davis wrote:
>
>> The author of one of the python database clients mentioned that
>> using one thread to retrieve the data from the oracle database and
>> another to insert the data into postgresql with something like a
>> pipe between the two threads might make
Sean Davis wrote:
> The author of one of the python database clients mentioned that
> using one thread to retrieve the data from the oracle database and
> another to insert the data into postgresql with something like a
> pipe between the two threads might make sense, keeping both IO
> streams bus
12 matches
Mail list logo
