After much work i was able to do it:
The psql script was changed to:
\o '/KakaoStats/bak/groupdup1.txt'
select
data,
usuario,
sum(pontos),
sum(wus)
from usuarios
where data <= 2056
group by data, usuario
;
\o
\o '/KakaoStats/bak/groupdup2.txt'
select
data,
usuario,
sum(pontos),
sum(wus)
from usuarios
where data > 2056 and data <= 2400
group by data, usuario
;
\o
...snip...
\o '/KakaoStats/bak/groupdup8.txt'
select
data,
usuario,
sum(pontos),
sum(wus)
from usuarios
where data > 2600
group by data, usuario
;
\o
Then I had 8 files which i concateneted and then i tried in pgadmin3:
truncate table usuarios2;
vacuum usuarios2;
copy usuarios2 (
data,
usuario,
pontos,
wus
)
from '/KakaoStats/bak/groupdup.txt'
;
Again out of memory error.
Tried to break it the same way i did with the select:
truncate table usuarios2;
vacuum usuarios2;
copy usuarios2 (
data,
usuario,
pontos,
wus
)
from '/KakaoStats/bak/groupdup1.txt'
;
...snip...
copy usuarios2 (
data,
usuario,
pontos,
wus
)
from '/KakaoStats/bak/groupdup8.txt'
;
And again out of memory error. ???
Then I added begin and commit to each of the copy commands and it worked !!!
truncate table usuarios2;
vacuum usuarios2;
begin;
copy usuarios2 (
data,
usuario,
pontos,
wus
)
from '/KakaoStats/bak/groupdup1.txt'
;
commit;
...snip...
begin;
copy usuarios2 (
data,
usuario,
pontos,
wus
)
from '/KakaoStats/bak/groupdup8.txt'
;
commit;
This was with pgadmin3. Does pgadmin3 turns the whole script in one
only transaction?
My hardware: Asus a7v8x-mx motherboard, 80GB Maxtor HD, XP2600/333,
2x512MB memory at the most conservative settings.
---------------------------(end of broadcast)---------------------------
TIP 9: the planner will ignore your desire to choose an index scan if your
joining column's datatypes do not match
