RE: Indexing about 40 Billion Entries

2012-06-21 Thread Rick James
Possibly worse than that, since it will rebuild the 'first' index again. > -Original Message- > From: mos [mailto:mo...@fastmail.fm] > Sent: Thursday, June 21, 2012 7:51 AM > To: mysql@lists.mysql.com > Subject: Re: Indexing about 40 Billion Entries > > A

Re: Indexing about 40 Billion Entries

2012-06-21 Thread Christian Koetteritzsch
Hi, thanks for the hint, I was testing the settings for the myisam_sort_buffer_size so I totally forgot it. But I have another three of these databases to do, so next time I do it with one ALTER statement Christian Am 21.06.2012 16:50, schrieb mos: At 02:04 AM 6/21/2012, you wrote: Thank yo

Re: Indexing about 40 Billion Entries

2012-06-21 Thread mos
At 02:04 AM 6/21/2012, you wrote: Thank you a lot. The first indexing process finished after about 13 hours, so I think the problem is solved now. I set the myisam_sort_bufffer_size to 10GB. The "first indexing process"??? You should have created all of your indexes with one Alter statement.

Re: Indexing about 40 Billion Entries

2012-06-21 Thread Brent Clark
On 20/06/2012 11:45, Christian Koetteritzsch wrote: Hi guys, As the title says I'm trying to index 40 billion entries with two indexes on a server with 16 cores and 128GB RAM. The table is the one below and it is a myisam table. The *.myd file is about 640GB Hiya I am unable to help. But

Re: Indexing about 40 Billion Entries

2012-06-21 Thread Christian Koetteritzsch
where PARTITION shines. (I have not seen such [yet] in your application.) -Original Message- From: Christian Koetteritzsch [mailto:ckoetteritz...@e-humanities.net] Sent: Wednesday, June 20, 2012 1:42 PM To: Rick James Cc: Ananda Kumar; mysql@lists.mysql.com Subject: Re: Indexing about 40

RE: Indexing about 40 Billion Entries

2012-06-20 Thread Rick James
t; > I hope the informations I gave you are helpful. If you have more > questions than you can ask them. > > Kind regards > > Christian > > > > > >> -Original Message- > >> From: Ananda Kumar [mailto:anan...@gmail.com] > >> Sent: Wednesday, Ju

RE: Indexing about 40 Billion Entries

2012-06-20 Thread Rick James
(ruid1, ruid2) will help for AND, but not at all for OR. > -Original Message- > From: Shawn Green [mailto:shawn.l.gr...@oracle.com] > Sent: Wednesday, June 20, 2012 2:30 PM > To: mysql@lists.mysql.com > Subject: Re: Indexing about 40 Billion Entries > > On 6/20/20

Re: Indexing about 40 Billion Entries

2012-06-20 Thread Shawn Green
On 6/20/2012 5:45 AM, Christian Koetteritzsch wrote: Hi guys, As the title says I'm trying to index 40 billion entries with two indexes on a server with 16 cores and 128GB RAM. The table is the one below and it is a myisam table. The *.myd file is about 640GB DROP TABLE IF EXISTS `l4_link`; CRE

Re: Indexing about 40 Billion Entries

2012-06-20 Thread Christian Koetteritzsch
ul. If you have more questions than you can ask them. Kind regards Christian -Original Message- From: Ananda Kumar [mailto:anan...@gmail.com] Sent: Wednesday, June 20, 2012 4:37 AM To: Christian Koetteritzsch Cc:mysql@lists.mysql.com Subject: Re: Indexing about 40 Billion Entries looks

Re: Indexing about 40 Billion Entries

2012-06-20 Thread Christian Koetteritzsch
com Subject: Re: Indexing about 40 Billion Entries looks like the value that you give for myisam_max_sort_size is not enough for the index creation and hence it doing a "REPAIR WITH KEYCACHE" Use the below query to set the min values required for myisam_max_sort_size to avoid "

RE: Indexing about 40 Billion Entries

2012-06-20 Thread Rick James
static? What are the semantics of the fields? > -Original Message- > From: Ananda Kumar [mailto:anan...@gmail.com] > Sent: Wednesday, June 20, 2012 4:37 AM > To: Christian Koetteritzsch > Cc: mysql@lists.mysql.com > Subject: Re: Indexing about 40 Billion Entries > &g

Re: Indexing about 40 Billion Entries

2012-06-20 Thread Ananda Kumar
looks like the value that you give for myisam_max_sort_size is not enough for the index creation and hence it doing a "REPAIR WITH KEYCACHE" Use the below query to set the min values required for myisam_max_sort_size to avoid "repair with keycache" select a.index_name as index_name,

Indexing about 40 Billion Entries

2012-06-20 Thread Christian Koetteritzsch
Hi guys, As the title says I'm trying to index 40 billion entries with two indexes on a server with 16 cores and 128GB RAM. The table is the one below and it is a myisam table. The *.myd file is about 640GB DROP TABLE IF EXISTS `l4_link`; CREATE TABLE `l4_link` ( `ruid1` int NOT NULL, `r