Hi,

The time of creating very long index is mostly reduced by disk IO.
In systems like MS-Windows it can be performance killer due the
very inefficients method of buffer flushing. Creating more threads
for simultaneous indexing only increases the problem becaue it
increase number of non linear buffer flushing dividing the available
cache memory to many threads so it's technical nonsense. Please forget
about it. It may help only if nearly whole operation is done in memory.
If you want to really reduce the time of indexing then you have to
reduce disk IO usage. The easiest method is migrating to system which
manage IO operation better and using server with big number of memory
so the whole operation can be done with very small disk IO usage.
For test you can try to use some new Linux distro with EXT4 with
enabled delayed write operation.
If you have to keep existing OS then you can try to increase the memory
used by Harbour RDD driver for buffers during indexing.
In src/rdd/dbfcd/dbfcdx1.c[9191] I described how DBFCDX allocates memory.
As you can see in the comment it should not allocate more then 35Mb memory.
For modern computers such memory block seems to be quite small so you can
try to increase it, i.e. to 256MB.
In line 9167:
   ulSize = ( 1L << 20 ) / ( iLen + 4 );
change the default memory block size, i.e.:
   ulSize = ( 1L << 27 ) / ( iLen + 4 );
recompile DBFCDX (all whole Harbour) and make speed tests.
Please only remember to not execute any other processes or
threads during indexing because it may cause that your indexing
process will not have access to physical memory and swap will
be used what can completely kill the performance.

best regards,
Przemek
_______________________________________________
Harbour mailing list (attachment size limit: 40KB)
Harbour@harbour-project.org
http://lists.harbour-project.org/mailman/listinfo/harbour

Reply via email to