Hello,
I don't know if this is the best place to post this.
Recently with Perl5 I had a big problem with big data structure using Perl5
HASHes. The problem was with a search in a big database, where to make this
search we used a group of threads with a random entry. Bascially we need to
search from number 010000 to 899999 in this DB, but to get good results,
since not all of this numbers has results, we used a random assortment of
this numbers with many threads as possible. But the problems is that we
can't sort and search the same number 2 times, so, we used a shared HASH
table to mark what numbers we already have used. This works for some time,
but the amount of memory used was huge, something bigger than 300Mb, and
increasing.
To fix that I changed the HASH table to a vec() trick, where we treated the
vector as an array of bits, with boolean values. So, each number were a
position in the vector. So memory usage was fixed and the vec() trick is
much more faster!
What I'm thinking is that in Parrot, as is in Perl, we have dynamic data
structures, like dynamic HASH and ARRAYs, with SCALARs that can grow as they
want. To write our code this help a lot, but some parts of our algorithms
need memory and speed, specially if we are working with big matrix and hash
tables. To solve that will be very interesting to can work directly with
"native data structures" (just a name for the basic data structures that we
find on C).
Will be nice to can write this directly on Perl6:
int the_matrix[10][10] ;
and can use the with normal "SCALAR"s or native values:
my $val = 123 ;
int x = 456 ;
the_matrix[0][0] = $val ;
the_matrix[0][1] = x ;
This is just a simple example, but let's imagine that this matrix can be
huge like the vector in the Perl5 example above.
Also, will be nice to can use the same idea with hash tables:
int , bool static_hash{1000} ;
The example above is a hash with 1000 keys, where we have integer keys and
boolean values.
The same idea can be used for int, char* (string), utf8_char*, float,
boolean (bits).
This will open a way to resolve with pure parrot code problems that today we
resolve with native code (XS) on Perl5. And the best thing is be able to
convert this parrot code to native code and keep the speed and know that
this is compiled using native resouces.
The idea is simple and is not so hard to implement and will solve a lot of
issues about memory and speed.
Have a happy new year! ;-P
Regards,
Graciliano M. P.
--
No virus found in this outgoing message.
Checked by AVG Anti-Virus.
Version: 7.0.298 / Virus Database: 265.6.5 - Release Date: 26/12/2004
