Re: Script to create huge sample files

Parag Kalra wrote: I am curious to know more on UTF First read perlunitut. -- Ruud -- To unsubscribe, e-mail: beginners-unsubscr...@perl.org For additional commands, e-mail: beginners-h...@perl.org http://learn.perl.org/

Re: Script to create huge sample files

Thanks Jeff. Cheers, Parag On Mon, Jan 4, 2010 at 7:41 AM, Jeff Peng wrote: > Parag Kalra: > > Hmmm - > http://search.cpan.org/~dankogai/Encode-2.39/lib/Encode/Guess.pm >> >> It says right at the bottom that below method w

Re: Script to create huge sample files

Parag Kalra: Hmmm - http://search.cpan.org/~dankogai/Encode-2.39/lib/Encode/Guess.pm It says right at the bottom that below method won't work to guess the encoding. :( Encode::Guess maybe work, but not so exactly. Because some Code Bits of an encoding are overlapped (for example,gb2312 and

Re: Script to create huge sample files

Hmmm - http://search.cpan.org/~dankogai/Encode-2.39/lib/Encode/Guess.pm It says right at the bottom that below method won't work to guess the encoding. :( Cheers, Parag On Sun, Jan 3, 2010 at 10:23 PM, Parag Kalra wrote: > Thanks a bunch Shlomi. > > Using your snippet now I am to create eve

Re: Script to create huge sample files

Thanks a bunch Shlomi. Using your snippet now I am to create even 1 Giga file. Previously it was throwing 'Out of Memory' message. :) Ok coming to UTF discussion, will the following work: use Encode; my @all_encodings = Encode->encodings(":all"); use Encode::Guess @all_encodings; while(<$sample

Re: Script to create huge sample files

On Sunday 03 Jan 2010 16:25:09 Parag Kalra wrote: > I am curious to know more on UTF and understand related issues that may > creep in my algorithm. Could someone please shed some light on it. > > Can I use following: > > use Encode; > Make sure you add "use strict;" and "use warnings;". > whi

Re: Script to create huge sample files

> What you got from <$sample_file_fh> is maybe different encoding chunk, for > example,iso-8859-1,gb2312 or UTF-8 etc. > You want to translate them to Perl's internal utf8 format firstly,which > includes a utf8 flag and the data part.After translation,utf8 flag should be > on and the data part is t

Re: Script to create huge sample files

Parag Kalra: I am curious to know more on UTF and understand related issues that may creep in my algorithm. Could someone please shed some light on it. Can I use following: use Encode; while(<$sample_file_fh>){ # Encoding into utf data $utf_data = encode("utf8", $_); For the line a

Re: Script to create huge sample files

I am curious to know more on UTF and understand related issues that may creep in my algorithm. Could someone please shed some light on it. Can I use following: use Encode; while(<$sample_file_fh>){ # Encoding into utf data $utf_data = encode("utf8", $_); $data_string = $data_string.

Re: Script to create huge sample files

On Sunday 03 Jan 2010 07:12:32 Parag Kalra wrote: > Thanks Shlomi for your expert comments and I must admit you have got a very > strong vision. :) > You're welcome. > Anyways coming to my first question: > > a.) What do I need to do to make sure that length of new file will > > increase > > >

Re: Script to create huge sample files

Thanks Shlomi for your expert comments and I must admit you have got a very strong vision. :) Anyways coming to my first question: > a.) What do I need to do to make sure that length of new file will increase > every time the step 4 is executed. Although it may have nothing to do with this algor

Re: Script to create huge sample files

Hi Parag! On Saturday 02 Jan 2010 19:56:02 Parag Kalra wrote: > Hello All, > > Major part of my Perl scripting goes in processing text files. And most of > the times I need huge sized text files ( 3 MB +) to perform benchmarking > tests. > > So I am planing to write a Perl script which will cre

Script to create huge sample files

Hello All, Major part of my Perl scripting goes in processing text files. And most of the times I need huge sized text files ( 3 MB +) to perform benchmarking tests. So I am planing to write a Perl script which will create huge sized text file of the sample file which it will receive as first In