Re: remove duplicate lines

Dr.Ruud wrote: while ( ) { next if $d{$_}; print; $d{$_} = 1; } Got the idea. Many thanks! -- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] http://learn.perl.org/

Re: remove duplicate lines

"Dr.Ruud" schreef: > beast: >> a 100 >> a 102 >> ... >> c 100 >> a 102 >> ... >> >> I would like to have a list (either array or hash) with unique line . > > perl -ne'$_{$_}||=print' datafile Jason Roth's version: perl -ne'$$_||=print' datafile > or > > perl -pe'$_ x=!$$_++' datafile -

Re: remove duplicate lines

"Dr.Ruud" <[EMAIL PROTECTED]> wrote:oryann9 schreef: > "Dr.Ruud" wrote: beast schreef: >> beast: >>> a 100 >>> a 102 >>> c 100 >>> a 102 >>> b 111 >>> c 100 >>> c 102 >>> c 100 >>> c 100 >>> a 102 >>> ... >>> >>> I would like to have a list (either array or hash) with unique line . >> >> per

Re: remove duplicate lines

If we're just going for confusing concise one liners then I would use perl -ne '$$_||=print' You save three whole characters by using the symbol table instead of a hash :) On 1/11/07, Dr.Ruud <[EMAIL PROTECTED]> wrote: oryann9 schreef: > "Dr.Ruud" <[EMAIL PROTECTED]> wrote: beast schreef: >>

Re: remove duplicate lines

oryann9 schreef: > "Dr.Ruud" <[EMAIL PROTECTED]> wrote: beast schreef: >> beast: >>> a 100 >>> a 102 >>> c 100 >>> a 102 >>> b 111 >>> c 100 >>> c 102 >>> c 100 >>> c 100 >>> a 102 >>> ... >>> >>> I would like to have a list (either array or hash) with unique line . >> >> perl -ne'$_{$_}||=print

Re: remove duplicate lines

"Dr.Ruud" <[EMAIL PROTECTED]> wrote: beast schreef: > a 100 > a 102 > c 100 > a 102 > b 111 > c 100 > c 102 > c 100 > c 100 > a 102 > ... > > I would like to have a list (either array or hash) with unique line . perl -ne'$_{$_}||=print' datafile or perl -pe'$_ x=!$$_++' datafile -- Affi

Re: remove duplicate lines

beast schreef: > a 100 > a 102 > c 100 > a 102 > b 111 > c 100 > c 102 > c 100 > c 100 > a 102 > ... > > I would like to have a list (either array or hash) with unique line . perl -ne'$_{$_}||=print' datafile or perl -pe'$_ x=!$$_++' datafile -- Affijn, Ruud "Gewoon is een tijger." --

Re: remove duplicate lines

beast wrote: > Mumia W. wrote: >> On 01/10/2007 01:35 AM, beast wrote: >>> [...] >>> It will only remove duplicate key. >>> >>> Is this still acceptable in perl (its very ugly =( >>> [...] >> >> This would remove duplicate lines: >> >> use List::MoreUtils qw(uniq); >> use File::Slurp; >>

Re: remove duplicate lines

Mumia W. wrote: On 01/10/2007 01:35 AM, beast wrote: [...] It will only remove duplicate key. Is this still acceptable in perl (its very ugly =( [...] This would remove duplicate lines: use List::MoreUtils qw(uniq); use File::Slurp; my @list = uniq read_file 'rm_dup_lines.txt';

Re: remove duplicate lines

On 01/10/2007 01:35 AM, beast wrote: [...] It will only remove duplicate key. Is this still acceptable in perl (its very ugly =( [...] This would remove duplicate lines: use List::MoreUtils qw(uniq); use File::Slurp; my @list = uniq read_file 'rm_dup_lines.txt'; print $_ for @

Re: remove duplicate lines

Owen Cook wrote: my %data_hash; while(){ chomp; # Get rid of line feeds my @bits = split; $data_hash{$bits[0]} = $bits[1]; } It will only remove duplicate key. Is this still acceptable in perl (its very ugly =( while (<>) { chomp; my ($k, $v) = split /,/; my $tmp_key =

Re: remove duplicate lines

On Wed, Jan 10, 2007 at 12:47:25PM +0700, beast wrote: > I have these following data: > > a 100 > a 102 > c 100 > a 102 > b 111 > c 100 > c 102 > c 100 > c 100 > a 102 > ... > > I would like to have a list (either array or hash) with unique line . > Any help would appreciated. > Thanks. > 1.

Re: remove duplicate lines

Mathew Snyder wrote: The way I would do it is to place the initial data instance into a hash if it doesn't already exist. Just do a next if it does. It will remove duplicate key, not line. my %hash=(); while (<>) { chomp; my ($key, $val) = split /,/; $hash{$key} = $val; } while ( m

Re: remove duplicate lines

beast wrote: > I have these following data: > > a 100 > a 102 > c 100 > a 102 > b 111 > c 100 > c 102 > c 100 > c 100 > a 102 > ... > > I would like to have a list (either array or hash) with unique line . > Any help would appreciated. > Thanks. > > > The way I would do it is to place the init

Re: remove duplicate lines

Tielman Koekemoer (TNE) wrote: I'm also very new to Perl and wrote a long and newbyish script that does exactly what the Unix command "sort FILENAME | uniq" does just to see how it can be done. How long? Because you can do that on one line in perl. :-) Hi John (we're not worthy - Wa

Re: remove duplicate lines

Chris Devers wrote: Or the forthcoming, more up-to-date _Perl Best Practices_: Though in this case you'll have to wait until July or August for it to be released, but I have no doubt that it'll be worth the wait. Yes, anything by Dr. Conway will b

RE: remove duplicate lines

On Mon, 13 Jun 2005, Thomas Bätzler wrote: > It also used to be part of a Book/CD set called the Perl Bookshelf. > AFAIK [_Perl Cookbook_ is] out of print but you might be able to pick > it up at a used book store (or onlinle at ABEbooks) cheaply. The first edition went out of print, but the seco

RE: remove duplicate lines

Tielman Koekemoer (TNE) <[EMAIL PROTECTED]> asked: > I've just come back to your post above (30/05/05) and I've > tried to get the internal workings of the code to agree with > my brain. Please help! > > > perl -e'print sort grep !$seen{$_}++, <>' FILENAME > > Initially I thought you were creat

RE: remove duplicate lines

On Mon, 13 Jun 2005, Tielman Koekemoer (TNE) wrote: > PS Will the Perl Cookbook have examples to common tools such as this? Yes. Oodles of them. > What does everyone think of the book - very helpful? Extremely. -- Chris Devers -- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional co

RE: remove duplicate lines

> > I'm also very new to Perl and wrote a long and newbyish script that > > does exactly what the Unix command "sort FILENAME | uniq" > does just to > > see how it can be done. > > How long? Because you can do that on one line in perl. :-) Hi John (we're not worthy - Wayne's World) ;-) I've ju

Re: remove duplicate lines

On 5/30/05, John W. Krahn <[EMAIL PROTECTED]> wrote: > Tielman Koekemoer (TNE) wrote: > > > >>Wow, I'm really confused. I'm trying to remove duplicate > >>lines from a marc21 text file. I have spent countless hours > >>searching for scripts etc. > > > > I'm also very new to Perl and wrote a long a

Re: remove duplicate lines

Tielman Koekemoer (TNE) wrote: Wow, I'm really confused. I'm trying to remove duplicate lines from a marc21 text file. I have spent countless hours searching for scripts etc. I'm also very new to Perl and wrote a long and newbyish script that does exactly what the Unix command "sort FILENA

RE: remove duplicate lines

> Wow, I'm really confused. I'm trying to remove duplicate > lines from a marc21 text file. I have spent countless hours > searching for scripts etc. I'm also very new to Perl and wrote a long and newbyish script that does exactly what the Unix command "sort FILENAME | uniq" does just to see

Re: remove duplicate lines

Hi John Doe, you wrote : *snip* To learn perl, you have to spend hours and hours studying books/docs/examples, thinking, trying etc. Depends on your own individual learning style. Some folk would breeze through the perldocs. Personally, I feel that they read like stereo instructions. I've

Re: remove duplicate lines (OT)

Am Samstag, 28. Mai 2005 00.06 schrieb Bakken, Luke: > > $ cat marc21textfile | uniq > outputfile > > Where's Randal for a UUOC award? Ok, for all using perl for everything and not beeing experienced with cmdline tools as me: $ uniq marc21textfile outputfile "Useless Use Of Cat" (I read the

RE: remove duplicate lines (OT)

> $ cat marc21textfile | uniq > outputfile Where's Randal for a UUOC award? -- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]

[OT] Re: remove duplicate lines

Am Freitag, 27. Mai 2005 23.31 schrieb Eric Walker: > On Friday 27 May 2005 01:22 pm, John Doe wrote: > > Am Freitag, 27. Mai 2005 17.15 schrieb Eric Walker: > > > Can't you run uniq from the command prompt to get rid of the duplicate > > > lines? > > > > > > perlnewbie.. > > > > [...] > > > > Yes

Re: remove duplicate lines

On Friday 27 May 2005 01:22 pm, John Doe wrote: > Am Freitag, 27. Mai 2005 17.15 schrieb Eric Walker: > > Can't you run uniq from the command prompt to get rid of the duplicate > > lines? > > > > perlnewbie.. > > [...] > > Yes I can, > > $ cat marc21textfile | uniq > outputfile > > but the OP expl

Re: remove duplicate lines

Am Freitag, 27. Mai 2005 17.15 schrieb Eric Walker: > Can't you run uniq from the command prompt to get rid of the duplicate > lines? > > perlnewbie.. [...] Yes I can, $ cat marc21textfile | uniq > outputfile but the OP explicitly wanted a perl script and states > For now, I just want to rem

Re: remove duplicate lines

Can't you run uniq from the command prompt to get rid of the duplicate lines? perlnewbie.. On Friday 27 May 2005 09:07 am, John Doe wrote: > Hello > > As an addition to my last post: > > Am Freitag, 27. Mai 2005 13.56 schrieb Jack Daniels (Butch): > > Wow, I'm really confused. I'm trying to remo

Re: remove duplicate lines

Hello As an addition to my last post: Am Freitag, 27. Mai 2005 13.56 schrieb Jack Daniels (Butch): > Wow, I'm really confused. I'm trying to remove duplicate lines from a > marc21 text file. I have spent countless hours searching for scripts etc. > > What I find frustrating while trying to learn

Re: remove duplicate lines

Am Freitag, 27. Mai 2005 13.56 schrieb Jack Daniels (Butch): > Wow, I'm really confused. I'm trying to remove duplicate lines from a > marc21 text file. I have spent countless hours searching for scripts etc. > > What I find frustrating while trying to learn Perl, is that most solutions > assume y

Re: remove duplicate lines

On Fri, 4 Oct 2002, waytech wrote: > hi everyone, > > Thanks for all replys about my question. > > I know here are lots of perl experts. > > it's amazing that someone can short > > that long program into one line. > > I am a perl newbie, i wonder > > whether someone can explain

Re: remove duplicate lines

hi everyone, Thanks for all replys about my question. I know here are lots of perl experts. it's amazing that someone can short that long program into one line. I am a perl newbie, i wonder whether someone can explain what these guys wrote(like what Sudarshan Raghavan wrote

Re: remove duplicate lines

John wrote: > > All, Hello, > This is the first time I've ever seen a bunch of guys reaching for bragging > rights on just how short theirs is...and to document it, too! Then you need to check out Perl golf. :-) http://perlgolf.sourceforge.net/ John -- use Perl; program fulfillment -- T

RE: remove duplicate lines

o pass up. ) Thanks! John-- [EMAIL PROTECTED] -Original Message- From: John W. Krahn [mailto:[EMAIL PROTECTED]] Sent: Wednesday, October 02, 2002 8:14 AM To: [EMAIL PROTECTED] Subject: Re: remove duplicate lines Sudarshan Ra

Re: remove duplicate lines

Sudarshan Raghavan wrote: > > On Wed, 2 Oct 2002, Sudarshan Raghavan wrote: > > > On Wed, 2 Oct 2002, John W. Krahn wrote: > > > > > "John W. Krahn" wrote: > > > > > > > > Sudarshan Raghavan wrote: > > > > > > > > > > On Wed, 2 Oct 2002, jontn_swift wrote: > > > > > > > > > > > This is a classic

Re: remove duplicate lines

On Wed, 2 Oct 2002, Sudarshan Raghavan wrote: > On Wed, 2 Oct 2002, John W. Krahn wrote: > > > "John W. Krahn" wrote: > > > > > > Sudarshan Raghavan wrote: > > > > > > > > On Wed, 2 Oct 2002, jontn_swift wrote: > > > > > > > > > This is a classic case for my favorite one-liner: > > > > > > > >

Re: remove duplicate lines

On Wed, 2 Oct 2002, John W. Krahn wrote: > "John W. Krahn" wrote: > > > > Sudarshan Raghavan wrote: > > > > > > On Wed, 2 Oct 2002, jontn_swift wrote: > > > > > > > This is a classic case for my favorite one-liner: > > > > > > > > perl -e'while(<>){print unless $seen{$_}++}' outfile > > > > > >

Re: remove duplicate lines

"John W. Krahn" wrote: > > Sudarshan Raghavan wrote: > > > > On Wed, 2 Oct 2002, jontn_swift wrote: > > > > > This is a classic case for my favorite one-liner: > > > > > > perl -e'while(<>){print unless $seen{$_}++}' outfile > > > > You favourite one-liner can be even shorter > > perl -n -e 'prin

Re: remove duplicate lines

Sudarshan Raghavan wrote: > > On Wed, 2 Oct 2002, jontn_swift wrote: > > > This is a classic case for my favorite one-liner: > > > > perl -e'while(<>){print unless $seen{$_}++}' outfile > > You favourite one-liner can be even shorter > perl -n -e 'print unless $seen{$_}++' infile >outfile You

Re: remove duplicate lines

On Wed, 2 Oct 2002, jontn_swift wrote: > This is a classic case for my favorite one-liner: > > perl -e'while(<>){print unless $seen{$_}++}' outfile You favourite one-liner can be even shorter perl -n -e 'print unless $seen{$_}++' infile >outfile -- To unsubscribe, e-mail: [EMAIL PROTECTED] F

Re: remove duplicate lines

This is a classic case for my favorite one-liner: perl -e'while(<>){print unless $seen{$_}++}' outfile On Wed, 02 Oct 2002 04:17:29 -0400, Sudarshan Raghavan wrote: > On Fri, 27 Sep 2002, waytech wrote: > >> hi, >> >> i want to remove duplicate lines from one file(original file), and sav

Re: remove duplicate lines

On Tue, 1 Oct 2002, Patricia Hinman wrote: > I wanted to do this without completely loading the > page into an array and then checking for duplicates. The only case the entire file is read into memory is when all the lines in the file are unique. But using a hash reduces the number of compariso

Re: remove duplicate lines

I wanted to do this without completely loading the page into an array and then checking for duplicates. So I just inserted some flags into my while statement. #!/usr/bin/perl -w $dupli_file = ; chomp ($dupli_file ); $res_file = ; chomp ($res_file ); open (DUPLI, "./$dupli_file") or die "Failed

Re: remove duplicate lines

On Fri, 27 Sep 2002, waytech wrote: > hi, > > i want to remove duplicate lines from one file(original file), and save > the result to another file. > > the origianl file like this: > > > > [EMAIL PROT