>From here: http://perldoc.perl.org/utf8.html
I wonder if utf8::upgrade() might do the trick. With write_file() in File::Slurp, passing it a hash with binmode set to :utf8 will get it to write UTF8 output to disk. Perl has in general very good internal UTF support. An aside: You know that leafnode basically does everything you're trying to write, right? -------- Paul Anderson -- VE3HOP On 2012-08-11, at 5:21 PM, Chris Knipe <sav...@savage.za.org> wrote: > Hi Paul, > > I will be inclined to disagree - it depends on whether or not the content > was encoded to begin with (perhaps I should not use the term 'binary' as > such, but rather refer to it as unreadable characters). File::Slurp did > make my life allot easier now yes, but my problem is still not solved. > Looking at the packet capture, it is as clear as daylight to me that either > Net::NNTP or File::Slurp is replacing all \r\n line breaks with \n. The > NNTP RFCs require all lines to be terminated by \r\n. I can't simply use a > regex to replace all instances of \n to \r\n as it will affect the > attachments (binary or not, encoded or not). I've attached an screen > capture out of Wireshark for a single packet (available at > http://www.savage.za.org/capture.png as well) which is showing my problem as > clearly as daylight. > > Line 1: 220 0 <message-id>\r\n This is sent by my application, and it is > properly terminated as required. > Line 2 through to 15 original headers as received by my upstream news > server. The lines WAS previously terminated with \r\n but either Net::NNTP > or File::Slurp, changed all the \r\n values to \n > Line 18 through to next packet is definitely not text anymore, and neither > is it terminated by \r\n anymore as it was previously... > > I am not trying to process, alter, or extract articles in any way, I simply > want to download them, store them, and forward them... > > -- > Chris. > > > > -----Original Message----- > From: Paul Anderson [mailto:wackyvor...@me.com] > Sent: 11 August 2012 18:17 > To: Chris Knipe > Cc: beginners@perl.org > Subject: Re: Net::NNTP inconsistent article retrieval > > Umm... Are you aware that binary attachments on usenet aren't actually *in* > binary? They're encoded in ASCII using one of a number of different methods. > They're just text, until decoded on the receiving end. > > I recommend looking into File::Slurp and CHI. CHI basically implements the > entire caching back-end for you. > > Sent from my iPhone > > On 2012-08-11, at 5:21 AM, Chris Knipe <sav...@savage.za.org> wrote: > >> Hi All, >> >> I'm using Net::NNTP to transfer articles from servers. My aim is to write >> an NNTP proxy. >> >> I obtain the article from my parent news server, write the file to disk, > and >> serve the current, as well as future requests for that article from the >> local file on the disk. My results are very inconsistent, and I suspect >> that it is related to the \r\n line breaks conflicting with the binary > data >> retrieved from the article (a line may include \r\n when it's not actually >> meant to indicate the END of a line). I am currently using: >> >> local $/ = "\r\n"; >> my $Article = $nntp->article($Command['1']); >> if ($Article) { >> # The parent has the article! Let's take it. >> open FILE, ">:raw", $File or die $!; >> binmode(FILE); >> foreach my $line (@$Article) { >> print FILE $line; >> } >> close(FILE); >> $ReturnStr = "220 0 " . $Command['1'] . "\r\n"; >> open FILE, "<:raw", $File; >> binmode(FILE); >> foreach my $line (<FILE>) { >> $ReturnStr .= $line; >> } >> close(FILE); >> $ReturnStr .= ".\r\n"; >> binmode(STDOUT); >> print $ReturnStr; >> >> For some articles, the above code is absolutely fine and no problems are >> returned. For others, the binary attachments to the article (regardless > of >> type of file), is corrupt, and cannot be opened. The results are also > very >> inconsistent, and being binary I'm not exactly sure how to provide samples >> of what works and what doesn't. I've analysed packet captures > excessively, >> and I am definitely getting all the data correctly, and consistently from > my >> parent news server - the problem is related to me writing the file to the >> local disk, and serving the content of that file from the local disk. >> >> Hopefully someone can assist and point me towards the right direction - >> after spending close to a week on this, I'm ready to pull out my hair! :-( >> -- >> Chris. >> >> >> >> -- >> To unsubscribe, e-mail: beginners-unsubscr...@perl.org >> For additional commands, e-mail: beginners-h...@perl.org >> http://learn.perl.org/ >> >> > <capture.png> -- To unsubscribe, e-mail: beginners-unsubscr...@perl.org For additional commands, e-mail: beginners-h...@perl.org http://learn.perl.org/
