Yes i was referring to node.
*Really just to illustrate*, (don't slap : ) this is what i had in head
when i started to dig this problem,
fs.createReadStream("some.file")
.on('error', console.error.bind(console))
.pipe(split(/\r?\n/))
.pipe(through(function (byteData, enc, next) {
next(null, byteData.toString().replace(/[^\x00-\x7F]/g, "?"))
}))
.pipe(through(function (byteData, enc, next) {
next(null, byteData.toString() + "\r\n")
}))
.pipe(iconv.encodeStream('win1252'))
.on('error', console.error.bind(console))
.pipe(fs.createWriteStream('file-in-win1252.txt'))
.on('error', console.error.bind(console))
as a simple developer, i worry much less about the size of the source, the
simple implementation helps me to reduce errors, the job is done, and this
is standard.
Now, IRL, this is not practicable without an helper like
missisippi.pipeline and this will work fine only if some assumptions are
met, and so on.
Better is preferable to worse when perfection is out of your scope, i guess.
> The equivalent of this in Go would be io.Reader and io.Writer and
friends. Transformers in text are lower-level and allow for easier to
implement, but above all, more efficient implementations of transforms.
> For text the latter is often quite important.
> Once you created a transform using Chain, you can convert it to a Reader
or Writer, for instance, using transform.Reader or transform.Writer.
> BTW, regarding your original problem, it is often more desirable to
replace non-ASCII by encoding.ASCIISub (U+001a). This is the default
behavior of the charmap.Windows1252 encoder. If you want to use "?"
instead, it may be better to replace U+001a with '?' instead of just
replacing non-ASCII.
Again, thanks!
I ll do more research based on all those hints.
Le mardi 28 juin 2016 12:54:37 UTC+2, Marcel van Lohuizen a écrit :
>
>
>
> On Tue, Jun 28, 2016 at 12:36 PM, mhhcbon <cpasmabo...@gmail.com
> <javascript:>> wrote:
>
>> > > This does not work in cases where someone want to use a Transformer
>> in a streaming context (e.g. to create a Reader or Writer or to include the
>> Transform in a Chain).
>>
>> This really is what i was looking to implement, somehow,
>>
>> src -> transform -> transform -> sink
>>
>>
>> > I've used bufio.Scanner to implement a custom transforming stream
>> reader.
>>
>> Indeed, that is a step forward for a much better implementation than
>> previous solution. thanks!
>>
>> Is there any formalized stream transform like apis in go that i missed ?
>> Something like another language implements :x
>>
> Are you referring to something like streams in NodeJS?
>
> The equivalent of this in Go would be io.Reader and io.Writer and
> friends. Transformers in text are lower-level and allow for easier to
> implement, but above all, more efficient implementations of transforms. For
> text the latter is often quite important.
>
> Once you created a transform using Chain, you can convert it to a Reader
> or Writer, for instance, using transform.Reader or transform.Writer.
>
> BTW, regarding your original problem, it is often more desirable to
> replace non-ASCII by encoding.ASCIISub (U+001a). This is the default
> behavior of the charmap.Windows1252 encoder. If you want to use "?"
> instead, it may be better to replace U+001a with '?' instead of just
> replacing non-ASCII.
>
>
>
>
>
>
>>
>>
>> Le mardi 28 juin 2016 11:54:25 UTC+2, Tamás Gulácsi a écrit :
>>>
>>> I've used bufio.Scanner to implement a custom transforming stream reader.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to golang-nuts...@googlegroups.com <javascript:>.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
