On Monday, 13 June 2022 at 01:55:11 UTC+2 Const V wrote:
> The way I'm using is "b" is a large buffer - hundreds of MB.
> I want to stop processing after the string is found using "quit".
>
> for i := 0; i < cores; i++ {
> go func(start, end, i int, quit chan string) {
> ch <- bytes.Contains(b.Bytes()[start:end], find)
> select {
> case <-quit:
> quit <- "bye"
> return
> }
> }(start, end, i, quit)
>
You missed
start := i * chunksize
end := min(start+chunksize+overlap, filelen)
But even if that's what you meant, I'm not understanding what you're trying
to do with this 'quit' channel. The two steps in the goroutine
(bytes.Contains, followed by <-quit) happen sequentially, one after the
other. "bytes.Contains" is not interruptible; it will scan the whole string
you've given, until either it finds the string or it exhausts the string.
You simply have to wait for it to complete.
Furthermore, I think that trying to parallize a string search this way is
almost certain not to bring any benefits, and is very likely to be slower.
This is because the bottleneck in string searching is not the CPU core;
unless the data is already in cache (which it won't be, if it's hundreds of
megabytes), the bottleneck is the bandwidth into main RAM (DRAM).
A single thread scanning from start to end of the buffer has the advantages
that (a) all accesses are sequential, which is an access pattern DRAMs are
optimised for, and (b) it can stop as soon as the search has found the
string. Splitting the search across (say) 4 cores will make things worse.
You'll still be limited by RAM bandwidth, but the access patterns will jump
around, and as I've just explained, you won't be able to terminate the
other searches early.
How do your two original benchmarks compare to doing the simple option,
i.e. just
bytes.Contains(b.Bytes(), find)
- with no goroutines?
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/58ec2a88-e5c5-4c0b-abf6-d404a81669d9n%40googlegroups.com.
