Reading from stdin significantly slower than reading file directly?

Hi, Relative beginner to D-lang here, and I'm very confused by the apparent performance disparity I've noticed between programs that do the following: 1) cat some-large-file | D-program-reading-stdin-byLine() 2) D-program-directly-reading-file-byLine() using File() struct The D-lang differe

Re: Reading from stdin significantly slower than reading file directly?

Thank you all very much for your detailed feedback! I wound up pulling the "TREE_GRM_ESTN.csv" file referred to by Jon and used it in subsequent tests. Created D-programs for reading directly through a File() structure, versus reading byLine() from the stdin alias. After copying the large CS

200-600x slower Dlang performance with nested foreach loop

Greetings Dlang wizards, I seek knowledge/understanding of a very frustrating phenomenon I've experienced over the past several days. The problem space: 1) Read a list of strings from a file 2) De-duplicate all strings into the subset of unique strings 3) Sort the subset of unique strings by

Re: 200-600x slower Dlang performance with nested foreach loop

On Tuesday, 26 January 2021 at 17:56:22 UTC, Paul Backus wrote: It would be much easier for us to help you with this if you could post the full program, or at the very least a reduced version that reproduces the same issue. [1] Since your attempts so far have failed to fix the problem, it is

Re: 200-600x slower Dlang performance with nested foreach loop

On Tuesday, 26 January 2021 at 18:17:31 UTC, H. S. Teoh wrote: Do not do this. Every time you call .array it allocates a new array and copies all its contents over. If this code runs frequently, it will cause a big performance hit, not to mention high GC load. The function you're looking for

Re: 200-600x slower Dlang performance with nested foreach loop

Greetings all, Many thanks for sharing your collective perspective and advice thus far! It has been very helpful and instructive. I return bearing live data and a minimally complete, compilable, and executable program to experiment with and potentially optimize. The dataset can be pulled from

Re: 200-600x slower Dlang performance with nested foreach loop

On Sunday, 31 January 2021 at 00:53:05 UTC, Steven Schveighoffer wrote: I'd suggest trying it in reverse. If you have the sequence "cba", "ba", "a", then determining "a" is in "ba" is probably cheaper than determining "a" is in "cba". I have user requirements that this application track string