> On 15 Apr 2016, at 4:59 PM, Roland Huettmann <roland.huettm...@gmail.com> > wrote: > > Though it does NOT work for me as I am reading much larger chunks of data > using "read until..." > > (It is still running for 60 minutes now... I think I have to force-stop it. > No... now it finished with 3,060,000. -:) > > I think it is important to know how to handle "big" data with files, in > memory and placing in fields. And of course in a future nearby 8.1 version > (or sooner?) those in need would love to see this taken care of in the > engine.
Roland are you ensuring to encode your sentinel string as the same encoding as the file? I’m interested to know how much difference that makes as I think this could be the main source of any difference you are seeing between LC 6 and 7/8 on this. One thing to remember is as much as we would like it to be the engine is not prescient. It does not know how much data it will need to read before it will find your sentinel string. What this means in practice is when reading until a string the engine needs to: - constantly resize the memory buffer when it nears overflow for data that is read in because it doesn’t know how big it needs to be - read one byte at a time then check if the last N bytes match your sentinel string Obviously this is significantly more expensive than read for because the buffer is allocated once and the read can read the number of bytes you want and give it to you straight away because it doesn’t need to check it anything matches. I second Mark’s recommendation to move the file parsing to a helper process if possible because no matter what this is going to be a bottleneck in your app. Cheers Monte _______________________________________________ use-livecode mailing list use-livecode@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-livecode
