storage types for big file

2023-05-30 Thread tom
Hello I have a big file after making changes in ram I need to write it back to disk. I know for text file store it's written line by line. But is there any better storage type for high performance read/writing? maybe binary? Thank you. -- Sent from https://dkinbox.com/ -- To unsubscribe,

Re: storage types for big file

2023-05-30 Thread Andrew Solomon
I haven't done this myself, but my first attempt would be with https://metacpan.org/pod/Path::Tiny using the spew_raw method. On Tue, May 30, 2023 at 1:28 PM wrote: > > Hello > > I have a big file after making changes in ram I need to write it back to > disk. > I know for text file store it's

RE: storage types for big file

2023-05-30 Thread Claude Brown via beginners
Do you have the option to "seek" to the correct place in the file to make your changes? For example, perhaps: - Your changes are few compared to writing out the whole file - Your changes do not change the size of the file (or you can pad line-end with spaces) It is an edge case, but just a tho

RE: storage types for big file

2023-05-30 Thread Tom Reed
> Do you have the option to "seek" to the correct place in the file to make > your changes? For example, perhaps: > > - Your changes are few compared to writing out the whole file > - Your changes do not change the size of the file (or you can pad line-end > with spaces) > 1. each time just ch

RE: storage types for big file

2023-05-30 Thread Claude Brown via beginners
That sounds positive. You should be able to avoid most of the overhead of writing the file. The general idea is that you are "updating in place" rather than writing out the whole file. Something like below is what I was thinking, but it isn't tested! Make sure you have a copy of your input

RE: storage types for big file

2023-05-30 Thread Tom Reed
Hello I am not sure, what does this mean? not changing the line length? but in most case I need to change the length of a line. Thanks again. Tom > That sounds positive. You should be able to avoid most of the overhead of > writing the file. The general idea is that yo

RE: storage types for big file

2023-05-30 Thread Claude Brown via beginners
This is where you need to pad spaces. So, for example: - old: "I am young and fast\n" (20 bytes) - new: "I am old and slow\n" (18 bytes) The second version is shorter, so it will need two spaces before the newline: - new: "I am old and slow__\n" (20 bytes; I put "_" instead of a space) Th