Using windows 11 + wsl
On Mon, 7 Jul 2025, 23:51 Robert Engels, <reng...@ix.netcom.com> wrote:
>
> I suspect you maybe have an anti virus software (or something like it)
> that is messing up by reading/interrupting the process. What OS?
>
> On Jul 7, 2025, at 1:14 PM, Kanak Bhatia <kanakbhati...@gmail.com> wrote:
>
>
> Its not networked
>
> // Read the data back
> fmt.Println("Fetching object back using GetObject...")
> r, err := c.GetObject(context.Background(), bucketName, objectName,
> minio.GetObjectOptions{})
> if err != nil {
> logError(testName, function, args, startTime, "", "GetObject failed",
> err)
> return
> }
> defer r.Close()
> fmt.Println("GetObject successful")
>
>
> // Stat the object
> st, err := r.Stat()
> if err != nil {
> logError(testName, function, args, startTime, "", "Stat object failed",
> err)
> return
> }
> fmt.Printf("Stat object successful. Object size: %d bytes\n", st.Size)
>
> if st.Size != int64(bufSize) { // bufSize is original object size (129 MB)
> logError(testName, function, args, startTime, "", fmt.Sprintf("Number of
> bytes does not match, expected %d, got %d", bufSize, st.Size), err)
> return
> }
> fmt.Println("Object size verified against expected buffer size")
>
>
> --------------------------------------------------------
>
>
> // Comparison function after seek
> cmpData := func(r io.Reader, start, end int) {
> if end-start == 0 {
> fmt.Printf("cmpData: no bytes to compare (start: %d, end: %d)\n",
> start, end)
> return
> }
> fmt.Printf("cmpData: comparing bytes from %d to %d...\n", start, end)
> buffer := bytes.NewBuffer([]byte{})
> // This is the critical line where the error occurs
> if _, err := io.CopyN(buffer, r, int64(bufSize)); err != nil {
> if err != io.EOF {
> logError(testName, function, args, startTime, "", "CopyN
> failed", err)
> return
> }
> // ... rest of cmpData function ...
> }
> if !bytes.Equal(buf[start:end], buffer.Bytes()) {
> logError(testName, function, args, startTime, "", "Incorrect read
> bytes v/s original buffer", err)
> return
> }
> fmt.Println("cmpData: byte comparison successful — data matches
> original buffer")
> }
>
>
>
> ----------------------------------------------------------------------------
>
> On Mon, Jul 7, 2025 at 11:29 PM Steven Hartland <stevenmhartl...@gmail.com>
> wrote:
>
>> What's the source and target, are they networked if so likely a
>> network issue.
>>
>> On Mon, 7 Jul 2025 at 18:12, Kanak Bhatia <kanakbhati...@gmail.com>
>> wrote:
>>
>>> Hi all,
>>>
>>> I'm encountering an intermittent issue while using io.CopyN to copy
>>> 129MB object from reader to writer.
>>>
>>> The problem:
>>> On some iterations, io.CopyN(dst, src, n) fails with unexpected EOF even
>>> though the same logic succeeds in most runs. The failure typically happens
>>> after 125–130 MB and returns ~1MB less data than expected.
>>>
>>> type debugReader struct {
>>> r io.Reader
>>> totalBytes int64
>>> nextLogMB int64
>>> }
>>>
>>> func (d *debugReader) Read(p []byte) (int, error) {
>>> n, err := d.r.Read(p)
>>> d.totalBytes += int64(n)
>>> if d.totalBytes >= 125*1024*1024 {
>>> currentMB := d.totalBytes / (1024 * 1024)
>>> if currentMB >= d.nextLogMB {
>>> fmt.Printf("🔵 Read %d MB so far\n", currentMB)
>>> d.nextLogMB = currentMB + 1
>>> }
>>> }
>>> return n, err
>>> }
>>>
>>> type debugWriter struct {
>>> w io.Writer
>>> totalBytes int64
>>> nextLogMB int64
>>> }
>>>
>>> func (d *debugWriter) Write(p []byte) (int, error) {
>>> n, err := d.w.Write(p)
>>> d.totalBytes += int64(n)
>>> if d.totalBytes >= 125*1024*1024 {
>>> currentMB := d.totalBytes / (1024 * 1024)
>>> if currentMB >= d.nextLogMB {
>>> fmt.Printf("🟡 Written %d MB so far\n", currentMB)
>>> d.nextLogMB = currentMB + 1
>>> }
>>> }
>>> return n, err
>>> }
>>>
>>> func SafeCopyN(dst *debugWriter, src *debugReader, n int64) (int64,
>>> error) {
>>> fmt.Printf("\n🔁 Starting SafeCopyN for %d bytes\n", n)
>>> written, err := io.CopyN(dst, src, n)
>>> fmt.Printf("📊 Total bytes read: %d\n", src.totalBytes)
>>> fmt.Printf("📊 Total bytes written: %d\n", dst.totalBytes)
>>>
>>> const allowedDrift = 128 * 1024
>>>
>>> if err != nil {
>>> diff := n - written
>>> if err == io.ErrUnexpectedEOF && diff <= allowedDrift {
>>> fmt.Printf("⚠ Accepting unexpected EOF: copied %d of %d
>>> bytes (drift: %d bytes)\n", written, n, diff)
>>> return written, nil
>>> }
>>>
>>> fmt.Printf("❌ CopyN failed: wrote %d of %d bytes, err: %v\n",
>>> written, n, err)
>>> if src.totalBytes != dst.totalBytes {
>>> fmt.Printf("📉 Mismatch: %d bytes read but not written\n",
>>> src.totalBytes-dst.totalBytes)
>>> }
>>> } else {
>>> fmt.Printf("✅ Successfully copied %d bytes\n", written)
>>> }
>>>
>>> return written, err
>>> }
>>>
>>>
>>> ---
>>>
>>> Here's a real log from one of the failed runs (test runs 20 times, fails
>>> 2):
>>>
>>>
>>> 🔁 Starting SafeCopyN for 135264256 bytes
>>> 🔵 Read 125 MB so far
>>> 🟡 Written 125 MB so far
>>> 🔵 Read 126 MB so far
>>> 🟡 Written 126 MB so far
>>> 🔵 Read 127 MB so far
>>> 🟡 Written 127 MB so far
>>> 📊 Total bytes read: 134215680
>>> 📊 Total bytes written: 134215680
>>> ❌ CopyN failed: wrote 134215680 of 135264256 bytes, err: unexpected EOF
>>> 📉 Mismatch: 0 bytes read but not written
>>>
>>> Notes:
>>>
>>> This only happens 1 to 2 times out of 20 iteration.
>>>
>>> No server-side encryption involved.
>>> ---
>>>
>>> Question:
>>>
>>> Has anyone seen something similar? Could this be:
>>>
>>> Timeout/internal truncation in io.CopyN?
>>>
>>> Should io.CopyN tolerate io.ErrUnexpectedEOF more gracefully in this
>>> case?
>>>
>>>
>>> Any insights or guidance appreciated!
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to golang-nuts+unsubscr...@googlegroups.com.
>>> To view this discussion visit
>>> https://groups.google.com/d/msgid/golang-nuts/56d2219e-32ab-4950-ab49-83b5fbbfc4fen%40googlegroups.com
>>> <https://groups.google.com/d/msgid/golang-nuts/56d2219e-32ab-4950-ab49-83b5fbbfc4fen%40googlegroups.com?utm_medium=email&utm_source=footer>
>>> .
>>>
>> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion visit
> https://groups.google.com/d/msgid/golang-nuts/CABXaED640tj-JF%2Bss0pH72WPxXR%3D4O0tRWE98XXzieHDCRbf_A%40mail.gmail.com
> <https://groups.google.com/d/msgid/golang-nuts/CABXaED640tj-JF%2Bss0pH72WPxXR%3D4O0tRWE98XXzieHDCRbf_A%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
>
>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion visit
https://groups.google.com/d/msgid/golang-nuts/CABXaED4Fkf9FsLWVFaYFYAHC78Lf_d4%3DQRqVD2BpR4vsWsZGYQ%40mail.gmail.com.
