I would try your program in a Linux vm. I highly suspect an OS issue. If it works, I would try on a pure Windows 10 VM. On Jul 7, 2025, at 1:25 PM, Kanak Bhatia <kanakbhati...@gmail.com> wrote:
I meant windows 11, but i am using wsl internally for everything Using windows 11 + wsl
I suspect you maybe have an anti virus software (or something like it) that is messing up by reading/interrupting the process. What OS? Its not networked // Read the data back fmt.Println("Fetching object back using GetObject...") r, err := c.GetObject(context.Background(), bucketName, objectName, minio.GetObjectOptions{}) if err != nil { logError(testName, function, args, startTime, "", "GetObject failed", err) return } defer r.Close() fmt.Println("GetObject successful") // Stat the object st, err := r.Stat() if err != nil { logError(testName, function, args, startTime, "", "Stat object failed", err) return } fmt.Printf("Stat object successful. Object size: %d bytes\n", st.Size) if st.Size != int64(bufSize) { // bufSize is original object size (129 MB) logError(testName, function, args, startTime, "", fmt.Sprintf("Number of bytes does not match, expected %d, got %d", bufSize, st.Size), err) return } fmt.Println("Object size verified against expected buffer size") --------------------------------------------------------
// Comparison function after seek
cmpData := func(r io.Reader, start, end int) {
if end-start == 0 {
fmt.Printf("cmpData: no bytes to compare (start: %d, end: %d)\n", start, end)
return
}
fmt.Printf("cmpData: comparing bytes from %d to %d...\n", start, end)
buffer := bytes.NewBuffer([]byte{})
// This is the critical line where the error occurs
if _, err := io.CopyN(buffer, r, int64(bufSize)); err != nil {
if err != io.EOF {
logError(testName, function, args, startTime, "", "CopyN failed", err)
return
}
// ... rest of cmpData function ...
}
if !bytes.Equal(buf[start:end], buffer.Bytes()) {
logError(testName, function, args, startTime, "", "Incorrect read bytes v/s original buffer", err)
return
}
fmt.Println("cmpData: byte comparison successful — data matches original buffer")
}
----------------------------------------------------------------------------
What's the source and target, are they networked if so likely a network issue.
Hi all,
I'm encountering an intermittent issue while using io.CopyN to copy 129MB object from reader to writer.
The problem:
On some iterations, io.CopyN(dst, src, n) fails with unexpected EOF even though the same logic succeeds in most runs. The failure typically happens after 125–130 MB and returns ~1MB less data than expected.
type debugReader struct {
r io.Reader
totalBytes int64
nextLogMB int64
}
func (d *debugReader) Read(p []byte) (int, error) {
n, err := d.r.Read(p)
d.totalBytes += int64(n)
if d.totalBytes >= 125*1024*1024 {
currentMB := d.totalBytes / (1024 * 1024)
if currentMB >= d.nextLogMB {
fmt.Printf("🔵 Read %d MB so far\n", currentMB)
d.nextLogMB = currentMB + 1
}
}
return n, err
}
type debugWriter struct {
w io.Writer
totalBytes int64
nextLogMB int64
}
func (d *debugWriter) Write(p []byte) (int, error) {
n, err := d.w.Write(p)
d.totalBytes += int64(n)
if d.totalBytes >= 125*1024*1024 {
currentMB := d.totalBytes / (1024 * 1024)
if currentMB >= d.nextLogMB {
fmt.Printf("🟡 Written %d MB so far\n", currentMB)
d.nextLogMB = currentMB + 1
}
}
return n, err
}
func SafeCopyN(dst *debugWriter, src *debugReader, n int64) (int64, error) {
fmt.Printf("\n🔁 Starting SafeCopyN for %d bytes\n", n)
written, err := io.CopyN(dst, src, n)
fmt.Printf("📊 Total bytes read: %d\n", src.totalBytes)
fmt.Printf("📊 Total bytes written: %d\n", dst.totalBytes)
const allowedDrift = 128 * 1024
if err != nil {
diff := n - written
if err == io.ErrUnexpectedEOF && diff <= allowedDrift {
fmt.Printf("⚠ Accepting unexpected EOF: copied %d of %d bytes (drift: %d bytes)\n", written, n, diff)
return written, nil
}
fmt.Printf("❌ CopyN failed: wrote %d of %d bytes, err: %v\n", written, n, err)
if src.totalBytes != dst.totalBytes {
fmt.Printf("📉 Mismatch: %d bytes read but not written\n", src.totalBytes-dst.totalBytes)
}
} else {
fmt.Printf("✅ Successfully copied %d bytes\n", written)
}
return written, err
}
---
Here's a real log from one of the failed runs (test runs 20 times, fails 2):
🔁 Starting SafeCopyN for 135264256 bytes
🔵 Read 125 MB so far
🟡 Written 125 MB so far
🔵 Read 126 MB so far
🟡 Written 126 MB so far
🔵 Read 127 MB so far
🟡 Written 127 MB so far
📊 Total bytes read: 134215680
📊 Total bytes written: 134215680
❌ CopyN failed: wrote 134215680 of 135264256 bytes, err: unexpected EOF
📉 Mismatch: 0 bytes read but not written
Notes:
This only happens 1 to 2 times out of 20 iteration.
No server-side encryption involved.
---
Question:
Has anyone seen something similar? Could this be:
Timeout/internal truncation in io.CopyN?
Should io.CopyN tolerate io.ErrUnexpectedEOF more gracefully in this case?
Any insights or guidance appreciated!
--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/golang-nuts/56d2219e-32ab-4950-ab49-83b5fbbfc4fen%40googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/golang-nuts/CABXaED640tj-JF%2Bss0pH72WPxXR%3D4O0tRWE98XXzieHDCRbf_A%40mail.gmail.com.
--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/golang-nuts/69B05896-F4BB-4D6B-83BB-C6B817BDC457%40ix.netcom.com.
|
