I meant windows 11, but i am using wsl internally for everything
On Mon, 7 Jul 2025, 23:52 Kanak Bhatia, <kanakbhati...@gmail.com> wrote:
> Using windows 11 + wsl
>
> On Mon, 7 Jul 2025, 23:51 Robert Engels, <reng...@ix.netcom.com> wrote:
>
>>
>> I suspect you maybe have an anti virus software (or something like it)
>> that is messing up by reading/interrupting the process. What OS?
>>
>> On Jul 7, 2025, at 1:14 PM, Kanak Bhatia <kanakbhati...@gmail.com> wrote:
>>
>>
>> Its not networked
>>
>> // Read the data back
>> fmt.Println("Fetching object back using GetObject...")
>> r, err := c.GetObject(context.Background(), bucketName, objectName,
>> minio.GetObjectOptions{})
>> if err != nil {
>> logError(testName, function, args, startTime, "", "GetObject failed",
>> err)
>> return
>> }
>> defer r.Close()
>> fmt.Println("GetObject successful")
>>
>>
>> // Stat the object
>> st, err := r.Stat()
>> if err != nil {
>> logError(testName, function, args, startTime, "", "Stat object failed",
>> err)
>> return
>> }
>> fmt.Printf("Stat object successful. Object size: %d bytes\n", st.Size)
>>
>> if st.Size != int64(bufSize) { // bufSize is original object size (129 MB)
>> logError(testName, function, args, startTime, "", fmt.Sprintf("Number of
>> bytes does not match, expected %d, got %d", bufSize, st.Size), err)
>> return
>> }
>> fmt.Println("Object size verified against expected buffer size")
>>
>>
>> --------------------------------------------------------
>>
>>
>> // Comparison function after seek
>> cmpData := func(r io.Reader, start, end int) {
>> if end-start == 0 {
>> fmt.Printf("cmpData: no bytes to compare (start: %d, end: %d)\n",
>> start, end)
>> return
>> }
>> fmt.Printf("cmpData: comparing bytes from %d to %d...\n", start, end)
>> buffer := bytes.NewBuffer([]byte{})
>> // This is the critical line where the error occurs
>> if _, err := io.CopyN(buffer, r, int64(bufSize)); err != nil {
>> if err != io.EOF {
>> logError(testName, function, args, startTime, "", "CopyN
>> failed", err)
>> return
>> }
>> // ... rest of cmpData function ...
>> }
>> if !bytes.Equal(buf[start:end], buffer.Bytes()) {
>> logError(testName, function, args, startTime, "", "Incorrect read
>> bytes v/s original buffer", err)
>> return
>> }
>> fmt.Println("cmpData: byte comparison successful — data matches
>> original buffer")
>> }
>>
>>
>>
>> ----------------------------------------------------------------------------
>>
>> On Mon, Jul 7, 2025 at 11:29 PM Steven Hartland <
>> stevenmhartl...@gmail.com> wrote:
>>
>>> What's the source and target, are they networked if so likely a
>>> network issue.
>>>
>>> On Mon, 7 Jul 2025 at 18:12, Kanak Bhatia <kanakbhati...@gmail.com>
>>> wrote:
>>>
>>>> Hi all,
>>>>
>>>> I'm encountering an intermittent issue while using io.CopyN to copy
>>>> 129MB object from reader to writer.
>>>>
>>>> The problem:
>>>> On some iterations, io.CopyN(dst, src, n) fails with unexpected EOF
>>>> even though the same logic succeeds in most runs. The failure typically
>>>> happens after 125–130 MB and returns ~1MB less data than expected.
>>>>
>>>> type debugReader struct {
>>>> r io.Reader
>>>> totalBytes int64
>>>> nextLogMB int64
>>>> }
>>>>
>>>> func (d *debugReader) Read(p []byte) (int, error) {
>>>> n, err := d.r.Read(p)
>>>> d.totalBytes += int64(n)
>>>> if d.totalBytes >= 125*1024*1024 {
>>>> currentMB := d.totalBytes / (1024 * 1024)
>>>> if currentMB >= d.nextLogMB {
>>>> fmt.Printf("🔵 Read %d MB so far\n", currentMB)
>>>> d.nextLogMB = currentMB + 1
>>>> }
>>>> }
>>>> return n, err
>>>> }
>>>>
>>>> type debugWriter struct {
>>>> w io.Writer
>>>> totalBytes int64
>>>> nextLogMB int64
>>>> }
>>>>
>>>> func (d *debugWriter) Write(p []byte) (int, error) {
>>>> n, err := d.w.Write(p)
>>>> d.totalBytes += int64(n)
>>>> if d.totalBytes >= 125*1024*1024 {
>>>> currentMB := d.totalBytes / (1024 * 1024)
>>>> if currentMB >= d.nextLogMB {
>>>> fmt.Printf("🟡 Written %d MB so far\n", currentMB)
>>>> d.nextLogMB = currentMB + 1
>>>> }
>>>> }
>>>> return n, err
>>>> }
>>>>
>>>> func SafeCopyN(dst *debugWriter, src *debugReader, n int64) (int64,
>>>> error) {
>>>> fmt.Printf("\n🔁 Starting SafeCopyN for %d bytes\n", n)
>>>> written, err := io.CopyN(dst, src, n)
>>>> fmt.Printf("📊 Total bytes read: %d\n", src.totalBytes)
>>>> fmt.Printf("📊 Total bytes written: %d\n", dst.totalBytes)
>>>>
>>>> const allowedDrift = 128 * 1024
>>>>
>>>> if err != nil {
>>>> diff := n - written
>>>> if err == io.ErrUnexpectedEOF && diff <= allowedDrift {
>>>> fmt.Printf("⚠ Accepting unexpected EOF: copied %d of %d
>>>> bytes (drift: %d bytes)\n", written, n, diff)
>>>> return written, nil
>>>> }
>>>>
>>>> fmt.Printf("❌ CopyN failed: wrote %d of %d bytes, err: %v\n",
>>>> written, n, err)
>>>> if src.totalBytes != dst.totalBytes {
>>>> fmt.Printf("📉 Mismatch: %d bytes read but not written\n",
>>>> src.totalBytes-dst.totalBytes)
>>>> }
>>>> } else {
>>>> fmt.Printf("✅ Successfully copied %d bytes\n", written)
>>>> }
>>>>
>>>> return written, err
>>>> }
>>>>
>>>>
>>>> ---
>>>>
>>>> Here's a real log from one of the failed runs (test runs 20 times,
>>>> fails 2):
>>>>
>>>>
>>>> 🔁 Starting SafeCopyN for 135264256 bytes
>>>> 🔵 Read 125 MB so far
>>>> 🟡 Written 125 MB so far
>>>> 🔵 Read 126 MB so far
>>>> 🟡 Written 126 MB so far
>>>> 🔵 Read 127 MB so far
>>>> 🟡 Written 127 MB so far
>>>> 📊 Total bytes read: 134215680
>>>> 📊 Total bytes written: 134215680
>>>> ❌ CopyN failed: wrote 134215680 of 135264256 bytes, err: unexpected EOF
>>>> 📉 Mismatch: 0 bytes read but not written
>>>>
>>>> Notes:
>>>>
>>>> This only happens 1 to 2 times out of 20 iteration.
>>>>
>>>> No server-side encryption involved.
>>>> ---
>>>>
>>>> Question:
>>>>
>>>> Has anyone seen something similar? Could this be:
>>>>
>>>> Timeout/internal truncation in io.CopyN?
>>>>
>>>> Should io.CopyN tolerate io.ErrUnexpectedEOF more gracefully in this
>>>> case?
>>>>
>>>>
>>>> Any insights or guidance appreciated!
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "golang-nuts" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to golang-nuts+unsubscr...@googlegroups.com.
>>>> To view this discussion visit
>>>> https://groups.google.com/d/msgid/golang-nuts/56d2219e-32ab-4950-ab49-83b5fbbfc4fen%40googlegroups.com
>>>> <https://groups.google.com/d/msgid/golang-nuts/56d2219e-32ab-4950-ab49-83b5fbbfc4fen%40googlegroups.com?utm_medium=email&utm_source=footer>
>>>> .
>>>>
>>> --
>> You received this message because you are subscribed to the Google Groups
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> To view this discussion visit
>> https://groups.google.com/d/msgid/golang-nuts/CABXaED640tj-JF%2Bss0pH72WPxXR%3D4O0tRWE98XXzieHDCRbf_A%40mail.gmail.com
>> <https://groups.google.com/d/msgid/golang-nuts/CABXaED640tj-JF%2Bss0pH72WPxXR%3D4O0tRWE98XXzieHDCRbf_A%40mail.gmail.com?utm_medium=email&utm_source=footer>
>> .
>>
>>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion visit
https://groups.google.com/d/msgid/golang-nuts/CABXaED7ADXf4H6TXpzSLo_9DxhqL_MMzrTaAQvOS0nnuZt3YtQ%40mail.gmail.com.
