Okay, a further update. While rewrite the code, did seem to make a marginal difference, the corruption issue persisted intermittently.
Just today, I did some more digging, and I remembered that I turned on Jumbo Frames for my ethernet port on the M1 Mac Mini. Turning that off, at first glance, seems to have resolved the corruption issue… I’m unclear if this is an Django, Rosetta 2, Big Sur, or M1 Mac mini hardware issue (or a combination of factors). I can’t check, but I’d swear that I had Jumbo frames turned on with My Catalina 2013 iMac. Any ideas or suggestions? - Benjamin > On Dec 22, 2020, at 6:46 AM, Benjamin Schollnick <bscholln...@schollnick.net> > wrote: > > Just an update. > > Working my way through the problem, it appears that either the data is being > written truncated to the database, or Pillow is generating a truncated image > preview (which is being stored to the database). > > The later is more likely, since I am seeing no other data corruption errors. > > Rewriting the thumbnail delivery code, seemed to reduce the amount of > downloading issues, but they are still not Non-Zero… > > But I may just have to live with this, since there are too many moving parts > to be able to point a finger at any one bit of code. (It could be rosetta 2 > overly optimizing the code in some way… etc.) > > Plus, It might be time for me to just simply start rewriting the code from > the ground up again, to better optimize and clean it up. > > - Benjamin > > > >> On Dec 21, 2020, at 10:52 PM, Benjamin Schollnick >> <bscholln...@schollnick.net <mailto:bscholln...@schollnick.net>> wrote: >> >> >>> Have you tried the download with various browsers? I wonder if Safari or >>> Chrome released at the same time as the M1 chips is causing different Range >>> header behavior than that code is designed to handle. So it might not be >>> the M1 chip, but just an unrelated change in browser software that got >>> pushed out at the same time. Speculating here. >> >> The only issue that I see, is that I started to use RangedFileResponse to >> allow the viewing of MP4, and other media files in the browser… ( >> >> So I know it works at least in that regard. >> >> But for debugging purposes, I’ll switch it around… >> >> - Benjamin >> >> >>> >>> On Mon, Dec 21, 2020 at 3:19 PM Benjamin Schollnick >>> <bscholln...@schollnick.net <mailto:bscholln...@schollnick.net>> wrote: >>> Folks, >>> >>> I’m using django for an online gallery application, and I’ve seen some >>> weird results since switching over to the M1 Mac Mini. >>> (Please note, I am running it under Rosetta / Intel Translation, since >>> pillow doesn’t seem to build under M1 native). >>> >>> So this could be a quirk of Rosetta, Django, or some combination of these. >>> >>> I don’t think so, but it could be. >>> >>> So What is happening? >>> >>> I never saw this under my 2013 intel iMac, but now under M1 (even under >>> Rosetta 2 / Intel translation) I am receiving a damaged / altered download >>> file roughly 6 out of 8 or 9 times (~25-33% successful). >>> >>> Just retrying the download will eventually be successful. >>> >>> This isn’t Pillow related, since the file is just being downloaded directly >>> (no thumbnails or alterations to the file is occurring). >>> >>> Does anyone have a suggestion on what to do at this time? I’m uncertain on >>> where to start to dig. Although removing ranged_response, might be a >>> starting point… >>> >>> URLs: >>> path("download/<uuid:d_uuid>", frontend.views.new_download, >>> name="downloads"), >>> path('thumbnails/<uuid:t_url_name>', frontend.views.thumbnails, >>> name="raw thumbnails"), >>> >>> >>> First, I am using these function for all graphics and/or file(s) being sent: >>> >>> def new_download(request, d_uuid=None): >>> page = request.GET.get('page', None) >>> if page is None: >>> download = index_data.objects.filter(uuid=d_uuid, >>> ignore=False, >>> delete_pending=False)[0] >>> else: >>> print ("Attempting to find page %s in archive" % page) >>> print("\tDownloading - %s, %s" % (download.fqpndirectory.lower(), >>> download.name >>> <http://download.name/>)) >>> return respond_as_inline(request, >>> "%s%s%s" % ( >>> configdata["locations"]["albums_path"], >>> os.sep, >>> download.fqpndirectory), >>> download.name <http://download.name/>) >>> >>> >>> def respond_as_inline(request, file_path, original_filename, ranged=False): >>> # https://stackoverflow.com/questions/36392510/django-download-a-file >>> <https://stackoverflow.com/questions/36392510/django-download-a-file> >>> # >>> https://stackoverflow.com/questions/27712778/video-plays-in-other-browsers-but-not-safari >>> >>> <https://stackoverflow.com/questions/27712778/video-plays-in-other-browsers-but-not-safari> >>> # >>> https://stackoverflow.com/questions/720419/how-can-i-find-out-whether-a-server-supports-the-range-header >>> >>> <https://stackoverflow.com/questions/720419/how-can-i-find-out-whether-a-server-supports-the-range-header> >>> filename = os.path.join(file_path, original_filename) >>> if os.path.exists(filename): >>> mtype, encoding = mimetypes.guess_type(original_filename) >>> if mtype is None: >>> mtype = 'application/octet-stream' >>> response = RangedFileResponse(request, file=open(filename, 'rb'), >>> as_attachment=False, filename=os.path.basename(filename)) >>> response["Content-Type"] = mtype >>> return response >>> raise Http404 >>> >>> >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "Django users" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email to django-users+unsubscr...@googlegroups.com >>> <mailto:django-users+unsubscr...@googlegroups.com>. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/django-users/75E15498-69D8-47B5-94D2-821C6B35CA92%40schollnick.net >>> >>> <https://groups.google.com/d/msgid/django-users/75E15498-69D8-47B5-94D2-821C6B35CA92%40schollnick.net?utm_medium=email&utm_source=footer>. >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "Django users" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email to django-users+unsubscr...@googlegroups.com >>> <mailto:django-users+unsubscr...@googlegroups.com>. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/django-users/CAD4ANxWgXyHODdqsD%3DCzawv9D08zSPEwfUWda54kG%2B5XrHZmLA%40mail.gmail.com >>> >>> <https://groups.google.com/d/msgid/django-users/CAD4ANxWgXyHODdqsD%3DCzawv9D08zSPEwfUWda54kG%2B5XrHZmLA%40mail.gmail.com?utm_medium=email&utm_source=footer>. >> > -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/C5A32221-1852-4DF5-A978-AD1334128C4E%40schollnick.net.