[ python-Feature Requests-1635335 ] Add registry functions to windows postinstall
Feature Requests item #1635335, was opened at 2007-01-14 21:00 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1635335&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Distutils Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: anatoly techtonik (techtonik) Assigned to: Nobody/Anonymous (nobody) Summary: Add registry functions to windows postinstall Initial Comment: It would be useful to add regkey_created() or regkey_modified() to windows postinstall scripts along with directory_created() and file_created(). Useful for adding installed package to App Paths. -- >Comment By: Martin v. Löwis (loewis) Date: 2007-01-20 11:55 Message: Logged In: YES user_id=21627 Originator: NO Can you please elaborate? Where should these functions be defined, what should they do, and when should they be invoked (by what code)? Also, what is a "windows postinstall script"? -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1635335&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1568240 ] Tix is not included in 2.5 for Windows
Bugs item #1568240, was opened at 2006-09-30 11:19 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1568240&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Build Group: Python 2.5 Status: Open Resolution: None Priority: 7 Private: No Submitted By: Christos Georgiou (tzot) Assigned to: Martin v. Löwis (loewis) Summary: Tix is not included in 2.5 for Windows Initial Comment: (I hope "Build" is more precise than "Extension Modules" and "Tkinter" for this specific bug.) At least the following files are missing from 2.5 for Windows: DLLs\tix8184.dll tcl\tix8184.lib tcl\tix8.1\* -- >Comment By: Martin v. Löwis (loewis) Date: 2007-01-20 14:16 Message: Logged In: YES user_id=21627 Originator: NO It seems that I can provide Tix binaries only for x86, not for AMD64 or Itanium. Is that sufficient? -- Comment By: Martin v. Löwis (loewis) Date: 2007-01-03 15:59 Message: Logged In: YES user_id=21627 Originator: NO Ah, ok. No, assigning this report to Neal or bumping its priority should not be done. -- Comment By: Christos Georgiou (tzot) Date: 2007-01-02 11:22 Message: Logged In: YES user_id=539787 Originator: YES Neal's message is this: http://mail.python.org/pipermail/python-dev/2006-December/070406.html and it refers to the 2.5.1 release, not prior to it. As you see, I refrained from both increasing the priority and assigning it to Neal, and actually just added a comment to the case with a related question, since I know you are the one responsible for the windows build and you already had assigned the bug to you. My adding this comment to the bug was nothing more or less than the action that felt appropriate, and still does feel appropriate to me (ie I didn't overstep any limits). The "we" was just all parties interested, and in this case, the ones I know are at least you (responsible for the windows build) and I (a user of Tix on windows). Happy new year, Martin! -- Comment By: Martin v. Löwis (loewis) Date: 2006-12-29 23:26 Message: Logged In: YES user_id=21627 Originator: NO I haven't read Neal's message yet, but I wonder what he could do about it. I plan to fix this with 2.5.1, there is absolutely no way to fix this earlier. I'm not sure who "we" is who would like to bump the bug, and what precisely this bumping would do; tzot, please refrain from changing the priority to higher than 7. These priorities are reserved to the release manager. -- Comment By: Christos Georgiou (tzot) Date: 2006-12-27 18:46 Message: Logged In: YES user_id=539787 Originator: YES Should we bump the bug up and/or assign it to Neal Norwitz as he requested on Python-Dev? -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1568240&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Feature Requests-1635335 ] Add registry functions to windows postinstall
Feature Requests item #1635335, was opened at 2007-01-14 20:00 Message generated for change (Comment added) made by techtonik You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1635335&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Distutils Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: anatoly techtonik (techtonik) Assigned to: Nobody/Anonymous (nobody) Summary: Add registry functions to windows postinstall Initial Comment: It would be useful to add regkey_created() or regkey_modified() to windows postinstall scripts along with directory_created() and file_created(). Useful for adding installed package to App Paths. -- >Comment By: anatoly techtonik (techtonik) Date: 2007-01-20 14:26 Message: Logged In: YES user_id=669020 Originator: YES Windows postinstall script is bundled with installation, launched after installation and just before uninstall. It is described here. http://docs.python.org/dist/postinstallation-script.html#SECTION00531 Where these should be defined? I do not know - there are already some functions that are said to be "available as additional built-in functions in the installation script." on the page above. The purpose is to be able to create/delete registry keys during installation. This should also be reflected in installation log file with appropriate status code so that users could be aware of what's going on. I think the functions needed are already defined in http://docs.python.org/lib/module--winreg.html but the module is very low-level. I'd rather use Autoit like API - http://www.autoitscript.com/autoit3/docs/functions/RegRead.htm http://www.autoitscript.com/autoit3/docs/functions/RegWrite.htm http://www.autoitscript.com/autoit3/docs/functions/RegDelete.htm -- Comment By: Martin v. Löwis (loewis) Date: 2007-01-20 10:55 Message: Logged In: YES user_id=21627 Originator: NO Can you please elaborate? Where should these functions be defined, what should they do, and when should they be invoked (by what code)? Also, what is a "windows postinstall script"? -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1635335&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Feature Requests-1635335 ] Add registry functions to windows postinstall
Feature Requests item #1635335, was opened at 2007-01-14 21:00 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1635335&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Distutils Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: anatoly techtonik (techtonik) >Assigned to: Thomas Heller (theller) Summary: Add registry functions to windows postinstall Initial Comment: It would be useful to add regkey_created() or regkey_modified() to windows postinstall scripts along with directory_created() and file_created(). Useful for adding installed package to App Paths. -- >Comment By: Martin v. Löwis (loewis) Date: 2007-01-20 19:07 Message: Logged In: YES user_id=21627 Originator: NO Thomas, what do you think? -- Comment By: anatoly techtonik (techtonik) Date: 2007-01-20 15:26 Message: Logged In: YES user_id=669020 Originator: YES Windows postinstall script is bundled with installation, launched after installation and just before uninstall. It is described here. http://docs.python.org/dist/postinstallation-script.html#SECTION00531 Where these should be defined? I do not know - there are already some functions that are said to be "available as additional built-in functions in the installation script." on the page above. The purpose is to be able to create/delete registry keys during installation. This should also be reflected in installation log file with appropriate status code so that users could be aware of what's going on. I think the functions needed are already defined in http://docs.python.org/lib/module--winreg.html but the module is very low-level. I'd rather use Autoit like API - http://www.autoitscript.com/autoit3/docs/functions/RegRead.htm http://www.autoitscript.com/autoit3/docs/functions/RegWrite.htm http://www.autoitscript.com/autoit3/docs/functions/RegDelete.htm -- Comment By: Martin v. Löwis (loewis) Date: 2007-01-20 11:55 Message: Logged In: YES user_id=21627 Originator: NO Can you please elaborate? Where should these functions be defined, what should they do, and when should they be invoked (by what code)? Also, what is a "windows postinstall script"? -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1635335&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1599254 ] mailbox: other programs' messages can vanish without trace
Bugs item #1599254, was opened at 2006-11-19 16:03 Message generated for change (Comment added) made by baikie You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1599254&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Library Group: Python 2.5 Status: Open Resolution: None Priority: 9 Private: No Submitted By: David Watson (baikie) Assigned to: A.M. Kuchling (akuchling) Summary: mailbox: other programs' messages can vanish without trace Initial Comment: The mailbox classes based on _singlefileMailbox (mbox, MMDF, Babyl) implement the flush() method by writing the new mailbox contents into a temporary file which is then renamed over the original. Unfortunately, if another program tries to deliver messages while mailbox.py is working, and uses only fcntl() locking, it will have the old file open and be blocked waiting for the lock to become available. Once mailbox.py has replaced the old file and closed it, making the lock available, the other program will write its messages into the now-deleted "old" file, consigning them to oblivion. I've caused Postfix on Linux to lose mail this way (although I did have to turn off its use of dot-locking to do so). A possible fix is attached. Instead of new_file being renamed, its contents are copied back to the original file. If file.truncate() is available, the mailbox is then truncated to size. Otherwise, if truncation is required, it's truncated to zero length beforehand by reopening self._path with mode wb+. In the latter case, there's a check to see if the mailbox was replaced while we weren't looking, but there's still a race condition. Any alternative ideas? Incidentally, this fixes a problem whereby Postfix wouldn't deliver to the replacement file as it had the execute bit set. -- >Comment By: David Watson (baikie) Date: 2007-01-20 18:20 Message: Logged In: YES user_id=1504904 Originator: YES Hang on. If a message's key changes after recreating _toc, that does not mean that another process has modified the mailbox. If the application removes a message and then (inadvertently) causes _toc to be regenerated, the keys of all subsequent messages will be decremented by one, due only to the application's own actions. That's what happens in the "broken locking" test case: the program intends to remove message 0, flush, and then remove message 1, but because _toc is regenerated in between, message 1 is renumbered as 0, message 2 is renumbered as 1, and so the program deletes message 2 instead. To clear _toc in such code without attempting to preserve the message keys turns possible data loss (in the case that another process modified the mailbox) into certain data loss. That's what I'm concerned about. -- Comment By: A.M. Kuchling (akuchling) Date: 2007-01-19 15:24 Message: Logged In: YES user_id=11375 Originator: NO After reflection, I don't think the potential changing actually makes things any worse. _generate() always starts numbering keys with 1, so if a message's key changes because of lock()'s, re-reading, that means someone else has already modified the mailbox. Without the ToC clearing, you're already fated to have a corrupted mailbox because the new mailbox will be written using outdated file offsets. With the ToC clearing, you delete the wrong message. Neither outcome is good, but data is lost either way. The new behaviour is maybe a little bit better in that you're losing a single message but still generating a well-formed mailbox, and not a randomly jumbled mailbox. I suggest applying the patch to clear self._toc, and noting in the documentation that keys might possibly change after doing a lock(). -- Comment By: David Watson (baikie) Date: 2007-01-18 18:15 Message: Logged In: YES user_id=1504904 Originator: YES This version passes the new tests (it fixes the length checking bug, and no longer clears self._toc), but goes back to failing test_concurrent_add. File Added: mailbox-unified2-module.diff -- Comment By: David Watson (baikie) Date: 2007-01-18 18:14 Message: Logged In: YES user_id=1504904 Originator: YES Unfortunately, there is a problem with clearing _toc: it's basically the one I alluded to in my comment of 2006-12-16. Back then I thought it could be caught by the test you issue the warning for, but the application may instead do its second remove() *after* the lock() (so that self._pending is not set at lock() time), using the key carried over from before it called unlock(). As before, this would result in the wrong message being delete
[ python-Bugs-1637943 ] Problem packaging wx application with py2exe.
Bugs item #1637943, was opened at 2007-01-17 10:10 Message generated for change (Comment added) made by josiahcarlson You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1637943&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: Python 2.5 Status: Closed Resolution: Invalid Priority: 5 Private: No Submitted By: Indy (indy90) Assigned to: Nobody/Anonymous (nobody) Summary: Problem packaging wx application with py2exe. Initial Comment: I have created a minimal wx application, which runs fine. However, when I package it with py2exe and I try to run the .exe file, an error occurs, the program crashes (before even starting) and a pop-up box says to look at the log file for the error trace. It says that wx/_core_.pyd failed to be loaded (this file exists in my filesystem - I have checked). When I skip "zipfile = None" in the setup() function, another pop-up box also appears, and says that a DLL failed to be loaded. Python 2.5 wxPython 2.8.0.1 py2exe 0.6.6 -- Comment By: Josiah Carlson (josiahcarlson) Date: 2007-01-20 10:27 Message: Logged In: YES user_id=341410 Originator: NO For reference, this may or may not be related to having gdiplus.dll and/or the msvcrt71.dll (or something like that) in the same path as the executable that runs your program. Try to move those into the same path as the executable you are trying to run and see if that helps. If it doesn't, feel free to post on the [EMAIL PROTECTED] mailing list. -- Comment By: Brett Cannon (bcannon) Date: 2007-01-17 10:40 Message: Logged In: YES user_id=357491 Originator: NO This is the bug tracker for the Python programming language. Please report this issue to the py2exe development team. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1637943&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Feature Requests-1637926 ] Empty class 'Object'
Feature Requests item #1637926, was opened at 2007-01-17 09:51
Message generated for change (Comment added) made by josiahcarlson
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1637926&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: kxroberto (kxroberto)
Assigned to: Nobody/Anonymous (nobody)
Summary: Empty class 'Object'
Initial Comment:
An empty class 'Object' in builtins, which can be instantiated (with optional
inline arguments as attributes (like dict)), and attributes added. Convenience
- Easy OO variable container - known to pickle etc.
http://groups.google.com/group/comp.lang.python/msg/3ff946e7da13dba9
http://groups.google.de/group/comp.lang.python/msg/a02f0eb4efb76b24
idea:
class X(object):
def __init__(self,_d={},**kwargs):
kwargs.update(_d)
self.__dict__=kwargs
class Y(X):
def __repr__(self):
return ''%self.__dict__
--
x=X(spam=1)
x.a=3
Robert
--
Comment By: Josiah Carlson (josiahcarlson)
Date: 2007-01-20 10:35
Message:
Logged In: YES
user_id=341410
Originator: NO
This has been requested in various forms over the years. See the "bunch"
discussion on the python-dev mailing list from over a year ago. There may
have even been a PEP. I believe the general consensus was "it would be
convenient sometimes, but it is *trivial* to implement it as necessary".
Also, not every X-line function or class should be included with Python.
--
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1637926&group_id=5470
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Feature Requests-1567331 ] logging.RotatingFileHandler has no "infinite" backupCount
Feature Requests item #1567331, was opened at 2006-09-28 14:36 Message generated for change (Comment added) made by josiahcarlson You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1567331&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Library Group: None Status: Closed Resolution: Fixed Priority: 5 Private: No Submitted By: Skip Montanaro (montanaro) Assigned to: Vinay Sajip (vsajip) Summary: logging.RotatingFileHandler has no "infinite" backupCount Initial Comment: It seems to me that logging.RotatingFileHandler should have a way to spell "never delete old log files". This is useful in situations where you want an external process (manual or automatic) make decisions about deleting log files. -- Comment By: Josiah Carlson (josiahcarlson) Date: 2007-01-20 10:39 Message: Logged In: YES user_id=341410 Originator: NO What about an optional different semantic for log renaming? Rather than log -> log.1, log -> log.+1, so if you have log, log.1, log.2; log -> log.3 and log gets created anew. I've used a similar semantic in other logging packages, and it works pretty well. It would also allow for users to have an "infinite" count of logfiles (if that is what they want). -- Comment By: Vinay Sajip (vsajip) Date: 2007-01-15 08:44 Message: Logged In: YES user_id=308438 Originator: NO The problem with this is that on rollover, RotatingFileHandler renames old logs: rollover.log.3 -> rollover.log.4, rollover.log.2 -> rollover.log.3, rollover.log.1 -> rollover.log.2, rollover.log -> rollover.log.1, and a new rollover.log is opened. With an arbitrary number of old log files, this leads to arbitrary renaming time - which could cause long pauses due to logging, not a good idea. If you are using e.g. logrotate or newsyslog, or a custom program to do logfile rotation, you can use the new logging.handlers.WatchedFileHandler handler (meant for use on Unix/Linux only - on Windows, logfiles can't be renamed or moved while in use and so the requirement doesn't arise) which watches the logged-to file to see when it changes. This has recently been checked into SVN trunk. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1567331&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Feature Requests-1639002 ] add type defintion support
Feature Requests item #1639002, was opened at 2007-01-18 13:53 Message generated for change (Comment added) made by josiahcarlson You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1639002&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Interpreter Core Group: Python 2.6 Status: Closed Resolution: None Priority: 5 Private: No Submitted By: djnet (djnet) Assigned to: Nobody/Anonymous (nobody) Summary: add type defintion support Initial Comment: Hi, I'm used to java language. When i use a good java ide, the autocompletion is very effective (python cannot be such effective). ex, if i enter following text: Date lDate=new Date(); lDate.[TAB_KEY] then the editor can display all the methods available for my 'lDate' object; it can because it knows the object's type. This is very convenient and allows to use a new API without browsing the API documentation ! I think such autocompletion could be achieved in python simply: it only need a "type definition" syntax. Of course, the type definition should NOT be MANDATORY ! So, is this a good idea ? David -- Comment By: Josiah Carlson (josiahcarlson) Date: 2007-01-20 10:43 Message: Logged In: YES user_id=341410 Originator: NO FYI, WingIDE and a few other Python IDEs/editors offer a pseudo-syntax for defining such things to help with such introspection. Sometimes it is code that is actually executed when the program is run, sometimes it is comments. You may consider looking into this stuff further before posting on the python-ideas list. -- Comment By: Georg Brandl (gbrandl) Date: 2007-01-19 10:10 Message: Logged In: YES user_id=849994 Originator: NO If what you're suggesting is static typing, please go to the python-ideas mailing list and discuss it there. Changes of a scope that large shouldn't be discussed in a issue tracker. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1639002&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1636950 ] Newline skipped in "for line in file"
Bugs item #1636950, was opened at 2007-01-16 10:56 Message generated for change (Comment added) made by amonthei You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1636950&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Library Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Andy Monthei (amonthei) Assigned to: Nobody/Anonymous (nobody) Summary: Newline skipped in "for line in file" Initial Comment: When processing huge fixed block files of about 7000 bytes wide and several hundred thousand lines long some pairs of lines get read as one long line with no line break when using "for line in file:". The problem is even worse when using the fileinput module and reading in five or six huge files consisting of 4.8 million records causes several hundred pairs of lines to be read as single lines. When a newline is skipped it is usually followed by several more in the next few hundred lines. I have not noticed any other characters being skipped, only the line break. O.S. Windows (5, 1, 2600, 2, 'Service Pack 2') Python 2.5 -- >Comment By: Andy Monthei (amonthei) Date: 2007-01-20 16:53 Message: Logged In: YES user_id=1693612 Originator: YES I have had no luck creating random data to reproduce the problem which leaves me to come to the conclusion that it was the data itself. Using a hex editor I find no problem with the line breaks. The data that triggers this bug is transferred several time before it gets to me. It originates on a Unix box, then goes to an IBM mainframe, then to my Windows machine and through many updates along the way. It may be an EBCDIC/ASCII conversion or possibly something to do with the mainframe to PC transfer. Whatever it is, it's in the data itself. The only thing that bothers me is that Java somehow is not affected by this bad data. -- Comment By: Andy Monthei (amonthei) Date: 2007-01-18 09:34 Message: Logged In: YES user_id=1693612 Originator: YES I am using open() for reading the file, no other features. I have also had fileinput.input(fileList) compound the problem. Each file that this has happened to is a fixed block file of either 6990 or 7700 bytes wide but this I think is insignificant. When looking at the file in a hex editor everything looks fine and a small Java program using a buffered reader will give me the correct line count when Python does not. Using something like fp.read(8192) I'm sure might temporarily solve my problem but I will keep working on getting a file I can upload. -- Comment By: Walter Dörwald (doerwalter) Date: 2007-01-18 03:23 Message: Logged In: YES user_id=89016 Originator: NO Are you using any of the unicode reading features (i.e. codecs.EncodedFile etc.) or are you using plain open() for reading the file? -- Comment By: Mark Roberts (mark-roberts) Date: 2007-01-18 01:12 Message: Logged In: YES user_id=1591633 Originator: NO I don't know if this helps: I spent the last little while creating / reading random files that all (seemingly) matched the description you gave us. None of these files failed to read properly. (e.g., have the right amount of rows with a line length that seemingly was the right line. Definitely no doubling lines). Perusing the file source code found a detailed discussion of fgets vs fgetc for finding the next line in the file. Have you tried reading the file with fp.read(8192) or similar? Hopefully you're able to reproduce the bug with scrubbed data (because I couldn't construct random data to do so). Good luck. -- Comment By: Mark Roberts (mark-roberts) Date: 2007-01-17 23:24 Message: Logged In: YES user_id=1591633 Originator: NO How wide are the min and max widths of the lines? This problem is of particular interest to me. -- Comment By: Andy Monthei (amonthei) Date: 2007-01-17 15:58 Message: Logged In: YES user_id=1693612 Originator: YES I can not upload the files that trigger this because of the data that is in them but I am working on getting around that. In my data line 617391 in a fixed block file of 6990 bytes wide gets read in with the next line after it. The line break is 0d0a (same as the others) where the bug happens so I am wondering if it is a buffer issue where the linebreak falls at the edge, however no other characters are ever missed. The total file is 888420 lines and this happens in four spots. I will hopefully have a file to send soon.
[ python-Bugs-1636950 ] Newline skipped in "for line in file"
Bugs item #1636950, was opened at 2007-01-16 08:56 Message generated for change (Comment added) made by bcannon You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1636950&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Library Group: Python 2.5 >Status: Closed >Resolution: Invalid Priority: 5 Private: No Submitted By: Andy Monthei (amonthei) Assigned to: Nobody/Anonymous (nobody) Summary: Newline skipped in "for line in file" Initial Comment: When processing huge fixed block files of about 7000 bytes wide and several hundred thousand lines long some pairs of lines get read as one long line with no line break when using "for line in file:". The problem is even worse when using the fileinput module and reading in five or six huge files consisting of 4.8 million records causes several hundred pairs of lines to be read as single lines. When a newline is skipped it is usually followed by several more in the next few hundred lines. I have not noticed any other characters being skipped, only the line break. O.S. Windows (5, 1, 2600, 2, 'Service Pack 2') Python 2.5 -- >Comment By: Brett Cannon (bcannon) Date: 2007-01-20 16:46 Message: Logged In: YES user_id=357491 Originator: NO Well, with Andy saying he can't reproduce the problem I am going to close as invalid. Andy, if you ever happen to be able to upload data that triggers it, then please re-open this bug. -- Comment By: Andy Monthei (amonthei) Date: 2007-01-20 14:53 Message: Logged In: YES user_id=1693612 Originator: YES I have had no luck creating random data to reproduce the problem which leaves me to come to the conclusion that it was the data itself. Using a hex editor I find no problem with the line breaks. The data that triggers this bug is transferred several time before it gets to me. It originates on a Unix box, then goes to an IBM mainframe, then to my Windows machine and through many updates along the way. It may be an EBCDIC/ASCII conversion or possibly something to do with the mainframe to PC transfer. Whatever it is, it's in the data itself. The only thing that bothers me is that Java somehow is not affected by this bad data. -- Comment By: Andy Monthei (amonthei) Date: 2007-01-18 07:34 Message: Logged In: YES user_id=1693612 Originator: YES I am using open() for reading the file, no other features. I have also had fileinput.input(fileList) compound the problem. Each file that this has happened to is a fixed block file of either 6990 or 7700 bytes wide but this I think is insignificant. When looking at the file in a hex editor everything looks fine and a small Java program using a buffered reader will give me the correct line count when Python does not. Using something like fp.read(8192) I'm sure might temporarily solve my problem but I will keep working on getting a file I can upload. -- Comment By: Walter Dörwald (doerwalter) Date: 2007-01-18 01:23 Message: Logged In: YES user_id=89016 Originator: NO Are you using any of the unicode reading features (i.e. codecs.EncodedFile etc.) or are you using plain open() for reading the file? -- Comment By: Mark Roberts (mark-roberts) Date: 2007-01-17 23:12 Message: Logged In: YES user_id=1591633 Originator: NO I don't know if this helps: I spent the last little while creating / reading random files that all (seemingly) matched the description you gave us. None of these files failed to read properly. (e.g., have the right amount of rows with a line length that seemingly was the right line. Definitely no doubling lines). Perusing the file source code found a detailed discussion of fgets vs fgetc for finding the next line in the file. Have you tried reading the file with fp.read(8192) or similar? Hopefully you're able to reproduce the bug with scrubbed data (because I couldn't construct random data to do so). Good luck. -- Comment By: Mark Roberts (mark-roberts) Date: 2007-01-17 21:24 Message: Logged In: YES user_id=1591633 Originator: NO How wide are the min and max widths of the lines? This problem is of particular interest to me. -- Comment By: Andy Monthei (amonthei) Date: 2007-01-17 13:58 Message: Logged In: YES user_id=1693612 Originator: YES I can not upload the files that trigger this because of the data that is in them but I am working on getting around that. In my data line 617391 in a
[ python-Bugs-1599254 ] mailbox: other programs' messages can vanish without trace
Bugs item #1599254, was opened at 2006-11-19 11:03 Message generated for change (Comment added) made by akuchling You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1599254&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Library Group: Python 2.5 Status: Open Resolution: None Priority: 9 Private: No Submitted By: David Watson (baikie) Assigned to: A.M. Kuchling (akuchling) Summary: mailbox: other programs' messages can vanish without trace Initial Comment: The mailbox classes based on _singlefileMailbox (mbox, MMDF, Babyl) implement the flush() method by writing the new mailbox contents into a temporary file which is then renamed over the original. Unfortunately, if another program tries to deliver messages while mailbox.py is working, and uses only fcntl() locking, it will have the old file open and be blocked waiting for the lock to become available. Once mailbox.py has replaced the old file and closed it, making the lock available, the other program will write its messages into the now-deleted "old" file, consigning them to oblivion. I've caused Postfix on Linux to lose mail this way (although I did have to turn off its use of dot-locking to do so). A possible fix is attached. Instead of new_file being renamed, its contents are copied back to the original file. If file.truncate() is available, the mailbox is then truncated to size. Otherwise, if truncation is required, it's truncated to zero length beforehand by reopening self._path with mode wb+. In the latter case, there's a check to see if the mailbox was replaced while we weren't looking, but there's still a race condition. Any alternative ideas? Incidentally, this fixes a problem whereby Postfix wouldn't deliver to the replacement file as it had the execute bit set. -- >Comment By: A.M. Kuchling (akuchling) Date: 2007-01-20 22:16 Message: Logged In: YES user_id=11375 Originator: NO I'm starting to lose track of all the variations on the bug. Maybe we should just add more warnings to the documentation about locking the mailbox when modifying it and not try to fix this at all. -- Comment By: David Watson (baikie) Date: 2007-01-20 13:20 Message: Logged In: YES user_id=1504904 Originator: YES Hang on. If a message's key changes after recreating _toc, that does not mean that another process has modified the mailbox. If the application removes a message and then (inadvertently) causes _toc to be regenerated, the keys of all subsequent messages will be decremented by one, due only to the application's own actions. That's what happens in the "broken locking" test case: the program intends to remove message 0, flush, and then remove message 1, but because _toc is regenerated in between, message 1 is renumbered as 0, message 2 is renumbered as 1, and so the program deletes message 2 instead. To clear _toc in such code without attempting to preserve the message keys turns possible data loss (in the case that another process modified the mailbox) into certain data loss. That's what I'm concerned about. -- Comment By: A.M. Kuchling (akuchling) Date: 2007-01-19 10:24 Message: Logged In: YES user_id=11375 Originator: NO After reflection, I don't think the potential changing actually makes things any worse. _generate() always starts numbering keys with 1, so if a message's key changes because of lock()'s, re-reading, that means someone else has already modified the mailbox. Without the ToC clearing, you're already fated to have a corrupted mailbox because the new mailbox will be written using outdated file offsets. With the ToC clearing, you delete the wrong message. Neither outcome is good, but data is lost either way. The new behaviour is maybe a little bit better in that you're losing a single message but still generating a well-formed mailbox, and not a randomly jumbled mailbox. I suggest applying the patch to clear self._toc, and noting in the documentation that keys might possibly change after doing a lock(). -- Comment By: David Watson (baikie) Date: 2007-01-18 13:15 Message: Logged In: YES user_id=1504904 Originator: YES This version passes the new tests (it fixes the length checking bug, and no longer clears self._toc), but goes back to failing test_concurrent_add. File Added: mailbox-unified2-module.diff -- Comment By: David Watson (baikie) Date: 2007-01-18 13:14 Message: Logged In: YES user_id=1504904 Originator: YES Unfortunately, there is a problem with clearing _to
