Hello, the s3cmd 'mv' command fails if the source contains non-ASCII
characters, for example:
$ s3cmd mv s3://bucket1/Thörn.wav s3://bucket2/
Problem: UnicodeEncodeError: ('ascii', u's3://bucket1/Th\xf6rn.wav', 15, 16,
'ordinal not in range(128)')
S3cmd: 0.9.9.91
Traceback (most recent call last):
File "/usr/local/bin/s3cmd", line 1736, in <module>
main()
File "/usr/local/bin/s3cmd", line 1681, in main
cmd_func(args)
File "/usr/local/bin/s3cmd", line 582, in cmd_mv
subcmd_cp_mv(args, s3.object_move, "move", "File %(src)s moved to %(dst)s")
File "/usr/local/bin/s3cmd", line 533, in subcmd_cp_mv
remote_list = fetch_remote_list(args, require_attribs = False)
File "/usr/local/bin/s3cmd", line 245, in fetch_remote_list
uri_str = str(uri)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position
15: ordinal not in range(128)
The problem isn't with the locale, but with the wildcard handling code here:
for uri in remote_uris:
uri_str = str(uri)
## Wildcards used in remote URI?
## If yes we'll need a bucket listing...
if uri_str.find('*') > -1 or uri_str.find('?') > -1:
str(uri) fails with a non-ASCII uri. I fixed it by changing this to
unicode(uri). I'm a newcomer to python and there might be a better fix. I also
added a command-line option to make the wildcard parsing optional. Otherwise
you'll get into trouble 'mv'ing files with '?' and '*' in the name. Here are
the total diffs (0.9.9.1):
+++ ./S3/Config.py 2010-04-08 12:40:30.000000000 -0400
@@ -71,0 +72 @@
+ wildcard = True
+++ ./s3cmd 2010-04-08 12:41:16.000000000 -0400
@@ -245 +245 @@
- uri_str = str(uri)
+ uri_str = unicode(uri)
@@ -248 +248 @@
- if uri_str.find('*') > -1 or uri_str.find('?') > -1:
+ if cfg.wildcard and (uri_str.find('*') > -1 or
uri_str.find('?') > -1):
@@ -1515,0 +1516,2 @@
+ optparser.add_option( "--wildcard", dest="wildcard",
action="store_true", help="Handle wildcards in remote file lists")
+ optparser.add_option( "--no-wildcard", dest="wildcard",
action="store_false", help="Disable handling wildcards in remote file lists")
Thanks for an otherwise excellent tool. We use s3cmd at Bandcamp to move
thousands of files each day to and from S3 without trouble.
Joe
------------------------------------------------------------------------------
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
S3tools-general mailing list
S3tools-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/s3tools-general
