Jay - thanks for the patch. I've made some stylistic changes, as well as
simplifies testing for xattr
module import success, passes the result into the Config object,
and ensures the correct namespace is used. Also added a test to the
test
suite, and the entry on the manpage.
I gave you credit in the changelog of course.
I've pushed this to my xattr branch, and into my merge branch as well, and
have filed a pull request for same. I've tested this with the test suite
and with manual tests to be sure it works.
https://github.com/mdomsch/s3cmd/tree/merge
Thanks much for the idea!
On Tue, May 14, 2013 at 4:11 PM, Jay McCanta <j.mcca...@f5.com> wrote:
> My apologies for cutting/pasting/posting without proofing.
>
> This patch allows s3cmd to utilize md5sums stored in the extended
> attributes of files. If for any reason, that doesn’t work, the usual
> md5sum mechanisms are used. This option only has impact if check md5 is
> enabled.
> The last patch block corrects a typo in the s3cmd ‘--acl-revoke’ option
> help text.
>
> diff -ru s3cmd-1.5.0-alpha3.orig/S3/Utils.py s3cmd-1.5.0-alpha3/S3/Utils.py
> --- s3cmd-1.5.0-alpha3.orig/S3/Utils.py 2013-03-10 15:35:09.000000000 -0700
> +++ s3cmd-1.5.0-alpha3/S3/Utils.py 2013-05-13 14:13:22.083721474 -0700
> @@ -35,6 +35,12 @@
> import elementtree.ElementTree as ET
> from xml.parsers.expat import ExpatError
>
> +try:
> + import xattr
> + hasXattr = True
> +except:
> + hasXattr = False
> +
> __all__ = []
> def parseNodes(nodes):
> ## WARNING: Ignores text nodes from mixed xml/text.
> @@ -226,7 +232,14 @@
> return mktmpsomething(prefix, randchars, createfunc)
> __all__.append("mktmpfile")
>
> -def hash_file_md5(filename):
> +def hash_file_md5(filename, xattrName=None):
> + if hasXattr and xattrName is not None:
> + try:
> + md5sum = xattr.get(filename, xattrName)
> + return md5sum
> + except:
> + pass
> +
> h = md5()
> f = open(filename, "rb")
> while True:
> diff -ru s3cmd-1.5.0-alpha3.orig/s3cmd s3cmd-1.5.0-alpha3/s3cmd
> --- s3cmd-1.5.0-alpha3.orig/s3cmd 2013-03-10 17:06:33.000000000 -0700
> +++ s3cmd-1.5.0-alpha3/s3cmd 2013-05-13 14:10:33.927609005 -0700
> @@ -32,6 +32,12 @@
> from logging import debug, info, warning, error
> from distutils.spawn import find_executable
>
> +try:
> + import xattr
> + hasXattr = True
> +except:
> + hasXattr = False
> +
> def output(message):
> sys.stdout.write(message + "\n")
> sys.stdout.flush()
> @@ -1507,9 +1513,9 @@
> ret_enc = gpg_encrypt(filename)
> ret_dec = gpg_decrypt(ret_enc[1], ret_enc[2],
> False)
> hash = [
> - Utils.hash_file_md5(filename),
> - Utils.hash_file_md5(ret_enc[1]),
> - Utils.hash_file_md5(ret_dec[1]),
> + Utils.hash_file_md5(filename, options.xattr),
> + Utils.hash_file_md5(ret_enc[1],
> options.xattr),
> + Utils.hash_file_md5(ret_dec[1],
> options.xattr),
> ]
> os.unlink(filename)
> os.unlink(ret_enc[1])
> @@ -1762,11 +1768,13 @@
> optparser.add_option( "--skip-existing", dest="skip_existing",
> action="store_true", help="Skip over files that exist at the destination
> (only for [get] and [sync] commands).")
> optparser.add_option("-r", "--recursive", dest="recursive",
> action="store_true", help="Recursive upload, download or removal.")
> optparser.add_option( "--check-md5", dest="check_md5",
> action="store_true", help="Check MD5 sums when comparing files for [sync].
> (default)")
> + if hasXattr:
> + optparser.add_option ("--xattr", dest="xattr", action="store",
> help="If possible, use extended file attribute named (default:%default)
> instead of calculating it [sync].")
> optparser.add_option( "--no-check-md5", dest="check_md5",
> action="store_false", help="Do not check MD5 sums when comparing files for
> [sync]. Only size will be compared. May significantly speed up transfer but
> may also miss some changed files.")
> optparser.add_option("-P", "--acl-public", dest="acl_public",
> action="store_true", help="Store objects with ACL allowing read for
> anyone.")
> optparser.add_option( "--acl-private", dest="acl_public",
> action="store_false", help="Store objects with default ACL allowing access
> for you only.")
> optparser.add_option( "--acl-grant", dest="acl_grants",
> type="s3acl", action="append", metavar="PERMISSION:EMAIL or
> USER_CANONICAL_ID", help="Grant stated permission to a given amazon user.
> Permission is one of: read, write, read_acp, write_acp, full_control, all")
> - optparser.add_option( "--acl-revoke", dest="acl_revokes",
> type="s3acl", action="append", metavar="PERMISSION:USER_CANONICAL_ID",
> help="Revoke stated permission for a given amazon user. Permission is one
> of: read, write, read_acp, wr ite_acp, full_control, all")
> + optparser.add_option( "--acl-revoke", dest="acl_revokes",
> type="s3acl", action="append", metavar="PERMISSION:USER_CANONICAL_ID",
> help="Revoke stated permission for a given amazon user. Permission is one
> of: read, write, read_acp, write_acp, full_control, all")
>
> optparser.add_option( "--delete-removed", dest="delete_removed",
> action="store_true", help="Delete remote objects with no corresponding
> local file [sync]")
> optparser.add_option( "--no-delete-removed",
> dest="delete_removed", action="store_false", help="Don't delete remote
> objects.")
>
>
>
> ------------------------------------------------------------------------------
> AlienVault Unified Security Management (USM) platform delivers complete
> security visibility with the essential security capabilities. Easily and
> efficiently configure, manage, and operate all of your security controls
> from a single console and one unified framework. Download a free trial.
> http://p.sf.net/sfu/alienvault_d2d
> _______________________________________________
> S3tools-general mailing list
> S3tools-general@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/s3tools-general
>
>
------------------------------------------------------------------------------
AlienVault Unified Security Management (USM) platform delivers complete
security visibility with the essential security capabilities. Easily and
efficiently configure, manage, and operate all of your security controls
from a single console and one unified framework. Download a free trial.
http://p.sf.net/sfu/alienvault_d2d
_______________________________________________
S3tools-general mailing list
S3tools-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/s3tools-general
