On Sun, May 12, 2013 at 06:36:54PM -0500, Jay McCanta wrote:
> I’m unsure of the proper protocol for proposing patches for the
> group. My apologies if this isn’t the correct way.
Thanks for the patch Jay.
Patches by email is fine, though pull requests filed in github to pull
from a specific bug or feature branch in your own github-forked
repository also works quite well, and makes it harder to forget over
time as the pull request remains open until merged or closed
manually. :-)
> We keep the md5 checksum of files in an extended attribute. When
> running s3cmd with --check-md5, this patch allow one to use the
> --xattr option and specify the name of the extended attribute to use
> for local md5sums. If the attribute is not there, a regular check
> is done, so no changes to existing procedures would be necessary.
> For us, the drastically improves check time since we are not
> re-reading every local file recalculating every md5sum.
Cool idea. What are you using to put such md5sums into the extended
attribute already?
> Also there was a typo in s3cmd for the ‘help’ field of --acl-revoke
> option definition. It’s corrected in this patch.
Ideally that would be split into a second patch.
> diff -ru s3cmd-1.5.0-alpha3/s3cmd s3cmd-1.5.0-alpha3.jtm/s3cmd
> --- s3cmd-1.5.0-alpha3/s3cmd 2013-03-10 17:06:33.000000000 -0700
> +++ s3cmd-1.5.0-alpha3.jtm/s3cmd 2013-05-12 15:20:44.013584123 -0700
> @@ -32,6 +32,13 @@
> from logging import debug, info, warning, error
> from distutils.spawn import find_executable
>
> +try:
> + hasXattr = False
> + import xattr
> + hasXattr = True
> +except:
> + pass
> +
This can be shortened to:
try: import xattr
except: pass
...
if 'xattr' in sys.modules.keys():
optparser.add_option ("--xattr", dest="xattr", action="store", help="If
possible, use extended file attribute named (default:%default) instead of
calculating it [sync].")
This should be default=False, action="store_true". That way we know
it exists later.
> def output(message):
> sys.stdout.write(message + "\n")
> sys.stdout.flush()
> @@ -1141,7 +1148,7 @@
>
> def _invalidate_on_cf(destination_base_uri):
> cf = CloudFront(cfg)
> - default_index_file = None
> + efault_index_file = None
keystroke typo?
> if cfg.invalidate_default_index_on_cf or
> cfg.invalidate_default_index_root_on_cf:
> info_response = s3.website_info(destination_base_uri,
> cfg.bucket_location)
> if info_response:
> @@ -1507,9 +1514,9 @@
> ret_enc = gpg_encrypt(filename)
> ret_dec = gpg_decrypt(ret_enc[1], ret_enc[2], False)
> hash = [
> - Utils.hash_file_md5(filename),
> - Utils.hash_file_md5(ret_enc[1]),
> - Utils.hash_file_md5(ret_dec[1]),
> + Utils.hash_file_md5(filename, options.xattr),
> + Utils.hash_file_md5(ret_enc[1], options.xattr),
> + Utils.hash_file_md5(ret_dec[1], options.xattr),
What's the corresponding change to S3/Utils.py please?
Given that this should also be a Config option, not just a command
line option, please stick it into the Config object, to be retrieved
in Utils.py.
Thanks,
Matt
--
Matt Domsch
Technology Strategist
Dell | Office of the CTO
------------------------------------------------------------------------------
Sponsored by Intel(R) XDK
Develop, test and display web and hybrid apps with a single code base.
Download it for free now!
http://pubads.g.doubleclick.net/gampad/clk?id=111408631&iu=/4140/ostg.clktrk
_______________________________________________
S3tools-general mailing list
S3tools-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/s3tools-general
