I was going to ask that, but felt I was being too greedy to do so. :)
It sure would make my transfer activities easier. I don't trust aws cli sync at
this stage. It doesn't sync all my files whereas s3cmd does and it gives me
feedback about what it is doing.
However, since I can't always (often not in my current set of transfer tasks)
use md5 checks in s3cmd, there is a not insignificant chance I am missing
modified/updated files.
Mike
On Apr 23, 2014, at 1:45 PM, Matt Domsch <m...@domsch.com> wrote:
> Yes, I believe it is accurate. it's also fair to ask why we don't follow
> suit and compare local mtime with S3 <LastModified>, and upload if mtime is
> newer.
>
> it's been that way since Michal first wrote the initial sync code back in
> September 2007. Doesn't mean it has to stay that way.
>
>
> On Wed, Apr 23, 2014 at 8:29 AM, WagnerOne <wag...@wagnerone.com> wrote:
> Thank you, Matt, for inspecting that and for the continued explanation.
>
> If I have a local and S3 file pair and the local file is modified such that
> size is not modified, but its date is, a sync with aws cli would copy that
> modified local file over the existing s3 counterpart (due to the source file
> having a newer mtime when compared to S3 LastModified), but a sync with s3cmd
> --no-check-md5 (which I unfortunately often have to use) would not.
>
> Is this statement accurate?
>
> Mike
>
> On Apr 14, 2014, at 6:51 PM, Matt Domsch <m...@domsch.com> wrote:
>
> > aws-cli (after a quick perusal of their source code) uses the LastModified
> > value (set by S3 to be the time the upload of the object occurred) on
> > objects in S3, which is obtainable from the <ListBucket> XML, without doing
> > a HEAD call. They then go on to calculate the difference between
> > LastModified and stat.mtime(), accounting for time zone differences.
> > local files use stat.mtime.
> >
> > aws-cli then proceeds to compare the two values; if LastModified is newer
> > than stat.mtime, syncing local->remote is skipped (the remote is newer than
> > local); likewise on download, if LastModified is older than the local file,
> > it too is skipped.
> >
> > Neither tool sets LastModified = stat.mtime on upload (nor can they).
> > s3cmd gets around this by setting stat.mtime into the file's metadata when
> > --preserve (the default) is used, but then would have to use a HEAD or GET
> > call to get it back. s3cmd does update the local on-disk mtime and atime
> > when downloading (GETting), because we get the header back, and that's free
> > then. Likewise, aws-cli sets both mtime and atime = LastModified on
> > download.
> >
> > So aside from aws-cli skipping over newer files in destination, I think
> > their behavior is identical.
> >
> >
> >
> >
> >
> > ------------------------------------------------------------------------------
> > Learn Graph Databases - Download FREE O'Reilly Book
> > "Graph Databases" is the definitive new guide to graph databases and their
> > applications. Written by three acclaimed leaders in the field,
> > this first edition is now available. Download your free book today!
> > http://p.sf.net/sfu/NeoTech_______________________________________________
> > S3tools-general mailing list
> > S3tools-general@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/s3tools-general
>
> --
> wag...@wagnerone.com
> "Never fall into the trap of judging that which you don't understand as
> nonsense. That error can destroy you."-Feist
>
>
>
> ------------------------------------------------------------------------------
> Start Your Social Network Today - Download eXo Platform
> Build your Enterprise Intranet with eXo Platform Software
> Java Based Open Source Intranet - Social, Extensible, Cloud Ready
> Get Started Now And Turn Your Intranet Into A Collaboration Platform
> http://p.sf.net/sfu/ExoPlatform
> _______________________________________________
> S3tools-general mailing list
> S3tools-general@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/s3tools-general
>
>
> ------------------------------------------------------------------------------
> Start Your Social Network Today - Download eXo Platform
> Build your Enterprise Intranet with eXo Platform Software
> Java Based Open Source Intranet - Social, Extensible, Cloud Ready
> Get Started Now And Turn Your Intranet Into A Collaboration Platform
> http://p.sf.net/sfu/ExoPlatform_______________________________________________
> S3tools-general mailing list
> S3tools-general@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/s3tools-general
--
wag...@wagnerone.com
"Any reviewer who expresses rage and loathing for a novel is preposterous. He
or she is like a person who has put on full armor and attacked a hot fudge
sundae."-Kurt Vonnegut
------------------------------------------------------------------------------
Start Your Social Network Today - Download eXo Platform
Build your Enterprise Intranet with eXo Platform Software
Java Based Open Source Intranet - Social, Extensible, Cloud Ready
Get Started Now And Turn Your Intranet Into A Collaboration Platform
http://p.sf.net/sfu/ExoPlatform
_______________________________________________
S3tools-general mailing list
S3tools-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/s3tools-general
