Hardened the script. Toleratres spaces
in file names and no-read-access files.
At 11:05 AM 8/10/2013 -0400, starli...@binnacle.cx wrote:
>This 'bash' script calculates the
>Amazon S3 Etag value for large files
>and/or files that were uploaded using the
>parallel transfer approach.
#!/bin/sh
#
# Calculate Amazon S3 object Etag for given part size.
#
# For files less than or equal to the part size,
# return the MD5 hash for the part.
#
# For files larger than one part, the MD5 hash of each
# part is calculated and appended in binary form
# to a hash string. Then MD5 hash of the resulting hash
# string is returned suffixed with the hyphen character
# followed by the part count.
#
set -e
SEGSZU=${1?}
shift
if test $(expr "${SEGSZU?}" : '[0-9]\+$') -eq 0 \
-o -z "${$}"
then
echo "usage: $0 part_size_mb file1 file2 file3 . . ."
exit 1
fi
UNITSZ=1048576
SEGSZY=$(expr ${SEGSZU:?} '*' ${UNITSZ:?})
EXITCODE=0
for FILE in "${@}"; do
if test ! -f "${FILE:?}" -o ! -r "${FILE:?}"; then
echo 1>&2 "\"${FILE:?}\" not found or not readable regular file"
EXITCODE=1
else
FSIZE=$(stat -c '%s' "${FILE:?}")
UNITCT=$(expr '(' ${FSIZE:?} + ${UNITSZ:?} - 1 ')' / ${UNITSZ:?})
SEGCNT=$(expr '(' ${FSIZE:?} + ${SEGSZY:?} - 1 ')' / ${SEGSZY:?})
if test ${SEGCNT:?} -le 1; then
md5sum --binary "${FILE:?}" \
| awk -v FILE="${FILE:?}" '{printf "%-35s %s\n", $1, FILE}'
else
SKIP=0
while test ${SKIP:?} -lt ${UNITCT:?}; do
dd bs=${UNITSZ:?} count=${SEGSZU:?} skip=${SKIP:?} \
if="${FILE:?}" 2>/dev/null \
| md5sum --binary \
| awk '{print $1}' \
| xxd -r -p
SKIP=$(expr ${SKIP:?} + ${SEGSZU:?})
done \
| md5sum --binary \
| awk -v SEGCNT=${SEGCNT:?} -v FILE="${FILE:?}" \
'{printf "%-35s %s\n", $1 "-" SEGCNT, FILE}'
fi
fi
done
exit ${EXITCODE:?}
------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite!
It's a free troubleshooting tool designed for production.
Get down to code-level detail for bottlenecks, with <2% overhead.
Download for free and get started troubleshooting in minutes.
http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
_______________________________________________
S3tools-general mailing list
S3tools-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/s3tools-general
