Public bug reported:
I have a duplicity job that runs on multiple hosts, and am now having failures
on my 20.04 hosts.
The syntax for the job (as I am currently testing to debug) is:
duplicity --dry-run list-current-files --name $name --s3-use-ia --archive-dir
$archiveDir --tempdir $tmpDir --s3-multipart-chunk-size 256
--s3-use-multiprocessing --volsize 1024 --verbosity 8
boto3+s3://s3.amazonaws.com/bucket.name
Using the duplicity packages from focal/main
duplicity=0.8.11.1612-1
And I installed the boto packages from focal/universe
python3-boto3=1.9.253-1
python3-botocore=1.14.14+repack-1
This runs into the error:
Using temporary directory $tmpDir/duplicity-iyhq3se4-tempdir
Traceback (innermost last):
File "/usr/bin/duplicity", line 106, in <module>
with_tempdir(main)
File "/usr/bin/duplicity", line 92, in with_tempdir
fn()
File "/usr/lib/python3/dist-packages/duplicity/dup_main.py", line 1525, in
main
action = commandline.ProcessCommandLine(sys.argv[1:])
File "/usr/lib/python3/dist-packages/duplicity/commandline.py", line 1175, in
ProcessCommandLine
globals.backend = backend.get_backend(args[0])
File "/usr/lib/python3/dist-packages/duplicity/backend.py", line 225, in
get_backend
obj = get_backend_object(url_string)
File "/usr/lib/python3/dist-packages/duplicity/backend.py", line 211, in
get_backend_object
return factory(pu)
File "/usr/lib/python3/dist-packages/duplicity/backends/s3_boto3_backend.py",
line 85, in __init__
self.reset_connection()
File "/usr/lib/python3/dist-packages/duplicity/backends/s3_boto3_backend.py",
line 96, in reset_connection
self.s3.meta.client.head_bucket(Bucket=self.bucket_name)
File "/home/ubuntu/.local/lib/python3.8/site-packages/botocore/client.py",
line 357, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/home/ubuntu/.local/lib/python3.8/site-packages/botocore/client.py",
line 676, in _make_api_call
raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (403) when calling the
HeadBucket operation: Forbidden
I then tried the duplicity snap from the snap store, both the stable and edge
builds, with the same issue:
Main action: list-current
================================================================================
duplicity 0.8.17
Args: /snap/duplicity/138/bin/duplicity --dry-run --ignore-errors
list-current-files --name $name --s3-use-ia --archive-dir $archiveDir --tempdir
$tmpDir --s3-multipart-chunk-size 256 --s3-use-multiprocessing --volsize 1024
--verbosity 8 boto3+s3://s3.amazonaws.com/bucket.name
Linux duplicity 5.4.0-53-generic #59-Ubuntu SMP Wed Oct 21 09:38:44 UTC 2020
x86_64 x86_64
/snap/duplicity/141/usr/bin/python3 3.6.9 (default, Oct 8 2020, 12:12:24)
[GCC 8.4.0]
================================================================================
Attempt 1 failed. ClientError: An error occurred (403) when calling the
HeadBucket operation: Forbidden
Main action: list-current
================================================================================
duplicity 0.8.18.dev11
Args: /snap/duplicity/141/bin/duplicity --dry-run --ignore-errors
list-current-files --name $name --s3-use-ia --archive-dir $archiveDir --tempdir
$tmpDir --s3-multipart-chunk-size 256 --s3-use-multiprocessing --volsize 1024
--verbosity 8 boto3+s3://s3.amazonaws.com/bucket.name
Linux duplicity 5.4.0-53-generic #59-Ubuntu SMP Wed Oct 21 09:38:44 UTC 2020
x86_64 x86_64
/snap/duplicity/141/usr/bin/python3 3.6.9 (default, Oct 8 2020, 12:12:24)
[GCC 8.4.0]
================================================================================
Attempt 1 failed. ClientError: An error occurred (403) when calling the
HeadBucket operation: Forbidden
I also tried to update the boto3/botocore packages by way of pip3:
pip3 freeze | grep boto
boto3==1.16.19
botocore==1.19.19
And lastly as a last ditch, I tried to pull the packages from the
groovy/universe repo, bringing them to:
python3-boto3 1.13.14-1
python3-botocore 1.17.22+repack-1_all
I have also tried in a 20.10 system as well to see if any changes there,
but same results.
Hoping for something I can look at to get this back working as it was in
18.04 since 14.04.
For good measure, I verified in a bionic build that using the same arguments as
above with duplicity=0.7.17
python-boto=2.44.0-1ubuntu2.18.04.1
I then verified by uninstalling the deb and installing the snap (0.8.17)
on bionic that the behavior stays the same.
And just to satisfy the 1-4 guidelines:
1. The main system I am having issues with:
$ lsb_release -rd
Description: Ubuntu 20.04.1 LTS
Release: 20.04
Also tested against
$ lsb_release -rd
Description: Ubuntu 18.04.5 LTS
Release: 18.04
$ lsb_release -rd
Description: Ubuntu 20.10
Release: 20.10
2. $ apt-cache policy duplicity
duplicity:
Installed: 0.8.11.1612-1
Candidate: 0.8.11.1612-1
Version table:
*** 0.8.11.1612-1 500
500 http://us.archive.ubuntu.com/ubuntu focal/main amd64 Packages
100 /var/lib/dpkg/status
3. Expect duplicity to function against my s3 bucket.
4. botocore.exceptions.ClientError: An error occurred (403) when calling the
HeadBucket operation: Forbidden
** Affects: duplicity (Ubuntu)
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Desktop
Packages, which is subscribed to duplicity in Ubuntu.
https://bugs.launchpad.net/bugs/1904622
Title:
boto3+s3 bucket fails
Status in duplicity package in Ubuntu:
New
Bug description:
I have a duplicity job that runs on multiple hosts, and am now having
failures on my 20.04 hosts.
The syntax for the job (as I am currently testing to debug) is:
duplicity --dry-run list-current-files --name $name --s3-use-ia
--archive-dir $archiveDir --tempdir $tmpDir --s3-multipart-chunk-size 256
--s3-use-multiprocessing --volsize 1024 --verbosity 8
boto3+s3://s3.amazonaws.com/bucket.name
Using the duplicity packages from focal/main
duplicity=0.8.11.1612-1
And I installed the boto packages from focal/universe
python3-boto3=1.9.253-1
python3-botocore=1.14.14+repack-1
This runs into the error:
Using temporary directory $tmpDir/duplicity-iyhq3se4-tempdir
Traceback (innermost last):
File "/usr/bin/duplicity", line 106, in <module>
with_tempdir(main)
File "/usr/bin/duplicity", line 92, in with_tempdir
fn()
File "/usr/lib/python3/dist-packages/duplicity/dup_main.py", line 1525, in
main
action = commandline.ProcessCommandLine(sys.argv[1:])
File "/usr/lib/python3/dist-packages/duplicity/commandline.py", line 1175,
in ProcessCommandLine
globals.backend = backend.get_backend(args[0])
File "/usr/lib/python3/dist-packages/duplicity/backend.py", line 225, in
get_backend
obj = get_backend_object(url_string)
File "/usr/lib/python3/dist-packages/duplicity/backend.py", line 211, in
get_backend_object
return factory(pu)
File
"/usr/lib/python3/dist-packages/duplicity/backends/s3_boto3_backend.py", line
85, in __init__
self.reset_connection()
File
"/usr/lib/python3/dist-packages/duplicity/backends/s3_boto3_backend.py", line
96, in reset_connection
self.s3.meta.client.head_bucket(Bucket=self.bucket_name)
File "/home/ubuntu/.local/lib/python3.8/site-packages/botocore/client.py",
line 357, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/home/ubuntu/.local/lib/python3.8/site-packages/botocore/client.py",
line 676, in _make_api_call
raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (403) when calling the
HeadBucket operation: Forbidden
I then tried the duplicity snap from the snap store, both the stable and edge
builds, with the same issue:
Main action: list-current
================================================================================
duplicity 0.8.17
Args: /snap/duplicity/138/bin/duplicity --dry-run --ignore-errors
list-current-files --name $name --s3-use-ia --archive-dir $archiveDir --tempdir
$tmpDir --s3-multipart-chunk-size 256 --s3-use-multiprocessing --volsize 1024
--verbosity 8 boto3+s3://s3.amazonaws.com/bucket.name
Linux duplicity 5.4.0-53-generic #59-Ubuntu SMP Wed Oct 21 09:38:44 UTC 2020
x86_64 x86_64
/snap/duplicity/141/usr/bin/python3 3.6.9 (default, Oct 8 2020, 12:12:24)
[GCC 8.4.0]
================================================================================
Attempt 1 failed. ClientError: An error occurred (403) when calling the
HeadBucket operation: Forbidden
Main action: list-current
================================================================================
duplicity 0.8.18.dev11
Args: /snap/duplicity/141/bin/duplicity --dry-run --ignore-errors
list-current-files --name $name --s3-use-ia --archive-dir $archiveDir --tempdir
$tmpDir --s3-multipart-chunk-size 256 --s3-use-multiprocessing --volsize 1024
--verbosity 8 boto3+s3://s3.amazonaws.com/bucket.name
Linux duplicity 5.4.0-53-generic #59-Ubuntu SMP Wed Oct 21 09:38:44 UTC 2020
x86_64 x86_64
/snap/duplicity/141/usr/bin/python3 3.6.9 (default, Oct 8 2020, 12:12:24)
[GCC 8.4.0]
================================================================================
Attempt 1 failed. ClientError: An error occurred (403) when calling the
HeadBucket operation: Forbidden
I also tried to update the boto3/botocore packages by way of pip3:
pip3 freeze | grep boto
boto3==1.16.19
botocore==1.19.19
And lastly as a last ditch, I tried to pull the packages from the
groovy/universe repo, bringing them to:
python3-boto3 1.13.14-1
python3-botocore 1.17.22+repack-1_all
I have also tried in a 20.10 system as well to see if any changes
there, but same results.
Hoping for something I can look at to get this back working as it was
in 18.04 since 14.04.
For good measure, I verified in a bionic build that using the same arguments
as above with duplicity=0.7.17
python-boto=2.44.0-1ubuntu2.18.04.1
I then verified by uninstalling the deb and installing the snap
(0.8.17) on bionic that the behavior stays the same.
And just to satisfy the 1-4 guidelines:
1. The main system I am having issues with:
$ lsb_release -rd
Description: Ubuntu 20.04.1 LTS
Release: 20.04
Also tested against
$ lsb_release -rd
Description: Ubuntu 18.04.5 LTS
Release: 18.04
$ lsb_release -rd
Description: Ubuntu 20.10
Release: 20.10
2. $ apt-cache policy duplicity
duplicity:
Installed: 0.8.11.1612-1
Candidate: 0.8.11.1612-1
Version table:
*** 0.8.11.1612-1 500
500 http://us.archive.ubuntu.com/ubuntu focal/main amd64 Packages
100 /var/lib/dpkg/status
3. Expect duplicity to function against my s3 bucket.
4. botocore.exceptions.ClientError: An error occurred (403) when calling the
HeadBucket operation: Forbidden
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/duplicity/+bug/1904622/+subscriptions
--
Mailing list: https://launchpad.net/~desktop-packages
Post to : desktop-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~desktop-packages
More help : https://help.launchpad.net/ListHelp
