[
https://issues.apache.org/jira/browse/LIBCLOUD-1007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16585816#comment-16585816
]
Gavin Williams commented on LIBCLOUD-1007:
------------------------------------------
PR raised - [https://github.com/apache/libcloud/pull/1232]
> RFC3339 date parsing in GCE driver fails
> ----------------------------------------
>
> Key: LIBCLOUD-1007
> URL: https://issues.apache.org/jira/browse/LIBCLOUD-1007
> Project: Libcloud
> Issue Type: Bug
> Components: Compute
> Reporter: Gavin Williams
> Priority: Major
>
> When attempting to use the `deprecated` param on `GCENodeImage.deprecate()`
> [1], the call was failing with:
> {noformat}
> 2018-08-20 11:52:08 - DEBUG: Delete at = 2018-08-23T10:52:08.061384+00:00
> Traceback (most recent call last):
> File
> "/Users/fatmcgav/.virtualenvs/packer_builder/lib/python3.6/site-packages/libcloud/compute/drivers/gce.py",
> line 6371, in ex_deprecate_image
> timestamp_to_datetime(value)
> File
> "/Users/fatmcgav/.virtualenvs/packer_builder/lib/python3.6/site-packages/libcloud/compute/drivers/gce.py",
> line 57, in timestamp_to_datetime
> ts = datetime.datetime.strptime(timestamp[:-10], '%Y-%m-%dT%H:%M:%S')
> File
> "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/_strptime.py",
> line 565, in _strptime_datetime
> tt, fraction = _strptime(data_string, format)
> File
> "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/_strptime.py",
> line 365, in _strptime
> data_string[found.end():])
> ValueError: unconverted data remains: .06
> During handling of the above exception, another exception occurred:
> Traceback (most recent call last):
> File "scripts/packer-image-manager.py", line 135, in <module>
> main()
> File "scripts/packer-image-manager.py", line 126, in main
> deprecate_images()
> File "scripts/packer-image-manager.py", line 56, in deprecate_images
> dry_run=args.dry_run)
> File "/Users/fatmcgav/Work/infra/infra/packer2.0/lib/utils.py", line 82, in
> deprecate_image
> if image.deprecate(replacement_image, 'DEPRECATED', deleted=delete_at):
> File
> "/Users/fatmcgav/.virtualenvs/packer_builder/lib/python3.6/site-packages/libcloud/compute/drivers/gce.py",
> line 699, in deprecate
> deprecated, obsolete, deleted)
> File
> "/Users/fatmcgav/.virtualenvs/packer_builder/lib/python3.6/site-packages/libcloud/compute/drivers/gce.py",
> line 6375, in ex_deprecate_image
> attribute)
> ValueError: deleted must be an RFC3339 timestamp
> {noformat}
> It would appear that the date parsing logic in `timestamp_to_datetime` [2] is
> flawed, assuming that the length of `microseconds + tz offset` will always be
> 10 characters, whereas the above example has 12 characters.
> Patch applied as follows:
> {code:java}
> diff --git a/libcloud/compute/drivers/gce.py b/libcloud/compute/drivers/gce.py
> index 3dfc87d3..efcdcb87 100644
> --- a/libcloud/compute/drivers/gce.py
> +++ b/libcloud/compute/drivers/gce.py
> @@ -52,9 +52,9 @@ def timestamp_to_datetime(timestamp):
> :return: Datetime object corresponding to timestamp
> :rtype: :class:`datetime.datetime`
> """
> - # We remove timezone offset and microseconds (Python 2.5 strptime doesn't
> - # support %f)
> - ts = datetime.datetime.strptime(timestamp[:-10], '%Y-%m-%dT%H:%M:%S')
> + # We remove timezone offset
> + ts = datetime.datetime.strptime(timestamp[:timestamp.index('+')],
> + '%Y-%m-%dT%H:%M:%S.%f')
> tz_hours = int(timestamp[-5:-3])
> tz_mins = int(timestamp[-2:]) * int(timestamp[-6:-5] + '1')
> tz_delta = datetime.timedelta(hours=tz_hours, minutes=tz_mins)
> {code}
>
> However once that issue was resolved, the Google Compute API call failed with:
> {noformat}
> 2018-08-20 12:04:59 - DEBUG: Delete at = 2018-08-23T11:04:59.525758+00:00
> Traceback (most recent call last):
> File "scripts/packer-image-manager.py", line 135, in <module>
> main()
> File "scripts/packer-image-manager.py", line 126, in main
> deprecate_images()
> File "scripts/packer-image-manager.py", line 56, in deprecate_images
> dry_run=args.dry_run)
> File "/Users/fatmcgav/Work/infra/infra/packer2.0/lib/utils.py", line 82, in
> deprecate_image
> if image.deprecate(replacement_image, 'DEPRECATED', deleted=delete_at):
> File
> "/Users/fatmcgav/.virtualenvs/packer_builder/lib/python3.6/site-packages/libcloud/compute/drivers/gce.py",
> line 699, in deprecate
> deprecated, obsolete, deleted)
> File
> "/Users/fatmcgav/.virtualenvs/packer_builder/lib/python3.6/site-packages/libcloud/compute/drivers/gce.py",
> line 6381, in ex_deprecate_image
> self.connection.request(request, method='POST', data=image_data).object
> File
> "/Users/fatmcgav/.virtualenvs/packer_builder/lib/python3.6/site-packages/libcloud/compute/drivers/gce.py",
> line 124, in request
> response = super(GCEConnection, self).request(*args, **kwargs)
> File
> "/Users/fatmcgav/.virtualenvs/packer_builder/lib/python3.6/site-packages/libcloud/common/google.py",
> line 808, in request
> *args, **kwargs)
> File
> "/Users/fatmcgav/.virtualenvs/packer_builder/lib/python3.6/site-packages/libcloud/common/base.py",
> line 637, in request
> response = responseCls(**kwargs)
> File
> "/Users/fatmcgav/.virtualenvs/packer_builder/lib/python3.6/site-packages/libcloud/common/base.py",
> line 155, in __init__
> self.object = self.parse_body()
> File
> "/Users/fatmcgav/.virtualenvs/packer_builder/lib/python3.6/site-packages/libcloud/common/google.py",
> line 288, in parse_body
> raise InvalidRequestError(message, self.status, code)
> libcloud.common.google.InvalidRequestError: {'domain': 'global', 'reason':
> 'invalid', 'message': "Invalid value for field 'deprecationStatus.deleted':
> '2018-08-23T11:04:59.525758'. Must be a string representation of a full-date
> or date-time in valid RFC3339 format with either 0 or 3 digits for fractional
> seconds"}
> {noformat}
> Tracked this down to the `ex_deprecate_image` [3] call, which appeared to be
> 'validating' the format of the timestamp, but not actually storing the
> converted result.
> Another patch applied:
> {code:java}
> diff --git a/libcloud/compute/drivers/gce.py b/libcloud/compute/drivers/gce.py
> index 3dfc87d3..a3329133 100644
> --- a/libcloud/compute/drivers/gce.py
> +++ b/libcloud/compute/drivers/gce.py
> @@ -6523,11 +6523,11 @@ class GCENodeDriver(NodeDriver):
> continue
> try:
> - timestamp_to_datetime(value)
> + value = timestamp_to_datetime(value)
> except:
> raise ValueError('%s must be an RFC3339 timestamp' %
> attribute)
> - image_data[attribute] = value
> + image_data[attribute] = value.isoformat(timespec='milliseconds')
> request = '/global/images/%s/deprecate' % (image.name)
> {code}
> Pull request incoming with both patches.
> [1]
> [https://github.com/apache/libcloud/blob/trunk/libcloud/compute/drivers/gce.py#L6485-L6486]
> [2]
> [https://github.com/apache/libcloud/blob/trunk/libcloud/compute/drivers/gce.py#L44-L61]
>
> [3]
> [https://github.com/apache/libcloud/blob/trunk/libcloud/compute/drivers/gce.py#L6525-L6530]
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
