[
https://issues.apache.org/jira/browse/LIBCLOUD-986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16408315#comment-16408315
]
Joshua Hawkinson commented on LIBCLOUD-986:
-------------------------------------------
Okay, done... Here are the diffs for your review:
{code:java}
diff --git a/libcloud/storage/drivers/azure_blobs.py
b/libcloud/storage/drivers/azure_blobs.py
index 82ee03a..2475793 100644
--- a/libcloud/storage/drivers/azure_blobs.py
+++ b/libcloud/storage/drivers/azure_blobs.py
@@ -381,11 +381,12 @@ class AzureBlobsStorageDriver(StorageDriver):
if not params['marker']:
break
- def iterate_container_objects(self, container):
+ def iterate_container_objects(self, container, ex_prefix=None):
"""
@inherits: :class:`StorageDriver.iterate_container_objects`
"""
- params = {'restype': 'container',
+ params = {'prefix': ex_prefix,
+ 'restype': 'container',
'comp': 'list',
'maxresults': RESPONSES_PER_REQUEST,
'include': 'metadata'}
@@ -416,6 +417,22 @@ class AzureBlobsStorageDriver(StorageDriver):
if not params['marker']:
break
+ def list_container_objects(self, container, ex_prefix=None):
+ """
+ Return a list of objects for the given container.
+
+ :param container: Container instance.
+ :type container: :class:`Container`
+
+ :param ex_prefix: Only return objects starting with ex_prefix
+ :type ex_prefix: ``str``
+
+ :return: A list of Object instances.
+ :rtype: ``list`` of :class:`Object`
+ """
+ return list(self.iterate_container_objects(container,
+ ex_prefix=ex_prefix))
+
def get_container(self, container_name):
"""
@inherits: :class:`StorageDriver.get_container`
diff --git a/libcloud/test/storage/test_azure_blobs.py
b/libcloud/test/storage/test_azure_blobs.py
index 95ab3f1..69bdebb 100644
--- a/libcloud/test/storage/test_azure_blobs.py
+++ b/libcloud/test/storage/test_azure_blobs.py
@@ -447,6 +447,28 @@ class AzureBlobsTests(unittest.TestCase):
self.assertTrue('content_encoding' in obj.extra)
self.assertTrue('content_language' in obj.extra)
+ def test_list_container_objects_with_prefix(self):
+ self.mock_response_klass.type = None
+ AzureBlobsStorageDriver.RESPONSES_PER_REQUEST = 2
+
+ container = Container(name='test_container', extra={},
+ driver=self.driver)
+ objects = self.driver.list_container_objects(container=container,
+ ex_prefix='test_prefix')
+ self.assertEqual(len(objects), 4)
+
+ obj = objects[1]
+ self.assertEqual(obj.name, 'object2.txt')
+ self.assertEqual(obj.hash, '0x8CFB90F1BA8CD8F')
+ self.assertEqual(obj.size, 1048576)
+ self.assertEqual(obj.container.name, 'test_container')
+ self.assertTrue('meta1' in obj.meta_data)
+ self.assertTrue('meta2' in obj.meta_data)
+ self.assertTrue('last_modified' in obj.extra)
+ self.assertTrue('content_type' in obj.extra)
+ self.assertTrue('content_encoding' in obj.extra)
+ self.assertTrue('content_language' in obj.extra)
+
{code}
And the before and after test results:
{code:java}
[root@localhost libcloud]# python3 -m pytest -k azure_blobs
=================================================================== test
session starts
====================================================================
platform linux -- Python 3.4.3, pytest-3.4.2, py-1.5.2, pluggy-0.6.0
rootdir: /usr/libcloud, inifile: pytest.ini
collected 9116 items
libcloud/test/storage/test_azure_blobs.py ....................F................
[100%]
=========================================================================
FAILURES
=========================================================================
_________________________________________________
AzureBlobsTests.test_list_container_objects_with_prefix
__________________________________________________
self = <libcloud.test.storage.test_azure_blobs.AzureBlobsTests
testMethod=test_list_container_objects_with_prefix>
def test_list_container_objects_with_prefix(self):
self.mock_response_klass.type = None
AzureBlobsStorageDriver.RESPONSES_PER_REQUEST = 2
container = Container(name='test_container', extra={},
driver=self.driver)
objects = self.driver.list_container_objects(container=container,
> ex_prefix='test_prefix')
E TypeError: list_container_objects() got an unexpected keyword argument
'ex_prefix'
libcloud/test/storage/test_azure_blobs.py:457: TypeError
================================================================== 9079 tests
deselected ===================================================================
=================================================== 1 failed, 36 passed, 9079
deselected in 3.39 seconds ===================================================
[root@localhost libcloud]# python3 -m pytest -k azure_blobs
=================================================================== test
session starts
====================================================================
platform linux -- Python 3.4.3, pytest-3.4.2, py-1.5.2, pluggy-0.6.0
rootdir: /usr/libcloud, inifile: pytest.ini
collected 9116 items
libcloud/test/storage/test_azure_blobs.py .....................................
[100%]
================================================================== 9079 tests
deselected ===================================================================
======================================================== 37 passed, 9079
deselected in 3.08 seconds
========================================================{code}
Some notes about the unit test for this feature... Ideally you'd have one
container that the Mock HTTP server would be able to return different results
for based on the prefix, I didn't see a way to simply accomplish this without
modifying some of the base mock server code. This test method was largely
pulled from the other driver's tests, but it seems to meet your base
requirements. Another note, I'm not sure why this Jira instance always strips
spaces, even in noformat, or code blocks (Sorry about the formatting).
> List / Iterate container with prefix for Azure Blob service (This support
> exists in the other drivers already)
> --------------------------------------------------------------------------------------------------------------
>
> Key: LIBCLOUD-986
> URL: https://issues.apache.org/jira/browse/LIBCLOUD-986
> Project: Libcloud
> Issue Type: Improvement
> Components: Storage
> Reporter: Joshua Hawkinson
> Priority: Major
>
> While testing out the libcloud storage abstraction I've come across a curious
> omission in driver support. It seems that for many drivers there is
> additional support for the {{list_container_objects()}} and
> {{iterate_container_objects()}} to support a prefix (or {{ex_prefix)}}.
> This functionality is not present in the Azure driver for some reason. I've
> gone and checked the list_blobs API and it also supports this functionality
> as seen here:
> [https://docs.microsoft.com/en-us/rest/api/storageservices/list-blobs]
>
> I've attempted a quick and dirty patch that seems to work as well.
> {code}
> azure_blobs.py Line 384
> - def iterate_container_objects(self, container):
> + def iterate_container_objects(self, container, ex_prefix=None):
> """
> @inherits: :class:`StorageDriver.iterate_container_objects`
> """
> params = {'restype': 'container',
> 'comp': 'list',
> + 'prefix': ex_prefix,
> 'maxresults': RESPONSES_PER_REQUEST,
> 'include': 'metadata'}
> Line 419 (Add new function)
> def list_container_objects(self, container, ex_prefix=None):
> """
> Return a list of objects for the given container.
>
> :param container: Container instance.
> :type container: :class:`Container`
>
> :param ex_prefix: Only return objects starting with ex_prefix
> :type ex_prefix: ``str``
>
> :return: A list of Object instances.
> :rtype: ``list`` of :class:`Object`
> """
> return list(self.iterate_container_objects(container,
> ex_prefix=ex_prefix))
>
> {code}
>
> I've just worked around this at a higher level for now. There isn't any
> urgency on my end, but it would be nice if the abstraction was more
> consistent so I could remove the workaround code.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
