On 13/07/14 13:48, Remo Mattei wrote:
As far as I know max size is 5gb into swift unless you setup a way to
split it
5GB is the default limit for a single request. However, swift provides a
couple of ways to upload larger single objects:
http://docs.openstack.org/developer/swift/overview_large_objects.html
OwnCloud should probably implement one of these methods to allow objects
greater than the default limit.
Regards,
Tom
Il giorno Jul 12, 2014, alle ore 16:42, Jason Bishop
<jason.bis...@gmail.com <mailto:jason.bis...@gmail.com>> ha scritto:
Hi Frank, I have to say ownCloud is really nicely done. i have been
in need of a file share which has native linux client support and a
really robust storage backend. ownCloud with swift meets the
requirements. I do have a comment which mirrors Hugo's comments I
think. Is it feasible to have the concept of users in ownCloud more
closely match up with swift's concept? It would be nice if the
backend storage for each user was stored in that user's swift account.
I'm sure it will add complexity to user management and authentication
but the advantages of separation of user data may offset that.
I did some testing on ubuntu trusty against an ubuntu trusty icehouse
openstack. I updated the config file and didn't get it quite right
the first time. The error in the admin gui told me what the problem
was (I had wrong string in for the region attribute) so that worked nice.
I stood up an instance of trusty and installed the owncloud client and
copied some files into the shared directory.
a 2gb file worked fine. when i tried a 7gb file I got this error
message, so I think the code in ownCloud does not handle > 5GB files
in swift. It also looks like it keeps retrying the upload over and
over. i'll watch it to see if it eventually gives up and errors out.
Fatal webdav Exception: #9 {main} 2014-07-12T16:11:43+00:00
Fatal webdav Exception: #8 /var/www/html/owncloud/remote.php(52):
require_once('/var/www/html/o...') 2014-07-12T16:11:43+00:00
Fatal webdav Exception: #7
/var/www/html/owncloud/apps/files/appinfo/remote.php(61):
Sabre\DAV\Server->exec() 2014-07-12T16:11:43+00:00
FatalwebdavException: #6
/var/www/html/owncloud/3rdparty/sabre/dav/lib/Sabre/DAV/Server.php(214):
Sabre\DAV\Server->invokeMethod('PUT',
'7gigafile-chunk...')2014-07-12T16:11:43+00:00
FatalwebdavException: #5
/var/www/html/owncloud/3rdparty/sabre/dav/lib/Sabre/DAV/Server.php(474):
call_user_func(Array, '7gigafile-chunk...')2014-07-12T16:11:43+00:00
FatalwebdavException: #4 [internal function]:
Sabre\DAV\Server->httpPut('7gigafile-chunk...')2014-07-12T16:11:43+00:00
FatalwebdavException: #3
/var/www/html/owncloud/3rdparty/sabre/dav/lib/Sabre/DAV/Server.php(900):
Sabre\DAV\Server->createFile('7gigafile-chunk...', Resource id #17,
NULL)2014-07-12T16:11:43+00:00
FatalwebdavException: #2
/var/www/html/owncloud/3rdparty/sabre/dav/lib/Sabre/DAV/Server.php(1647):
OC_Connector_Sabre_Directory->createFile('7gigafile-chunk...',
Resource id #17)2014-07-12T16:11:43+00:00
FatalwebdavException: #1
/var/www/html/owncloud/lib/private/connector/sabre/directory.php(76):
OC_Connector_Sabre_File->put(Resource id #17)2014-07-12T16:11:43+00:00
FatalwebdavSabre\DAV\Exception: Could not rename part file assembled
from chunks2014-07-12T16:11:43+00:00
Errorwebdav\OC\Files\Filesystem::rename() failed2014-07-12T16:11:43+00:00
ErrorobjectstoreCould not create object: Client error response [status
code] 413 [reason phrase] Request Entity Too Large [url]
http://openstack-swiftproxy.stanford.edu:8080/v1/AUTH_78043e536430433fae0318b0ac1040ff/owncloud2/urn:oid:1163
I will continue testing and let you know if I find anything else. I
am still using just http connection for keystone and swift services.
I will pull proper https certificates and check ssl plumbing is
working fine.
After that my next steps will probably be authentication and LDAP related.
Jason
On Mon, Jul 7, 2014 at 9:09 AM, Frank Karlitschek <fr...@owncloud.com
<mailto:fr...@owncloud.com>> wrote:
Hi Hugo,
thanks a lot for the recommendations. This is super helpful.
We will look into the upload problem you found.
Cheers
Frank
> On 07.07.2014, at 05:09, Kuo Hugo <tonyt...@gmail.com
<mailto:tonyt...@gmail.com>> wrote:
>
> Hi Frank,
>
> I had a try with it on my box here. Most of OwnCloud's features
are done by OwnCloud host. Not too much functions rely on Swift's
feature. It simply uses CRUD methods.
>
> [Suggestions]
> • Multiple containers/accounts for Swift as primary
storage. The performance will degrade while over 1million within a
container.
> • Since all metadata is stored in the DB on OwnCloud host.
And OwnCloud remaps the filename as object_id in Swift. That would
be good to store some mapping metadata in Swift for safety. If the
DB crashed, no chance to get the proper data from Swift. User has
no knowledge about object name "urn:oid:153".
> • For large object(100MB+), OwnCloud chunks it into pieces
and upload each segment to Swift as single object. Swift provide
two methods for handling large object DLO/SLO. It might be good to
leverage it for reduce some loading on OwnCloud. And more
consistency of how large object been stored in Swift.
> • Data traffic passthrough the OwnCloud server will cause
the bottleneck/loading on OwnCloud host. To upload data to Swift
API from client app directly may bring you some benefits.
> • OwnCloud can have the shared link from Swift directly
via TempURL feature.
> • The Object expired feature of Swift is a plus for
OwnCloud to use.
> • Pre-create folders for user in objectstore as primary
storage mode.
> • When downloading file, seems OwnCloud will cache it
somewhere. Once the entire object were loaded into OwnCloud then
sending to user. To streaming it is an ideally way to do so from
my perspective.
>
> Renaming is easy on OwnCloud by overwriting the value in DB.
> Thumbnail is good idea.
> To edit txt file on Web is awesome.
>
> I got a problem to upload a 470MB object from OSX client app. It
just showed me timeout but no any further clues here.
>
> Cheers // Hugo
>
>
>
>
> 2014-07-04 10:31 GMT+08:00 John Dickinson <m...@not.mn
<mailto:m...@not.mn>>:
> That's fantastic news! Hearing more companies and applications
supporting Swift is great for the community, and it gives us as
contributors yet another use case to reference.
>
> I work with Hugo, so I'll peek over his shoulder to check it
out. Meanwhile, if there's anything I can do to help answer
questions or get involved in the community (from contributing to
getting issues addressed), please let me know.
>
> --John
>
>
> > On Jul 3, 2014, at 6:01 PM, Frank Karlitschek
<fr...@owncloud.com <mailto:fr...@owncloud.com>> wrote:
> >
> > Hi everyone,
> >
> >
> > Tom Fifield recommended that I send a message to this list.
> >
> > As some of you might know we at ownCloud just added Swift
support as primary storage to ownCloud. This will be released as
part of our upcoming release 7 in about 3 weeks.
> >
> > The code can be seen here.
> > https://github.com/owncloud/core/pull/8383
> >
> >
> > ownCloud is a fully open source solution that can be used a
self hosted Dropbox or Google Drive alternative. The new Swift
backend is an important step for us to mask ownCloud more
scalable, easier to deploy in bigger setups and integrate it
better into OpenStack.
> >
> > Tom suggested to ask here for testers who are willing to try
ownCloud 7 together with Swift.
> > We just announced the RC1 a few minutes ago:
> >
http://mailman.owncloud.org/pipermail/announcements/2014-July/000049.html
> >
> >
> > So it would be great if you guys could give it a test and give
some feedback.
> >
> >
> > Thanks a lot
> >
> >
> > Cheers
> > Frank
> > _______________________________________________
> > Mailing list:
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> > Post to : openstack@lists.openstack.org
<mailto:openstack@lists.openstack.org>
> > Unsubscribe :
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>
> _______________________________________________
> Mailing list:
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> Post to : openstack@lists.openstack.org
<mailto:openstack@lists.openstack.org>
> Unsubscribe :
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>
_______________________________________________
Mailing list:
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
<mailto:openstack@lists.openstack.org>
Unsubscribe :
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
!DSPAM:1,53c1cac3265432135880153!
_______________________________________________
Mailing list:
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
<mailto:openstack@lists.openstack.org>
Unsubscribe :
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
!DSPAM:1,53c1cac3265432135880153!
_______________________________________________
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
_______________________________________________
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
