Re: [Openstack] Using Foreign Keys
On Thu, Apr 12, 2012 at 7:35 AM, J. Daniel Schmidt wrote: > In our point of view foreign keys should be used in all possible places. This > would not harm any database that does not support them but helps all of us to > find data inconsistencies and related bugs, which leads to faster development > with fewer bugs. > FWIW, I recently learned that you can enforce foreign keys in sqlite > 3.6.19 with a pragma by setting up and event/listener on session create. http://www.sqlite.org/foreignkeys.html We've had some good success with it. -clayg ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] nova volume attachment issue
On Fri, Jun 22, 2012 at 11:18 AM, jsb wrote: > Hello, > I am able to create a volume (Essex released version). But, I am not able to > attach it to an instance. > > I am using the following in nova.conf: > --iscsi_ip_address=10.1.2.0 Is that correct? shouldn't it be like 10.1.2.* > --iscsi_helper=tgtadm > > But, still, I am getting "iscsiadm: No portal found" error in > nova-compute.log. What about if you try to discovery the targets directly? do you know the host the volume is on? iscsiadm -m discovery -t st -p > I have open-iscsi, iscsitarget and tgt started. You probably don't want iscsitarget & tgt - if you're nova.conf is set to tgt - try to purge iscsitarget & the dkms. Good luck! -clayg ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] create volume hangs at creating.
On Thu, Jun 28, 2012 at 4:52 AM, Milind wrote: > Thanks for reply. > > When I ran following command > > tgtadm --lld iscsi --mode target --op show > > It hangs forever > So I guess the command to create the export probably hangs too, and the greenthread in nova-volume never joins... which is maybe why you didn't get a traceback? Sadly I've not seen that before from tgt. But I guess it sounds like tgt isn't installed/setup properly. There's a number of things you could check, maybe try and restart the service and watch syslog for errors. Jay makes a good suggestion, about running the commands manually - some understanding of what the volume-manager does to provision volumes and expose them via iscsi would help you understand where things break down. You could also try to install iscsitarget. If your os has packages it's pretty painless, but there is a dkms - just uninstall/purge tgt first. Then try setting nova-volume to use iet as the iscsi_helper in your conf/flags. Good Luck, -clayg ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [openstack-dev][Cinder]Management of Block storage by nova-volume through iSCSI
On Fri, Sep 28, 2012 at 4:37 AM, Ahmed Al-Mehdi wrote: > Who manages LVM VG - "nova-volumes". The operator is responsible for creating the LVM volume group, from the physical volumes's available on the system. This options specifies the name of the volume group that the volume-manager will use to provision volumes and snapshots. You get an error if it does not exist. > My thinking is nova-volume creates LV > out of this VG and feeds it into iSCSI Target using utilites - tgt-admin, > tgt-setup-lun (etc.). This LV is then exported as a block storage by the > iSCSI Target. This block storage is then attached to / seen by a VM > instance on the same or another physical host which has iSCSI connection to > the SCSI Tgt. Would that be correct Sounds right to me. > If so, I have a few questions: > - What if I want to "feed" additional disks/block devices or VGs to > nova-volume module? How and where would I specify that. Or do I have to > modify the nova-volume code to handle that. (One solution I can think of is > to pool the additional storage into the existing VG - nova-volumes.)> vgextend should do the trick. > - What if I don't want to feed any VG to nova-volume, but rather want > nova-volume to call into the iSCSI target to get block storage. > - I would like to understand the interaction/interface/API of nova-volume > (Cinder) that calls into iSCSI Target to expose storage or API to storage > appliance to get block storage.Is this a generic standardized API that > can call into any type of block storage - iSCSI , FCoE, etcv? If so, > pointers to API would be highly appreciated. None of the current drives support this to my knowledge. You would have to write a custom driver. -clayg ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] How to deal with 'tangential' bugs?
Unittest2 lets you define a test case that is expected to fail: http://docs.python.org/library/unittest.html#unittest.expectedFailure new in 2.7, but it could be possible to backport - or do something similar... May have issues with nose: http://code.google.com/p/python-nose/issues/detail?id=325 ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] testing and deploying swift?
Hey Jon, Specially regarding "expanding the ring" - one of the Cloud Files ops wrote some tips in a lp answer awhile back: https://answers.launchpad.net/swift/+question/152024 Clay Gerrard Software Developer Rackspace > -Original Message- > From: openstack-bounces+clay.gerrard=rackspace@lists.launchpad.net > [mailto:openstack- > bounces+clay.gerrard=rackspace@lists.launchpad.net] On Behalf Of > Jon Slenk > Sent: Tuesday, May 17, 2011 5:34 PM > To: openstack@lists.launchpad.net > Subject: [Openstack] testing and deploying swift? > > hi, > > So what are people's processes for tracking Swift releases, on > production systems? > > I'm guessing Rackspace is probably the most serious deployment to > date. If anybody there could comment on what release of Swift is being > run and how you expect to deploy newer versions, that would be fun and > educational to hear about and mull over. > > If anybody working on core Swift could comment on which parts of the > system are more vs. less dangerous to muck with, that would be great, > too. For one example, we're still trying to grok the implications of > significantly changing the Rings (expanding them, usually). Like, what > even qualifies as "significant" vs. not. > > thanks for sharing any experiences, > -Jon. > > ___ > Mailing list: https://launchpad.net/~openstack > Post to : openstack@lists.launchpad.net > Unsubscribe : https://launchpad.net/~openstack > More help : https://help.launchpad.net/ListHelp Confidentiality Notice: This e-mail message (including any attached or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at ab...@rackspace.com, and delete the original message. Your cooperation is appreciated. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Thinking about Backups/Snapshots in Nova Volume
> Andiabes > Sent: Thursday, July 21, 2011 5:57 AM > > I think vish pointed out the main differences between the 2 entities, > and maybe that can lead to name disambiguation... > > Backup is a full copy, and usable without the original object being > available in any state ( original or modified). It's expensive, since > it's a full copy. Main use cases are dr and recovery. > > Snapshot represents a point in time state of the object. It's > relatively cheap ( with the expectation that some copy-on-write or > differencing technique is used). Only usable if the reference point of > the snapshot is available (could be thought of as an incremental > backup); what that reference point is depends on the underlying > implementation technology. Main use case is rewinding to so a historic > state some time in the future. > I think this is a pretty good description of the fundamental differences in the characteristics of what have classically been called backups and snapshots. IMO, EC2/EBS muddied the waters on the distinction between backups vs. snapshot in the cloud, and since I don't really *know* what they do in the background internally - I feel like OpenStack has an opportunity to do better. There's a difference between a fast copy-on-write-ish volume "snapshot" and an archived disk-image-ish dump of a volume to an object store "backup". > That said, with the prereqs met, both can probably be used to mount a > new volume. Exactly. > Reasonable? > So should the distinction be purely and implementation detail or will the OSAPI support a way to expose the difference to the consumer? Naming aside, there's uses cases where one may be better suited to the task-at-hand than the other. Clay Gerrard Confidentiality Notice: This e-mail message (including any attached or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at ab...@rackspace.com, and delete the original message. Your cooperation is appreciated. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Getting approved to submit code for review for OpenStack Swift
Hi Peter, No, nothing special should be needed - sometimes it's tricky to get your launchpad account hooked into to review.openstack.org - but if that's giving you trouble just ask for help in #openstack-dev (we've all been there). So yeah, if you've got some code that helps fix a problem or is small improvement that works for you - just push it on up - we'd love to take a look at it! We'll give you feedback if you have any questions. http://wiki.openstack.org/GerritWorkflow should be mostly all you need. OTOH, if you have a bigger idea for something new or different, it makes the review process easier if you can talk a little bit about what you're doing with some of the swift core devs - context really helps on something big or new - plus if you can explain what you're thinking before you get too far down in the weeds, sometimes they can warn you about any big gotchas that may not be obvious at first but could complicate the implementation just cause of some weird corner of the swift internals. Technically I think there's also like a blueprint "process" ( http://wiki.openstack.org/Blueprints ?), but I'm not sure it's strictly necessary. But it'd be more paperwork for notmyname - so that's fun. Either way, we'd love to hear about what you're doing with swift, you can post back here on the openstack mailing list, or even better - easiest place to find some swifters is in #openstack-swift on Freenode! Welcome! -clayg On Wed, Nov 14, 2012 at 12:48 PM, Peter Portante wrote: > Good afternoon, > > I was wondering if there is anything else I must do in order to get > approved to submit code for review in Gerrit for OpenStack Swift. > > Thanks, > > -peter > -- > This message was sent from Launchpad by > Peter Portante (https://launchpad.net/~peter-a-portante) > using the "Contact this team's admins" link on the OpenStack Contributors > team page (https://launchpad.net/~openstack-cla). > For more information see > https://help.launchpad.net/YourAccount/ContactingPeople > ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Object Replication fails
Did you get this worked out? Starting the replicators on the new nodes should create the objects directory. On Sat, Apr 13, 2013 at 2:26 AM, Philip wrote: > Hi, > > I just tried to add two new servers into the ring. Only the containers > were replicated to the new servers but there are no objects beeing > replicated. The disks don't even have a objects folder yet. On the old > servers there are plenty of log entries that indicate that something is > going wrong: > > Apr 13 11:14:45 z1-n1 object-replicator Bad rsync return code: ['rsync', > '--recursive', '--whole-file', '--human-readable', '--xattrs', > '--itemize-changes', '--ignore-existing', '--timeout=30', > '--contimeout=30', '/srv/node/sdq1/objects/80058/b70', > '/srv/node/sdq1/objects/80058/ff9', '/srv/node/sdq1/objects/80058/5d3', > '/srv/node/sdq1/objects/80058/389', '/srv/node/sdq1/objects/80058/473', > '/srv/node/sdq1/objects/80058/81a', '/srv/node/sdq1/objects/80058/a67', > '/srv/node/sdq1/objects/80058/b72', '/srv/node/sdq1/objects/80058/8f5', > '/srv/node/sdq1/objects/80058/ed3', '/srv/node/sdq1/objects/80058/8db', > '/srv/node/sdq1/objects/80058/4e5', '/srv/node/sdq1/objects/80058/fbf', > '/srv/node/sdq1/objects/80058/5cc', '/srv/node/sdq1/objects/80058/318', > '172.16.100.4::object/sdg1/objects/80058'] -> 12 > > Apr 13 11:14:46 z1-n1 object-replicator rsync: mkdir "/sdl1/objects/75331" > (in object) failed: No such file or directory (2) > > Apr 13 11:14:46 z1-n1 object-replicator rsync error: error in file IO > (code 11) at main.c(605) [Receiver=3.0.9] > Apr 13 11:14:46 z1-n1 object-replicator rsync: read error: Connection > reset by peer (104) > > What could be the reason for this? > > ___ > Mailing list: https://launchpad.net/~openstack > Post to : openstack@lists.launchpad.net > Unsubscribe : https://launchpad.net/~openstack > More help : https://help.launchpad.net/ListHelp > > ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Swift container's rwx permissions
Wow, so I glanced quickly at the github project to try and get some context, but I think I actually start getting *more* confused when I see "swift" in the same class name as "file-system" ;) I'd like you (or maybe vaidy, hi vaidy!) to correct me if I'm wrong, but this hadoop integration will *not* access the filesystem of the object servers directly? Everything will happen on a pool of processing boxes that will talk to swift via HTTP - same as any other client? In that case, the error message is just a leaky abstraction showing through. HDFS probably has permission errors that it tries to helpfully map back to file system constructs which just don't apply when you're trying to "simulate" a file system on object storage. You'll have to get down the the HTTP to understand what's causing the error. Presumably a 401 from Swift, so access to swift logs would be helpful. OTOH, if we're * actually* talking about filesystem permissions; then I'm totally confused. But ACL's definitely won't help. They're just a row sitting in a sqlite database - probably on a totally different server from where the one replica of this object is sitting on the filesystem. Nothing you can set in the api will change the filesystem permissions of the directory structure or files on the object servers. Maybe do you have some more overview info on the general approach? I don't really have any Hadoop experience, so maybe it'd be better if there's a hadoop expert out there that also has some experience with swift and can help get you on the right track... -Clay In On Fri, Apr 26, 2013 at 1:11 AM, Shashank Sahni wrote: > Hi everyone, > > I've been experimenting with using Swift(1.8) as Hadoop's DFS backend. > > https://github.com/DmitryMezhensky/Hadoop-and-Swift-integration > > After a few glitches, I'm able to create/access/delete objects/containers > using hadoop's cli fs tool. But whenever I'm trying to run the job it fails > with the following error. > > ERROR security.UserGroupInformation: PriviledgedActionException > as:dharmesh cause:java.io.IOException: The ownership/permissions on the > staging directory > swift://hadooptest.rackspace/tmp/app/mapred/staging/dharmesh/.staging is > not as expected. It is owned by and permissions are rwxrwxrwx. The > directory must be owned by the submitter dharmesh or by dharmesh and > permissions must be rwx-- > > Note that, the local system username is "dharmesh" and the openstack > account and associated tenant is "dharmesh" too. > > I tried setting the permissions by creating "tmp" container using "swift > post -r 'dharmesh:dharmesh'", but unfortunately ended up with the same > result. Is there an other way to set rwx ACLs in swift? > > -- > Shashank Sahni > > ___ > Mailing list: https://launchpad.net/~openstack > Post to : openstack@lists.launchpad.net > Unsubscribe : https://launchpad.net/~openstack > More help : https://help.launchpad.net/ListHelp > > ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Swift] Is it possible to use tempauth and tempurl together?
Yes tempurl and and tempauth are tessted together, your pipeline looks good. "allow_overrides = false" in the tempauth section could cause this, but it's default is true. Are you creating the tempurl's with "bin/swift-temp-url" ??? clayg@swift:~$ swift post -m temp-url-key:asdf clayg@swift:~$ swift stat -v StorageURL: http://localhost:8080/v1/AUTH_test Auth Token: AUTH_tk63d7b585c939428380a97abd1f36292a Account: AUTH_test Containers: 2 Objects: 2 Bytes: 348 Meta Temp-Url-Key: asdf X-Timestamp: 1374006741.08564 Content-Type: text/plain; charset=utf-8 Accept-Ranges: bytes clayg@swift:~$ swift-temp-url GET 1200 /v1/AUTH_test/mycontainer/swift.tar asdf /v1/AUTH_test/mycontainer/swift.tar?temp_url_sig=b9e76c3caa12d1f47b9255a359c31a95b44e99fe&temp_url_expires=1374049125 clayg@swift:~$ curl " http://localhost:8080/v1/AUTH_test/mycontainer/swift.tar?temp_url_sig=b9e76c3caa12d1f47b9255a359c31a95b44e99fe&temp_url_expires=1374049125"; -I HTTP/1.1 200 OK Content-Length: 21073920 Accept-Ranges: bytes Last-Modified: Wed, 17 Jul 2013 07:58:57 GMT Etag: a3bef56fdf0721d07920e32fe4e65737 X-Timestamp: 1374047937.12682 Content-Type: application/x-tar Date: Wed, 17 Jul 2013 07:59:20 GMT On Wed, Jul 17, 2013 at 12:39 AM, Jonathan Lu wrote: > Hi, all stackers, > Can I use the tempauth and tempurl at the same time? I will set a > long tempauth timeout time for internal service and a short tempurl timeout > for the ohters? > > My proxy-server.conf is quite easy: > [pipeline:main] > pipeline = healthcheck cache tempurl tempauth proxy-server > > [app:proxy-server] > use = egg:swift#proxy > allow_account_management = true > account_autocreate = true > > [filter:tempauth] > use = egg:swift#tempauth > ... > > [filter:tempurl] > use = egg:swifttempurl#tempurl > > It works well in tempauth. But after I set the > X-Account-Meta-Temp-URL-Key header on the Swift account and try to use > tempurl, I got the "401 Unauthorized: Temp URL invalid" response. > > Thanks, > Jonathan Lu > > ___ > Mailing list: https://launchpad.net/~openstack > Post to : openstack@lists.launchpad.net > Unsubscribe : https://launchpad.net/~openstack > More help : https://help.launchpad.net/ListHelp > > ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] clay.gerrard
http://cgtek.it/uyrsoyrj/jivjynwm.bnligkqbxewrhs ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
