30.11.2011 15:51, Vadim Bulst wrote:
> @Vladislav
>
> Where and how can I set the switch for the cluster manager if it runs as
> a resource.
Ahm, I use my own RA for clvmd, and don't remember if upstream has that
possibility.
Please find attached.
I can't say it is perfect, it is just a quick hack over accidentally
found one, but it does its function for me.
Set 'avoid_lck' to 1.
Best,
Vladislav
>
>
> Am 30.11.2011 13:10, schrieb Vadim Bulst:
>> Am 30.11.2011 12:22, schrieb Vladislav Bogdanov:
>>> 30.11.2011 14:08, Vadim Bulst wrote:
>>>> Hello,
>>>>
>>>> first of all I'd like to ask you a general question:
>>>>
>>>> Does somebody successfully set up a clvm cluster with pacemaker and run
>>>> it in productive mode?
>>> I will say yes after I finally resolve remaining dlm&fencing issues.
>>>
>>>> Now back to the concrete problem:
>>>>
>>>> I configured two interfaces for corosync:
>>>>
>>>> root@bbzclnode04:~# corosync-cfgtool -s
>>>> Printing ring status.
>>>> Local node ID 897624256
>>>> RING ID 0
>>>> id = 192.168.128.53
>>>> status = ring 0 active with no faults
>>>> RING ID 1
>>>> id = 192.168.129.23
>>>> status = ring 1 active with no faults
>>>>
>>>> RRD set to passive
>>>>
>>>> I also made some changes to my cib:
>>>>
>>>> node bbzclnode04
>>>> node bbzclnode06
>>>> node bbzclnode07
>>>> primitive clvm ocf:lvm2:clvmd \
>>>> params daemon_timeout="30" \
>>>> meta target-role="Started"
>>> Please instruct clvmd to use corosync stack instead of openais (-I
>>> corosync): otherwise it uses LCK service which is not mature and I
>>> observed major problems with it.
>>>
>>>> primitive dlm ocf:pacemaker:controld \
>>>> meta target-role="Started"
>>>> group dlm-clvm dlm clvm
>>>> clone dlm-clvm-clone dlm-clvm \
>>>> meta interleave="true" ordered="true"
>>>> property $id="cib-bootstrap-options" \
>>>> dc-version="1.1.5-01e86afaaa6d4a8c4836f68df80ababd6ca3902f" \
>>>> cluster-infrastructure="openais" \
>>>> expected-quorum-votes="3" \
>>>> no-quorum-policy="ignore" \
>>>> stonith-enabled="false" \
>>>> last-lrm-refresh="1322643084"
>>>>
>>>> I cleaned and restarted the resources - nothing! :
>>>>
>>>> crm(live)resource# cleanup dlm-clvm-clone
>>>> Cleaning up dlm:0 on bbzclnode04
>>>> Cleaning up dlm:0 on bbzclnode06
>>>> Cleaning up dlm:0 on bbzclnode07
>>>> Cleaning up clvm:0 on bbzclnode04
>>>> Cleaning up clvm:0 on bbzclnode06
>>>> Cleaning up clvm:0 on bbzclnode07
>>>> Cleaning up dlm:1 on bbzclnode04
>>>> Cleaning up dlm:1 on bbzclnode06
>>>> Cleaning up dlm:1 on bbzclnode07
>>>> Cleaning up clvm:1 on bbzclnode04
>>>> Cleaning up clvm:1 on bbzclnode06
>>>> Cleaning up clvm:1 on bbzclnode07
>>>> Cleaning up dlm:2 on bbzclnode04
>>>> Cleaning up dlm:2 on bbzclnode06
>>>> Cleaning up dlm:2 on bbzclnode07
>>>> Cleaning up clvm:2 on bbzclnode04
>>>> Cleaning up clvm:2 on bbzclnode06
>>>> Cleaning up clvm:2 on bbzclnode07
>>>> Waiting for 19 replies from the CRMd................... OK
>>>>
>>>> crm_mon:
>>>>
>>>> ============
>>>> Last updated: Wed Nov 30 10:15:09 2011
>>>> Stack: openais
>>>> Current DC: bbzclnode04 - partition with quorum
>>>> Version: 1.1.5-01e86afaaa6d4a8c4836f68df80ababd6ca3902f
>>>> 3 Nodes configured, 3 expected votes
>>>> 1 Resources configured.
>>>> ============
>>>>
>>>> Online: [ bbzclnode04 bbzclnode06 bbzclnode07 ]
>>>>
>>>>
>>>> Failed actions:
>>>> clvm:1_start_0 (node=bbzclnode06, call=11, rc=1, status=complete):
>>>> unknown error
>>>> clvm:0_start_0 (node=bbzclnode04, call=11, rc=1, status=complete):
>>>> unknown error
>>>> clvm:2_start_0 (node=bbzclnode07, call=11, rc=1, status=complete):
>>>> unknown error
>>>>
>>>>
>>>> When I look in the log - there is a message which tells me that may be
>>>> another clvm process is already running - but it isn't so.
>>>>
>>>> "clvmd could not create local socket Another clvmd is probably already
>>>> running"
>>>>
>>>> Or is it a permission problem - writing to the filesystem? Is there a
>>>> way to get rid of it?
>>> You can try to run it manually under strace. It will show you what
>>> happens.
>>
>>
>> Here we go:
>>
>> root@bbzclnode07:~# strace clvmd -d -I cororsync
>> execve("/usr/sbin/clvmd", ["clvmd", "-d", "-I", "cororsync"], [/* 18
>> vars */]) = 0
>> brk(0) = 0x12f7000
>> access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or
>> directory)
>> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
>> 0) = 0x7f9f09dad000
>> access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or
>> directory)
>> open("/etc/ld.so.cache", O_RDONLY) = 3
>> fstat(3, {st_mode=S_IFREG|0644, st_size=25864, ...}) = 0
>> mmap(NULL, 25864, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f9f09da6000
>> close(3) = 0
>> access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or
>> directory)
>> open("/lib/x86_64-linux-gnu/libdl.so.2", O_RDONLY) = 3
>> read(3,
>> "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\340\r\0\0\0\0\0\0"...., 832)
>> = 832
>> fstat(3, {st_mode=S_IFREG|0644, st_size=14768, ...}) = 0
>> mmap(NULL, 2109704, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
>> 0) = 0x7f9f0998b000
>> mprotect(0x7f9f0998d000, 2097152, PROT_NONE) = 0
>> mmap(0x7f9f09b8d000, 8192, PROT_READ|PROT_WRITE,
>> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x7f9f09b8d000
>> close(3) = 0
>> access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or
>> directory)
>> open("/lib/libdevmapper-event.so.1.02.1", O_RDONLY) = 3
>> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0
>> \24\0\0\0\0\0\0"..., 832) = 832
>> fstat(3, {st_mode=S_IFREG|0644, st_size=18704, ...}) = 0
>> mmap(NULL, 2113872, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
>> 0) = 0x7f9f09786000
>> mprotect(0x7f9f0978a000, 2093056, PROT_NONE) = 0
>> mmap(0x7f9f09989000, 8192, PROT_READ|PROT_WRITE,
>> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x3000) = 0x7f9f09989000
>> close(3) = 0
>> access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or
>> directory)
>> open("/lib/libdevmapper.so.1.02.1", O_RDONLY) = 3
>> read(3,
>> "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0`L\0\0\0\0\0\0"...,
>> 832) = 832
>> fstat(3, {st_mode=S_IFREG|0644, st_size=143296, ...}) = 0
>> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
>> 0) = 0x7f9f09da5000
>> mmap(NULL, 2238488, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
>> 0) = 0x7f9f09563000
>> mprotect(0x7f9f09583000, 2097152, PROT_NONE) = 0
>> mmap(0x7f9f09783000, 12288, PROT_READ|PROT_WRITE,
>> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x20000) = 0x7f9f09783000
>> close(3) = 0
>> access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or
>> directory)
>> open("/usr/lib/libcman.so.3", O_RDONLY) = 3
>> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0
>> \23\0\0\0\0\0\0"..., 832) = 832
>> fstat(3, {st_mode=S_IFREG|0644, st_size=22544, ...}) = 0
>> mmap(NULL, 2117800, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
>> 0) = 0x7f9f0935d000
>> mprotect(0x7f9f09361000, 2097152, PROT_NONE) = 0
>> mmap(0x7f9f09561000, 8192, PROT_READ|PROT_WRITE,
>> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4000) = 0x7f9f09561000
>> close(3) = 0
>> access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or
>> directory)
>> open("/usr/lib/libconfdb.so.4", O_RDONLY) = 3
>> read(3,
>> "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0P\35\0\0\0\0\0\0"...,
>> 832) = 832
>> fstat(3, {st_mode=S_IFREG|0644, st_size=51680, ...}) = 0
>> mmap(NULL, 2148992, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
>> 0) = 0x7f9f09150000
>> mprotect(0x7f9f0915b000, 2097152, PROT_NONE) = 0
>> mmap(0x7f9f0935b000, 8192, PROT_READ|PROT_WRITE,
>> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0xb000) = 0x7f9f0935b000
>> close(3) = 0
>> access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or
>> directory)
>> open("/usr/lib/libdlm.so.3", O_RDONLY) = 3
>> read(3,
>> "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\260\25\0\0\0\0\0\0"....,
>> 832) = 832
>> fstat(3, {st_mode=S_IFREG|0644, st_size=18624, ...}) = 0
>> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
>> 0) = 0x7f9f09da4000
>> mmap(NULL, 2113912, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
>> 0) = 0x7f9f08f4b000
>> mprotect(0x7f9f08f4f000, 2093056, PROT_NONE) = 0
>> mmap(0x7f9f0914e000, 8192, PROT_READ|PROT_WRITE,
>> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x3000) = 0x7f9f0914e000
>> close(3) = 0
>> access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or
>> directory)
>> open("/lib/x86_64-linux-gnu/libpthread.so.0", O_RDONLY) = 3
>> read(3,
>> "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0Pl\0\0\0\0\0\0"...,
>> 832) = 832
>> fstat(3, {st_mode=S_IFREG|0755, st_size=135500, ...}) = 0
>> mmap(NULL, 2212920, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
>> 0) = 0x7f9f08d2e000
>> mprotect(0x7f9f08d46000, 2093056, PROT_NONE) = 0
>> mmap(0x7f9f08f45000, 8192, PROT_READ|PROT_WRITE,
>> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x17000) = 0x7f9f08f45000
>> mmap(0x7f9f08f47000, 13368, PROT_READ|PROT_WRITE,
>> MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f9f08f47000
>> close(3) = 0
>> access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or
>> directory)
>> open("/usr/lib/libcpg.so.4", O_RDONLY) = 3
>> read(3,
>> "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0`\21\0\0\0\0\0\0"...,
>> 832) = 832
>> fstat(3, {st_mode=S_IFREG|0644, st_size=26928, ...}) = 0
>> mmap(NULL, 2122104, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
>> 0) = 0x7f9f08b27000
>> mprotect(0x7f9f08b2d000, 2093056, PROT_NONE) = 0
>> mmap(0x7f9f08d2c000, 8192, PROT_READ|PROT_WRITE,
>> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x5000) = 0x7f9f08d2c000
>> close(3) = 0
>> access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or
>> directory)
>> open("/usr/lib/libSaLck.so.3", O_RDONLY) = 3
>> read(3,
>> "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0p\16\0\0\0\0\0\0"...,
>> 832) = 832
>> fstat(3, {st_mode=S_IFREG|0644, st_size=26952, ...}) = 0
>> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
>> 0) = 0x7f9f09da3000
>> mmap(NULL, 2122128, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
>> 0) = 0x7f9f08920000
>> mprotect(0x7f9f08926000, 2093056, PROT_NONE) = 0
>> mmap(0x7f9f08b25000, 8192, PROT_READ|PROT_WRITE,
>> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x5000) = 0x7f9f08b25000
>> close(3) = 0
>> access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or
>> directory)
>> open("/usr/lib/libquorum.so.4", O_RDONLY) = 3
>> read(3,
>> "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\360\v\0\0\0\0\0\0"...., 832)
>> = 832
>> fstat(3, {st_mode=S_IFREG|0644, st_size=14480, ...}) = 0
>> mmap(NULL, 2109656, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
>> 0) = 0x7f9f0871c000
>> mprotect(0x7f9f0871f000, 2093056, PROT_NONE) = 0
>> mmap(0x7f9f0891e000, 8192, PROT_READ|PROT_WRITE,
>> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x7f9f0891e000
>> close(3) = 0
>> access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or
>> directory)
>> open("/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY) = 3
>> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0
>> \24\2\0\0\0\0\0"..., 832) = 832
>> fstat(3, {st_mode=S_IFREG|0755, st_size=1677624, ...}) = 0
>> mmap(NULL, 3793768, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
>> 0) = 0x7f9f0837d000
>> mprotect(0x7f9f08512000, 2093056, PROT_NONE) = 0
>> mmap(0x7f9f08711000, 20480, PROT_READ|PROT_WRITE,
>> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x194000) = 0x7f9f08711000
>> mmap(0x7f9f08716000, 21352, PROT_READ|PROT_WRITE,
>> MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f9f08716000
>> close(3) = 0
>> access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or
>> directory)
>> open("/lib/x86_64-linux-gnu/libselinux.so.1", O_RDONLY) = 3
>> read(3,
>> "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0`Q\0\0\0\0\0\0"...,
>> 832) = 832
>> fstat(3, {st_mode=S_IFREG|0644, st_size=117720, ...}) = 0
>> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
>> 0) = 0x7f9f09da2000
>> mmap(NULL, 2217512, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
>> 0) = 0x7f9f0815f000
>> mprotect(0x7f9f0817b000, 2093056, PROT_NONE) = 0
>> mmap(0x7f9f0837a000, 8192, PROT_READ|PROT_WRITE,
>> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1b000) = 0x7f9f0837a000
>> mmap(0x7f9f0837c000, 1576, PROT_READ|PROT_WRITE,
>> MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f9f0837c000
>> close(3) = 0
>> access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or
>> directory)
>> open("/lib/x86_64-linux-gnu/libudev.so.0", O_RDONLY) = 3
>> read(3,
>> "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\2401\0\0\0\0\0\0"....,
>> 832) = 832
>> fstat(3, {st_mode=S_IFREG|0644, st_size=55176, ...}) = 0
>> mmap(NULL, 2150448, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
>> 0) = 0x7f9f07f51000
>> mprotect(0x7f9f07f5e000, 2093056, PROT_NONE) = 0
>> mmap(0x7f9f0815d000, 8192, PROT_READ|PROT_WRITE,
>> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0xc000) = 0x7f9f0815d000
>> close(3) = 0
>> access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or
>> directory)
>> open("/usr/lib/libcoroipcc.so.4", O_RDONLY) = 3
>> read(3,
>> "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\240\23\0\0\0\0\0\0"....,
>> 832) = 832
>> fstat(3, {st_mode=S_IFREG|0644, st_size=22872, ...}) = 0
>> mmap(NULL, 2118040, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
>> 0) = 0x7f9f07d4b000
>> mprotect(0x7f9f07d50000, 2093056, PROT_NONE) = 0
>> mmap(0x7f9f07f4f000, 8192, PROT_READ|PROT_WRITE,
>> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4000) = 0x7f9f07f4f000
>> close(3) = 0
>> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
>> 0) = 0x7f9f09da1000
>> access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or
>> directory)
>> open("/lib/x86_64-linux-gnu/librt.so.1", O_RDONLY) = 3
>> read(3,
>> "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\200!\0\0\0\0\0\0"....,
>> 832) = 832
>> fstat(3, {st_mode=S_IFREG|0644, st_size=31752, ...}) = 0
>> mmap(NULL, 2129016, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
>> 0) = 0x7f9f07b43000
>> mprotect(0x7f9f07b4a000, 2093056, PROT_NONE) = 0
>> mmap(0x7f9f07d49000, 8192, PROT_READ|PROT_WRITE,
>> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6000) = 0x7f9f07d49000
>> close(3) = 0
>> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
>> 0) = 0x7f9f09da0000
>> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
>> 0) = 0x7f9f09d9e000
>> arch_prctl(ARCH_SET_FS, 0x7f9f09d9e7c0) = 0
>> mprotect(0x7f9f07d49000, 4096, PROT_READ) = 0
>> mprotect(0x7f9f07f4f000, 4096, PROT_READ) = 0
>> mprotect(0x7f9f0815d000, 4096, PROT_READ) = 0
>> mprotect(0x7f9f0837a000, 4096, PROT_READ) = 0
>> mprotect(0x7f9f08711000, 16384, PROT_READ) = 0
>> mprotect(0x7f9f0891e000, 4096, PROT_READ) = 0
>> mprotect(0x7f9f08b25000, 4096, PROT_READ) = 0
>> mprotect(0x7f9f08d2c000, 4096, PROT_READ) = 0
>> mprotect(0x7f9f08f45000, 4096, PROT_READ) = 0
>> mprotect(0x7f9f0914e000, 4096, PROT_READ) = 0
>> mprotect(0x7f9f0935b000, 4096, PROT_READ) = 0
>> mprotect(0x7f9f09561000, 4096, PROT_READ) = 0
>> mprotect(0x7f9f09783000, 4096, PROT_READ) = 0
>> mprotect(0x7f9f09989000, 4096, PROT_READ) = 0
>> mprotect(0x7f9f09b8d000, 4096, PROT_READ) = 0
>> mprotect(0x67f000, 4096, PROT_READ) = 0
>> mprotect(0x7f9f09daf000, 4096, PROT_READ) = 0
>> munmap(0x7f9f09da6000, 25864) = 0
>> set_tid_address(0x7f9f09d9ea90) = 2847
>> set_robust_list(0x7f9f09d9eaa0, 0x18) = 0
>> futex(0x7ffff6d7cb7c, FUTEX_WAKE_PRIVATE, 1) = 0
>> futex(0x7ffff6d7cb7c, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME,
>> 1, NULL, 7f9f09d9e7c0) = -1 EAGAIN (Resource temporarily unavailable)
>> rt_sigaction(SIGRTMIN, {0x7f9f08d346c0, [], SA_RESTORER|SA_SIGINFO,
>> 0x7f9f08d3e060}, NULL, 8) = 0
>> rt_sigaction(SIGRT_1, {0x7f9f08d34750, [],
>> SA_RESTORER|SA_RESTART|SA_SIGINFO, 0x7f9f08d3e060}, NULL, 8) = 0
>> rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
>> getrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM_INFINITY}) = 0
>> statfs("/selinux", {f_type="EXT2_SUPER_MAGIC", f_bsize=4096,
>> f_blocks=8285134, f_bfree=7803671, f_bavail=7382807, f_files=2105344,
>> f_ffree=2016160, f_fsid={-1739579276, -1310995585}, f_namelen=255,
>> f_frsize=4096}) = 0
>> brk(0) = 0x12f7000
>> brk(0x1318000) = 0x1318000
>> open("/proc/filesystems", O_RDONLY) = 3
>> fstat(3, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
>> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
>> 0) = 0x7f9f09dac000
>> read(3, "nodev\tsysfs\nnodev\trootfs\nnodev\tb"..., 1024) = 325
>> read(3, "", 1024) = 0
>> close(3) = 0
>> munmap(0x7f9f09dac000, 4096) = 0
>> getuid() = 0
>> geteuid() = 0
>> socket(PF_FILE, SOCK_STREAM, 0) = 3
>> connect(3, {sa_family=AF_FILE, path="/var/run/lvm/clvmd.sock"}, 110) =
>> -1 ENOENT (No such file or directory)
>> close(3) = 0
>> open("/etc/localtime", O_RDONLY) = 3
>> fstat(3, {st_mode=S_IFREG|0644, st_size=2309, ...}) = 0
>> fstat(3, {st_mode=S_IFREG|0644, st_size=2309, ...}) = 0
>> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
>> 0) = 0x7f9f09dac000
>> read(3,
>> "TZif2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\10\0\0\0\10\0\0\0\0"...,
>> 4096) = 2309
>> lseek(3, -1467, SEEK_CUR) = 842
>> read(3,
>> "TZif2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\t\0\0\0\t\0\0\0\0"...,
>> 4096) = 1467
>> close(3) = 0
>> munmap(0x7f9f09dac000, 4096) = 0
>> write(2, "CLVMD[9d9e7c0]: Nov 30 12:51:22 ", 32CLVMD[9d9e7c0]: Nov 30
>> 12:51:22 ) = 32
>> write(2, "CLVMD started\n", 14CLVMD started
>> ) = 14
>> unlink("/var/run/lvm/clvmd.sock") = -1 ENOENT (No such file or
>> directory)
>> umask(077) = 022
>> socket(PF_FILE, SOCK_STREAM, 0) = 3
>> fcntl(3, F_SETFD, FD_CLOEXEC) = 0
>> fcntl(3, F_GETFL) = 0x2 (flags O_RDWR)
>> fcntl(3, F_SETFL, O_RDWR|O_NONBLOCK) = 0
>> bind(3, {sa_family=AF_FILE, path="/var/run/lvm/clvmd.sock"}, 110) = -1
>> ENOENT (No such file or directory)
>> write(2, " ", 2 ) = 2
>> write(2, "can't bind local socket: No such"..., 50can't bind local
>> socket: No such file or directory) = 50
>> write(2, "\n", 1
>> ) = 1
>> close(3) = 0
>> unlink("/var/run/lvm/clvmd.sock") = -1 ENOENT (No such file or
>> directory)
>> umask(022) = 077
>> exit_group(2) = ?
>>
>>
>>
>> I created now the directory "/var/run/lvm" . It wasn't there - work
>> for the package maintainer.
>>
>> IT WORKS NOW - thanks for Your help Vladislav :-))
>>
>>>
>>>> Shell I use a different distro - our install from source?
>>>>
>>>>
>>>> Am 24.11.2011 22:59, schrieb Andreas Kurz:
>>>>> Hello,
>>>>>
>>>>> On 11/24/2011 10:12 PM, Vadim Bulst wrote:
>>>>>> Hi Andreas,
>>>>>>
>>>>>> I changed my cib:
>>>>>>
>>>>>> node bbzclnode04
>>>>>> node bbzclnode06
>>>>>> node bbzclnode07
>>>>>> primitive clvm ocf:lvm2:clvmd \
>>>>>> params daemon_timeout="30"
>>>>>> primitive dlm ocf:pacemaker:controld
>>>>>> group g_lock dlm clvm
>>>>>> clone g_lock-clone g_lock \
>>>>>> meta interleave="true"
>>>>>> property $id="cib-bootstrap-options" \
>>>>>>
>>>>>> dc-version="1.1.5-01e86afaaa6d4a8c4836f68df80ababd6ca3902f" \
>>>>>> cluster-infrastructure="openais" \
>>>>>> expected-quorum-votes="3" \
>>>>>> no-quorum-policy="ignore" \
>>>>>> stonith-enabled="false" \
>>>>>> last-lrm-refresh="1322049979
>>>>>>
>>>>>> but no luck at all.
>>>>> I assume you did at least a cleanup on clvm and it still does not work
>>>>> ... next step would be to grep for ERROR in your cluster log and look
>>>>> for other suspicious messages to find out why clvm is not that
>>>>> motivated
>>>>> to start.
>>>>>
>>>>>> "And use Corosync 1.4.x with redundant rings and automatic ring
>>>>>> recovery
>>>>>> feature enabled."
>>>>>>
>>>>>> I got two interfaces per server - there are bonded together and
>>>>>> bridged
>>>>>> for virtualization. Only one untagged vlan. I tried to give a tagged
>>>>>> Vlan Bridge a Address but didn't worked. My network conf looks
>>>>>> like that:
>>>>> One ore two extra nics are quite affordable today to build e.g. a
>>>>> direct
>>>>> connection between the nodes (if possible)
>>>>>
>>>>> Regards,
>>>>> Andreas
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
>>>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>>>
>>>>> Project Home: http://www.clusterlabs.org
>>>>> Getting started:
>>>>> http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>>>> Bugs: http://bugs.clusterlabs.org
>>>>
>>>> --
>>>> Mit freundlichen Grüßen
>>>>
>>>> Vadim Bulst
>>>> Systemadministrator BBZ
>>>>
>>>> Biotechnologisch-Biomedizinisches Zentrum
>>>> Universität Leipzig
>>>> Deutscher Platz 5, 04103 Leipzig
>>>> Tel.: 0341 97 - 31 307
>>>> Fax : 0341 97 - 31 309
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
>>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>>
>>>> Project Home: http://www.clusterlabs.org
>>>> Getting started:
>>>> http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>>> Bugs: http://bugs.clusterlabs.org
>>>
>>> _______________________________________________
>>> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>>>
>>> Project Home: http://www.clusterlabs.org
>>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>> Bugs: http://bugs.clusterlabs.org
>>>
>>
>>
>
>
#!/bin/bash
# Copyright (c) 2008 Xinwei Hu
# All Rights Reserved.
# Copyright (c) 2011 Vladislav Bogdanov
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of version 2 of the GNU General Public License as
# published by the Free Software Foundation.
#
# This program is distributed in the hope that it would be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
#
# Further, this software is distributed without any warranty that it is
# free of the rightful claim of any third person regarding infringement
# or the like. Any license provided herein, whether implied or
# otherwise, applies only to this software file. Patent licenses, if
# any, provided herein do not apply to combinations of this program with
# other software, or any other product whatsoever.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write the Free Software Foundation,
# Inc., 59 Temple Place - Suite 330, Boston MA 02111-1307, USA.
#
#######################################################################
# OCF initialization
: ${OCF_FUNCTIONS_DIR=${OCF_ROOT}/lib/heartbeat}
. ${OCF_FUNCTIONS_DIR}/ocf-shellfuncs
# Parameter defaults
: ${OCF_RESKEY_CRM_meta_globally_unique:="false"}
: ${OCF_RESKEY_daemon_timeout:="80"}
: ${OCF_RESKEY_daemon_options:="-d0"}
: ${OCF_RESKEY_avoid_lck:=0}
# Common variables
DAEMON="/usr/sbin/clvmd"
status_daemon()
{
PID=`pidof "$DAEMON"`
if [ -n "$PID" ]; then
if grep -q "State:.*zombie" /proc/$PID/status >/dev/null 2>&1 ; then
ocf_log err "$DAEMON is zombie"
return $OCF_ERR_GENERIC
fi
return $OCF_SUCCESS
fi
return $OCF_NOT_RUNNING
}
bringup_daemon()
{
if [ ! -e "$DAEMON" ]; then
ocf_log err "Required binary not found: $DAEMON"
return $OCF_ERR_INSTALLED
fi
CLUSTER_TYPE=$( crm_attribute --type crm_config --name
cluster-infrastructure --query --quiet )
if [ "X${CLUSTER_TYPE}" = "Xopenais" ] && ocf_is_true
${OCF_RESKEY_avoid_lck} ; then
CLUSTER_TYPE="corosync"
fi
# Remove -I ... from daemon options
# And run it with real-time priority
chrt -r 99 "$DAEMON" $( echo $OCF_RESKEY_daemon_options | sed -r 's/-I
[^[:space:]]+ *//' ) -I ${CLUSTER_TYPE}
rc=$?
if [ $rc != 0 ] && [ $rc != 5 ]; then
ocf_log err "Could not start $DAEMON"
return $OCF_ERR_GENERIC
fi
sleep 1
COUNT=0
rc=$OCF_NOT_RUNNING
while [ $rc = $OCF_NOT_RUNNING ]; do
COUNT=`expr $COUNT + 1`
if [ $COUNT -gt $OCF_RESKEY_daemon_timeout ]; then
ocf_log err "`basename $DAEMON` did not come up"
return $OCF_ERR_GENERIC
fi
status_daemon; rc=$?
sleep 1
done
return $rc
}
kill_daemon()
{
status_daemon; rc=$?
if [ $rc != $OCF_SUCCESS ]; then
return rc
fi
ocf_log info "Stopping `basename "$DAEMON"`"
killall -INT "$DAEMON"
sleep 1
status_daemon; rc=$?
if [ $rc = $OCF_ERR_GENERIC ]; then
return ${rc}
fi
COUNT=0
while [ $rc != $OCF_NOT_RUNNING ]; do
COUNT=`expr $COUNT + 1`
if [ $COUNT -gt $OCF_RESKEY_daemon_timeout ]; then
ocf_log err "`basename $DAEMON` shutdown emergency"
killall -9 "$DAEMON"
status_daemon; rc=$?
if [ $rc = $OCF_NOT_RUNNING ]; then
return $OCF_SUCCESS
else
return $OCF_ERR_GENERIC
fi
fi
killall -INT "$DAEMON"
sleep 1
status_daemon; rc=$?
if [ $rc = $OCF_ERR_GENERIC ]; then
return ${rc}
fi
done
return $OCF_SUCCESS
}
clvmd_start() {
clvmd_monitor; rc=$?
if [ $rc != $OCF_NOT_RUNNING ]; then
return $rc
fi
ocf_log info "Starting $OCF_RESOURCE_INSTANCE"
bringup_daemon
return $?
}
clvmd_stop() {
clvmd_monitor; rc=$?
case $rc in
$OCF_NOT_RUNNING) return $OCF_SUCCESS;;
esac
ocf_log info "Stopping $OCF_RESOURCE_INSTANCE"
kill_daemon
return $?
}
clvmd_monitor() {
clvmd_validate
status_daemon
return $?
}
clvmd_usage() {
echo "usage: $0 {start|stop|monitor|validate-all|meta-data}"
echo " Expects to have a fully populated OCF RA-compliant environment set."
echo " In particualr, a value for OCF_ROOT"
}
clvmd_validate() {
: TODO: check for globally_unique=true and return OCF_ERR_CONFIGURED
case ${OCF_RESKEY_CRM_meta_globally_unique} in
yes|Yes|true|True|1)
ocf_log err "$OCF_RESOURCE_INSTANCE must be configured with the
globally_unique=false meta attribute"
exit $OCF_ERR_CONFIGURED
;;
esac
return $OCF_SUCCESS
}
meta_data() {
cat <<END
<?xml version="1.0"?>
<!DOCTYPE resource-agent SYSTEM "ra-api-1.dtd">
<resource-agent name="clvmd">
<version>1.0</version>
<longdesc lang="en">
This is a clvmd Resource Agent.
It starts clvmd as anonymous clones.
</longdesc>
<shortdesc lang="en">clvmd resource agent</shortdesc>
<parameters>
<parameter name="daemon_timeout" unique="0">
<longdesc lang="en">
Number of seconds to allow the control daemon to come up and down
</longdesc>
<shortdesc lang="en">Daemon Timeout</shortdesc>
<content type="string" default="80"/>
</parameter>
<parameter name="daemon_options" unique="0">
<longdesc lang="en">
Options to clvmd. Refer to clvmd.8 for detailed descriptions.
</longdesc>
<shortdesc lang="en">Daemon Options</shortdesc>
<content type="string" default="-d0"/>
</parameter>
<parameter name="avoid_lck" unique="0">
<longdesc lang="en">
Try to avoid use of OpenAIS lck service in favor of corosync (only if
cluster is running on top of openais). This will force clvmd run using
corosync service even if cluster uses openais.
</longdesc>
<shortdesc lang="en">Avoid use of OpenAIS lck service</shortdesc>
<content type="boolean" default="0"/>
</parameter>
</parameters>
<actions>
<action name="start" timeout="90" />
<action name="stop" timeout="100" />
<action name="monitor" timeout="20" depth="0"/>
<action name="meta-data" timeout="5" />
<action name="validate-all" timeout="30" />
</actions>
</resource-agent>
END
}
case $__OCF_ACTION in
meta-data)
meta_data
exit $OCF_SUCCESS
;;
start)
clvmd_start
;;
stop)
clvmd_stop
;;
monitor)
clvmd_monitor
;;
validate-all)
clvmd_validate
;;
usage|help)
clvmd_usage
exit $OCF_SUCCESS
;;
*)
clvmd_usage
exit $OCF_ERR_UNIMPLEMENTED
;;
esac
exit $?
_______________________________________________
Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
http://oss.clusterlabs.org/mailman/listinfo/pacemaker
Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org
