I used asyncBackupCount=3 before and behavior was the same.
Regarding UDP I may try to use it but it should work with tcpIpEnabled too
so it's not an issue.
I guess this problem is related to docker environment somehow. I don't have
an access to configuration of VM. Probably I have to expose some additional
port(s) on docker. I found thread about dockerized env of CAS with
hazelcast in this group but this env is based on docker swarm but my env
doesn't support docker swarm.
I saw this property in the documentation of CAS
# ${configurationKey}.cluster.outboundPorts[0]=45000
But container stops with exception if I add it to properties file:
Binding to target [Bindable@2dd2e270 type =
> org.apereo.cas.configuration.CasConfigurationProperties, value =
> 'provided', annotations =
> array<Annotation>[@org.springframework.boot.context.properties.ConfigurationProperties(ignoreInvalidFields=false,
>
> ignoreUnknownFields=false, prefix=cas, value=cas)]] failed:
> Property: cas.ticket.registry.hazelcast.cluster.outboundports[0]
> Value: 33000-33100
> Origin: class path resource [application-dev.properties]:82:56
> Reason: The elements
> [cas.ticket.registry.hazelcast.cluster.outboundports[0]] were left unbound.
On Friday, February 7, 2020 at 12:31:02 AM UTC+2, David Curry wrote:
>
> I believe, if you have 4 members, that asyncBackupCount should be 3.
> Because a node doesn't back itself up. (Hazelcast might be smart enough to
> fix that itself, but I don't know.)
>
> Also, I'm curious as to why you have tcpIpEnabled set to true? You'd be
> much better off, from a performance standpoint, setting it to false and
> using UDP. TCP blocks, UDP doesn't.
>
> I've never run this in docker, so I don't know what, if anything, you need
> to do differently. But I would be at least a little suspicious of the
> interactions there -- it should work of course, but could you be missing
> something in the docker config, or the config on the host?
>
>
> --
>
> DAVID A. CURRY, CISSP
> *DIRECTOR • INFORMATION SECURITY & PRIVACY*
> THE NEW SCHOOL • INFORMATION TECHNOLOGY
>
> 71 FIFTH AVE., 9TH FL., NEW YORK, NY 10003
> +1 646 909-4728 • [email protected] <javascript:>
>
>
> On Thu, Feb 6, 2020 at 4:36 PM 'Maksim Kopeyka' via CAS Community <
> [email protected] <javascript:>> wrote:
>
>> Hi David,
>>
>> I tried similar config with 4 nodes:
>>
>>>
>>> cas.ticket.registry.hazelcast.cluster.members=${HAZELCAST_CLUSTER_MEMBERS}
>>> cas.ticket.registry.hazelcast.cluster.asyncBackupCount=4
>>> cas.ticket.registry.hazelcast.cluster.backupCount=0
>>> cas.ticket.registry.hazelcast.cluster.port=5701
>>> cas.ticket.registry.hazelcast.cluster.portAutoIncrement=false
>>> cas.ticket.registry.hazelcast.cluster.instanceName=localhost
>>>
>>> cas.ticket.registry.hazelcast.cluster.publicAddress=${HAZELCAST_PUBLIC_ADDRESS}
>>> cas.ticket.registry.hazelcast.cluster.tcpipEnabled=true
>>> cas.ticket.registry.hazelcast.crypto.enabled=false
>>
>>
>> I see this output on each node i.e. hazelcast creates a cluster and sees
>> all nodes:
>>
>> 2020-02-06 21:20:49,235 INFO
>>> [com.hazelcast.internal.cluster.ClusterService] -
>>> <[ecdc-rant-affiliateidp-dev-1]:5701 [dev] [3.12.4]
>>> Members {size:4, ver:4} [
>>> Member [wcdc-rant-affiliateidp-dev-1]:5701 -
>>> a245c93b-beb0-4929-b831-e40a323cad8b
>>> Member [ecdc-rant-affiliateidp-dev-2]:5701 -
>>> bcbcd799-8cb8-4e5d-8802-5d95d4015ffd
>>> Member [wcdc-rant-affiliateidp-dev-2]:5701 -
>>> 9d3f52c9-1475-462e-844a-1b534efdca73
>>> Member [ecdc-rant-affiliateidp-dev-1]:5701 -
>>> e9f81f52-7a99-4428-a402-5a2f48cba838 this
>>> ]
>>> >
>>
>>
>> However tickets distribution doesn't work. Nodes 1, 2, 3 don't know about
>> session on Node 4.
>>
>> I don't see any errors in the logs related to hazelcast but this one
>> appears time to time
>>
>>> 2020-02-06 17:31:56,248 ERROR
>>> [org.apereo.cas.web.flow.executor.EncryptedTranscoder] - <Null input buffer>
>>> java.lang.IllegalArgumentException: Null input buffer
>>> at javax.crypto.Cipher.doFinal(Unknown Source) ~[?:?]
>>> at
>>> org.apereo.cas.util.cipher.BaseBinaryCipherExecutor.decode(BaseBinaryCipherExecutor.java:92)
>>>
>>> ~[cas-server-core-util-api-6.1.3.jar!/:6.1.3]
>>
>>
>>
>> On Wednesday, February 5, 2020 at 9:28:43 PM UTC+2, David Curry wrote:
>>>
>>> Maksim,
>>>
>>> If you don't want to ever lose tickets, then you would want all nodes to
>>> back up all other nodes. So if you have 3 member nodes, you would want 2
>>> async backup nodes (asyncBackupCount) and also you'd probably want to
>>> disable the default sync backup (backupCount) node since it will block.
>>> Here are the settings we're running with in production (although this is
>>> CAS 5):
>>>
>>> cas.ticket.registry.hazelcast.cluster.members:
>>> cas01.newschool.edu,cas02.newschool.edu,cas03.newschool.edu,
>>> cas04.newschool.edu,cas05.newschool.edu
>>> cas.ticket.registry.hazelcast.cluster.asyncBackupCount: 4
>>> cas.ticket.registry.hazelcast.cluster.backupCount: 0
>>> cas.ticket.registry.hazelcast.cluster.port: 5701
>>> cas.ticket.registry.hazelcast.cluster.portAutoIncrement: false
>>> cas.ticket.registry.hazelcast.crypto.encryption.key:
>>> xxxIoXN6SBU5bF+iAVTKgw==
>>> cas.ticket.registry.hazelcast.crypto.signing.key:
>>>
>>> xxxmEbPGT_MXg0JWYLTe4oFaOaklocCqlY2VuHBdAHuh0V6-PdQxmgi4tTA3CZZos8TUbzg-L9nYHJpA5RqcvA
>>> cas.ticket.registry.hazelcast.crypto.enabled: true
>>>
>>> This works well for us behind an F5 load balancer; we do not use sticky
>>> sessions. We can (and do) reboot servers in the pool without anyone getting
>>> re-prompted to log in (just don't reboot them all at once).
>>>
>>> The crypto stuff (last three lines) is not needed for this to work, but
>>> you (arguably) might want it in production. You can leave it off while
>>> getting things to work and enable it later.
>>>
>>> One other thing -- did you remember to open 5701 in the firewall on all
>>> the servers?
>>>
>>> --
>>>
>>> DAVID A. CURRY, CISSP
>>> *DIRECTOR • INFORMATION SECURITY & PRIVACY*
>>> THE NEW SCHOOL • INFORMATION TECHNOLOGY
>>>
>>> 71 FIFTH AVE., 9TH FL., NEW YORK, NY 10003
>>> +1 646 909-4728 • [email protected]
>>>
>>>
>>> On Wed, Feb 5, 2020 at 1:40 PM Ray Bon <[email protected]> wrote:
>>>
>>>> Maksim,
>>>>
>>>> There is this config setting
>>>> cas.ticket.registry.hazelcast.cluster.members=
>>>> Add IPs of all members to the list.
>>>> https://apereo.github.io/cas/6.1.x/configuration/Configuration-Properties.html#hazelcast-ticket-registry
>>>>
>>>> and the link to common settings.
>>>>
>>>> There are some hazelcast loggers in log4j2.xml
>>>>
>>>> <AsyncLogger name="com.hazelcast"
>>>> level="${sys:hazelcast.log.level}" includeLocation="true" />
>>>>
>>>> Ray
>>>>
>>>> On Wed, 2020-02-05 at 09:16 -0800, 'Maksim Kopeyka' via CAS Community
>>>> wrote:
>>>>
>>>> Ray,
>>>>
>>>> I asked about CAS functionality to distribute tickets across nodes. I
>>>> need specific CAS functionality based on Hazelcast and seems to me this
>>>> functionality doesn't work as expected so I need to check it somehow.
>>>> Maybe
>>>> with some debug logging.
>>>> I have a cluster with several nodes of CAS with hazelcast ticket
>>>> registry and I have a load balancer. If I turn off sticky sessions CAS
>>>> asks
>>>> me about username/password every time. This cluster works in the same way
>>>> without hazelcast ticket registry. So seems to me this functionality
>>>> doesn't work and I don't see any errors in the logs.
>>>>
>>>> On Wednesday, February 5, 2020 at 7:03:55 PM UTC+2, rbon wrote:
>>>>
>>>> Maksim,
>>>>
>>>> I do not know if there is a stand alone client for accessing hazelcast
>>>> data. In the docs,
>>>> https://docs.hazelcast.org/docs/4.0/manual/html-single/index.html,
>>>> there is a section on clients and one on management. It looks like you
>>>> would have to create an application yourself but someone has probably done
>>>> that already.
>>>>
>>>> Ray
>>>>
>>>> On Wed, 2020-02-05 at 08:11 -0800, 'Maksim Kopeyka' via CAS Community
>>>> wrote:
>>>>
>>>> Hi Ray,
>>>>
>>>> Seems to me Hazelcast doesn't distribute data across all nodes because
>>>> each node doesn't have information about sessions on other nodes.
>>>> How to check data distribution?
>>>>
>>>> On Friday, January 31, 2020 at 11:02:05 PM UTC+2, rbon wrote:
>>>>
>>>> Maksim,
>>>>
>>>> Hazelcast is distributed but not replicated. Thus, when a server goes
>>>> down, the tickets on that server are lost. You have to relogin only if
>>>> your
>>>> ticket was on that server.
>>>> Hazelcast has some mechanism of determining which node has which
>>>> ticket. It may also be possible to make hazelcast replicated but I have
>>>> not
>>>> tried.
>>>>
>>>> Ray
>>>>
>>>> P.S. you have two node3s in your config.
>>>>
>>>> On Fri, 2020-01-31 at 11:51 -0800, 'Maksim Kopeyka' via CAS Community
>>>> wrote:
>>>>
>>>> Hi Andy,
>>>>
>>>> Your example is very helpful. Thank you.
>>>> I see how hazelcast tickets registry works on my local env. I turned
>>>> off active container and another container continues to work with my
>>>> active
>>>> session without relogin.
>>>>
>>>> However on my remote env. with CAS 6.1.3 it doesn't work in this way.
>>>> Load balancer ask me to re-login If I turn off active container.
>>>> Each node with hazelcast sees other nodes. I see such messages on all
>>>> nodes in case node1 is turned off:
>>>>
>>>> WARN [com.hazelcast.nio.tcp.TcpIpConnectionErrorHandler] - <[node2]:
>>>> 5701 [dev] [3.12.4] Removing connection to endpoint [node1]:5701 Cause
>>>> => java.net.SocketException {Connection refused to address node1/xx.xx.
>>>> xx.xx:5701}, Error-Count: 5>
>>>> WARN [com.hazelcast.internal.cluster.impl.MembershipManager] - <[node2
>>>> ]:5701 [dev] [3.12.4] Member [node1]:5701 -
>>>> b1fba639-dfff-4536-b5f4-a8681920594d
>>>> is suspected to be dead for reason: No connection>
>>>> WARN [com.hazelcast.nio.tcp.TcpIpConnectionErrorHandler] - <[node2]:
>>>> 5701 [dev] [3.12.4] Removing connection to endpoint [node1]:5701 Cause
>>>> => java.net.SocketException {Connection refused to address node1/xx.xx.
>>>> xx.xx:5701}, Error-Count: 6>
>>>> WARN [com.hazelcast.nio.tcp.TcpIpConnectionErrorHandler] - <[node2]:
>>>> 5701 [dev] [3.12.4] Removing connection to endpoint [node1]:5701 Cause
>>>> => java.net.SocketException {Connection refused to address node1/xx.xx.
>>>> xx.xx:5701}, Error-Count: 7>
>>>> WARN [com.hazelcast.nio.tcp.TcpIpConnectionErrorHandler] - <[node2]:
>>>> 5701 [dev] [3.12.4] Removing connection to endpoint [node1]:5701 Cause
>>>> => java.net.SocketException {Connection refused to address node1/xx.xx.
>>>> xx.xx:57001}, Error-Count: 8>
>>>>
>>>>
>>>>
>>>> This is my hazelcast settings:
>>>>
>>>> cas.ticket.registry.hazelcast.cluster.members=node1:5701,node2:5701,
>>>> node3:5701,node3:5701
>>>> cas.ticket.registry.hazelcast.cluster.asyncBackupCount=3
>>>> cas.ticket.registry.hazelcast.cluster.port=5701
>>>> cas.ticket.registry.hazelcast.cluster.portAutoIncrement=false
>>>> cas.ticket.registry.hazelcast.cluster.instanceName=localhost
>>>> cas.ticket.registry.hazelcast.cluster.publicAddress=node1:5701
>>>> cas.ticket.registry.hazelcast.cluster.tcpipEnabled=true
>>>>
>>>>
>>>> Why hazelcast doesn't share data across the cluster?
>>>>
>>>> These messages I see on startup
>>>>
>>>> WARN [com.hazelcast.instance.AddressPicker] - <[LOCAL] [dev] [3.12.4]
>>>> You configured your member address as host name. Please be aware of
>>>> that your dns can be spoofed. Make sure that your dns configurations
>>>> are correct.>
>>>> WARN [com.hazelcast.instance.AddressPicker] - <[LOCAL] [dev] [3.12.4]
>>>> You configured your member address as host name. Please be aware of
>>>> that your dns can be spoofed. Make sure that your dns configurations
>>>> are correct.>
>>>> WARN [com.hazelcast.instance.AddressPicker] - <[LOCAL] [dev] [3.12.4]
>>>> You configured your member address as host name. Please be aware of
>>>> that your dns can be spoofed. Make sure that your dns configurations
>>>> are correct.>
>>>> WARN [com.hazelcast.instance.AddressPicker] - <[LOCAL] [dev] [3.12.4]
>>>> You configured your member address as host name. Please be aware of
>>>> that your dns can be spoofed. Make sure that your dns configurations
>>>> are correct.>
>>>> WARN [com.hazelcast.instance.AddressPicker] - <[LOCAL] [dev] [3.12.4]
>>>> Could not find a matching address to start with! Picking one of
>>>> non-loopback
>>>> addresses.>
>>>> INFO [org.apereo.cas.util.CoreTicketUtils] - <Ticket registry
>>>> encryption/signing is turned off. This MAY NOT be safe in a clustered
>>>> production environment. Consider using other choices to handle
>>>> encryption, signing and verification of ticket registry tickets, and
>>>> verify the chosen ticket registry does support this behavior.>
>>>>
>>>>
>>>> On Wednesday, January 22, 2020 at 3:18:34 AM UTC+2, Andy Ng wrote:
>>>>
>>>> Hi Maksim,
>>>>
>>>> Pretty sure:
>>>> cas.ticket.registry.hazelcast.cluster.public-address
>>>> and
>>>> cas.ticket.registry.hazelcast.cluster.publicAddress
>>>>
>>>> Both works the same, since spring property allows both camelCase and
>>>> kebak-case.
>>>>
>>>>
>>>> And I did successfully use docker CAS and use Hazelcast as ticketing
>>>> system, however I am using it for demo so I just included a whole bunch of
>>>> private IP so it works......
>>>>
>>>> here's my CAS properties:
>>>>
>>>>
>>>> cas.ticket.registry.hazelcast.cluster.members=172.20.0.1,172.20.0.2,172.20.0.3,172.20.0.4,172.20.0.5,172.20.0.6,172.20.0.7,172.20.0.8,172.20.0.9,172.20.0.10
>>>> cas.ticket.registry.hazelcast.cluster.instanceName=localhost
>>>> my project link as well so you can reference if want to:
>>>> https://github.com/NgSekLong/SelectUrCAS/blob/master/source/ticket-registry/hazelcast/cas.yml
>>>>
>>>>
>>>> Cheers!
>>>> - Andy
>>>>
>>>> --
>>>>
>>>>
>>>> Ray Bon
>>>> Programmer Analyst
>>>> Development Services, University Systems
>>>> 2507218831 | CLE 019 | [email protected]
>>>>
>>>> I respectfully acknowledge that my place of work is located within the
>>>> ancestral, traditional and unceded territory of the Songhees, Esquimalt
>>>> and
>>>> WSÁNEĆ Nations.
>>>>
>>>> --
>>>>
>>>>
>>>> Ray Bon
>>>> Programmer Analyst
>>>> Development Services, University Systems
>>>> 2507218831 | CLE 019 | [email protected]
>>>>
>>>> I respectfully acknowledge that my place of work is located within the
>>>> ancestral, traditional and unceded territory of the Songhees, Esquimalt
>>>> and
>>>> WSÁNEĆ Nations.
>>>>
>>>> --
>>>>
>>>> Ray Bon
>>>> Programmer Analyst
>>>> Development Services, University Systems
>>>> 2507218831 | CLE 019 | [email protected]
>>>>
>>>> I respectfully acknowledge that my place of work is located within the
>>>> ancestral, traditional and unceded territory of the Songhees, Esquimalt
>>>> and
>>>> WSÁNEĆ Nations.
>>>>
>>>> --
>>>> - Website: https://apereo.github.io/cas
>>>> - Gitter Chatroom: https://gitter.im/apereo/cas
>>>> - List Guidelines: https://goo.gl/1VRrw7
>>>> - Contributions: https://goo.gl/mh7qDG
>>>> ---
>>>> You received this message because you are subscribed to the Google
>>>> Groups "CAS Community" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to [email protected].
>>>> To view this discussion on the web visit
>>>> https://groups.google.com/a/apereo.org/d/msgid/cas-user/cfe74ef4f3aeb0c1dfc506a7257e659418e41f0e.camel%40uvic.ca
>>>>
>>>> <https://groups.google.com/a/apereo.org/d/msgid/cas-user/cfe74ef4f3aeb0c1dfc506a7257e659418e41f0e.camel%40uvic.ca?utm_medium=email&utm_source=footer>
>>>> .
>>>>
>>> --
>> - Website: https://apereo.github.io/cas
>> - Gitter Chatroom: https://gitter.im/apereo/cas
>> - List Guidelines: https://goo.gl/1VRrw7
>> - Contributions: https://goo.gl/mh7qDG
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "CAS Community" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected] <javascript:>.
>> To view this discussion on the web visit
>> https://groups.google.com/a/apereo.org/d/msgid/cas-user/8d82a0af-84cb-4e9e-96e9-7e1e688eba1d%40apereo.org
>>
>> <https://groups.google.com/a/apereo.org/d/msgid/cas-user/8d82a0af-84cb-4e9e-96e9-7e1e688eba1d%40apereo.org?utm_medium=email&utm_source=footer>
>> .
>>
>
--
- Website: https://apereo.github.io/cas
- Gitter Chatroom: https://gitter.im/apereo/cas
- List Guidelines: https://goo.gl/1VRrw7
- Contributions: https://goo.gl/mh7qDG
---
You received this message because you are subscribed to the Google Groups "CAS
Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/a/apereo.org/d/msgid/cas-user/ca1a66d0-d8c3-41e3-b2ba-2a5d35ca4c3b%40apereo.org.
