[Mosquitto-users] Bridging issues

[Frisch, Michael]

I'm trying to configure bridging in a master-slave type configuration where the 
slaves connection to the masters and subscribe to the topics in the IN 
direction only.  I was having issues with 0.15 but seeing as the release of 1.0 
is soon I decided to check out the branch and build it.  I seem to be having 
two different issues at the moment.

First issue is if I use a username/password for the bridge.  I see that the 
slave connects, connects to the three topics that I want it listening to, and 
then a socket error.
Slave output:
1343748661: Connecting bridge mosquitto_master
1343748661: Bridge test-us-east-mqtt-40e9eb38.swypeconnect.com.mosquitto_master 
sending CONNECT
1343748661: Received CONNACK on connection 
test-us-east-mqtt-40e9eb38.swypeconnect.com.mosquitto_master.
1343748661: Bridge test-us-east-mqtt-40e9eb38.swypeconnect.com.mosquitto_master 
sending SUBSCRIBE (Mid: 154, Topic: a/#, QoS: 0)
1343748661: Bridge test-us-east-mqtt-40e9eb38.swypeconnect.com.mosquitto_master 
sending SUBSCRIBE (Mid: 155, Topic: b/#, QoS: 0)
1343748661: Bridge test-us-east-mqtt-40e9eb38.swypeconnect.com.mosquitto_master 
sending SUBSCRIBE (Mid: 156, Topic: d/#, QoS: 0)
1343748661: Received PUBREC from 
test-us-east-mqtt-40e9eb38.swypeconnect.com.mosquitto_master (Mid: 11)
1343748661: Socket read error on client 
test-us-east-mqtt-40e9eb38.swypeconnect.com.mosquitto_master, disconnecting.
Master output:
1343748692: New connection from 127.0.0.1.
1343748692: New client connected from 127.0.0.1 as 
test-us-east-mqtt-40e9eb38.swypeconnect.com.mosquitto_master.
1343748692: Sending CONNACK to 
test-us-east-mqtt-40e9eb38.swypeconnect.com.mosquitto_master (0)
1343748692: Sending PUBREC to 
test-us-east-mqtt-40e9eb38.swypeconnect.com.mosquitto_master (Mid: 11)
1343748692: Sending PUBREC to 
test-us-east-mqtt-40e9eb38.swypeconnect.com.mosquitto_master (Mid: 12)
1343748692: Sending PUBREC to 
test-us-east-mqtt-40e9eb38.swypeconnect.com.mosquitto_master (Mid: 13)
1343748692: Sending PUBREC to 
test-us-east-mqtt-40e9eb38.swypeconnect.com.mosquitto_master (Mid: 14)
1343748692: Sending PUBREC to 
test-us-east-mqtt-40e9eb38.swypeconnect.com.mosquitto_master (Mid: 15)
1343748692: Sending PUBREC to 
test-us-east-mqtt-40e9eb38.swypeconnect.com.mosquitto_master (Mid: 16)
1343748692: Sending PUBREC to 
test-us-east-mqtt-40e9eb38.swypeconnect.com.mosquitto_master (Mid: 17)
1343748692: Sending PUBREC to 
test-us-east-mqtt-40e9eb38.swypeconnect.com.mosquitto_master (Mid: 18)
1343748692: Received PUBLISH from 
test-us-east-mqtt-40e9eb38.swypeconnect.com.mosquitto_master (d0, q1, r1, m157, 
'$SYS/broker/connection/test-us-east-mqtt-40e9eb38.swypeconnect.com.mosquitto_master/state',
 ... (2 bytes))
1343748692: Sending PUBACK to 
test-us-east-mqtt-40e9eb38.swypeconnect.com.mosquitto_master (Mid: 157)
1343748692: Socket error on client 
test-us-east-mqtt-40e9eb38.swypeconnect.com.mosquitto_master, disconnecting.

The second issue presents itself if I don't have a username/password set for 
the bridge, I have an ACL still set, but anonymous users can read from the 
topics I wish to subscribe to.  I can successfully bridge the slave to the 
master and messages that are published to the master are in turn published to 
the slave but the slave does not publish those messages to other clients 
listening to those messages.  If I publish directly to the slave the clients do 
get the messages that they are subscribed to.

I'm using CentOS 6.2 on both slave and master.
-- 
Mailing list: https://launchpad.net/~mosquitto-users
Post to : mosquitto-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~mosquitto-users
More help   : https://help.launchpad.net/ListHelp

Re: [Mosquitto-users] Bridging issues

[Frisch, Michael]

Roger,

That worked.

Maybe send a patch file that I could apply locally and test.  If nothing else 
it'll point me at where in the code base to try making a general fix.

Thanks,
Michael

From: mosquitto-users-bounces+michael.frisch=nuance@lists.launchpad.net 
[mosquitto-users-bounces+michael.frisch=nuance@lists.launchpad.net] on 
behalf of Roger Light [ro...@atchoo.org]
Sent: Tuesday, July 31, 2012 6:56 PM
To: mosquitto-users@lists.launchpad.net
Subject: Re: [Mosquitto-users] Bridging issues

Hi Michael,

> First issue is if I use a username/password for the bridge.  I see that the
> slave connects, connects to the three topics that I want it listening to,
> and then a socket error.
>
> Slave output:

...

> 1343748661: Received PUBREC from
> test-us-east-mqtt-40e9eb38.swypeconnect.com.mosquitto_master (Mid: 11)

I'm a bit concerned as to where this is coming from and the same on
the master side. I wonder if the problem is related to undelivered
messages stored on the master. Could you try putting your mosquitto.db
to one side on the master and the slave and restarting them?

If this works it only fixes the symptom not the problem, but it's
useful to know what's going on.

> The second issue presents itself if I don’t have a username/password set for
> the bridge, I have an ACL still set, but anonymous users can read from the
> topics I wish to subscribe to.  I can successfully bridge the slave to the
> master and messages that are published to the master are in turn published
> to the slave but the slave does not publish those messages to other clients
> listening to those messages.  If I publish directly to the slave the clients
> do get the messages that they are subscribed to.

This is down to ACLs being applied to local bridges but having no
mechanism to set a local username/password as well as a remote
username/password. I did fix it so that local bridges aren't
constrained by local ACLs but it looks like that change has gone
astray in some code rearrangement. I've put the fix back in if you
want to give it a try.

Cheers,

Roger

--
Mailing list: https://launchpad.net/~mosquitto-users
Post to : mosquitto-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~mosquitto-users
More help   : https://help.launchpad.net/ListHelp
-- 
Mailing list: https://launchpad.net/~mosquitto-users
Post to : mosquitto-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~mosquitto-users
More help   : https://help.launchpad.net/ListHelp

Re: [Mosquitto-users] Build up a MQTT server with about 20, 000+ subscribers

[Frisch, Michael]

Hello all,

I’ll answer Joe’s questions based on my experience:

-You’ll hit kernel limits pretty early on but many of those are easy to 
modify with ulimit and sysctl, those will be hurdles before Mosquitto is

-There’s not much special, EC2 mediums can do 200k+ (yes, two hundred) 
with QoS 2, SSL off, and a moderate number of messages publishing constantly.

-You can have a client that listens to $SYS/# or specific subtopics; a 
UI is a totally different matter though

-Not out of the box

Sharon, the problem that I can already tell you that will arise with SSL is 
memory usage.  I forget the exact numbers but calling the consumption 
noticeable would be an understatement.  With SSL off Mosquitto uses such a 
small amount of RAM which is why EC2 high CPU mediums are perfect.


-Mike

From: mosquitto-users-bounces+michael.frisch=nuance@lists.launchpad.net 
[mailto:mosquitto-users-bounces+michael.frisch=nuance@lists.launchpad.net] 
On Behalf Of Sharon Ben-Asher
Sent: Wednesday, November 14, 2012 5:17 AM
To: Joe Zhu; mosquitto-users@lists.launchpad.net
Subject: Re: [Mosquitto-users] Build up a MQTT server with about 20, 000+ 
subscribers

Hi Joe,

We are investigating MQTT and mosquitto for the same purpose (push 
notifications for Android)
As I said earlier. I was able to achieve 100K concurrent connections to one 
broker installed on medium sized AWS EC2 instance.
However, for the purpose of benchmark, I used the most basic scenario: QoS 0 
only, no persistence, no SSL/encryption.
I imagine that if you add those features, the number will go down.
I will test with SSL connection in the upcoming days and will report my 
findings.
I also had to tweak the OS configuration (Ubuntu 12.04) to support such high 
number of TCP connections.  If needed, I will share the settings I used.
I also found a working way to connect from Eclipse Paho to mosquitto using SSL 
and server and client authentication.  I will publish this as git gist shortly.

For any more info, will be glad to help.

Sharon.

From: 
mosquitto-users-bounces+sharon.ben-asher=avg@lists.launchpad.net
 
[mailto:mosquitto-users-bounces+sharon.ben-asher=avg@lists.launchpad.net]
 On Behalf Of Joe Zhu
Sent: Monday, November 12, 2012 6:18 PM
To: 
mosquitto-users@lists.launchpad.net
Subject: [Mosquitto-users] Build up a MQTT server with about 20, 000+ 
subscribers

Our project need to setup and Android push notification service. I noticed many 
of projects are using MQTT protocol and I found this mosquitto.

  *   Any people has this experience for this? Will it be a problem while the 
subscribers grow up？
  *   What configuration do I need to handle for 20,000+ subscribers with QoS 
>1?
  *   Any dashboard and UI for mosquitto to monitor?
  *   Is it possible to integrate DB/Memcache/Mongodb for message tracking?
I'm not sure if it is a right place to ask this type of question in this 
mailing list,  if not, sorry about it.

Thanks.


--
Joe Zhu
-- 
Mailing list: https://launchpad.net/~mosquitto-users
Post to : mosquitto-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~mosquitto-users
More help   : https://help.launchpad.net/ListHelp

Re: [Mosquitto-users] Build up a MQTT server with about 20, 000+ subscribers

[Frisch, Michael]

My test involved 200k clients subscribed to 1 of 10 topics and publishing on 
the random interval of 0 to 1800 seconds to one of the 10 topics.  Everything 
was done at QoS 2, SSL off.

An easy workaround that we've taken is to encrypt the payload over the 
non-secure connection.  The burden is then placed on the message 
producing/consuming software but that's something that just has to be dealt 
with when maintaining so many open connections.

- Mike

-Original Message-
From: mosquitto-users-bounces+michael.frisch=nuance@lists.launchpad.net 
[mailto:mosquitto-users-bounces+michael.frisch=nuance@lists.launchpad.net] 
On Behalf Of Roger Light
Sent: Wednesday, November 14, 2012 12:36 PM
To: mosquitto-users@lists.launchpad.net
Subject: Re: [Mosquitto-users] Build up a MQTT server with about 20, 000+ 
subscribers

Hi Mike,

> -There’s not much special, EC2 mediums can do 200k+ (yes, two
> hundred) with QoS 2, SSL off, and a moderate number of messages 
> publishing constantly.

Can you offer any further details on what "moderate" means here? Just a rough 
order of magnitude would be nice to know.

> Sharon, the problem that I can already tell you that will arise with 
> SSL is memory usage.  I forget the exact numbers but calling the 
> consumption noticeable would be an understatement.  With SSL off 
> Mosquitto uses such a small amount of RAM which is why EC2 high CPU mediums 
> are perfect.

Yes... It would be nice to try something less heavyweight than openssl, but the 
nice obvious alternatives like cyassl are gpl/commercial licensed.

Cheers,

Roger

--
Mailing list: https://launchpad.net/~mosquitto-users
Post to : mosquitto-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~mosquitto-users
More help   : https://help.launchpad.net/ListHelp
-- 
Mailing list: https://launchpad.net/~mosquitto-users
Post to : mosquitto-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~mosquitto-users
More help   : https://help.launchpad.net/ListHelp

Re: [Mosquitto-users] SSL memory usage

[Frisch, Michael]

That's a tremendous improvement over SSL compression enabled, 
SSL_MODE_RELEASE_BUFFERS disabled and a very welcome change.

- Mike

-Original Message-
From: mosquitto-users-bounces+michael.frisch=nuance@lists.launchpad.net 
[mailto:mosquitto-users-bounces+michael.frisch=nuance@lists.launchpad.net] 
On Behalf Of Roger Light
Sent: Monday, November 26, 2012 12:31 PM
To: mosquitto-users@lists.launchpad.net
Subject: [Mosquitto-users] SSL memory usage

Hi,

I've just done some crude testing with massif (the valgrind heap profiling 
tool) and mosquitto in various configurations. I'm connecting 1000 
mosquitto_sub clients, all subscribing to $SYS/#. I'm not controlling any 
timing of when things happen, so there will be some small differences across 
tests.

Test 1: Mosquitto 1.0.5 without SSL. Peak memory usage hit 2.871MB.
Test 2: 1.0.5 with SSL, no client certificates. Peak memory of 617.3MB.
Test 3: Modified 1.0.5 with SSL compression disabled. Peak memory 41.93MB.
Test 4: Modified test 3 with SSL_MODE_RELEASE_BUFFERS enabled as well.
Peak memory of 11.49MB.

Quite an improvement I think you'll agree.

I'm planning on disabling SSL compression in version 1.1, with no option for 
enabling it. It makes a huge difference to memory usage and also mitigates 
against possible CRIME like attacks:
http://arstechnica.com/security/2012/09/many-ways-to-break-ssl-with-crime-attacks-experts-warn/

I haven't found any background on the possible downsides to using 
SSL_MODE_RELEASE_BUFFERS so it is difficult to say but it seems like a good 
candidate for inclusion.

Cheers,

Roger

--
Mailing list: https://launchpad.net/~mosquitto-users
Post to : mosquitto-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~mosquitto-users
More help   : https://help.launchpad.net/ListHelp

-- 
Mailing list: https://launchpad.net/~mosquitto-users
Post to : mosquitto-users@lists.launchpad.net
Unsubscribe : https://launchpad.net/~mosquitto-users
More help   : https://help.launchpad.net/ListHelp