Hi Aldo, No, its not a bug; Its an issue with data representation as defined in the script that processes responses;
check more on forum discussions involving charset and coding; You'll get your answer there. Saludos Ombongi Moraa fe On 15 February 2013 03:20, <users-requ...@kannel.org> wrote: > Send users mailing list submissions to > users@kannel.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://www.kannel.org/mailman/listinfo/users > or, via email, send a message with subject or body 'help' to > users-requ...@kannel.org > > You can reach the person managing the list at > users-ow...@kannel.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of users digest..." > > > Today's Topics: > > 1. Re: Issue with bearerbox (Rinor Hoxha) > 2. Re: Issue with bearerbox (spameden) > 3. RE: smsbox: Failed to convert msgdata from charset <UTF-8> to > <UTF-8> (Aldo Zavala) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Thu, 14 Feb 2013 18:19:30 +0100 > From: Rinor Hoxha <rinorho...@gmail.com> > To: Willy Mularto <sangpr...@gmail.com> > Cc: "users@kannel.org" <users@kannel.org> > Subject: Re: Issue with bearerbox > Message-ID: > <CAFvba4i1E10EJC= > vxabygwxxo9fi5exwd3r9gkyr8axdww4...@mail.gmail.com> > Content-Type: text/plain; charset="windows-1252" > > Sure Willy, > I'll post them in one hour, the time to remove the sensitive information, > but I can tell you in advance there is nothing fancy on them. > > One scenario is the following: > 4xSMSbox -> 1xSQLbox -> 1xBEARERbox => 10xConnections to 4xMNOs. > (7xSMPPxTRANSCEIVER + 3 custom http connections to interface some smsc > proxy that is not included on default kannel smscs) > > 1 SMSbox handles services for 1 MNO. > Don't remember why I did that, probably because of the retry delay queue > issue on smsbox, but not really sure. > > Each SMSbox has one specific config file that includes a common file, for > all SMSboxes, that contains the users and the services. > > Kannel rev.5014 > Kannel code has a small modification to activate smpp relative_time_format > for validity (since some of MNOs refused to go with absolute_time_format). > I agree with the list, in some previous debates, that relative time should > not be used, but when you have to, kannel is there for you :) > relative-time-format = 1 is the customized directive for this. > > Sqlbox rev.49 > Sqlbox has a small modification in mysql to insert MO,MT,DLR in separate > TABLES - that could be done also with triggers, but on high load I didn't > wanted triggers so thought it was better to put it inside the code. > sql-log-table-split = 1 is the customized directive for this. > > This is in one production system, however the tests have been done also > with unmodified versions of those revisions. (keep in mind that the small > changes made to the code have nothing to do with the "mentioned issue"). > Test were done both on debian and windows. > > I'll provide more info later if needed/interested. > > Br, Rinor > > > On Thu, Feb 14, 2013 at 2:11 AM, Willy Mularto <sangpr...@gmail.com> > wrote: > > > May we know your Kannel and SQLBox version also please share your Kannel > > scenario configuration, including SQLBox position > (Bearerbox-SQLBox-SMSBox > > or Bearbox-SMSBox-SQLBox or other?) > > > > > > > > Willy Mularto > > sangpr...@gmail.com > > > > On Feb 14, 2013, at 12:20 AM, Rinor Hoxha <rinorho...@gmail.com> wrote: > > > > I was concerned in the first time I read about this "issue", > > since I'm using sqlbox in different scenarios, with different load (1-20+ > > sms/sec) with at least 10 smsc connections (smpp and http), with 4 > > smsboxes, with delivery tracking, with/without sms services, > with/without > > opensmppbox > > and the only times that sqlbox "failed" was because sql server reaching > > max connections (on very high load and bad programmed services) or > services > > script issues or the MNO never sent dlr report or other issues not > related > > to sqlbox itself. > > I tried myself on lab to make sqlbox "loose" sms, but it never happened. > I > > went further and compiled a cygwin version on windows and it never > > happened. The last test was done on windows with rev.5014 with variable > > rate 6-30 sms with approx 300.000 sms with dlr=3. And everything was > there. > > No matter if one or more than one smsboxes, routing works fine. > > Yes, it may be some room for improvement on sqlbox (as with any > software), > > but not sure if it has a "bug" causing missing sms. > > I may be wrong, but that my 2 cents, since I see this "issue" coming up > > from time to time. If others think that there is really a bug, lets help > > each other and the programmer to fix it. > > > > Br, Rinor > > > > > > > > On Tue, Feb 12, 2013 at 8:14 PM, Rene Kluwen <rene.klu...@chimit.nl > >wrote: > > > >> Please indicate if you are using sqlbox or not.**** > >> > >> I?ve tried to debug this problem in the past, but I could not reproduce > >> this problem.**** > >> > >> ** ** > >> > >> ** ** > >> > >> *From:* Willy Mularto [mailto:sangpr...@gmail.com] > >> *Sent:* donderdag 7 februari 2013 14:05 > >> *To:* michael osakede > >> *Cc:* Rene Kluwen; kannel users@kannel.org > >> *Subject:* Re: Issue with bearerbox**** > >> > >> ** ** > >> > >> > >> Do you use current version with SQLBox configured to service MO? If yes > >> then it could be thee old bug still there. **** > >> > >> SlyDroid**** > >> > >> On Feb 7, 2013 7:44 PM, "michael osakede" <ire...@yahoo.com> wrote:**** > >> > >> Hi,**** > >> > >> ** ** > >> > >> I have setup an sms service**** > >> > >> ** ** > >> > >> group = sms-service**** > >> > >> keyword = default**** > >> > >> get-url = "http://localhost/insms/insms.php?sender=%p&text=%b&to=%P > "**** > >> > >> accept-x-kannel-headers = true**** > >> > >> max-messages = 3**** > >> > >> concatenation = true**** > >> > >> catch-all = true**** > >> > >> accepted-smsc = "xxx;xxx1;xxx2;xxx3"**** > >> > >> omit-empty = true**** > >> > >> ** ** > >> > >> and corresponding SMPP connction with smsc-ids.**** > >> > >> ** ** > >> > >> I noticed that only about 40% of messages are routed to the sms service, > >> any idea why this is a problem? and is there anyway to check and see > what > >> happened?**** > >> > >> ** ** > >> > >> ** ** > >> > >> Michael**** > >> > >> ** ** > >> > > > > > > > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: < > http://www.kannel.org/pipermail/users/attachments/20130214/848bac86/attachment-0001.html > > > > ------------------------------ > > Message: 2 > Date: Fri, 15 Feb 2013 03:28:10 +0400 > From: spameden <spame...@gmail.com> > To: Rinor Hoxha <rinorho...@gmail.com> > Cc: "users@kannel.org" <users@kannel.org> > Subject: Re: Issue with bearerbox > Message-ID: > < > cahcalewjhppxdacwc+fqfeyj1f8noyn4eq8uwv48qwec5sf...@mail.gmail.com> > Content-Type: text/plain; charset=windows-1252 > > Hey Rinor. > > Your idea about splitting MO, MT, DLR in separate tables is just great! > > How do you do matching against MT's btw? Do you invoke everytime > script or just update table via unique dlr_url id in the MT table? > > Would love to see your patch against SQLBox. > > I'm also wondering if you solved the issue of DLR missing when DB goes > down. > > Thanks. > > 2013/2/14 Rinor Hoxha <rinorho...@gmail.com>: > > Sure Willy, > > I'll post them in one hour, the time to remove the sensitive information, > > but I can tell you in advance there is nothing fancy on them. > > > > One scenario is the following: > > 4xSMSbox -> 1xSQLbox -> 1xBEARERbox => 10xConnections to 4xMNOs. > > (7xSMPPxTRANSCEIVER + 3 custom http connections to interface some smsc > proxy > > that is not included on default kannel smscs) > > > > 1 SMSbox handles services for 1 MNO. > > Don't remember why I did that, probably because of the retry delay queue > > issue on smsbox, but not really sure. > > > > Each SMSbox has one specific config file that includes a common file, for > > all SMSboxes, that contains the users and the services. > > > > Kannel rev.5014 > > Kannel code has a small modification to activate smpp > relative_time_format > > for validity (since some of MNOs refused to go with > absolute_time_format). I > > agree with the list, in some previous debates, that relative time should > not > > be used, but when you have to, kannel is there for you :) > > relative-time-format = 1 is the customized directive for this. > > > > Sqlbox rev.49 > > Sqlbox has a small modification in mysql to insert MO,MT,DLR in separate > > TABLES - that could be done also with triggers, but on high load I didn't > > wanted triggers so thought it was better to put it inside the code. > > sql-log-table-split = 1 is the customized directive for this. > > > > This is in one production system, however the tests have been done also > with > > unmodified versions of those revisions. (keep in mind that the small > changes > > made to the code have nothing to do with the "mentioned issue"). Test > were > > done both on debian and windows. > > > > I'll provide more info later if needed/interested. > > > > Br, Rinor > > > > > > On Thu, Feb 14, 2013 at 2:11 AM, Willy Mularto <sangpr...@gmail.com> > wrote: > >> > >> May we know your Kannel and SQLBox version also please share your Kannel > >> scenario configuration, including SQLBox position > (Bearerbox-SQLBox-SMSBox > >> or Bearbox-SMSBox-SQLBox or other?) > >> > >> > >> > >> Willy Mularto > >> sangpr...@gmail.com > >> > >> On Feb 14, 2013, at 12:20 AM, Rinor Hoxha <rinorho...@gmail.com> wrote: > >> > >> I was concerned in the first time I read about this "issue", > >> since I'm using sqlbox in different scenarios, with different load > (1-20+ > >> sms/sec) with at least 10 smsc connections (smpp and http), with 4 > smsboxes, > >> with delivery tracking, with/without sms services, with/without > opensmppbox > >> and the only times that sqlbox "failed" was because sql server reaching > >> max connections (on very high load and bad programmed services) or > services > >> script issues or the MNO never sent dlr report or other issues not > related > >> to sqlbox itself. > >> I tried myself on lab to make sqlbox "loose" sms, but it never > happened. I > >> went further and compiled a cygwin version on windows and it never > happened. > >> The last test was done on windows with rev.5014 with variable rate 6-30 > sms > >> with approx 300.000 sms with dlr=3. And everything was there. No matter > if > >> one or more than one smsboxes, routing works fine. > >> Yes, it may be some room for improvement on sqlbox (as with any > software), > >> but not sure if it has a "bug" causing missing sms. > >> I may be wrong, but that my 2 cents, since I see this "issue" coming up > >> from time to time. If others think that there is really a bug, lets help > >> each other and the programmer to fix it. > >> > >> Br, Rinor > >> > >> > >> > >> On Tue, Feb 12, 2013 at 8:14 PM, Rene Kluwen <rene.klu...@chimit.nl> > >> wrote: > >>> > >>> Please indicate if you are using sqlbox or not. > >>> > >>> I?ve tried to debug this problem in the past, but I could not reproduce > >>> this problem. > >>> > >>> > >>> > >>> > >>> > >>> From: Willy Mularto [mailto:sangpr...@gmail.com] > >>> Sent: donderdag 7 februari 2013 14:05 > >>> To: michael osakede > >>> Cc: Rene Kluwen; kannel users@kannel.org > >>> Subject: Re: Issue with bearerbox > >>> > >>> > >>> > >>> > >>> Do you use current version with SQLBox configured to service MO? If yes > >>> then it could be thee old bug still there. > >>> > >>> SlyDroid > >>> > >>> On Feb 7, 2013 7:44 PM, "michael osakede" <ire...@yahoo.com> wrote: > >>> > >>> Hi, > >>> > >>> > >>> > >>> I have setup an sms service > >>> > >>> > >>> > >>> group = sms-service > >>> > >>> keyword = default > >>> > >>> get-url = "http://localhost/insms/insms.php?sender=%p&text=%b&to=%P"; > >>> > >>> accept-x-kannel-headers = true > >>> > >>> max-messages = 3 > >>> > >>> concatenation = true > >>> > >>> catch-all = true > >>> > >>> accepted-smsc = "xxx;xxx1;xxx2;xxx3" > >>> > >>> omit-empty = true > >>> > >>> > >>> > >>> and corresponding SMPP connction with smsc-ids. > >>> > >>> > >>> > >>> I noticed that only about 40% of messages are routed to the sms > service, > >>> any idea why this is a problem? and is there anyway to check and see > what > >>> happened? > >>> > >>> > >>> > >>> > >>> > >>> Michael > >>> > >>> > >> > >> > >> > > > > > > ------------------------------ > > Message: 3 > Date: Fri, 15 Feb 2013 00:19:42 +0000 > From: Aldo Zavala <a...@flatwireless.com> > To: Juan Luis G?mez Correa <juanluis.go...@blipblip.com.co> > Cc: "users@kannel.org" <users@kannel.org> > Subject: RE: smsbox: Failed to convert msgdata from charset <UTF-8> to > <UTF-8> > Message-ID: > <f55928c6562c5f458cee0144fb2cfebd825c0...@flt-amaclu-ex02.ct.local > > > Content-Type: text/plain; charset="iso-8859-1" > > Its fine, my system is working like a charm now and you are right about > the ""<UTF-8> to <UTF-8>" errors, I can ignore them for now... > > Sorry for my late response, I had two surgeries recently. > > > > Aldo Zavala > Network Operations > ________________________________ > From: Juan Luis G?mez Correa [juanluis.go...@blipblip.com.co] > Sent: Saturday, December 29, 2012 8:08 AM > To: Aldo Zavala > Cc: users@kannel.org > Subject: RE: smsbox: Failed to convert msgdata from charset <UTF-8> to > <UTF-8> > > > Sorry Aldo, no clue with that, what is the output of the service when you > make a manual request? > > Anyway, I think you can safely ignore that error. > > Regards, > > Juan L > > On Dec 28, 2012 11:07 PM, "Aldo Zavala" <a...@flatwireless.com<mailto: > a...@flatwireless.com>> wrote: > Sure, this is the smsbox.log output of a service SMS. > > 2012-12-28 20:20:05 [46897] [2] DEBUG: HTTP: Creating HTTPClient for > `192.168.8.188'. > 2012-12-28 20:20:05 [46897] [2] DEBUG: HTTP: Created HTTPClient area > 0x28b6b070. > 2012-12-28 20:20:05 [46897] [3] INFO: smsbox: Got HTTP request > </cgi-bin/sendsms> from <192.168.8.188> > 2012-12-28 20:20:05 [46897] [3] INFO: sendsms used by <aldo_user107> > 2012-12-28 20:20:05 [46897] [3] INFO: sendsms sender:<aldo_user107:2265> > (192.168.8.188) to:<5038845319> msg:<This is a TEST message.> > 2012-12-28 20:20:05 [46897] [3] DEBUG: Stored UUID > 1becd6dd-0d47-465d-8cc7-7e1e0e149c5b > 2012-12-28 20:20:05 [46897] [3] DEBUG: message length 81, sending 1 > messages > 2012-12-28 20:20:05 [46897] [3] DEBUG: Status: 202 Answer: <Sent.> > 2012-12-28 20:20:05 [46897] [3] DEBUG: Delayed reply - wait for bearerbox > 2012-12-28 20:20:05 [46897] [0] DEBUG: Got ACK (0) of > 1becd6dd-0d47-465d-8cc7-7e1e0e149c5b > 2012-12-28 20:20:05 [46897] [0] DEBUG: HTTP: Resetting HTTPClient for > `192.168.8.188'. > 2012-12-28 20:20:05 [46897] [1] DEBUG: HTTP: Destroying HTTPClient area > 0x28b6b070. > 2012-12-28 20:20:05 [46897] [1] DEBUG: HTTP: Destroying HTTPClient for > `192.168.8.188'. > 2012-12-28 20:20:05 [46897] [8] DEBUG: HTTP: Status line: <HTTP/1.1 200 OK> > 2012-12-28 20:20:05 [46897] [8] DEBUG: HTTP: Received response: > 2012-12-28 20:20:05 [46897] [8] DEBUG: Octet string at 0x28ac19f0: > 2012-12-28 20:20:05 [46897] [8] DEBUG: len: 148 > 2012-12-28 20:20:05 [46897] [8] DEBUG: size: 1024 > 2012-12-28 20:20:05 [46897] [8] DEBUG: immutable: 0 > 2012-12-28 20:20:05 [46897] [8] DEBUG: data: 58 2d 50 6f 77 65 72 65 64 > 2d 42 79 3a 20 50 48 X-Powered-By: PH > 2012-12-28 20:20:05 [46897] [8] DEBUG: data: 50 2f 35 2e 34 2e 36 0d 0a > 43 6f 6e 74 65 6e 74 P/5.4.6.<http://5.4.6.>.Content > 2012-12-28 20:20:05 [46897] [8] DEBUG: data: 2d 74 79 70 65 3a 20 74 65 > 78 74 2f 68 74 6d 6c -type: text/html > 2012-12-28 20:20:05 [46897] [8] DEBUG: data: 3b 20 63 68 61 72 73 65 74 > 3d 55 54 46 2d 38 0d ; charset=UTF-8. > 2012-12-28 20:20:05 [46897] [8] DEBUG: data: 0a 43 6f 6e 74 65 6e 74 2d > 4c 65 6e 67 74 68 3a .Content-Length: > 2012-12-28 20:20:05 [46897] [8] DEBUG: data: 20 30 0d 0a 44 61 74 65 3a > 20 53 61 74 2c 20 32 0..Date: Sat, 2 > 2012-12-28 20:20:05 [46897] [8] DEBUG: data: 39 20 44 65 63 20 32 30 31 > 32 20 30 34 3a 32 30 9 Dec 2012 04:20 > 2012-12-28 20:20:05 [46897] [8] DEBUG: data: 3a 31 37 20 47 4d 54 0d 0a > 53 65 72 76 65 72 3a :17 GMT..Server: > 2012-12-28 20:20:05 [46897] [8] DEBUG: data: 20 6c 69 67 68 74 74 70 64 > 2f 31 2e 34 2e 33 31 lighttpd/1.4.31 > 2012-12-28 20:20:05 [46897] [8] DEBUG: data: 0d 0a 0d 0a > .... > 2012-12-28 20:20:05 [46897] [8] DEBUG: Octet string dump ends. > 2012-12-28 20:20:05 [46897] [5] ERROR: Failed to convert msgdata from > charset <UTF-8> to <UTF-8>, will leave as is. > 2012-12-28 20:20:05 [46897] [5] INFO: No reply sent, denied. > 2012-12-28 20:20:11 [46897] [8] DEBUG: HTTP: Server closed connection, > destroying it <192.168.8.188:80:0::><0x28af4e80><fd:28>. > > > Aldo Zavala > ________________________________________ > From: Juan Luis G?mez Correa [juanluis.go...@blipblip.com.co<mailto: > juanluis.go...@blipblip.com.co>] > Sent: Friday, December 28, 2012 7:57 PM > To: Aldo Zavala > Cc: users@kannel.org<mailto:users@kannel.org> > Subject: Re: smsbox: Failed to convert msgdata from charset <UTF-8> to > <UTF-8> > > Glad that helped! > > Sadly I don't have an explanation for the error you are seeing, can > you paste a typical output from your service? Is it possible for your > service to not output any answer and see if that fixes your problem? > > Regards, > > Juan L > > On 28/12/2012, at 22:37, Aldo Zavala <a...@flatwireless.com<mailto: > a...@flatwireless.com>> wrote: > > > I used to have this: > > max-messages = 2 > > omit-empty = true > > > > And now after changed it to max-messages = 0 it finally made the trick > you are right... I failed to read again the docs... I just did't consider > that it's necessarily to read it again after upgrade my opensmpp, > bearerbox, sqlbox to latest SVN... but considering that many things can > change from one *old* version to latest... I should do it... again thanks! > > > > By the way the smsbox log line: "ERROR: Failed to convert msgdata from > charset <UTF-8> to <UTF-8>, will leave as is." is still appearing.. even > when I setup all my php.ini and inetrnal variables to use UTF-8, also I put > this on kannel: "alt-charset = UTF-8" > > > > Any idea what causes that? > > > > Thanks again! > > > > > > Aldo Zavala > > > > ________________________________________ > > From: Juan Luis G?mez Correa [juanluis.go...@blipblip.com.co<mailto: > juanluis.go...@blipblip.com.co>] > > Sent: Friday, December 28, 2012 7:00 PM > > To: Aldo Zavala; users@kannel.org<mailto:users@kannel.org> > > Subject: Re: smsbox: Failed to convert msgdata from charset <UTF-8> to > <UTF-8> > > > > Hi Aldo, as you can see from the docs: > > > > "This service grabs any SMS with two words and 'www' as the first > > word, and then does an HTTP request to an URL which is taken from > > the rest of the message. Any result is sent back to the phone > > (or requester), but is truncated to the 160 characters that will > > fit into an SMS message, naturally." > > > > So, the problem you're facing is that kannel is trying to "parse" your > > service answer as an SMS, and assuming it's an html, then kannel can't > > convert this answer to an SMS hence the error SMS you're receiving. > > > > To avoid this problem your service must not output any answer, then add > > this variables to the sms-service group: > > > > max-messages = 0 > > > > omit-empty = true > > > > > > > > Hope this helps! > > > > > > Juan L > > > > > > On 12/28/12 7:57 PM, "Aldo Zavala" <a...@flatwireless.com<mailto: > a...@flatwireless.com>> wrote: > > > >> am I the only one that has faced this situation...? come on... > >> > >> thanks guys! > >> > >> > >> Aldo Zavala > >> ________________________________________ > >> From: Aldo Zavala > >> Sent: Monday, December 17, 2012 11:25 AM > >> To: users@kannel.org<mailto:users@kannel.org> > >> Subject: smsbox: Failed to convert msgdata from charset <UTF-8> to > <UTF-8> > >> > >> Hi, I am constantly getting this message: "Failed to convert msgdata > from > >> charset <UTF-8> to <UTF-8>" from the smsbox.log, it happens after > execute > >> a sms service, the process flow works as bearerbox executes the remote > >> server script, the remote server process the information and request a > >> sms submission back to bearerbox and then smsbox generated these log > >> files and it sends the service SMS plus an annoying message that says: > >> "Result cannot be represented as an SMS message" every time. Could this > >> be a bug? > >> > >> Thanks in advance! > >> > >> Aldo Zavala > > > > > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: < > http://www.kannel.org/pipermail/users/attachments/20130215/653ffd02/attachment.html > > > > ------------------------------ > > _______________________________________________ > users mailing list > users@kannel.org > http://www.kannel.org/mailman/listinfo/users > > > End of users Digest, Vol 78, Issue 28 > ************************************* >
