There’s a few times gwthread_sleep(1.0) in the code. As a test, you can try to change those to 2.0, one by one.
And if your message throughput drops to 0.5 messages per second, then you know where to look to improve it. == Rene From: users [mailto:users-boun...@kannel.org] On Behalf Of Michael Epstein Sent: maandag 27 april 2015 20:45 To: Beck, Stuart (ADE-MNT); users@kannel.org Subject: Re: getting consistent 1 msg/sec sending with Kannel + opensmpp box Hi Beck, We switch to HTTP the problem is definitely related to opensmppbox. I would just suggest switching and maybe using multiple SMSBOXs if you have too (we were able to switch to HTTP with only SMSBOX). Regards, Mike On 4/27/15 12:28 AM, Beck, Stuart (ADE-MNT) wrote: Hi List, We are running into a similar issue where opensmppbox does not seem to deliver much more than 1 msg / sec In our case we are wanting to avoid having a third party application handling traffic so have multiple opensmppbox applications setup for each of our individual carriers, each with their own configuration file and routing entries. I've tried the suggestions listed here for Michael, but have had a similar experience (i.e. no change) In our case, we have also found messages queueing both in SMS store and in the smsbox connection, once messages become queued, they rarely get processed, a restart of the processes may cause one or two to get processed through. Unfortunately while I can confirm all the queued SMS messages are our originating MO's, I do not know how to view messages queued in the opensmppbox to see what these are or what state they may be in. Kannel bearerbox version `1.4.4'. Build `Mar 23 2015 15:48:36', compiler `4.5.2'. System SunOS, release 5.11, version 11.1, machine i86pc. Hostname xxxxx, IP xxxxx. Libxml version 2.7.6. Using OpenSSL 1.0.0j 10 May 2012. Compiled with PostgreSQL 9.2.1. Using native malloc. Status: running, uptime 4d 1h 44m 23s WDP: received 0 (0 queued), sent 0 (0 queued) SMS: received 63100 (0 queued), sent 461 (0 queued), store size 263 <---- queued messages here SMS: inbound (1.52,0.07,0.18) msg/sec, outbound (0.00,0.00,0.00) msg/sec DLR: received 10960, sent 0 DLR: inbound (0.00,0.01,0.03) msg/sec, outbound (0.00,0.00,0.00) msg/sec DLR: 130507 queued, using pgsql storage Box connections: smsbox:localsmsbox, IP 127.0.0.1 (0 queued), (on-line 4d 1h 44m 9s) smsbox:smpp1, IP 127.0.0.1 (0 queued), (on-line 0d 1h 0m 8s) smsbox:smpp2, IP 127.0.0.1 (364 queued), (on-line 0d 1h 0m 7s) <---- queued messages here smsbox:smpp3, IP 127.0.0.1 (0 queued), (on-line 0d 1h 0m 6s) smsbox:smpp1, IP 127.0.0.1 (0 queued), (on-line 0d 1h 0m 5s) smsbox:smpp2, IP 127.0.0.1 (0 queued), (on-line 0d 1h 0m 4s) smsbox:smpp3, IP 127.0.0.1 (0 queued), (on-line 0d 0h 57m 50s) SMSC connections: I've tried looking at the code to see what might be happening to cause the delays (I'm presuming that IF I can fix the throughput issue, the queueing issue will go away), however I dont have enough experience with the code to see what is going on. Hopefully this will be enough information to help someone sort this one out. Stuart. On Fri, 2015-04-10 at 14:30 -0700, Michael Epstein wrote: Hi Again, Thanks for response I have run my test again using the suggested changes. With no effect still a consistent 1 msg /sec. Any further suggestions, please? complete conf below just in case there is something weird in the mysql conf or kannel.conf. Thanks again, Mike kannel.conf: # Default kannel configuration file ## CORE config group = core admin-port = 13000 admin-password = XXXXXX admin-deny-ip = "*.*.*.*" admin-allow-ip = "127.0.0.1" smsbox-port = 13001 smsbox-max-pending = 500 box-deny-ip = "*.*.*.*" box-allow-ip = "127.0.0.1" wdp-interface-name = "*" log-file = "/var/log/kannel/bearerbox.log" log-level = 0 access-log = "/var/log/kannel/bearerbox-access.log" store-type = file store-location = "/var/spool/kannel/message-storage/store.file" dlr-storage = mysql sms-resend-freq = 1 ## MySQL config #### connection group = mysql-connection id = mydlr host = 173.194.254.79 username = YYYYYY password = XXXXXXX database = kannel max-connections = 300 #### table field mappings group = dlr-db id = mydlr table = dlr field-smsc = smsc field-timestamp = ts field-destination = destination field-source = source field-service = service field-url = url field-mask = mask field-status = status field-boxc-id = boxc #### FAKE group = smsc smsc = smpp smsc-id = FAKE allowed-smsc-id = FAKE host = 146.148.57.201 port = 4401 transceiver-mode = 1 system-type = SMPP smsc-username = XXXXXX smsc-password = XXXXX source-addr-ton = 0 source-addr-npi = 1 dest-addr-ton = 0 dest-addr-npi = 1 address-range = "" max-pending-submits = 500 throughput = 500 wait-ack = 1 log-level = 0 log-file = "/var/log/kannel/smsc-FAKE.log" ## SMSBOX config group = smsbox smsbox-id = smsbox1 bearerbox-host = 127.0.0.1 sendsms-port = 13131 log-file = "/var/log/kannel/smsbox.log" access-log = "/var/log/kannel/smsbox-access.log" log-level = 0 max-pending-requests = 4096 ## FAKE ROUTE group = smsbox-route smsbox-id = smsbox1 smsc-id = FAKE ## SENDSMS-USER config group = sendsms-user username = XXXXXX password = YYYYYY my.conf: auto_increment_increment 1 auto_increment_offset 1 autocommit ON automatic_sp_privileges ON back_log 50 basedir /sfs/cf-infrastructure/kanneltest/ big_tables OFF binlog_cache_size 32768 binlog_direct_non_transactional_updates OFF binlog_format STATEMENT binlog_stmt_cache_size 32768 bulk_insert_buffer_size 8388608 character_set_client utf8 character_set_connection utf8 character_set_database utf8 character_set_filesystem binary character_set_results utf8 character_set_server utf8 character_set_system utf8 character_sets_dir /static/share/charsets/ collation_connection utf8_general_ci collation_database utf8_general_ci collation_server utf8_general_ci completion_type NO_CHAIN concurrent_insert AUTO connect_timeout 60 datadir /sfs/cf-infrastructure/kanneltest/data/ date_format %Y-%m-%d datetime_format %Y-%m-%d %H:%i:%s default_storage_engine InnoDB default_week_format 0 delay_key_write ON delayed_insert_limit 100 delayed_insert_timeout 300 delayed_queue_size 1000 div_precision_increment 4 engine_condition_pushdown ON error_count 0 event_scheduler OFF expire_logs_days 0 external_user flush OFF flush_time 0 foreign_key_checks ON ft_boolean_syntax + -><()~*:""&| ft_max_word_len 84 ft_min_word_len 4 ft_query_expansion_limit 20 ft_stopword_file (built-in) general_log OFF general_log_file /sfs/cf-infrastructure/kanneltest/data/localhost.log group_concat_max_len 1024 have_compress YES have_crypt YES have_csv YES have_dynamic_loading YES have_geometry YES have_innodb YES have_ndbcluster NO have_openssl DISABLED have_partitioning YES have_profiling YES have_query_cache YES have_rtree_keys YES have_ssl DISABLED have_symlink YES hostname localhost identity 0 ignore_builtin_innodb OFF init_connect init_file init_slave innodb_adaptive_flushing ON innodb_adaptive_hash_index ON innodb_additional_mem_pool_size 8388608 innodb_autoextend_increment 8 innodb_autoinc_lock_mode 1 innodb_buffer_pool_instances 1 innodb_buffer_pool_size 3221225472 innodb_change_buffering all innodb_checksums ON innodb_commit_concurrency 0 innodb_concurrency_tickets 500 innodb_data_file_path ibdata1:10M:autoextend innodb_data_home_dir innodb_doublewrite OFF innodb_fast_shutdown 1 innodb_file_format Antelope innodb_file_format_check ON innodb_file_format_max Antelope innodb_file_per_table OFF innodb_flush_log_at_trx_commit 1 innodb_flush_method innodb_force_load_corrupted OFF innodb_force_recovery 0 innodb_io_capacity 200 innodb_large_prefix OFF innodb_lock_wait_timeout 50 innodb_locks_unsafe_for_binlog OFF innodb_log_buffer_size 8388608 innodb_log_file_size 134217728 innodb_log_files_in_group 2 innodb_log_group_home_dir ./ innodb_max_dirty_pages_pct 75 innodb_max_purge_lag 0 innodb_mirrored_log_groups 1 innodb_old_blocks_pct 37 innodb_old_blocks_time 0 innodb_open_files 300 innodb_print_all_deadlocks OFF innodb_purge_batch_size 20 innodb_purge_threads 0 innodb_random_read_ahead OFF innodb_read_ahead_threshold 56 innodb_read_io_threads 4 innodb_replication_delay 0 innodb_rollback_on_timeout OFF innodb_rollback_segments 128 innodb_spin_wait_delay 6 innodb_stats_method nulls_equal innodb_stats_on_metadata ON innodb_stats_sample_pages 8 innodb_strict_mode OFF innodb_support_xa OFF innodb_sync_spin_loops 30 innodb_table_locks ON innodb_thread_concurrency 0 innodb_thread_sleep_delay 10000 innodb_use_native_aio OFF innodb_use_sys_malloc ON innodb_version 5.5.38 innodb_write_io_threads 4 insert_id 0 interactive_timeout 20 join_buffer_size 131072 keep_files_on_create OFF key_buffer_size 8388608 key_cache_age_threshold 300 key_cache_block_size 1024 key_cache_division_limit 100 large_files_support ON large_page_size 0 large_pages OFF last_insert_id 0 lc_messages en_US lc_messages_dir /static/share/ lc_time_names en_US license GPL local_infile ON lock_wait_timeout 31536000 locked_in_memory OFF log OFF log_bin OFF log_bin_trust_function_creators OFF log_error log_output TABLE log_queries_not_using_indexes OFF log_slave_updates OFF log_slow_queries OFF log_warnings 1 long_query_time 10.000000 low_priority_updates OFF lower_case_file_system OFF lower_case_table_names 0 max_allowed_packet 1048576 max_binlog_cache_size 18446744073709547520 max_binlog_size 1073741824 max_binlog_stmt_cache_size 18446744073709547520 max_connect_errors 10 max_connections 1000 max_delayed_threads 20 max_error_count 64 max_heap_table_size 16777216 max_insert_delayed_threads 20 max_join_size 18446744073709551615 max_length_for_sort_data 1024 max_long_data_size 1048576 max_prepared_stmt_count 16382 max_relay_log_size 0 max_seeks_for_key 18446744073709551615 max_sort_length 1024 max_sp_recursion_depth 0 max_tmp_tables 32 max_user_connections 0 max_write_lock_count 18446744073709551615 metadata_locks_cache_size 1024 min_examined_row_limit 0 multi_range_count 256 myisam_data_pointer_size 6 myisam_max_sort_file_size 9223372036853727232 myisam_mmap_size 18446744073709551615 myisam_recover_options OFF myisam_repair_threads 1 myisam_sort_buffer_size 8388608 myisam_stats_method nulls_unequal myisam_use_mmap OFF net_buffer_length 16384 net_read_timeout 30 net_retry_count 10 net_write_timeout 60 new OFF old OFF old_alter_table OFF old_passwords OFF open_files_limit 4500000 optimizer_prune_level 1 optimizer_search_depth 62 optimizer_switch index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,engine_condition_pushdown=on pid_file /sfs/cf-infrastructure/kanneltest/data/localhost.pid plugin_dir /sfs/cf-infrastructure/kanneltest/lib/plugin/ port 0 preload_buffer_size 32768 profiling OFF profiling_history_size 15 protocol_version 10 proxy_user pseudo_slave_mode OFF pseudo_thread_id 643 query_alloc_block_size 8192 query_cache_limit 1048576 query_cache_min_res_unit 4096 query_cache_size 0 query_cache_type ON query_cache_wlock_invalidate OFF query_prealloc_size 8192 rand_seed1 0 rand_seed2 0 range_alloc_block_size 4096 read_buffer_size 131072 read_only OFF read_rnd_buffer_size 262144 relay_log relay_log_index relay_log_info_file relay-log.info relay_log_purge ON relay_log_recovery OFF relay_log_space_limit 0 report_host report_password report_port 0 report_user rpl_recovery_rank 0 secure_auth OFF secure_file_priv server_id 2205575229 skip_external_locking ON skip_name_resolve OFF skip_networking ON skip_show_database OFF slave_compressed_protocol OFF slave_exec_mode STRICT slave_load_tmpdir /cloudsqltmp slave_max_allowed_packet 1073741824 slave_net_timeout 3600 slave_skip_errors OFF slave_transaction_retries 10 slave_type_conversions slow_launch_time 2 slow_query_log OFF slow_query_log_file /sfs/cf-infrastructure/kanneltest/data/localhost-slow.log socket sort_buffer_size 2097152 sql_auto_is_null OFF sql_big_selects ON sql_big_tables OFF sql_buffer_result OFF sql_log_bin ON sql_log_off OFF sql_low_priority_updates OFF sql_max_join_size 18446744073709551615 sql_mode sql_notes ON sql_quote_show_create ON sql_safe_updates OFF sql_select_limit 18446744073709551615 sql_slave_skip_counter 0 sql_warnings OFF ssl_ca ssl_capath ssl_cert ssl_cipher ssl_key storage_engine InnoDB stored_program_cache 256 sync_binlog 0 sync_frm ON sync_master_info 0 sync_relay_log 0 sync_relay_log_info 0 system_time_zone UTC table_definition_cache 400 table_open_cache 400 thread_cache_size 20 thread_concurrency 10 thread_handling one-thread-per-connection thread_stack 262144 time_format %H:%i:%s time_zone SYSTEM timed_mutexes OFF timestamp 1428688227 tmp_table_size 16777216 tmpdir /cloudsqltmp transaction_alloc_block_size 8192 transaction_prealloc_size 4096 tx_isolation REPEATABLE-READ unique_checks ON updatable_views_with_limit YES version 5.5.38 version_comment (Google) version_compile_machine x86_64 version_compile_os Linux wait_timeout 20 warning_count 0 On 4/10/15 7:53 AM, Ali Kashif wrote: your problem is Mysql, and not max-pending-submits ## MySQL config #### connection group = mysql-connection id = mydlr host = localhost username = YYYYYY password = XXXXXXX database = kannel max-connections = 1 you are telling that kannel should have max 1 mysql connection, increase your mysql max-connections for example , and you will get required results, max-connections = 300 please also see mysql configuration and see what number of connection are allowed. so that you may not run out of connections and face error of "To many Connections" and set your required number of connections in /etc/my.cnf (or locate your mysql config) (be careful while setting a large number, see machine resources as well ) for a heavy load make sure to set your time out of sleeping connections so that you get free connections more quickly. for example, ## /etc/my.cnf max_connections = 400 wait_timeout=20 interactive_timeout=20 i Think this will solve your issue. Thanks Ali On Thu, Apr 9, 2015 at 9:54 PM, Michael Epstein <mepst...@callfire.com> wrote: So I tried my same test but sending over http with sendsms and was able to push over 200mgs/sec to kannel and 100 msg/sec downstream to my test SMPP server. So the problem is somewhere in opensmppbox. Any ideas on how to fix this? I saw one similar post to the mailing list http://www.kannel.org/pipermail/users/2012-July/018185.html but no solution. Thanks for you help, Mike On 4/8/15 10:54 AM, Michael Epstein wrote: Hi Alvaro, Thanks for the response had not tried setting max-pending-requests before. I tried max-pending-requests = 2048 still getting exactly 1 msg/sec. we already had max-pending-submits = 500 new config below. Thanks, Mike ## SMSBOX config group = smsbox smsbox-id = smsbox1 bearerbox-host = 127.0.0.1 sendsms-port = 13131 log-file = "/var/log/kannel/smsbox.log" access-log = "/var/log/kannel/smsbox-access.log" log-level = 0 max-pending-requests = 2048 On 4/8/15 10:41 AM, Alvaro Cornejo wrote: Hi Check max pending submit and max pending requests parameter. It holds Tx queue for replies from smsc until confirmations are below it. Regards Alvaro |-----------------------------------------------------------------------------------------------------------------| Envíe y Reciba Datos y mensajes de Texto (SMS) hacia y desde cualquier celular y Nextel en el Perú, México y en mas de 180 paises. Use aplicaciones 2 vias via SMS y GPRS online Visitenos en www.perusms.com On Wed, Apr 8, 2015 at 12:30 PM, Michael Epstein <mepst...@callfire.com> wrote: Hi Again, Tried moving to bigger better hardware 8 core 30gb machine still same 1 msg/sec. Which points to some sort of purposeful limit. Any ideas? Thank again, Mike On 4/7/15 8:36 PM, Michael Epstein wrote: Hi All, I have an SMPP server setup and am using Kannel as a proxy with opensmpp box from app. When I connect directly to the SMPP server the throughput is over 10x higher than going through Kannel. When going through Kannel I always get exactly 1 msg/sec. I have been tweaking and testing kannel configs with no luck. any help would be greatly appreciated. Thanks, Mike kannel.conf: # Default kannel configuration file ## CORE config group = core admin-port = 13000 admin-password = XXXXXXX admin-deny-ip = "*.*.*.*" admin-allow-ip = "127.0.0.1" smsbox-port = 13001 box-deny-ip = "*.*.*.*" box-allow-ip = "127.0.0.1" wdp-interface-name = "*" log-file = "/var/log/kannel/bearerbox.log" log-level = 0 access-log = "/var/log/kannel/bearerbox-access.log" store-type = spool store-location = "/var/spool/kannel/message-storage" dlr-storage = mysql ## MySQL config #### connection group = mysql-connection id = mydlr host = localhost username = YYYYYY password = XXXXXXX database = kannel max-connections = 1 #### table field mappings group = dlr-db id = mydlr table = dlr field-smsc = smsc field-timestamp = ts field-destination = destination field-source = source field-service = service field-url = url field-mask = mask field-status = status field-boxc-id = boxc #### FAKE group = smsc smsc = smpp smsc-id = FAKE allowed-smsc-id = FAKE host = 146.148.57.201 port = 4401 transceiver-mode = 1 system-type = SMPP smsc-username = XXXXX smsc-password = YYYYYY source-addr-ton = 0 source-addr-npi = 1 dest-addr-ton = 0 dest-addr-npi = 1 address-range = "" max-pending-submits = 500 wait-ack = 10 log-level = 0 log-file = "/var/log/kannel/smsc-FAKE.log" ## SMSBOX config group = smsbox smsbox-id = smsbox1 bearerbox-host = 127.0.0.1 sendsms-port = 13131 log-file = "/var/log/kannel/smsbox.log" access-log = "/var/log/kannel/smsbox-access.log" log-level = 0 ## SENDSMS-USER config group = sendsms-user username = sender password = YYYYYYYY opensmpp.conf: # OpenSmppBox config ## include the core kannel config ## OpenSmppBox needs dlr configs include = "/etc/kannel/kannel.conf" ## OPENSMSBOX config group = opensmppbox opensmppbox-id = FAKE log-level = 0 log-file = /var/log/kannel/opensmppbox-FAKE.log our-system-id = FAKE use-systemid-as-smsboxid = true route-to-smsc = FAKE smpp-logins = "/var/lib/kannel/smpplogins.txt" opensmppbox-port=4400
