>but I guess ASSP is waiting for it to complete or something Yes, this is a configuration task - assp will (has to) wait for the result of the script.
>way for you to code ASSP to actually run the external >calls externally as a separate thread This make no sense for several reasons. 1. scheduled starting an external program without wating for it, is the same like using the system scheduler - no need to 2. fork a new process in Perl will copy the complete code and data instance - wasted resources and very BIG problems if any database is used 3. having an extra thread - same like 1 4. calling the progam in any other thread - if the program stucks, the thread stucks In any case - assp has absolutely no control to an externaly started program. See it the same way like clamd and freshclam are working together. freshclam is running totaly independend, it only tells the clamd that it has done something. clamd than reloads the pattern database. Thomas Von: K Post <nntp.p...@gmail.com> An: ASSP development mailing list <assp-test@lists.sourceforge.net> Datum: 14.09.2015 16:21 Betreff: Re: [Assp-test] MainThread stuck occasionally calling external script I'll change the workflow ASAP so that an externally scheduled task generates the files and the files are included in the group definition file. That's no problem and it's a good idea. Thinking bigger picture though: Might there be a way for you to code ASSP to actually run the external calls externally as a separate thread such that ASSP continues even if the external program gets hung? I'm calling perl, so it's a separate perl instance, but I guess ASSP is waiting for it to complete or something? On Mon, Sep 14, 2015 at 4:20 AM, Thomas Eckardt <thomas.ecka...@thockar.com> wrote: > You should change the logic to prevent the MainThread from doing > DNS-queries in an external script. > > Read the help text at the bottom of the GUI: > > "Fields marked with at least one asterisk (*) accept .... > .... > # include filename > where filename is the relative path (from C:/assp) to the included file > like files/inc1.txt or inc1.txt > (one file per line). The line will be internaly replaced by the contents > of the included file!" > > solution: > Run the script that builds the list (doing DNS queries) in a cron job and > store the result in a text file - lets say 'c:/assp/files/spfIPs.txt' > > In the 'Groups' file define the result file as included file > > [GROUP-CONSTANTCONTACT-IPS] > #include files/spfIPs.txt > > ASSP constantly watches all defined files and all included files for > updates. If the cron job changes the include file, assp will reload all > groups. > > I found an issue for encrypted include-files. With the current release, > one restart after making this changes may be required. An updated dev > version will be released today. > > Thomas > > > > > > > Von: K Post <nntp.p...@gmail.com> > An: ASSP development mailing list <assp-test@lists.sourceforge.net> > Datum: 13.09.2015 16:31 > Betreff: [Assp-test] MainThread stuck occasionally calling external > script > > > > I wrote a scrpt to populate ASSP config files with IP address ranges based > on SPF values. For example, I call my GetDomainIPSfromSPF.pl script to > get > all of the IP's that ConstantContact.com says they can send from. It > works > well. > > HOWEVER, on rare occasion, maybe once every 2-3 weeks, MainThread gets > stuck calling the script, always when I do it with the ConstantContact > parameter. My guess is that there's something quirky with the > ConstantContact SPF, I'm doing something wrong, or something is broken. > That's on me to figure out, but the problem is that when the script hangs, > so does ASSP's SMTP processing. > > My Group Definitions config file has this entry: > [GROUP-CONSTANTCONTACT-IPS] > exec:c:/perl/bin/perl.exe "c:\scripts\GetDomainIPSfromSPF.pl" > constantcontact.com > > That calls the perl script, telling it to get the ip's for > constantcontact.com. The group looks fine upon visual inspection from the > gui. > > > Here's the log from this morning's temporary SMTP outage. SMTP was > unresponsive for 16 minutes according to my monitors, consistent with the > ASSP log. > > > Sep-13-15 08:41:16 Saving config > Sep-13-15 08:41:16 Info: no configuration changes detected - nothing to > save - file c:/ASSP/assp.cfg is unchanged > Sep-13-15 08:41:16 Adminupdate: [root x.x.x.9] file > c:/ASSP/cfg/GroupDefinitions.cfg' for config 'Groups' was changed > Sep-13-15 08:41:16 Option list file: c:/ASSP/cfg/GroupDefinitions.cfg' > reloaded (Groups) with 43 records > Sep-13-15 08:41:16 Info: loading groups from groupsfile > c:/ASSP/cfg/GroupDefinitions.cfg > Sep-13-15 08:41:16 Info: group GROUP-OUR-IPS loaded with 2 records > Sep-13-15 08:41:55 Info: Name Server x.x.x.51: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:41:55 Info: Name Server x.x.x.52: ResponseTime = 2 ms for > sourceforge.net > Sep-13-15 08:41:55 Info: Name Server x.x.x.53: ResponseTime = 3 ms for > sourceforge.net > Sep-13-15 08:42:17 Info: notification message queued to sent to > assp-not...@ourcharity.org > *Sep-13-15 08:42:17 Error: MainThread stuck for 61 seconds after: MainLoop > start - last debug step was: ConfigMakeGroupRe - exec:c:/perl/bin/perl.exe > "c:\scripts\GetDomainIPSfromSPF.pl" constantcontact.com > <http://constantcontact.com>!* > Sep-13-15 08:42:20 Info: successful sent file > c:/ASSP/messages/resendmail/n1000000000320.txt to x.x.x.35:25 > (smtpDestination) > Sep-13-15 08:42:28 Info: synchronizing all BerkeleyDB hashes to disk > Sep-13-15 08:42:28 Info: compacting all BerkeleyDB hashes on disk > Sep-13-15 08:42:55 Info: Name Server x.x.x.51: ResponseTime = 2 ms for > sourceforge.net > Sep-13-15 08:42:55 Info: Name Server x.x.x.52: ResponseTime = 17 ms for > sourceforge.net > Sep-13-15 08:42:55 Info: Name Server x.x.x.53: ResponseTime = 3 ms for > sourceforge.net > Sep-13-15 08:43:11 IPNumTries: cleaning cache finished: IP's before=1, > deleted=1 > Sep-13-15 08:43:11 SMTPdomainIP: cleaning cache finished: domain's > before=9, deleted=0 > Sep-13-15 08:43:11 SSLfailedCache: cleaning cache finished: IP's before=1, > deleted=0 > Sep-13-15 08:43:11 SubjectFrequency: cleaning cache finished: subjects > before=6, deleted=4 > Sep-13-15 08:43:55 Info: Name Server x.x.x.51: ResponseTime = 18 ms for > sourceforge.net > Sep-13-15 08:43:55 Info: Name Server x.x.x.52: ResponseTime = 3 ms for > sourceforge.net > Sep-13-15 08:43:55 Info: Name Server x.x.x.53: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:44:55 Info: Name Server x.x.x.51: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:44:55 Info: Name Server x.x.x.52: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:44:55 Info: Name Server x.x.x.53: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:45:55 Info: Name Server x.x.x.51: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:45:55 Info: Name Server x.x.x.52: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:45:55 Info: Name Server x.x.x.53: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:46:55 Info: Name Server x.x.x.51: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:46:55 Info: Name Server x.x.x.52: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:46:55 Info: Name Server x.x.x.53: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:47:55 Info: Name Server x.x.x.51: ResponseTime = 2 ms for > sourceforge.net > Sep-13-15 08:47:55 Info: Name Server x.x.x.52: ResponseTime = 19 ms for > sourceforge.net > Sep-13-15 08:47:55 Info: Name Server x.x.x.53: ResponseTime = 19 ms for > sourceforge.net > Sep-13-15 08:48:11 SMTPdomainIP: cleaning cache finished: domain's > before=9, deleted=0 > Sep-13-15 08:48:11 SSLfailedCache: cleaning cache finished: IP's before=1, > deleted=0 > Sep-13-15 08:48:11 SubjectFrequency: cleaning cache finished: subjects > before=2, deleted=2 > Sep-13-15 08:48:55 Info: Name Server x.x.x.51: ResponseTime = 23 ms for > sourceforge.net > Sep-13-15 08:48:55 Info: Name Server x.x.x.52: ResponseTime = 6 ms for > sourceforge.net > Sep-13-15 08:48:55 Info: Name Server x.x.x.53: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:49:55 Info: Name Server x.x.x.51: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:49:55 Info: Name Server x.x.x.52: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:49:55 Info: Name Server x.x.x.53: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:50:55 Info: Name Server x.x.x.51: ResponseTime = 3 ms for > sourceforge.net > Sep-13-15 08:50:55 Info: Name Server x.x.x.52: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:50:55 Info: Name Server x.x.x.53: ResponseTime = 4 ms for > sourceforge.net > Sep-13-15 08:51:55 Info: Name Server x.x.x.51: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:51:55 Info: Name Server x.x.x.52: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:51:55 Info: Name Server x.x.x.53: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:52:55 Info: Name Server x.x.x.51: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:52:55 Info: Name Server x.x.x.52: ResponseTime = 18 ms for > sourceforge.net > Sep-13-15 08:52:55 Info: Name Server x.x.x.53: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:53:11 SMTPdomainIP: cleaning cache finished: domain's > before=9, deleted=0 > Sep-13-15 08:53:11 SSLfailedCache: cleaning cache finished: IP's before=1, > deleted=0 > Sep-13-15 08:53:55 Info: Name Server x.x.x.51: ResponseTime = 10 ms for > sourceforge.net > Sep-13-15 08:53:55 Info: Name Server x.x.x.52: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:53:55 Info: Name Server x.x.x.53: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:54:55 Info: Name Server x.x.x.51: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:54:55 Info: Name Server x.x.x.52: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:54:55 Info: Name Server x.x.x.53: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:55:55 Info: Name Server x.x.x.51: ResponseTime = 2 ms for > sourceforge.net > Sep-13-15 08:55:55 Info: Name Server x.x.x.52: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:55:55 Info: Name Server x.x.x.53: ResponseTime = 3 ms for > sourceforge.net > Sep-13-15 08:56:55 Info: Name Server x.x.x.51: ResponseTime = 18 ms for > sourceforge.net > Sep-13-15 08:56:55 Info: Name Server x.x.x.52: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:56:55 Info: Name Server x.x.x.53: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:57:28 Info: synchronizing all BerkeleyDB hashes to disk > Sep-13-15 08:57:28 Info: compacting all BerkeleyDB hashes on disk > Sep-13-15 08:57:55 Info: Name Server x.x.x.51: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:57:55 Info: Name Server x.x.x.52: ResponseTime = 3 ms for > sourceforge.net > Sep-13-15 08:57:55 Info: Name Server x.x.x.53: ResponseTime = 18 ms for > sourceforge.net > Sep-13-15 08:58:11 SMTPdomainIP: cleaning cache finished: domain's > before=9, deleted=0 > Sep-13-15 08:58:11 SSLfailedCache: cleaning cache finished: IP's before=1, > deleted=0 > Sep-13-15 08:58:30 Info: group GROUP-CONSTANTCONTACT-IPS loaded 24 > addresses via exec(line 1) > Sep-13-15 08:58:30 Info: group GROUP-CONSTANTCONTACT-IPS loaded with 24 > records > Sep-13-15 08:58:31 Info: group GROUP-MANDRILL-IPS loaded 9 addresses via > exec(line 1) > Sep-13-15 08:58:31 Info: group GROUP-MANDRILL-IPS loaded with 9 records > Sep-13-15 08:58:31 Info: group GROUP-MICROSOFT-IPS loaded 83 addresses via > exec(line 1) > Sep-13-15 08:58:32 Info: group GROUP-MICROSOFT-IPS loaded 83 addresses via > exec(line 2) > Sep-13-15 08:58:32 Info: group GROUP-MICROSOFT-IPS loaded with 166 records > Sep-13-15 08:58:32 Info: group GROUP-GOOGLE-IPS loaded 12 addresses via > exec(line 1) > Sep-13-15 08:58:32 Info: group GROUP-GOOGLE-IPS loaded with 12 records > Sep-13-15 08:58:33 Info: group GROUP-AMAZON-IPS loaded 20 addresses via > exec(line 1) > Sep-13-15 08:58:33 Info: group GROUP-AMAZON-IPS loaded with 20 records > Sep-13-15 08:58:34 Info: group GROUP-FACEBOOK-IPS loaded 8 addresses via > exec(line 1) > Sep-13-15 08:58:34 Info: group GROUP-FACEBOOK-IPS loaded 6 addresses via > exec(line 2) > Sep-13-15 08:58:34 Info: group GROUP-FACEBOOK-IPS loaded with 14 records > Sep-13-15 08:58:34 Info: group GROUP-HONEYPOT-EMAILS loaded with 12 > records > Sep-13-15 08:58:34 Option list file: > c:/ASSP/cfg/SpamLoverSpecific/SpamLovers-NoDelaying.cfg' reloaded > (delaySpamLovers) with 2 records > Sep-13-15 08:58:34 Option list file: c:/ASSP/cfg/ValidMailboxes.cfg' > reloaded (LocalAddresses_Flat) with 214 records > Sep-13-15 08:58:34 Option list file: c:/ASSP/cfg/SpamOnlyAddresses.cfg' > reloaded (spamaddresses) with 48 records > Sep-13-15 08:58:34 Option list file: c:/ASSP/cfg/noBlockingIPs.cfg' > reloaded (noBlockingIPs) with 5 records > Sep-13-15 08:58:34 Option list file: c:/ASSP/cfg/LocalDomains.cfg' > reloaded > (localDomains) with 58 records > Sep-13-15 08:58:34 Option list file: c:/ASSP/cfg/noBlockingIPs.cfg' > reloaded (noBlockingIPs) with 5 records > Sep-13-15 08:58:34 Option list file: > c:/ASSP/cfg/PenaltyDoNotPenalizeIPs.cfg' reloaded (noPB) with 10 records > Sep-13-15 08:58:34 Option list file: > c:/ASSP/cfg/PenaltyDoNotPBWhiteTheseIPs.cfg' reloaded (noPBwhite) with 4 > records > Sep-13-15 08:58:34 Option list file: c:/ASSP/cfg/NoDelayIPs.cfg' reloaded > (noDelay) with 9 records > Sep-13-15 08:58:34 Option list file: c:/ASSP/cfg/DNSBLIgnoreTheseIPs.cfg' > reloaded (noRBL) with 6 records > Sep-13-15 08:58:34 Option list file: c:/ASSP/cfg/NoDelayIPs.cfg' reloaded > (noDelay) with 9 records > Sep-13-15 08:58:34 Option list file: c:/ASSP/cfg/DNSBLIgnoreTheseIPs.cfg' > reloaded (noRBL) with 6 records > Sep-13-15 08:58:35 Option list file: c:/ASSP/cfg/noBlockingIPs.cfg' > reloaded (noBlockingIPs) with 5 records > Sep-13-15 08:58:35 Option list file: > c:/ASSP/cfg/PenaltyDoNotPenalizeIPs.cfg' reloaded (noPB) with 10 records > Sep-13-15 08:58:35 Option list file: > c:/ASSP/cfg/PenaltyDoNotPBWhiteTheseIPs.cfg' reloaded (noPBwhite) with 4 > records > Sep-13-15 08:58:35 Option list file: c:/ASSP/cfg/NoDelayIPs.cfg' reloaded > (noDelay) with 9 records > Sep-13-15 08:58:35 Option list file: c:/ASSP/cfg/DNSBLIgnoreTheseIPs.cfg' > reloaded (noRBL) with 6 records > Sep-13-15 08:58:35 Option list file: c:/ASSP/cfg/DNSBLIgnoreTheseIPs.cfg' > reloaded (noRBL) with 6 records > Sep-13-15 08:58:35 Option list file: c:/ASSP/cfg/noBlockingIPs.cfg' > reloaded (noBlockingIPs) with 5 records > Sep-13-15 08:58:35 Option list file: > c:/ASSP/cfg/PenaltyDoNotPenalizeIPs.cfg' reloaded (noPB) with 10 records > Sep-13-15 08:58:35 Option list file: > c:/ASSP/cfg/PenaltyDoNotPBWhiteTheseIPs.cfg' reloaded (noPBwhite) with 4 > records > Sep-13-15 08:58:35 Option list file: c:/ASSP/cfg/NoDelayIPs.cfg' reloaded > (noDelay) with 9 records > Sep-13-15 08:58:35 Option list file: c:/ASSP/cfg/DNSBLIgnoreTheseIPs.cfg' > reloaded (noRBL) with 6 records > Sep-13-15 08:58:35 Option list file: c:/ASSP/cfg/noBlockingIPs.cfg' > reloaded (noBlockingIPs) with 5 records > Sep-13-15 08:58:35 Option list file: > c:/ASSP/cfg/PenaltyDoNotPenalizeIPs.cfg' reloaded (noPB) with 10 records > Sep-13-15 08:58:35 Option list file: > c:/ASSP/cfg/PenaltyDoNotPBWhiteTheseIPs.cfg' reloaded (noPBwhite) with 4 > records > Sep-13-15 08:58:35 Option list file: c:/ASSP/cfg/NoDelayIPs.cfg' reloaded > (noDelay) with 9 records > Sep-13-15 08:58:35 Option list file: c:/ASSP/cfg/DNSBLIgnoreTheseIPs.cfg' > reloaded (noRBL) with 6 records > Sep-13-15 08:58:35 Info: MainThread has retured to normal state after > stuck > [[ email processing resumes here ]] > > So, is there some way to have ASSP continue on without MainThread getting > stuck if there's a problem with a script called when updating a group > definition? > > > Note "rutured" is spelled wrong above in the log. > > > THANKS! > > ------------------------------------------------------------------------------ > _______________________________________________ > Assp-test mailing list > Assp-test@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/assp-test > > > > > > > DISCLAIMER: > ******************************************************* > This email and any files transmitted with it may be confidential, legally > privileged and protected in law and are intended solely for the use of the > > individual to whom it is addressed. > This email was multiple times scanned for viruses. There should be no > known virus in this email! > ******************************************************* > > > ------------------------------------------------------------------------------ > _______________________________________________ > Assp-test mailing list > Assp-test@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/assp-test > ------------------------------------------------------------------------------ _______________________________________________ Assp-test mailing list Assp-test@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/assp-test DISCLAIMER: ******************************************************* This email and any files transmitted with it may be confidential, legally privileged and protected in law and are intended solely for the use of the individual to whom it is addressed. This email was multiple times scanned for viruses. There should be no known virus in this email! ******************************************************* ------------------------------------------------------------------------------ _______________________________________________ Assp-test mailing list Assp-test@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/assp-test
