I'll change the workflow ASAP so that an externally scheduled task generates the files and the files are included in the group definition file. That's no problem and it's a good idea.
Thinking bigger picture though: Might there be a way for you to code ASSP to actually run the external calls externally as a separate thread such that ASSP continues even if the external program gets hung? I'm calling perl, so it's a separate perl instance, but I guess ASSP is waiting for it to complete or something? On Mon, Sep 14, 2015 at 4:20 AM, Thomas Eckardt <thomas.ecka...@thockar.com> wrote: > You should change the logic to prevent the MainThread from doing > DNS-queries in an external script. > > Read the help text at the bottom of the GUI: > > "Fields marked with at least one asterisk (*) accept .... > .... > # include filename > where filename is the relative path (from C:/assp) to the included file > like files/inc1.txt or inc1.txt > (one file per line). The line will be internaly replaced by the contents > of the included file!" > > solution: > Run the script that builds the list (doing DNS queries) in a cron job and > store the result in a text file - lets say 'c:/assp/files/spfIPs.txt' > > In the 'Groups' file define the result file as included file > > [GROUP-CONSTANTCONTACT-IPS] > #include files/spfIPs.txt > > ASSP constantly watches all defined files and all included files for > updates. If the cron job changes the include file, assp will reload all > groups. > > I found an issue for encrypted include-files. With the current release, > one restart after making this changes may be required. An updated dev > version will be released today. > > Thomas > > > > > > > Von: K Post <nntp.p...@gmail.com> > An: ASSP development mailing list <assp-test@lists.sourceforge.net> > Datum: 13.09.2015 16:31 > Betreff: [Assp-test] MainThread stuck occasionally calling external > script > > > > I wrote a scrpt to populate ASSP config files with IP address ranges based > on SPF values. For example, I call my GetDomainIPSfromSPF.pl script to > get > all of the IP's that ConstantContact.com says they can send from. It > works > well. > > HOWEVER, on rare occasion, maybe once every 2-3 weeks, MainThread gets > stuck calling the script, always when I do it with the ConstantContact > parameter. My guess is that there's something quirky with the > ConstantContact SPF, I'm doing something wrong, or something is broken. > That's on me to figure out, but the problem is that when the script hangs, > so does ASSP's SMTP processing. > > My Group Definitions config file has this entry: > [GROUP-CONSTANTCONTACT-IPS] > exec:c:/perl/bin/perl.exe "c:\scripts\GetDomainIPSfromSPF.pl" > constantcontact.com > > That calls the perl script, telling it to get the ip's for > constantcontact.com. The group looks fine upon visual inspection from the > gui. > > > Here's the log from this morning's temporary SMTP outage. SMTP was > unresponsive for 16 minutes according to my monitors, consistent with the > ASSP log. > > > Sep-13-15 08:41:16 Saving config > Sep-13-15 08:41:16 Info: no configuration changes detected - nothing to > save - file c:/ASSP/assp.cfg is unchanged > Sep-13-15 08:41:16 Adminupdate: [root x.x.x.9] file > c:/ASSP/cfg/GroupDefinitions.cfg' for config 'Groups' was changed > Sep-13-15 08:41:16 Option list file: c:/ASSP/cfg/GroupDefinitions.cfg' > reloaded (Groups) with 43 records > Sep-13-15 08:41:16 Info: loading groups from groupsfile > c:/ASSP/cfg/GroupDefinitions.cfg > Sep-13-15 08:41:16 Info: group GROUP-OUR-IPS loaded with 2 records > Sep-13-15 08:41:55 Info: Name Server x.x.x.51: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:41:55 Info: Name Server x.x.x.52: ResponseTime = 2 ms for > sourceforge.net > Sep-13-15 08:41:55 Info: Name Server x.x.x.53: ResponseTime = 3 ms for > sourceforge.net > Sep-13-15 08:42:17 Info: notification message queued to sent to > assp-not...@ourcharity.org > *Sep-13-15 08:42:17 Error: MainThread stuck for 61 seconds after: MainLoop > start - last debug step was: ConfigMakeGroupRe - exec:c:/perl/bin/perl.exe > "c:\scripts\GetDomainIPSfromSPF.pl" constantcontact.com > <http://constantcontact.com>!* > Sep-13-15 08:42:20 Info: successful sent file > c:/ASSP/messages/resendmail/n1000000000320.txt to x.x.x.35:25 > (smtpDestination) > Sep-13-15 08:42:28 Info: synchronizing all BerkeleyDB hashes to disk > Sep-13-15 08:42:28 Info: compacting all BerkeleyDB hashes on disk > Sep-13-15 08:42:55 Info: Name Server x.x.x.51: ResponseTime = 2 ms for > sourceforge.net > Sep-13-15 08:42:55 Info: Name Server x.x.x.52: ResponseTime = 17 ms for > sourceforge.net > Sep-13-15 08:42:55 Info: Name Server x.x.x.53: ResponseTime = 3 ms for > sourceforge.net > Sep-13-15 08:43:11 IPNumTries: cleaning cache finished: IP's before=1, > deleted=1 > Sep-13-15 08:43:11 SMTPdomainIP: cleaning cache finished: domain's > before=9, deleted=0 > Sep-13-15 08:43:11 SSLfailedCache: cleaning cache finished: IP's before=1, > deleted=0 > Sep-13-15 08:43:11 SubjectFrequency: cleaning cache finished: subjects > before=6, deleted=4 > Sep-13-15 08:43:55 Info: Name Server x.x.x.51: ResponseTime = 18 ms for > sourceforge.net > Sep-13-15 08:43:55 Info: Name Server x.x.x.52: ResponseTime = 3 ms for > sourceforge.net > Sep-13-15 08:43:55 Info: Name Server x.x.x.53: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:44:55 Info: Name Server x.x.x.51: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:44:55 Info: Name Server x.x.x.52: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:44:55 Info: Name Server x.x.x.53: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:45:55 Info: Name Server x.x.x.51: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:45:55 Info: Name Server x.x.x.52: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:45:55 Info: Name Server x.x.x.53: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:46:55 Info: Name Server x.x.x.51: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:46:55 Info: Name Server x.x.x.52: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:46:55 Info: Name Server x.x.x.53: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:47:55 Info: Name Server x.x.x.51: ResponseTime = 2 ms for > sourceforge.net > Sep-13-15 08:47:55 Info: Name Server x.x.x.52: ResponseTime = 19 ms for > sourceforge.net > Sep-13-15 08:47:55 Info: Name Server x.x.x.53: ResponseTime = 19 ms for > sourceforge.net > Sep-13-15 08:48:11 SMTPdomainIP: cleaning cache finished: domain's > before=9, deleted=0 > Sep-13-15 08:48:11 SSLfailedCache: cleaning cache finished: IP's before=1, > deleted=0 > Sep-13-15 08:48:11 SubjectFrequency: cleaning cache finished: subjects > before=2, deleted=2 > Sep-13-15 08:48:55 Info: Name Server x.x.x.51: ResponseTime = 23 ms for > sourceforge.net > Sep-13-15 08:48:55 Info: Name Server x.x.x.52: ResponseTime = 6 ms for > sourceforge.net > Sep-13-15 08:48:55 Info: Name Server x.x.x.53: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:49:55 Info: Name Server x.x.x.51: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:49:55 Info: Name Server x.x.x.52: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:49:55 Info: Name Server x.x.x.53: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:50:55 Info: Name Server x.x.x.51: ResponseTime = 3 ms for > sourceforge.net > Sep-13-15 08:50:55 Info: Name Server x.x.x.52: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:50:55 Info: Name Server x.x.x.53: ResponseTime = 4 ms for > sourceforge.net > Sep-13-15 08:51:55 Info: Name Server x.x.x.51: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:51:55 Info: Name Server x.x.x.52: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:51:55 Info: Name Server x.x.x.53: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:52:55 Info: Name Server x.x.x.51: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:52:55 Info: Name Server x.x.x.52: ResponseTime = 18 ms for > sourceforge.net > Sep-13-15 08:52:55 Info: Name Server x.x.x.53: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:53:11 SMTPdomainIP: cleaning cache finished: domain's > before=9, deleted=0 > Sep-13-15 08:53:11 SSLfailedCache: cleaning cache finished: IP's before=1, > deleted=0 > Sep-13-15 08:53:55 Info: Name Server x.x.x.51: ResponseTime = 10 ms for > sourceforge.net > Sep-13-15 08:53:55 Info: Name Server x.x.x.52: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:53:55 Info: Name Server x.x.x.53: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:54:55 Info: Name Server x.x.x.51: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:54:55 Info: Name Server x.x.x.52: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:54:55 Info: Name Server x.x.x.53: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:55:55 Info: Name Server x.x.x.51: ResponseTime = 2 ms for > sourceforge.net > Sep-13-15 08:55:55 Info: Name Server x.x.x.52: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:55:55 Info: Name Server x.x.x.53: ResponseTime = 3 ms for > sourceforge.net > Sep-13-15 08:56:55 Info: Name Server x.x.x.51: ResponseTime = 18 ms for > sourceforge.net > Sep-13-15 08:56:55 Info: Name Server x.x.x.52: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:56:55 Info: Name Server x.x.x.53: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:57:28 Info: synchronizing all BerkeleyDB hashes to disk > Sep-13-15 08:57:28 Info: compacting all BerkeleyDB hashes on disk > Sep-13-15 08:57:55 Info: Name Server x.x.x.51: ResponseTime = 1 ms for > sourceforge.net > Sep-13-15 08:57:55 Info: Name Server x.x.x.52: ResponseTime = 3 ms for > sourceforge.net > Sep-13-15 08:57:55 Info: Name Server x.x.x.53: ResponseTime = 18 ms for > sourceforge.net > Sep-13-15 08:58:11 SMTPdomainIP: cleaning cache finished: domain's > before=9, deleted=0 > Sep-13-15 08:58:11 SSLfailedCache: cleaning cache finished: IP's before=1, > deleted=0 > Sep-13-15 08:58:30 Info: group GROUP-CONSTANTCONTACT-IPS loaded 24 > addresses via exec(line 1) > Sep-13-15 08:58:30 Info: group GROUP-CONSTANTCONTACT-IPS loaded with 24 > records > Sep-13-15 08:58:31 Info: group GROUP-MANDRILL-IPS loaded 9 addresses via > exec(line 1) > Sep-13-15 08:58:31 Info: group GROUP-MANDRILL-IPS loaded with 9 records > Sep-13-15 08:58:31 Info: group GROUP-MICROSOFT-IPS loaded 83 addresses via > exec(line 1) > Sep-13-15 08:58:32 Info: group GROUP-MICROSOFT-IPS loaded 83 addresses via > exec(line 2) > Sep-13-15 08:58:32 Info: group GROUP-MICROSOFT-IPS loaded with 166 records > Sep-13-15 08:58:32 Info: group GROUP-GOOGLE-IPS loaded 12 addresses via > exec(line 1) > Sep-13-15 08:58:32 Info: group GROUP-GOOGLE-IPS loaded with 12 records > Sep-13-15 08:58:33 Info: group GROUP-AMAZON-IPS loaded 20 addresses via > exec(line 1) > Sep-13-15 08:58:33 Info: group GROUP-AMAZON-IPS loaded with 20 records > Sep-13-15 08:58:34 Info: group GROUP-FACEBOOK-IPS loaded 8 addresses via > exec(line 1) > Sep-13-15 08:58:34 Info: group GROUP-FACEBOOK-IPS loaded 6 addresses via > exec(line 2) > Sep-13-15 08:58:34 Info: group GROUP-FACEBOOK-IPS loaded with 14 records > Sep-13-15 08:58:34 Info: group GROUP-HONEYPOT-EMAILS loaded with 12 > records > Sep-13-15 08:58:34 Option list file: > c:/ASSP/cfg/SpamLoverSpecific/SpamLovers-NoDelaying.cfg' reloaded > (delaySpamLovers) with 2 records > Sep-13-15 08:58:34 Option list file: c:/ASSP/cfg/ValidMailboxes.cfg' > reloaded (LocalAddresses_Flat) with 214 records > Sep-13-15 08:58:34 Option list file: c:/ASSP/cfg/SpamOnlyAddresses.cfg' > reloaded (spamaddresses) with 48 records > Sep-13-15 08:58:34 Option list file: c:/ASSP/cfg/noBlockingIPs.cfg' > reloaded (noBlockingIPs) with 5 records > Sep-13-15 08:58:34 Option list file: c:/ASSP/cfg/LocalDomains.cfg' > reloaded > (localDomains) with 58 records > Sep-13-15 08:58:34 Option list file: c:/ASSP/cfg/noBlockingIPs.cfg' > reloaded (noBlockingIPs) with 5 records > Sep-13-15 08:58:34 Option list file: > c:/ASSP/cfg/PenaltyDoNotPenalizeIPs.cfg' reloaded (noPB) with 10 records > Sep-13-15 08:58:34 Option list file: > c:/ASSP/cfg/PenaltyDoNotPBWhiteTheseIPs.cfg' reloaded (noPBwhite) with 4 > records > Sep-13-15 08:58:34 Option list file: c:/ASSP/cfg/NoDelayIPs.cfg' reloaded > (noDelay) with 9 records > Sep-13-15 08:58:34 Option list file: c:/ASSP/cfg/DNSBLIgnoreTheseIPs.cfg' > reloaded (noRBL) with 6 records > Sep-13-15 08:58:34 Option list file: c:/ASSP/cfg/NoDelayIPs.cfg' reloaded > (noDelay) with 9 records > Sep-13-15 08:58:34 Option list file: c:/ASSP/cfg/DNSBLIgnoreTheseIPs.cfg' > reloaded (noRBL) with 6 records > Sep-13-15 08:58:35 Option list file: c:/ASSP/cfg/noBlockingIPs.cfg' > reloaded (noBlockingIPs) with 5 records > Sep-13-15 08:58:35 Option list file: > c:/ASSP/cfg/PenaltyDoNotPenalizeIPs.cfg' reloaded (noPB) with 10 records > Sep-13-15 08:58:35 Option list file: > c:/ASSP/cfg/PenaltyDoNotPBWhiteTheseIPs.cfg' reloaded (noPBwhite) with 4 > records > Sep-13-15 08:58:35 Option list file: c:/ASSP/cfg/NoDelayIPs.cfg' reloaded > (noDelay) with 9 records > Sep-13-15 08:58:35 Option list file: c:/ASSP/cfg/DNSBLIgnoreTheseIPs.cfg' > reloaded (noRBL) with 6 records > Sep-13-15 08:58:35 Option list file: c:/ASSP/cfg/DNSBLIgnoreTheseIPs.cfg' > reloaded (noRBL) with 6 records > Sep-13-15 08:58:35 Option list file: c:/ASSP/cfg/noBlockingIPs.cfg' > reloaded (noBlockingIPs) with 5 records > Sep-13-15 08:58:35 Option list file: > c:/ASSP/cfg/PenaltyDoNotPenalizeIPs.cfg' reloaded (noPB) with 10 records > Sep-13-15 08:58:35 Option list file: > c:/ASSP/cfg/PenaltyDoNotPBWhiteTheseIPs.cfg' reloaded (noPBwhite) with 4 > records > Sep-13-15 08:58:35 Option list file: c:/ASSP/cfg/NoDelayIPs.cfg' reloaded > (noDelay) with 9 records > Sep-13-15 08:58:35 Option list file: c:/ASSP/cfg/DNSBLIgnoreTheseIPs.cfg' > reloaded (noRBL) with 6 records > Sep-13-15 08:58:35 Option list file: c:/ASSP/cfg/noBlockingIPs.cfg' > reloaded (noBlockingIPs) with 5 records > Sep-13-15 08:58:35 Option list file: > c:/ASSP/cfg/PenaltyDoNotPenalizeIPs.cfg' reloaded (noPB) with 10 records > Sep-13-15 08:58:35 Option list file: > c:/ASSP/cfg/PenaltyDoNotPBWhiteTheseIPs.cfg' reloaded (noPBwhite) with 4 > records > Sep-13-15 08:58:35 Option list file: c:/ASSP/cfg/NoDelayIPs.cfg' reloaded > (noDelay) with 9 records > Sep-13-15 08:58:35 Option list file: c:/ASSP/cfg/DNSBLIgnoreTheseIPs.cfg' > reloaded (noRBL) with 6 records > Sep-13-15 08:58:35 Info: MainThread has retured to normal state after > stuck > [[ email processing resumes here ]] > > So, is there some way to have ASSP continue on without MainThread getting > stuck if there's a problem with a script called when updating a group > definition? > > > Note "rutured" is spelled wrong above in the log. > > > THANKS! > > ------------------------------------------------------------------------------ > _______________________________________________ > Assp-test mailing list > Assp-test@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/assp-test > > > > > > > DISCLAIMER: > ******************************************************* > This email and any files transmitted with it may be confidential, legally > privileged and protected in law and are intended solely for the use of the > > individual to whom it is addressed. > This email was multiple times scanned for viruses. There should be no > known virus in this email! > ******************************************************* > > > ------------------------------------------------------------------------------ > _______________________________________________ > Assp-test mailing list > Assp-test@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/assp-test > ------------------------------------------------------------------------------ _______________________________________________ Assp-test mailing list Assp-test@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/assp-test
