[fpc-pascal] Processing passwords etc.
Is my understanding correct that when a string or a dynamic array is extended it might result in its existing content being released to the heap? If so, is it possible to ensure that this is zeroed or randomised first, without having to do it manually? -- Mark Morgan Lloyd markMLl .AT. telemetry.co .DOT. uk [Opinions above are the author's, not those of his employers or colleagues] ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Processing passwords etc.
On Fri, 11 Apr 2014, Mark Morgan Lloyd wrote: Is my understanding correct that when a string or a dynamic array is extended it might result in its existing content being released to the heap? If so, is it possible to ensure that this is zeroed or randomised first, without having to do it manually? Currently not, although such behaviour could easile be introduced as an option. Current HeartBleed frenzy getting you (or your bosses) scared ? :) Michael. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Processing passwords etc.
On 11 Apr 2014, at 09:36, Mark Morgan Lloyd wrote: > Is my understanding correct that when a string or a dynamic array is extended > it might result in its existing content being released to the heap? > > If so, is it possible to ensure that this is zeroed or randomised first, > without having to do it manually? You can install a memory manager that does this for all (de)allocations and then calls through to the original memory manager. There is no way to only do this for strings and dynamic arrays, and I don't think it would be a good idea to do so. Not all passwords are strings, so that would probably mostly give a false sense of security. Jonas ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Processing passwords etc.
Jonas Maebe wrote: On 11 Apr 2014, at 09:36, Mark Morgan Lloyd wrote: Is my understanding correct that when a string or a dynamic array is extended it might result in its existing content being released to the heap? If so, is it possible to ensure that this is zeroed or randomised first, without having to do it manually? You can install a memory manager that does this for all (de)allocations and then calls through to the original memory manager. There is no way to only do this for strings and dynamic arrays, and I don't think it would be a good idea to do so. Not all passwords are strings, so that would probably mostly give a false sense of security. Using a memory manager would reliably wipe strings etc. when reallocated (i.e rather than when an assignment didn't trigger reallocation). On the other hand it would have the overhead of also overwriting blocks that the user knew were being freed and could audit first, as well as stuff that was being freed as part of the RTL operation. It's that latter case- where the RTL is copying something without the user being aware- that I think is significant. -- Mark Morgan Lloyd markMLl .AT. telemetry.co .DOT. uk [Opinions above are the author's, not those of his employers or colleagues] ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Processing passwords etc.
Michael Van Canneyt wrote: On Fri, 11 Apr 2014, Mark Morgan Lloyd wrote: Is my understanding correct that when a string or a dynamic array is extended it might result in its existing content being released to the heap? If so, is it possible to ensure that this is zeroed or randomised first, without having to do it manually? Currently not, although such behaviour could easile be introduced as an option. Current HeartBleed frenzy getting you (or your bosses) scared ? :) :-) No, but I don't think enough people are focussing on the real problem which is that the OpenSSL developers were letting sensitive data leak to the freelist. If, when they wrote the code some years ago, they'd been rigorous in their handling of passwords and private keys then the current bug- introduced in 2012- would have been far less serious. -- Mark Morgan Lloyd markMLl .AT. telemetry.co .DOT. uk [Opinions above are the author's, not those of his employers or colleagues] ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Processing passwords etc.
On 11 Apr 2014, at 10:10, Mark Morgan Lloyd wrote: > Jonas Maebe wrote: >> On 11 Apr 2014, at 09:36, Mark Morgan Lloyd wrote: >>> Is my understanding correct that when a string or a dynamic array is >>> extended it might result in its existing content being released to the heap? >>> >>> If so, is it possible to ensure that this is zeroed or randomised first, >>> without having to do it manually? >> You can install a memory manager that does this for all (de)allocations and >> then calls through to the original memory manager. There is no way to only >> do this for strings and dynamic arrays, and I don't think it would be a good >> idea to do so. Not all passwords are strings, so that would probably mostly >> give a false sense of security. > > Using a memory manager would reliably wipe strings etc. when reallocated (i.e > rather than when an assignment didn't trigger reallocation). On the other > hand it would have the overhead of also overwriting blocks that the user knew > were being freed and could audit first, as well as stuff that was being freed > as part of the RTL operation. Such stuff from RTL operations may include getmem'd buffers used to read private key data from disk. > It's that latter case- where the RTL is copying something without the user > being aware- that I think is significant. Either the memory manager guarantees trashing, or it does not. Having half-solutions are worse than nothing at all, because it gives a false sense of security. Even the best audits are guaranteed to miss things (as in case of the person who wrote and the person who reviewed/audited the patch that introduced the heartbleed bug), and as you mentioned there is no control over the internals of the RTL (which has not been written at all from the point of view that buffers that might sometimes contain sensitive data should always be cleansed). Jonas ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Processing passwords etc.
On Fri, 11 Apr 2014, Mark Morgan Lloyd wrote: Michael Van Canneyt wrote: On Fri, 11 Apr 2014, Mark Morgan Lloyd wrote: Is my understanding correct that when a string or a dynamic array is extended it might result in its existing content being released to the heap? If so, is it possible to ensure that this is zeroed or randomised first, without having to do it manually? Currently not, although such behaviour could easile be introduced as an option. Current HeartBleed frenzy getting you (or your bosses) scared ? :) :-) No, but I don't think enough people are focussing on the real problem which is that the OpenSSL developers were letting sensitive data leak to the freelist. If, when they wrote the code some years ago, they'd been rigorous in their handling of passwords and private keys then the current bug- introduced in 2012- would have been far less serious. Correct. Having looked at the openssl library, I can say it's a miracle it works at all. Rarely seen such a mess of macros and whatnot, a good showcase of why I think C is not a good language choice. So, not surprised that it contains leaks :( OTOH, I think people are hugely exaggerating the problem, considering it was introduced relatively recently and that I got my security update before it hit the newspapers. That is of course not to say that it shouldn't be fixed and people shouldn't bother. But the way it is presented is more about scaring people than anything else. Hysterics... Michael. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Processing passwords etc.
On 11 Apr 2014, at 10:26, Michael Van Canneyt wrote: > OTOH, I think people are hugely exaggerating the problem, considering it was > introduced relatively recently and that I got my security update before it > hit the newspapers. The exploit code was also on github before news about the bug hit the newspapers. There is even some evidence it may have been exploited for at least 3 months already, maybe longer (because unless you used some special intrusion detection system rules, it left no traces at all in the log files, so there's only very little data to go on). Also, the fact that you updated your server so quickly, doesn't mean that everyone did. Our university's mail servers were only patched yesterday morning (more than 24 hours after the story broke), because they needed time to prepare the patching (don't ask, I don't know the details). I bet tons of credentials and private data has been accessed over the past days all over the world. > That is of course not to say that it shouldn't be fixed and people shouldn't > bother. > But the way it is presented is more about scaring people than anything else. > Hysterics... I very strongly disagree. All certificates and login data used with vulnerable services over the past year or so should be considered compromised. It will probably take months before all affected certificates are replaced (if that ever happens for most of them), and many of the replaced and hence potentially compromised certificates will probably never be revoked. The result is a huge increase in chances for man-in-the-middle attacks, not to mention all the compromised login data and private information (emails, bank statements, ...). Jonas ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Processing passwords etc.
On Fri, 11 Apr 2014, Jonas Maebe wrote: On 11 Apr 2014, at 10:26, Michael Van Canneyt wrote: OTOH, I think people are hugely exaggerating the problem, considering it was introduced relatively recently and that I got my security update before it hit the newspapers. That is of course not to say that it shouldn't be fixed and people shouldn't bother. But the way it is presented is more about scaring people than anything else. Hysterics... I very strongly disagree. All certificates and login data used with vulnerable services over the past year or so should be considered compromised. It will probably take months before all affected certificates are replaced (if that ever happens for most of them), and many of the replaced and hence potentially compromised certificates will probably never be revoked. The result is a huge increase in chances for man-in-the-middle attacks, not to mention all the compromised login data and private information (emails, bank statements, ...). Like I said, this is not to say that no action should be taken. I expect that all sensitive sites (banks, google, etc) have taken immediate action. That the login of my local tennis/pool/golf club was compromised is not really so scary, sorry. Anyway, getting off topic. The main point is that in FPC you can install a memory manager that wipes out any memory when getting or releasing it, if you want to make your software more secure that way. Michael. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
[fpc-pascal] https support; call for testers
Hello, I've just committed support for SSL in the ssockets unit of FPC. I also made the OpenSSL unit more thread-safe. One consequence of this is that the fphttpclient unit now has support for the https:// protocol. I have tested on windows and unix, the support for client-side SSL support works. Server-side support (as could be done in the embedded httpserver) is not yet tested. I would like to invite people to test and report if they find problems or missing features. That this support is released around the same time as the heartbleed leak issue, is entirely coincidental and definitely not intended. Michael. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] https support; call for testers
On 11/04/2014 15:00, Michael Van Canneyt wrote: > One consequence of this is that the fphttpclient unit now has support > for the https:// protocol. > I have tested on windows and unix, the support for client-side SSL > support works. > > I would like to invite people to test and report if they find problems > or missing features. Thanks a lot - I'll test client side encryption against a CGI application on Apache... almost posted this morning asking whether support was upcoming ;) > That this support is released around the same time as the heartbleed > leak issue, > is entirely coincidental and definitely not intended. Of course. I'll just make sure to use the proper openssl version when traversing the big bad internet ;) ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] https support; call for testers
On Fri, 11 Apr 2014, Reinier Olislagers wrote: On 11/04/2014 15:00, Michael Van Canneyt wrote: One consequence of this is that the fphttpclient unit now has support for the https:// protocol. I have tested on windows and unix, the support for client-side SSL support works. I would like to invite people to test and report if they find problems or missing features. Thanks a lot - I'll test client side encryption against a CGI application on Apache... almost posted this morning asking whether support was upcoming ;) Well, I committed just in time then :) That this support is released around the same time as the heartbleed leak issue, is entirely coincidental and definitely not intended. Of course. I'll just make sure to use the proper openssl version when traversing the big bad internet ;) People that want to be 100% safe should pull out the internet cable, and go sit in a Faraday cage. Watch 'enemy of the state' if you need convincing ;) Michael. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Processing passwords etc.
On 4/11/2014 5:03 AM, Michael Van Canneyt wrote: The main point is that in FPC you can install a memory manager that wipes out any memory when getting or releasing it, if you want to make your software more secure that way. how would one go about doing this? i learned in my TP3/6 days to use fillchar on everything to ensure that it was filled with 0x00... especially my data files... when looking at them with a hex editor, my OCD would hit strongly because the data file was "not clean and holding only my data"... i don't know how one would go about cleaning released memory as someone else asked about (eg: extending an array or string or etc)... once the memory is released, it is no longer accessible, right? -- NOTE: No off-list assistance is given without prior approval. Please keep mailing list traffic on the list unless private contact is specifically requested and granted. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Processing passwords etc.
On 4/11/2014 4:10 AM, Mark Morgan Lloyd wrote: Using a memory manager would reliably wipe strings etc. when reallocated (i.e rather than when an assignment didn't trigger reallocation). On the other hand it would have the overhead of also overwriting blocks that the user knew were being freed and could audit first, as well as stuff that was being freed as part of the RTL operation. It's that latter case- where the RTL is copying something without the user being aware- that I think is significant. agreed... -- NOTE: No off-list assistance is given without prior approval. Please keep mailing list traffic on the list unless private contact is specifically requested and granted. ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
[fpc-pascal] Watching for file change
Hi, Does FPC have some file watching solution? I need this only for linux. I need notifications: - File/folder changed / removed / added in watched directory. I have idea how to do this in thread loop but maybe FPC has OS solution Regards ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] https support; call for testers
Hi, Στις 11/4/2014 4:00 μμ, ο/η Michael Van Canneyt έγραψε: I've just committed support for SSL in the ssockets unit of FPC. I also made the OpenSSL unit more thread-safe. I would like to invite people to test and report if they find problems or missing features. the "Writeln(pchar(@buffer));" at line 417 in sslsockets i assume is a leftover from debugging ? regards, -- Dimitrios Chr. Ioannidis smime.p7s Description: Κρυπτογραφημένη υπογραφή S/MIME ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Watching for file change
On 4/11/2014 3:06 PM, Krzysztof wrote:
Hi,
Does FPC have some file watching solution? I need this only for linux.
I need notifications:
- File/folder changed / removed / added in watched directory. I have
idea how to do this in thread loop but maybe FPC has OS solution
i'm interested in this, as well... i have an app in perl that i'm considering
rewriting in FPC... in that app, i had to specifically grab the attributes of
the files the app watches and compare those values to previously stored values...
in perl:
if ($check_file eq $alert_file) {# If we're testing the alert
file and
($dev,$ino,$mode,$nlink,$uid,$gid,$rdev,$size,$atime,$mtime,$ctime,$blksize,$blocks)
= stat($check_file);
if ($size < $prev_alert_size) { # the filesize is less than
last we checked,
&write_log (2,"Alert filename changed. Reopening $alert_file");
close (ALERT); # we need to close and
open (ALERT, "$alert_file"); # reopen it.
$prev_alert_size = $size;
} else {
$prev_alert_size = $size;
}
}
hopefully there is something similar to perl's stat() routine...
--
NOTE: No off-list assistance is given without prior approval.
Please keep mailing list traffic on the list unless
private contact is specifically requested and granted.
___
fpc-pascal maillist - fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] https support; call for testers
Michael Van Canneyt<http://www.mail-archive.com/search?l=fpc-pascal@lists.freepascal.org&q=from:%22Michael+Van+Canneyt%22> Fri, 11 Apr 2014 06:01:08 -0700<http://www.mail-archive.com/search?l=fpc-pascal@lists.freepascal.org&q=date:20140411> > > > Hello, > > I've just committed support for SSL in the ssockets unit of FPC. > I also made the OpenSSL unit more thread-safe. > > One consequence of this is that the fphttpclient unit now has support > for the https:// protocol. > > I have tested on windows and unix, the support for client-side SSL support > works. > Server-side support (as could be done in the embedded httpserver) is not yet > tested. > > > I would like to invite people to test and report if they find problems or > missingfeatures. > > > That this support is released around the same time as the heartbleed leak > issue, > is entirely coincidental and definitely not intended. > > Michael. > > Thanks a lot Michael! I'll test it in my new TDropbox class! -- Silvio Clécio My public projects - github.com/silvioprog ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Processing passwords etc.
waldo kitty wrote: On 4/11/2014 5:03 AM, Michael Van Canneyt wrote: The main point is that in FPC you can install a memory manager that wipes out any memory when getting or releasing it, if you want to make your software more secure that way. how would one go about doing this? i learned in my TP3/6 days to use fillchar on everything to ensure that it was filled with 0x00... especially my data files... when looking at them with a hex editor, my OCD would hit strongly because the data file was "not clean and holding only my data"... The ideal is to overwrite sensitive data with random bytes, since even the length of a zero block can be useful to an attacker. i don't know how one would go about cleaning released memory as someone else asked about (eg: extending an array or string or etc)... once the memory is released, it is no longer accessible, right? But since the deallocated memory is going to a local heap, sooner or later you're likely to get that back as a new block. That, as I understand it, is what happened in OpenSSL. The worst case would be if a cautious programmer zeroed everything that he was freeing explicitly, without realising that any strings he extended were going back into the heap intact so now stood out like a sore thumb. Anybody who was able to inspect the heap would see only strings that had subsequently been expanded: password := getFromUser(); // Probably about 7 chars password += #$00 + systemName();// Leaves password on heap saveToDB(Tiger2(password)); zeroString(password)// Length doesn't change end; // Zeroed block freed to heap -- Mark Morgan Lloyd markMLl .AT. telemetry.co .DOT. uk [Opinions above are the author's, not those of his employers or colleagues] ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Watching for file change
On 11 Apr 2014 20:07, "Krzysztof" wrote: > > Hi, > > Does FPC have some file watching solution? I need this only for linux. > I need notifications: > - File/folder changed / removed / added in watched directory. I have > idea how to do this in thread loop but maybe FPC has OS solution You can use the inotify* functions in linux unit. See the man pages or Wikipedia for how they work. Henry ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Watching for file change
waldo kitty wrote: On 4/11/2014 3:06 PM, Krzysztof wrote: Hi, Does FPC have some file watching solution? I need this only for linux. I need notifications: - File/folder changed / removed / added in watched directory. I have idea how to do this in thread loop but maybe FPC has OS solution i'm interested in this, as well... i have an app in perl that i'm considering rewriting in FPC... in that app, i had to specifically grab the attributes of the files the app watches and compare those values to previously stored values... I believe Linux has dnotify, inotify and possibly File Alteration Monitor, but that these are version-dependant and extremely dependant on distro policies. Because of this uncertainty it would be necessary to load the support libraries dynamically. -- Mark Morgan Lloyd markMLl .AT. telemetry.co .DOT. uk [Opinions above are the author's, not those of his employers or colleagues] ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Watching for file change
See: http://www.freepascal.org/~michael/articles/dirwatch/dirwatch.pdf -- Silvio Clécio My public projects - github.com/silvioprog ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Watching for file change
2014-04-11 17:40 GMT-03:00 silvioprog : > See: http://www.freepascal.org/~michael/articles/dirwatch/dirwatch.pdf > With sources and demos: http://www.freepascal.org/~michael/articles/ -- Silvio Clécio My public projects - github.com/silvioprog ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Watching for file change
On 11 Apr 2014 21:39, "Mark Morgan Lloyd" < markmll.fpc-pas...@telemetry.co.uk> wrote: > > waldo kitty wrote: >> >> On 4/11/2014 3:06 PM, Krzysztof wrote: >>> >>> Hi, >>> >>> Does FPC have some file watching solution? I need this only for linux. >>> I need notifications: >>> - File/folder changed / removed / added in watched directory. I have >>> idea how to do this in thread loop but maybe FPC has OS solution >> >> >> i'm interested in this, as well... i have an app in perl that i'm considering rewriting in FPC... in that app, i had to specifically grab the attributes of the files the app watches and compare those values to previously stored values... > > > I believe Linux has dnotify, inotify and possibly File Alteration Monitor, but that these are version-dependant and extremely dependant on distro policies. Because of this uncertainty it would be necessary to load the support libraries dynamically. inotify replaced dnotify many years ago (both are in kernel, so no libraries needed). I don't know what FAM is useful for. Henry ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Re: [fpc-pascal] Watching for file change
Thanks to all for tips. They are very helpfull. I'll test it, wait for feedback :) ___ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
