openSSL 0.9.7 and COMP_{zlib,rle}

[Andrew Marlow]

I would like to report a bug in openSSL version 0.9.7.
I cannot get on-the-fly compression to work using
SSL_COMP_add_compression_method. The performance
of the ssltest program is the same with or without
the -zlib command line option. My own program behaves
in a similar way. I have added trace to the ZLIB compress
function and the trace does not come out. By hacking the
code of s3_{clnt,srv} to force it into thinking that
compression is being used the trace statements DO get
executed, but just by calling SSL_COMP_add_compression_method
they do not.

I understand that as of 0.9.7 SSL_COMP_add_compression_method
is all that is needed (i.e no explicit negotiation is
needed unlike 0.9.6) but I do not think it is working
properly.

Regards,

Andrew Marlow.


__
OpenSSL Project http://www.openssl.org
User Support Mailing List[EMAIL PROTECTED]
Automated List Manager   [EMAIL PROTECTED]

Re: openSSL 0.9.7 and COMP_{zlib,rle}

[Richard Levitte - VMS Whacker]

In message <[EMAIL PROTECTED]> on Wed, 15 
Jan 2003 10:20:46 +, Andrew Marlow <[EMAIL PROTECTED]> said:

apm35> I would like to report a bug in openSSL version 0.9.7.
apm35> I cannot get on-the-fly compression to work using
apm35> SSL_COMP_add_compression_method. The performance
apm35> of the ssltest program is the same with or without
apm35> the -zlib command line option. My own program behaves
apm35> in a similar way. I have added trace to the ZLIB compress
apm35> function and the trace does not come out. By hacking the
apm35> code of s3_{clnt,srv} to force it into thinking that
apm35> compression is being used the trace statements DO get
apm35> executed, but just by calling SSL_COMP_add_compression_method
apm35> they do not.
apm35> 
apm35> I understand that as of 0.9.7 SSL_COMP_add_compression_method
apm35> is all that is needed (i.e no explicit negotiation is
apm35> needed unlike 0.9.6) but I do not think it is working
apm35> properly.

Actually, the way it's done in 0.9.7, there should be no need to call
SSL_COMP_add_compression_method() at all (i.e. there should be no need
at all for the -zlib flag in ssltest).

However, for this to work, you MUST configure with the options 'zlib'
or 'zlib-dynamic'.  Something like this:

  ./config zlib

If you don't, it's assumed you don't care about or don't have the z
library (same as if you gave the configuration option 'no-zlib').

If you want to trace the actual calls, you should do it in
crypto/comp/c_zlib.c.

I'm pondering making 'zlib-dynamic' the default instead of 'no-zlib'.
Does that sound like a good idea.  What it means is that the build
environment must have zlib.h reachable by the compiler.  We *could*
have a copy of the relevant lines from zlib.h as part of OpenSSL...

-- 
Richard Levitte   \ Spannvägen 38, II \ [EMAIL PROTECTED]
Redakteur@Stacken  \ S-168 35  BROMMA  \ T: +46-8-26 52 47
\  SWEDEN   \ or +46-708-26 53 44
Procurator Odiosus Ex Infernis-- [EMAIL PROTECTED]
Member of the OpenSSL development team: http://www.openssl.org/

Unsolicited commercial email is subject to an archival fee of $400.
See  for more info.
__
OpenSSL Project http://www.openssl.org
User Support Mailing List[EMAIL PROTECTED]
Automated List Manager   [EMAIL PROTECTED]

Re: openSSL 0.9.7 and COMP_{zlib,rle}

[Andrew Marlow]

[EMAIL PROTECTED] writes:
>In message
><[EMAIL PROTECTED]> on Wed,
>15 Jan 2003 10:20:46 +, Andrew Marlow <[EMAIL PROTECTED]> said:
>
>apm35> I would like to report a bug in openSSL version 0.9.7.
>apm35> I cannot get on-the-fly compression to work using
>apm35> SSL_COMP_add_compression_method. 
[snip]
>Actually, the way it's done in 0.9.7, there should be no need to call
>SSL_COMP_add_compression_method() at all (i.e. there should be no need
>at all for the -zlib flag in ssltest).

Maybe, but SSL_COMP_add_compression_method is used by ssltest and if
I go and do likewise then I should get the same result and the result
should be that on-the-fly compression is performed.

>
>However, for this to work, you MUST configure with the options 'zlib'
>or 'zlib-dynamic'.  Something like this:
>
>  ./config zlib

Indeed. If I don't then the macro ZLIB is not defined and
compilation chaos ensues. I *have* configured the build this
and for good measure I have hacked c_zlib.c to force on
the macros ZLIB and DEBUG_ZLIB. I have also added
additional trace statements. Use of ZLIB is definately on.
However, the compress/uncompress functions do not get called.
The variable 'j', used to denote whether the SSL message
has been compressed, is always set to zero. If you force it on
then the ZLIB trace is executed but this is an extreme hack.
The question is "why does ssl_lib decide that compression
is not required?".

>If you want to trace the actual calls, you should do it in
>crypto/comp/c_zlib.c.

Done that.

>
>I'm pondering making 'zlib-dynamic' the default instead of 'no-zlib'.
>Does that sound like a good idea.  

It does to me for a couple of reasons:

1) you can always turn it off, if you don't want it.
   Given how common zlib and dynamic libraries are
   these days it seems like a reasonable default to me.

2) Just because compression support is there doesn't
   mean you *have* to use it.

What it means is that the build
>environment must have zlib.h reachable by the compiler.  We *could*
>have a copy of the relevant lines from zlib.h as part of OpenSSL...

No. Sod's Law says the same information held in 2 different places
*will* be different. If we include the real zlib.h then we would
have to do what most automake/autoconf configuration procedures do,
and that is to check that zlib is available and fail to produce
Makefiles if it is missing (unless --zlib=no has been specified).

Regards,

Andrew Marlow.


__
OpenSSL Project http://www.openssl.org
User Support Mailing List[EMAIL PROTECTED]
Automated List Manager   [EMAIL PROTECTED]

RE: trouble compiling

[Patrick Best-TM]

what are you compiling with? (ver)

-Original Message-
From: Wayne Thomas [mailto:[EMAIL PROTECTED]]
Sent: January 10, 2003 8:33 PM
To: [EMAIL PROTECTED]
Subject: trouble compiling


I am attempting to compile openssl-0.9.7 on my Solaris 8 Sun Blade 100
with simply ./config and make. The following error occurs:

"/usr/ucbinclude/signal.h", line 49: syntax error before or at: int
"/usr/ucbinclude/signal.h", line 49: warning: undefined or missing type
for: int
*** Error code 2
make: Fatal error: Command failed for target `speed.o'
Current working directory /usr/local/src/openssl-0.9.7/apps
*** Error code 1
make: Fatal error: Command failed for target `sub_all'

Any suggestions on how to get past this?

thanks,
Wayne Thomas
__
OpenSSL Project http://www.openssl.org
User Support Mailing List[EMAIL PROTECTED]
Automated List Manager   [EMAIL PROTECTED]

__
OpenSSL Project http://www.openssl.org
User Support Mailing List[EMAIL PROTECTED]
Automated List Manager   [EMAIL PROTECTED]

ssltest and on-the-fly ZLIB compression

[Andrew Marlow]

This post is about my attempts to discover why
the ssltest program does not use compression when -zlib
is given on the command line. My openSSL is version 0.9.7
and was built via the command './Configure shared zlib'.
I have proved that compression does not occur.
I proved it via trace statements in z_clib.c.
The trace is never executed. The performance is
the same whether or not -zlib is given on the 
command line.

I have received a disappointing response to this query so far.
I am grateful that people have responded but the responses
have been along the lines of 'use 0.9.7' and 'make sure
you say 'Configure shared zlib'. Well I am and I have
but it doesn't work. What follows below is my attempt
to analyse the code. Hopefully this will shed light as
to what is wrong.

Compression happens if s3->tmp.new_compression 
points to a compression method structure.
It never does. There are FOUR files that use
this variable; s3_clnt, s3_srvr, s3_enc and t1_enc.
We will deal with each in turn:

s3_clnt
===
in ssl3_connect for the states SSL3_ST_CW_CHANGE_{A,B}
when there is a change of cipher spec. 

in ssl3_get_server_hello where it reads the compression
algorithm number set by the server. It looks up the algorithm
in its own structures if the incoming algorithm number is
non-zero. The server always sends zero.

s3_srvr
===
in ssl3_get_client_hello where it reads the compression
algorithm number set by the client.

In ssl3_send_server_hello after putting the cipher byte
it sends the compression algorithm byte if new_compression
is non-null (otherwise it sends zero). When it checks
new_compression it is null all the time.

s3_enc
==
It is examined in ssl3_change_cipher_state.
ssltest does not call this function but even if it did,
new_compression still has not been set.

In ssl3_setup_key_block it does do a lookup using
ssl_cipher_get_evp. It looks like this is what
should happen. However, ssl_setup_key_block
is not called! (in ssltest).

t1_enc
==
It is examined in tls1_change_cipher but
ssltest does not call this function.

IMO the ssltest program should end up calling either
ssl3_setup_key_block or ssl3_change_cipher_state.
I am not absolutely sure about this, since it does
not enter any state for which this would be the
correct thing to do, but it seems the only way
it is going to get the compression method established.

Please help me.

Regards,

Andrew M.


__
OpenSSL Project http://www.openssl.org
User Support Mailing List[EMAIL PROTECTED]
Automated List Manager   [EMAIL PROTECTED]

Re: ssltest and on-the-fly ZLIB compression

[Richard Levitte - VMS Whacker]

In message <[EMAIL PROTECTED]> on Wed, 15 
Jan 2003 17:55:05 +, Andrew Marlow <[EMAIL PROTECTED]> said:

apm35> I have received a disappointing response to this query so far.

Here's a less disappointing one: I'm looking into it.  I've observed
the same as you, that the compression methods aren't called...

I'll look through the rest of your mail later.

-- 
Richard Levitte   \ Spannvägen 38, II \ [EMAIL PROTECTED]
Redakteur@Stacken  \ S-168 35  BROMMA  \ T: +46-8-26 52 47
\  SWEDEN   \ or +46-708-26 53 44
Procurator Odiosus Ex Infernis-- [EMAIL PROTECTED]
Member of the OpenSSL development team: http://www.openssl.org/

Unsolicited commercial email is subject to an archival fee of $400.
See  for more info.
__
OpenSSL Project http://www.openssl.org
User Support Mailing List[EMAIL PROTECTED]
Automated List Manager   [EMAIL PROTECTED]

Re: ssltest and on-the-fly ZLIB compression

[Neil Nelson]

Andrew Marlow,

I executed zlib external to SSL which is fairly simple and
allows greater control over, e.g., the zlib compression level
parameter.


This post is about my attempts to discover why
the ssltest program does not use compression when -zlib
is given on the command line. My openSSL is version 0.9.7
and was built via the command './Configure shared zlib'.
I have proved that compression does not occur.
I proved it via trace statements in z_clib.c.
The trace is never executed. The performance is
the same whether or not -zlib is given on the 
command line.
 


// Sending before SSL
#define ZLIB_PACKET_MIN 1000
#define MAX_PACKET_SIZE 16000
#define PACKET_SIZE 15000

   // msg_state is the object carrying the packet to be sent.
   // First two packet bytes contain the packet length as an unsigned 
short.
   // Third packet byte (packet[2]) contains flags. One of those flags 
is compressed or not.
   if (msg_state->packet_bytes > ZLIB_PACKET_MIN)
   {
 uLongf 
bufzip_size=(uLongf)(((float)msg_state->packet_bytes-3)*1.1+12.5);
 char bufzip[(int)bufzip_size];
 int zlib_rslt=0;

 if ((zlib_rslt=compress2((Bytef*)bufzip, &bufzip_size,
  (Bytef*)msg_state->packet+3,
  (unsigned int)msg_state->packet_bytes-3,
  icsa_config.zlib_compression_level))
   != Z_OK)
 {
   string err_msg = "Error assemble_std_packet - Unable to 
compress2 - ";
   if (zlib_rslt == Z_ERRNO)
 err_msg += "Z_ERRNO";
   else if (zlib_rslt == Z_STREAM_ERROR)
 err_msg += "Z_STREAM_ERROR";
   else if (zlib_rslt == Z_DATA_ERROR)
 err_msg += "Z_DATA_ERROR";
   else if (zlib_rslt == Z_MEM_ERROR)
 err_msg += "Z_MEM_ERROR";
   else if (zlib_rslt == Z_BUF_ERROR)
 err_msg += "Z_BUF_ERROR";
   else if (zlib_rslt == Z_VERSION_ERROR)
 err_msg += "Z_VERSION_ERROR";
   else if (zlib_rslt == Z_STREAM_END)
 err_msg += "Z_STREAM_END";
   else if (zlib_rslt == Z_NEED_DICT)
 err_msg += "Z_NEED_DICT";
   else
 err_msg += "unknown error - zlib_rslt=" + itostr(zlib_rslt);

  icsa_log << err_msg << endl;
  // Continue without modification of packet.
 }

 // Require at least 95% compression.
 else if ( (float(bufzip_size) / float(msg_state->packet_bytes-3)) 
< 0.95)
 {
   msg_state->packet_bytes = bufzip_size+3;
   msg_state->packet_type |= ZIPPED;
   memcpy(msg_state->packet, &msg_state->packet_bytes, USHORT_SIZE);
   memcpy(msg_state->packet+USHORT_SIZE, &msg_state->packet_type, 1);
   memcpy(msg_state->packet+3, bufzip, bufzip_size);
 }
   }

 // Receiving after SSL
 if (packet_type & ZIPPED)
 {

   uLongf bufout_size = MAX_PACKET_SIZE;
   char bufout[MAX_PACKET_SIZE];
   int zlib_rslt=0;

   if ((zlib_rslt=uncompress((Bytef *)bufout, &bufout_size,
 (Bytef*)recv_skt->packet+3,
 (unsigned int)recv_skt->packet_len-3))
 != Z_OK)
   {
 string err_msg = "Error decode_packet - Unable to uncompress - ";
 if (zlib_rslt == Z_ERRNO)
   err_msg += "Z_ERRNO";
 else if (zlib_rslt == Z_STREAM_ERROR)
   err_msg += "Z_STREAM_ERROR";
 else if (zlib_rslt == Z_DATA_ERROR)
   err_msg += "Z_DATA_ERROR";
 else if (zlib_rslt == Z_MEM_ERROR)
   err_msg += "Z_MEM_ERROR";
 else if (zlib_rslt == Z_BUF_ERROR)
   err_msg += "Z_BUF_ERROR";
 else if (zlib_rslt == Z_VERSION_ERROR)
   err_msg += "Z_VERSION_ERROR";
 else if (zlib_rslt == Z_STREAM_END)
   err_msg += "Z_STREAM_END";
 else if (zlib_rslt == Z_NEED_DICT)
   err_msg += "Z_NEED_DICT";
 else
   err_msg += "unknown error - zlib_rslt=" + itostr(zlib_rslt);

 icsa_log << err_msg << endl;
 return 0;
   }
 }


__
OpenSSL Project http://www.openssl.org
User Support Mailing List[EMAIL PROTECTED]
Automated List Manager   [EMAIL PROTECTED]

Re: ssltest and on-the-fly ZLIB compression

[Richard Levitte - VMS Whacker]

In message <[EMAIL PROTECTED]> on Wed, 15 Jan 2003 10:26:59 -0800, Neil 
Nelson <[EMAIL PROTECTED]> said:

news_replies> I executed zlib external to SSL which is fairly simple and
news_replies> allows greater control over, e.g., the zlib compression level
news_replies> parameter.

Of course, one can do that.  But that has nothing to do with the
SSL/TLS protocols.

-- 
Richard Levitte   \ Spannvägen 38, II \ [EMAIL PROTECTED]
Redakteur@Stacken  \ S-168 35  BROMMA  \ T: +46-8-26 52 47
\  SWEDEN   \ or +46-708-26 53 44
Procurator Odiosus Ex Infernis-- [EMAIL PROTECTED]
Member of the OpenSSL development team: http://www.openssl.org/

Unsolicited commercial email is subject to an archival fee of $400.
See  for more info.
__
OpenSSL Project http://www.openssl.org
User Support Mailing List[EMAIL PROTECTED]
Automated List Manager   [EMAIL PROTECTED]

Re: ssltest and on-the-fly ZLIB compression

[Andrew Marlow]

[EMAIL PROTECTED] writes:
>In message
><[EMAIL PROTECTED]> on Wed,
>15 Jan 2003 17:55:05 +, Andrew Marlow <[EMAIL PROTECTED]> said:
>
>apm35> I have received a disappointing response to this query so far.
>
>Here's a less disappointing one: I'm looking into it.  I've observed
>the same as you, that the compression methods aren't called...

I have done some more investigation and have found that ssltest
will compress when the TLS1 protocol is explicitly selected.
So this could all be a mistake on my part. I assumed that
on-the-fly-compression was available with ssl version 3 and later.
According to RFC 2246 the compression code is an integral part
of the TLS v1 standard (although the numbers still have to be
formally agreed). It does not seem to be mentioned in the
original Netscape SSL v3 document. 

I note that the compression byte is sent when using the
SSL v3 protocol but the value is always zero.
The question is: should this byte be there and if so,
does that mean that openSSL is wrong not to set it when
SSL_COMP_add_compression_method has been called?


Regards,

Andrew M.



__
OpenSSL Project http://www.openssl.org
User Support Mailing List[EMAIL PROTECTED]
Automated List Manager   [EMAIL PROTECTED]

Re: ssltest and on-the-fly ZLIB compression

[Geoff Thorpe]

* Andrew Marlow ([EMAIL PROTECTED]) wrote:
> 
> I have done some more investigation and have found that ssltest
> will compress when the TLS1 protocol is explicitly selected.

I also took a look - it seems the problem is the v23 SSL/TLS method,
it's there to provide a handshake that can negotiate any protocol level,
but it also seems to preclude any negotiation of compression. Eg. if
you've built with "zlib", you can change into the apps/ directory and in
one shell run;

./openssl s_server

you'll find that (in another shell) both of the following result in
compression;

./openssl s_client -ssl3
./openssl s_client -tls1

but the following does not;

./openssl s_client -no_ssl2 -no_ssl3

As for why - this could be impossible to get around because of the
implicit constraints of SSLv2 compatibility, I'm not sure. Certainly if
you use the SSLv3 or TLSv1 client methods (and thus give up on talking
with any SSLv2 servers), then you'll probably be OK w.r.t. compression
unless you hit an SSLv2 server. The crap way to address this (something
Lutz mentioned in another thread) is to try connecting with an
SSLv3/TLSv1 method first and if that fails on protocol troubles, retry
with SSLv2. Yes I know, bleurgh.

Cheers,
Geoff

-- 
Geoff Thorpe
[EMAIL PROTECTED]
http://www.openssl.org/

__
OpenSSL Project http://www.openssl.org
User Support Mailing List[EMAIL PROTECTED]
Automated List Manager   [EMAIL PROTECTED]

Re: ssltest and on-the-fly ZLIB compression

[Andrew Marlow]

[EMAIL PROTECTED] writes:
>* Andrew Marlow ([EMAIL PROTECTED]) wrote:
>> 
>> I have done some more investigation and have found that ssltest
>> will compress when the TLS1 protocol is explicitly selected.
>
>I also took a look - it seems the problem is the v23 SSL/TLS method,
>it's there to provide a handshake that can negotiate any protocol level,
>but it also seems to preclude any negotiation of compression. 

This is what we find :-(
Running ssltest and explicitly selecting the protocol to be
other than sslv2 seems to work.

>As for why - this could be impossible to get around because of the
>implicit constraints of SSLv2 compatibility, I'm not sure. Certainly if
>you use the SSLv3 or TLSv1 client methods (and thus give up on talking
>with any SSLv2 servers), then you'll probably be OK w.r.t. compression
>unless you hit an SSLv2 server. 

I reckon you are right.

>The crap way to address this (something
>Lutz mentioned in another thread) is to try connecting with an
>SSLv3/TLSv1 method first and if that fails on protocol troubles, retry
>with SSLv2. Yes I know, bleurgh.
>
>Cheers,
>Geoff

Not ideal but it should work.
That's great Geoff, thanks for looking into this, my app now works!

Regards,

Andrew M.


__
OpenSSL Project http://www.openssl.org
User Support Mailing List[EMAIL PROTECTED]
Automated List Manager   [EMAIL PROTECTED]