Hi,
I am having my client server on Windows. The server is concurrent and
having each thread for each connection. When the number of connection
increases to 400-500 i.e having high thread load, my server crashes. I
debuged it and found that it gives error (“random number
generator:FIPS_RAND:prng error") when it tries to invoke SSL_accept(). My
server is Fips compliant. I looked furthur inside openssl code and found
issue with fips_rand() method (fips/rand/fips_rand.c). Following is my
observation.
The error is generated at following point
fips_rand()
{
.............................
............................
if (!ctx->test_mode)
fips_get_dt(ctx);
AES_encrypt(ctx->DT, I, &ctx->ks);
for (i = 0; i < AES_BLOCK_LENGTH; i++)
tmp[i] = I[i] ^ ctx->V[i];
AES_encrypt(tmp, R, &ctx->ks);
for (i = 0; i < AES_BLOCK_LENGTH; i++)
tmp[i] = R[i] ^ I[i];
AES_encrypt(tmp, ctx->V, &ctx->ks);
/* Continuous PRNG test */
if (ctx->second)
{
if (fips_prng_fail){
memcpy(ctx->last, R, AES_BLOCK_LENGTH);
RANDerr(RAND_F_FIPS_RAND,RAND_
R_PRNG_STUCK);
}
if (!memcmp(R, ctx->last, AES_BLOCK_LENGTH))
<------------------------------------------- The check is failing as the
current encrypted and last one are same
{
RANDerr(RAND_F_FIPS_RAND,RAND_R_PRNG_STUCK);
ctx->error = 1;
fips_set_selftest_fail();
return 0;
}
}
memcpy(ctx->last, R, AES_BLOCK_LENGTH);
.......................................................................
......................................................................
}
I think under heavy load openssl continous PRNG test is failing. It might be
generating the same values as it applies AES encryption over the data taken
from fips_get_dt(ctx). For windows platform this function takes
GetSystemTimeAsFileTime(). like
..........
.........
#ifdef OPENSSL_SYS_WIN32
GetSystemTimeAsFileTime(&ft);
buf[0] = (unsigned char) (ft.dwHighDateTime & 0xff);
buf[1] = (unsigned char) ((ft.dwHighDateTime >> 8) & 0xff);
buf[2] = (unsigned char) ((ft.dwHighDateTime >> 16) & 0xff);
buf[3] = (unsigned char) ((ft.dwHighDateTime >> 24) & 0xff);
buf[4] = (unsigned char) (ft.dwLowDateTime & 0xff);
buf[5] = (unsigned char) ((ft.dwLowDateTime >> 8) & 0xff);
buf[6] = (unsigned char) ((ft.dwLowDateTime >> 16) & 0xff);
buf[7] = (unsigned char) ((ft.dwLowDateTime >> 24) & 0xff);
.........................
.........................
Please help in this regard. I am using openssl version 0.9.8o.
Regards,
Alok
