Re: [HACKERS] Re: CRC

2000-12-11 Thread Bruce Guenter
On Mon, Dec 11, 2000 at 10:09:01AM -0800, Mikheev, Vadim wrote: > > One thing we should look at before going with a 64-bit method is the > > extra storage space for the larger checksum. We can clearly afford > > an extra 32 bits for a checksum on an 8K disk page, but if Vadim is > > envisioning c

RE: [HACKERS] Re: CRC

2000-12-11 Thread Mikheev, Vadim
> One thing we should look at before going with a 64-bit method is the > extra storage space for the larger checksum. We can clearly afford > an extra 32 bits for a checksum on an 8K disk page, but if Vadim is > envisioning checksumming each individual XLOG record then the extra > space is more a

Re: [HACKERS] Re: CRC

2000-12-11 Thread Bruce Guenter
On Sun, Dec 10, 2000 at 02:36:38PM -0500, Tom Lane wrote: > > MD4 would be a better choice than MD5, despite that a theoretical attack > > on MD4 has been described (albeit never executed). We don't even care > > about real attacks, never mind theoretical ones. What matters is that > > MD4 is

Re: [HACKERS] Re: CRC

2000-12-11 Thread Thomas Lockhart
> Interesting. I was under the impression that virtually no RISC CPU had > a rotate instruction. Do any others? (fyi; doesn't really contribute to the thread :/ Most or all do. There are no "pure RISC" chips in production; all have had some optimized complex operations added for performance an

Re: [HACKERS] Re: CRC

2000-12-11 Thread Erich
PG should include support for SHA1 anyway. MD5 is not being used in new stuff anymore, I think. I actually have an SHA1 implementation that links into PG if anyone is interested (especially if it could get included in a future release). e

Re: [HACKERS] Re: CRC

2000-12-10 Thread Bruce Guenter
On Sun, Dec 10, 2000 at 02:53:43PM -0500, Tom Lane wrote: > > On my Celeron, the timing for those six opcodes is almost whopping 13 > > cycles per byte. Obviously there's some major performance hit to do the > > memory instructions, because there's no more than 4 cycles worth of > > dependant ins

Re: [HACKERS] Re: CRC

2000-12-10 Thread Bruce Guenter
On Sun, Dec 10, 2000 at 12:24:59PM -0800, Alfred Perlstein wrote: > I would try unrolling the loop some (if possible) and retesting. The inner loop was already unrolled, but was only processing single bytes at a time. By loading in 32-bit words at once, it reduced the cost to only 7 cycles per b

Re: [HACKERS] Re: CRC

2000-12-10 Thread Alfred Perlstein
* Tom Lane <[EMAIL PROTECTED]> [001210 12:00] wrote: > Bruce Guenter <[EMAIL PROTECTED]> writes: > >> A good theory, but unfortunately not a correct theory. PA-RISC can do a > >> circular shift in one cycle using the "shift right double" instruction, > > > Interesting. I was under the impressio

Re: [HACKERS] Re: CRC

2000-12-10 Thread Tom Lane
[EMAIL PROTECTED] (Nathan Myers) writes: > Minutiae aside, it's clear that the MD5 and CRC are "comparable", > regardless of CPU. We've established that the inner loops are pretty comparable. I'm still concerned about the startup/termination costs of MD5 on short records. The numbers Bruce and

Re: [HACKERS] Re: CRC

2000-12-10 Thread Tom Lane
Bruce Guenter <[EMAIL PROTECTED]> writes: >> A good theory, but unfortunately not a correct theory. PA-RISC can do a >> circular shift in one cycle using the "shift right double" instruction, > Interesting. I was under the impression that virtually no RISC CPU had > a rotate instruction. Do an

Re: [HACKERS] Re: CRC

2000-12-09 Thread Nathan Myers
On Sat, Dec 09, 2000 at 06:46:23PM -0500, Tom Lane wrote: > I'm at a loss to see how a Pentium would arrive at a better result for > MD5 than for CRC. For one thing, it's going to be at a disadvantage > because it hasn't got enough registers. I'd be interested to see the > assembly code... Minu

Re: [HACKERS] Re: CRC

2000-12-09 Thread Bruce Guenter
On Sat, Dec 09, 2000 at 06:46:23PM -0500, Tom Lane wrote: > Bruce Guenter <[EMAIL PROTECTED]> writes: > > The difference is likely because PA-RISC (like most other RISC > > architectures) lack a "roll" opcode that is very prevalent in the MD5 > > algorithm. > > A good theory, but unfortunately no

Re: [HACKERS] Re: CRC

2000-12-09 Thread Bruce Guenter
On Fri, Dec 08, 2000 at 10:17:00PM -0500, Tom Lane wrote: > A couple further observations while playing with this benchmark --- > > 1. This MD5 implementation is not too robust. On my machine it dumps > core if given a non-word-aligned data buffer. We could probably work > around that, but it b

Re: [HACKERS] Re: CRC

2000-12-09 Thread Tom Lane
Bruce Guenter <[EMAIL PROTECTED]> writes: >> This is a lot closer than I'd have expected, but it sure ain't >> "MD5 40% faster" as you reported. I wonder why the difference >> in results between your platform and mine? > The difference is likely because PA-RISC (like most other RISC > architectu

Re: [HACKERS] Re: CRC

2000-12-08 Thread Bruce Guenter
On Fri, Dec 08, 2000 at 09:28:38PM -0500, Tom Lane wrote: > Bruce Guenter <[EMAIL PROTECTED]> writes: > >> I agree, don't send it to the whole list. But I'd like a copy. > > Here you go. > As near as I could tell, the test as you have it (one CRC computation per > fread) is purely I/O bound. Nop

Re: [HACKERS] Re: CRC

2000-12-08 Thread Tom Lane
A couple further observations while playing with this benchmark --- 1. This MD5 implementation is not too robust. On my machine it dumps core if given a non-word-aligned data buffer. We could probably work around that, but it bespeaks a little *too* much hand optimization... 2. It's a bad idea

Re: [HACKERS] Re: CRC

2000-12-08 Thread Tom Lane
Bruce Guenter <[EMAIL PROTECTED]> writes: >> I agree, don't send it to the whole list. But I'd like a copy. > Here you go. As near as I could tell, the test as you have it (one CRC computation per fread) is purely I/O bound. I changed the main loop to this: int main() { static char buf[8192

Re: [HACKERS] Re: CRC

2000-12-08 Thread Tom Lane
Bruce Guenter <[EMAIL PROTECTED]> writes: > Would you like to see the simple benchmarking setup I used? The amount > of code involved (once all the hashes are factored in) is fairly large, > so I'm somewhat hesitant to just send it to the mailing list. I agree, don't send it to the whole list.

Re: [HACKERS] Re: CRC

2000-12-08 Thread Bruce Guenter
On Fri, Dec 08, 2000 at 04:30:58PM -0500, Tom Lane wrote: > Bruce Guenter <[EMAIL PROTECTED]> writes: > >> Are you really saying MD5 was faster than CRC-32? > > Yes. I expect it's because the operations used in MD5 are easily > > parallelized, and operate on blocks of 64-bytes at a time, while th

Re: [HACKERS] Re: CRC

2000-12-08 Thread Nathan Myers
On Fri, Dec 08, 2000 at 11:10:19AM -0800, I wrote: > Current evidence suggests that MD4 would be a good choice for a hash > algorithm. Thinking about it, I suspect that any CRC implementation that can't outrun MD5 by a wide margin is seriously sub-optimal. Can you post any more details about ho

Re: [HACKERS] Re: CRC

2000-12-08 Thread Tom Lane
[EMAIL PROTECTED] (Nathan Myers) writes: > Thinking about it, I suspect that any CRC implementation that can't outrun > MD5 by a wide margin is seriously sub-optimal. I was finding that hard to believe, too, at least for CRC-32 (CRC-64 would take more code, so I'm not so sure about it). Is that

Re: [HACKERS] Re: CRC

2000-12-08 Thread Bruce Guenter
On Fri, Dec 08, 2000 at 04:21:21PM -0500, Tom Lane wrote: > [EMAIL PROTECTED] (Nathan Myers) writes: > > Thinking about it, I suspect that any CRC implementation that can't outrun > > MD5 by a wide margin is seriously sub-optimal. > I was finding that hard to believe, too, at least for CRC-32 (CR

Re: [HACKERS] Re: CRC

2000-12-08 Thread Tom Lane
Bruce Guenter <[EMAIL PROTECTED]> writes: >> Are you really saying MD5 was faster than CRC-32? > Yes. I expect it's because the operations used in MD5 are easily > parallelized, and operate on blocks of 64-bytes at a time, while the CRC > is mostly non-parallelizable, uses a table lookup, and op

Re: [HACKERS] Re: CRC

2000-12-08 Thread Bruce Guenter
On Fri, Dec 08, 2000 at 11:10:19AM -0800, Nathan Myers wrote: > This is very interesting. MD4 is faster than MD5. (MD5, described as > "MD4 with suspenders on", does some extra stuff to protect against more- > obscure attacks, of no interest to us.) Which 64-bit CRC code did you > use, Mark M