> On Feb. 25, 2014, 6:22 p.m., Jun Rao wrote: > > Could we extend ProducerSendTest to test sending/reading compressed > > messages?
Yep > On Feb. 25, 2014, 6:22 p.m., Jun Rao wrote: > > clients/src/main/java/org/apache/kafka/common/record/CompressedMemoryRecords.java, > > lines 72-73 > > <https://reviews.apache.org/r/18299/diff/4/?file=502748#file502748line72> > > > > Does it really do the rewinding? Ack > On Feb. 25, 2014, 6:22 p.m., Jun Rao wrote: > > clients/src/main/java/org/apache/kafka/common/record/CompressedMemoryRecords.java, > > lines 74-76 > > <https://reviews.apache.org/r/18299/diff/4/?file=502748#file502748line74> > > > > Not sure we need to do this since we don't expect any other exceptions. If we do not catch IOException and wrap with KafkaException, we need to declare throwables in the function signature, and any functions that call it. > On Feb. 25, 2014, 6:22 p.m., Jun Rao wrote: > > clients/src/main/java/org/apache/kafka/common/record/CompressionType.java, > > lines 61-71 > > <https://reviews.apache.org/r/18299/diff/4/?file=502749#file502749line61> > > > > Could we just catch Exception? Ack > On Feb. 25, 2014, 6:22 p.m., Jun Rao wrote: > > clients/src/main/java/org/apache/kafka/clients/producer/internals/RecordAccumulator.java, > > lines 145-155 > > <https://reviews.apache.org/r/18299/diff/4/?file=502744#file502744line145> > > > > When this happens, should we close the RecordBatch immediately after > > this record is appended? Talked with Jay about closing a record batch, and it is actually not an issue. Since the appending into a MemoryRecords and the getting of a MemoryRecords from a dequeue is sycnchronized, appending and writing to socket will not happen at the same time. - Guozhang ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18299/#review35422 ----------------------------------------------------------- On Feb. 24, 2014, 9:34 p.m., Guozhang Wang wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/18299/ > ----------------------------------------------------------- > > (Updated Feb. 24, 2014, 9:34 p.m.) > > > Review request for kafka. > > > Bugs: KAFKA-1253 > https://issues.apache.org/jira/browse/KAFKA-1253 > > > Repository: kafka > > > Description > ------- > > Incorporated Jun's comments. > > In-place compression with Crc32 pre-computation. > > In-place decompression embedded in nested iterator. > > Dynamic class loader for Snappy. > > TODOs: > > 1. Mix CompressedMemoryRecords into MemoryRecords. > 2. Integration Test with GZIP. > 3. Class loader for Snappy. > > > Diffs > ----- > > clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java > e4bc97279585818860487a39a93b6481742b91db > clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java > d8e35e7d0e4cd27aad9a8d4bf14bc97458da9417 > > clients/src/main/java/org/apache/kafka/clients/producer/internals/RecordAccumulator.java > ce5cf27efa08b79e501439cf79bc8666054a5429 > > clients/src/main/java/org/apache/kafka/clients/producer/internals/RecordBatch.java > eb16f6d236e07b16654623606294a051531b5f58 > > clients/src/main/java/org/apache/kafka/common/record/ByteBufferInputStream.java > PRE-CREATION > > clients/src/main/java/org/apache/kafka/common/record/ByteBufferOutputStream.java > PRE-CREATION > > clients/src/main/java/org/apache/kafka/common/record/CompressedMemoryRecords.java > PRE-CREATION > clients/src/main/java/org/apache/kafka/common/record/CompressionType.java > 906da02d02c03aadd8ab73ed2fc9a1898acb8d72 > clients/src/main/java/org/apache/kafka/common/record/MemoryRecords.java > 9d8935fa3beeb2a78b109a41ed76fd4374239560 > clients/src/main/java/org/apache/kafka/common/record/Record.java > f1dc9778502cbdfe982254fb6e25947842622239 > clients/src/main/java/org/apache/kafka/common/utils/Utils.java > 9c34e7dc82f33df7406cad0e64eb6a896d068dc6 > clients/src/test/java/org/apache/kafka/common/record/MemoryRecordsTest.java > b0745b528cef929c4273f7e2ac4de1476cfc25ad > clients/src/test/java/org/apache/kafka/common/record/RecordTest.java > ae54d67da9907b0a043180c7395a1370b3d0528d > clients/src/test/java/org/apache/kafka/common/utils/CrcTest.java > PRE-CREATION > clients/src/test/java/org/apache/kafka/test/TestUtils.java > 36cfc0fda742eb501af2c2c0330e3f461cf1f40c > > Diff: https://reviews.apache.org/r/18299/diff/ > > > Testing > ------- > > > Thanks, > > Guozhang Wang > >
