----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35445/ -----------------------------------------------------------
Review request for samza. Repository: samza Description ------- SAMZA-693: Very basic HDFS Producer service for Samza Diffs ----- build.gradle a5f54106a822dc91ff82270df27217a8765a0d80 samza-hdfs/src/main/scala/org/apache/samza/system/hdfs/HdfsConfig.scala PRE-CREATION samza-hdfs/src/main/scala/org/apache/samza/system/hdfs/HdfsSystemAdmin.scala PRE-CREATION samza-hdfs/src/main/scala/org/apache/samza/system/hdfs/HdfsSystemFactory.scala PRE-CREATION samza-hdfs/src/main/scala/org/apache/samza/system/hdfs/HdfsSystemProducer.scala PRE-CREATION samza-hdfs/src/main/scala/org/apache/samza/system/hdfs/HdfsSystemProducerMetrics.scala PRE-CREATION samza-hdfs/src/test/org/apache/samza/system/hdfs/TestHdfsSystemProducer.scala PRE-CREATION samza-hdfs/src/test/resources/samza-hdfs-test-job.properties PRE-CREATION settings.gradle bb07a3b84b14dcef94da1bb166eab6aa3d0026bb Diff: https://reviews.apache.org/r/35445/diff/ Testing ------- New unit test, but it's fairly rudimentary. Passes "./gradlew test" and "./gradlew check" This only supplies an HDFS Producer, and this producer only writes SequenceFiles of ByteWriteables so far. If the patch were accepted as-is, I'd suggest future tickets for a matching HDFS Consumer, and a pluggable set of output formats, configurable via HdfsConfig settings. On the upside, this patch has been tested on a real cluster with real data, using several serdes, with good results. Thanks, Eli Reisman
