[GitHub] nifi issue #2292: NIFI-4639: fresh writer for each output record

[markap14]

Github user markap14 commented on the issue:

    https://github.com/apache/nifi/pull/2292
  
    @joewitt I agree that this seems correct but have concerns about 
performance as well. I built this PR and then copied in the kafka-0.11 
processors from NiFi 1.4.0 in order to do a side-by-side comparison. The 
results are shown below (left-hand-side being the new one):
    
    ![screen shot 2017-12-07 at 1 33 41 
pm](https://user-images.githubusercontent.com/184268/33732137-6fdf77ee-db53-11e7-94e8-1640b516644b.png)
    
    So we can see that the performance dropped by at 15%. This was the JSON 
writer, specifically. Haven't tried with the Avro writer, but in any case we 
need to ensure that this has high performance for all cases.
    
    I wonder if a better option may be to add some sort of 
`rebind(OuptutStream)` method to RecordWriter, so that it would reset itself to 
write to a new Output Stream. This would avoid having to recreate potentially 
expensive objects for every single record. Then, instead of creating a new 
writer each time, we can just re-bind the existing writer to a new 
OutputStream, write the record, flush, and repeat.


---