Heya, Playing around with the 0.8.2-beta producer client. One of my test cases is to ensure producers can deal with Kafka being down when the producer is created. My tests failed miserably because of the default blocking in the producer with regard to metadata.fetch.timeout.ms. The first line of new producer is waitOnMetadata which is blocking.
I can handle this case by loading topic meta on init and setting the timeout value to very low metadata.fetch.timeout.ms and either throwing away messages or creating my own internal queue to buffer. I'm surprised the metasync isn't done async. If it fails, return that in the future/callback. This way the API could actually be considered safely async and the producer buffer could try to hold on to things until block.on.buffer.full kicks in. You'd probably need a partition callback since numPartitions wouldn't be available. The implication is that people's apps will work fine if first messages are sent while kafka server is up, however, if kafka is down and they restart their app, the new producer will block all sends and blow things up if you haven't written your app to be aware of this edge case. Thanks, Paul This email and any attachments may contain confidential and proprietary information of Blackboard that is for the sole use of the intended recipient. If you are not the intended recipient, disclosure, copying, re-distribution or other use of any of this information is strictly prohibited. Please immediately notify the sender and delete this transmission if you received this email in error.
