[SPARK STREAMING ] Sending data to ElasticSearch

[Nipun Arora]

Hi,

I am sending data to an elasticsearch deployment. The printing to file
seems to work fine, but I keep getting no-node found for ES when I send
data to it. I suspect there is some special way to handle the connection
object? Can anyone explain what should be changed here?

Thanks
Nipun

The following is the code block where I send the data




addMTSUnmatched.foreach(
        new Function<JavaRDD<String>, Void>() {
            @Override
        public Void call(JavaRDD<String> stringJavaRDD) throws Exception {
                List<String> list = stringJavaRDD.collect();
                for (String str : list){
                    if(OnlineUtils.ESFlag) {
                        OnlineUtils.printToFile(str, 1,
type1_outputFile, OnlineUtils.client);
                    }else{
                        OnlineUtils.printToFile(str, 1, type1_outputFile);
                    }
                }
                return null;
            }
        }
);


public static void printToFile(String str, int type, File fileID,
Client client) throws IOException {
    JSONObject obj = new JSONObject(str);
    if (obj.has("message")) {
        String message = obj.get("message").toString();
        obj.put("message", message.replace("'", ""));
    }
    obj.put("anomaly_type", type);

    ESUtils.putMessage(client,"necla","demo",obj.toString());

    Files.append(str + "\n", fileID, Charset.defaultCharset());
}