You can utilize commons-httpclient and push result of RDD to the server. BTW specify commons-httpclient version which is consistent with the hadoop version you use. e.g. the following is brought in along with hadoop 2.6:
[INFO] | | | +- commons-httpclient:commons-httpclient:jar:3.1:compile Cheers On Mon, Aug 31, 2015 at 11:45 AM, Cassa L <lcas...@gmail.com> wrote: > Hi Ted, > My server is expecting JSON. Can I just write HttpClient in spark job > and push result of RDD action to the server? I'm trying to figure out how > to achieve this. > > LCassa > > On Fri, Aug 28, 2015 at 9:45 PM, Ted Yu <yuzhih...@gmail.com> wrote: > >> What format does your REST server expect ? >> >> You may have seen this: >> >> https://www.paypal-engineering.com/2014/02/13/hello-newman-a-rest-client-for-scala/ >> >> On Fri, Aug 28, 2015 at 9:35 PM, Cassa L <lcas...@gmail.com> wrote: >> >>> Hi, >>> If I have RDD that counts something e.g.: >>> >>> JavaPairDStream<String, Integer> successMsgCounts = successMsgs >>> .flatMap(buffer -> Arrays.asList(buffer.getType())) >>> .mapToPair(txnType -> new Tuple2<String, >>> Integer>("Success " + txnType, 1)) >>> .reduceByKey((count1, count2) -> count1 + count2); >>> >>> successMsgCounts.print(); >>> >>> Instead of printing this output how can I push it to REST API? I have a >>> server which needs this information to be fed via REST. >>> >>> >>> thanks >>> LCassa >>> >> >> >
