Aw: Re: Re: Split Kafka JSON String

[Daniela S]

Hi,

 

unfortunately it does not work. Do I need this step? Or is it also possible to extract the value of a specific field out of my JSON string?

So my JSON string is for example

  {"timestamp": "23462626", "ID1": "1", "ID2": "100", "command": "START"}

 

I would like to use a filter bolt to filter for START and END. So I have to access the command field within the JSON string.

 

Afterwards I would like to store the START JSONs in Redis. The key should be ID1 and ID2. Is that possible? What would be the best way to store it in Redis and to define two keys for each value?

 

The END JSONs should be deleted from Redis.

 

Thank you in advance.

 

Regards,

Daniela

 

 

Gesendet: Montag, 16. Mai 2016 um 14:08 Uhr
Von: "Henry Hottelet" <hotte...@gmail.com>
An: user@storm.apache.org
Betreff: Re: Re: Split Kafka JSON String

 

https://code.google.com/archive/p/json-simple/

 

import org.json.simple.JSONObject;

 

void main() {

String yourString;


String key;

JSONObject j = yourString;

 

Redis.put(key,j);

 

 

}

 










— 

Henry Hottelet

646-543-6104 (Google Voice: cell/sms)

hotte...@gmail.com (Email)

http://www.linkedin.com/in/hottelet (LinkedIn)

https://technologyventureslimited.appointlet.com (Schedule appointment)

 

 

This transmission may contain information that is privileged,

confidential, legally privileged, and/or exempt from disclosure

under applicable law. If you are not the intended recipient, you

are hereby notified that any disclosure, copying, distribution, or

use of the information contained herein (including any reliance

thereon) is STRICTLY PROHIBITED. Although this transmission and any

attachments are believed to be free of any virus or other defect

that might affect any computer system into which it is received and

opened, it is the responsibility of the recipient to ensure that it

is virus free and no responsibility is accepted by Henry Hottelet

as applicable, for any loss or damage arising in any way from its use.

If you received this transmission in error, please immediately contact

the sender and destroy the material in its entirety, whether in

electronic or hard copy format. Thank you.







 



On May 16, 2016, at 4:11 AM, Daniela S <daniela_4...@gmx.at> wrote:
 




Henry,

 

thank you for your help. I did not use any github link for my project. I tried to build the project on my own, with the help of some books and available code parts from the Internet. I try to explain what I am doing:

 

1. I have generated random JSON data with a JSON data generator from the Internet

    the JSON data looks as follows:

    {"timestamp": "nowTimestamp()",

      "ID1": "integer(1,1000)",

      "ID2": "integer(1,100)",

      "command": "random('START','END')"

   }

   The JSON generator simulates a data stream, sending the data to a specific Kafka topic as String

 

2. I would like to have a KafkaSpout to fetch the data from the Kafka topic.

 

3. Then I would like to store all the events containing "START" into Redis and to delete all the events containing "END" from Redis. The key should be a combination of ID1 and ID2.

 

4. In the next step I would like to pull all the entries stored in Redis and to add a specific value for every entry (this values is located in a database and I have to look it up for every entry). Then I would like to use these values to build a sum over all entries. This should be repeated every minute, as there might have been new start or end events. I thought I could use a sliding window for that.

 

But unfortunately I have no idea how to proceed with the JSON String, I receive from Kafka. I already read a lot about the possibility to store JSON Strings in Redis, but I have no idea how to build the key with both IDs and how to access the fields in the JSON String.

 

Hope you can help me.

 

Thank you so much in advance!

 

Regards,

Daniela 

 

 

 

Gesendet: Sonntag, 15. Mai 2016 um 22:07 Uhr
Von: "Henry Hottelet" <hotte...@gmail.com>
An: user@storm.apache.org
Betreff: Re: Aw: Re: Split Kafka JSON String



Daniella,

 

I am curious about your architecture with Kafka and storm.  Can you send the github link you used for that project?

 

I would like to be able to see your example. I use IntelliJ.

 

Henry


Sent from my iPad


On May 15, 2016, at 2:14 PM, Daniela S <daniela_4...@gmx.at> wrote:
 




Hi,

 

thank you very much for your answers. But unfortunately I do not have a JSON object yet, I only have a String, containing a JSON object. I receive the Strings from Kafka but I do not know how to get the information out of the String.

 

example: {"timestamp": 1463335828342, "ID":570,"command":"START"}

 

I would like to extract the three fields out of the String. Then I would like to store it into Redis.

Afterwards I would like to pull all the stored (active) devices out of Redis and to build a sum over all the devices every minute. For building the sum I have to add another value for every device. I would like to use Redis as a kind of cache.

 

Regards,

Daniela

 

Gesendet: Sonntag, 15. Mai 2016 um 20:06 Uhr
Von: "Henry Hottelet" <hotte...@gmail.com>
An: user@storm.apache.org
Betreff: Re: Split Kafka JSON String


Try using intellij plug in that converts json to pojo classes

On May 15, 2016 2:04 PM, "Nathan Leung" <ncle...@gmail.com> wrote:


Easiest way is to map your job to a pojo and use Jackson or gson to convert the json.

On May 15, 2016 1:48 PM, "Daniela S" <daniela_4...@gmx.at> wrote:




Hi

 

I am receiving Strings of JSON from Kafka. I would like to split the string to get each field from the JSON object to store it into Redis.

How can I split the string and assign the parts to variables?

 

Thank you in advance.

 

Regards,

Daniela