Hi Satendra, if I understand correctly you are using the RestSourceConnector to pull data into a Kafka topic and that seems to work - at least as far as the topic getting created.
But are you saying that you tried to read data with 'cat filename.log' on a file within your kafka data directory? I assume I misunderstood this part, can you please add some detail around what you did there? Best regards, Sönke On Thu, Nov 29, 2018 at 7:27 PM Satendra Pratap Singh <sameerp...@gmail.com> wrote: > Hi Sönke, > > I have single node kafka cluster, i have setup web server locally, many > apps are sending logs to this server. I have Rest API which send/receive > client request to/from server. > > here is my connector configuration : curl -i -X POST -H > "Accept:application/json" -H "Content-Type: application/json" > http://$CONNECT_HOST_SOURCE:8083/connectors/ > -d '{ "name": "source_rest_telemetry_data”, "config": { > "key.converter":"org.apache.kafka.connect.storage.StringConverter", > "value.converter":"org.apache.kafka.connect.storage.StringConverter", > "connector.class": "com.tm.kafka.connect.rest.RestSourceConnector", > "tasks.max": "1", "rest.source.poll.interval.ms": "900000", > "rest.source.method": "GET", "rest.source.url": " > http://abc.server.in:0000/api/Dia/getDiaData";, > "rest.source.payload.converter.class": > "com.tm.kafka.connect.rest.converter.StringPayloadConverter", > "rest.source.properties": > "Content-Type:application/json,Accept::application/json", > "rest.source.topic.selector": > "com.tm.kafka.connect.rest.selector.SimpleTopicSelector", > "rest.source.destination.topics": “telemetry_data” } }' > > After running this command connector runs and and create a topic called > telemetry_data. Inside telemetry data topic got 4 files with extension > called .log, .index, .timeindex and leader-epoch-checkpoint. > > i used cat command and open .log file and got following error : > > > [\!�qg_z��g_z�����������������(�"{"error":{"name":"Error","status":404,"message":"Shared > class \"Dia\" has no method handling GET > /getDia","statusCode":404,"stack":"Error: Shared class \"Dia\" has no > method handling GET /getDia\n at restRemoteMethodNotFound > > (/Users/satendra/Api/node_modules/loopback/node_modules/strong-remoting/lib/rest-adapter.js:371:17)\n > at Layer.handle [as handle_request] > > (/Users/satendra/Api/node_modules/loopback/node_modules/express/lib/router/layer.js:95:5)\n > at trim_prefix > > (/Users/satendra/Api/node_modules/loopback/node_modules/express/lib/router/index.js:317:13)\n > at > > /Users/satendra/Api/node_modules/loopback/node_modules/express/lib/router/index.js:284:7\n > at Function.process_params > > (/Users/satendra/Api/node_modules/loopback/node_modules/express/lib/router/index.js:335:12)\n > at next > > (/Users/satendra/Api/node_modules/loopback/node_modules/express/lib/router/index.js:275:10)\n > at Function.handle > > (/Users/satendra/Api/node_modules/loopback/node_modules/express/lib/router/index.js:174:3)\n > at router > > (/Users/satendra/Api/node_modules/loopback/node_modules/express/lib/router/index.js:47:12)\n > at Layer.handle [as handle_request] > > (/Users/satendra/Api/node_modules/loopback/node_modules/express/lib/router/layer.js:95:5)\n > at trim_prefix > > (/Users/satendra/Api/node_modules/loopback/node_modules/express/lib/router/index.js:317:13)\n > at > > /Users/satendra/Api/node_modules/loopback/node_modules/express/lib/router/index.js:284:7\n > at Function.process_params > > (/Users/satendra/Api/node_modules/loopback/node_modules/express/lib/router/index.js:335:12)\n > at next > > (/Users/satendra/Api/node_modules/loopback/node_modules/express/lib/router/index.js:275:10)\n > at jsonParser > > (/Users/satendra/analyticsApi/node_modules/body-parser/lib/types/json.js:109:7)\n > at Layer.handle [as handle_request] > > (/Users/satendra/analyticsApi/node_modules/loopback/node_modules/express/lib/router/layer.js:95:5)\n > at trim_prefix > > (/Users/satendra/analyticsApi/node_modules/loopback/node_modules/express/lib/router/index.js:317:13)"}}@access-control-allow-credentials["true"]connection["keep-alive"]content-typeF["application/json; > charset=utf-8"dateB["Thu, 29 Nov 2018 12:38:59 > > GMT"etagR["W/\"89a-6BIHHr4YrwjSfEHOpFjIThnXbCY\""]"transfer-encoding["chunked"vary6["Origin, > > Accept-Encoding"],x-content-type-options["nosniff"]$x-download-options["noopen"]x-frame-options["DENY"] > x-xss-protection"["1; mode=block"] > > but couldn't get data what looking for. but when i run curl -X POST > http://localhost:0000/api/dia/diadata by terminal/postman it's fetching > desired result. > > Please help me. > > looking > > On Thu, Nov 29, 2018 at 8:42 PM Sönke Liebau > <soenke.lie...@opencore.com.invalid> wrote: > > > Hi Satendra, > > > > feel free to share the error and I'll see if I can help. > > > > Best regards, > > Sönke > > > > > > Am Do., 29. Nov. 2018, 14:59 hat Satendra Pratap Singh < > > sameerp...@gmail.com> > > geschrieben: > > > > > Hi Sönke, > > > > > > Thanks for your response. The reason for asking about rest bcoz I m > stack > > > while pulling pulling request from rest api. I have been using Kafka > rest > > > source connect to pull the request, connected is pulling some error > > instead > > > actual data. > > > > > > Would you like to help me. I will share that err once you respond. > > > > > > Sent from my iPhone > > > > > > > On 29-Nov-2018, at > > > > > > > > > > 18:11, Sönke Liebau <soenke.lie...@opencore.com.INVALID> wrote: > > > > > > > > Hi Satendra, > > > > > > > > in addition to what Ryanne said, maybe a graphical REST tool would > > better > > > > suit your needs, there is a couple of good ones out there: > > > > Postman - https://www.getpostman.com/ > > > > Insomnia - https://insomnia.rest/ > > > > > > > > are two that I have used myself, but just googling for "rest gui" > > should > > > > turn up many alternatives. > > > > > > > > Best regards, > > > > Sönke > > > > > > > >> On Wed, Nov 28, 2018 at 5:06 PM Ryanne Dolan <ryannedo...@gmail.com > > > > > wrote: > > > >> > > > >> I think you might be looking for cURL? > > > >> > > > >> On Fri, Nov 23, 2018 at 10:28 AM Satendra Pratap Singh < > > > >> sameerp...@gmail.com> > > > >> wrote: > > > >> > > > >>> Hi Ryan, > > > >>> > > > >>> Thanks. Since I am new to Kafka don’t understand how to configures > > rest > > > >>> api and how to reconfigure connectors. In general I don’t know how > to > > > run > > > >>> GET /connector command and where to run. > > > >>> > > > >>> Please elaborate. > > > >>> > > > >>> Sent from my iPhone > > > >>> > > > >>>> On 23-Nov-2018, at 21:43, Ryanne Dolan <ryannedo...@gmail.com> > > wrote: > > > >>>> > > > >>>> Satendra, the Connect REST API enables starting, stopping, and > > > >>>> reconfiguring Connecters in a distributed Connect cluster. Kafka > > > >> Connect, > > > >>>> more generally, is a framework for connecting Kafka to external > data > > > >>>> sources, e.g. Change Data Capture from relational databases, etc. > > > >>>> > > > >>>> Ryanne > > > >>>> > > > >>>> On Fri, Nov 23, 2018, 6:48 AM Satendra Pratap Singh < > > > >>> sameerp...@gmail.com> > > > >>>> wrote: > > > >>>> > > > >>>>> Hi Team, > > > >>>>> > > > >>>>> Please click on this link : > > > >>>>> https://kafka.apache.org/documentation/#connect_rest > > > >>>>> > > > >>>>> I am kafka beginner recently started learning kafka. i want to > get > > my > > > >>>>> hands dirty in kafka. most of the topics have explained well but > i > > > >>>>> couldn't understand Rest API. what is used for? how to use it? > > > >>>>> all commands were listed under documentation i couldn't get that > > how > > > >>>>> to run them. if it's itself framework how to work on it. > > > >>>>> > > > >>>>> Please explain me. I am keen to learn about this. > > > >>>>> > > > >>>>> Looking forward to hear from you. > > > >>>>> > > > >>>>> > > > >>>>> Sent from my iPhone > > > >>>>> > > > >>> > > > >> > > > > > > > > > > > > -- > > > > Sönke Liebau > > > > Partner > > > > Tel. +49 179 7940878 > > > > OpenCore GmbH & Co. KG - Thomas-Mann-Straße 8 - 22880 Wedel - Germany > > > > > > -- Sönke Liebau Partner Tel. +49 179 7940878 OpenCore GmbH & Co. KG - Thomas-Mann-Straße 8 - 22880 Wedel - Germany
