Hi, Thanks for the suggestion. I will take a look at Blob Handler.
On Thu, Aug 31, 2017 at 1:28 PM, Archana Ravindran <srarchan...@gmail.com> wrote: > You can try using blob handler in http source. Though blob is for binary, > we were able to push relatively large event. > > For size limit, in our case we were not able to find default support in > flume. So we customized the code with a small function for checking the > size. > You can try similar. > > Thanks, > Archana. > > > On Aug 31, 2017 12:29 PM, "Muhammad Yaseen" <yaseenaftab...@gmail.com> > wrote: > > Hello, > > I am using an http source for my Flume agent and would like to know if > there a size limit (explicit or implicit) on POST request body/data > (content length) for Flume's HTTP Source ? > > I have a handler which is throwing* "MalformedJsonException: Unterminated > string at line 1 column 966657*" (number varies) . I know for a fact that > the JSON is correct, for some reason the Handler is dropping the POST body. > This leads to *"HTTPBadRequestException: Request has invalid JSON > Syntax."* > > The source receives a huge request with an array of JSON objects as events > (10000-30000 events per request body). > This problem occurs when the content length ( request.getContentLength() ) > is large e.g. 2142675 but works fine when the length is smaller, e.g. 38320. > > Has anyone experienced this ? > > I am interested in the reason causing this problem. > > Regards, > Muhammad Yaseen > > On Thu, Aug 31, 2017 at 2:45 PM, Muhammad Yaseen <yaseenaftab...@gmail.com> wrote: > Hi, > > Thanks for the suggestion. I will take a look at Blob Handler. > > On Thu, Aug 31, 2017 at 1:28 PM, Archana Ravindran <srarchan...@gmail.com> > wrote: > >> You can try using blob handler in http source. Though blob is for binary, >> we were able to push relatively large event. >> >> For size limit, in our case we were not able to find default support in >> flume. So we customized the code with a small function for checking the >> size. >> You can try similar. >> >> Thanks, >> Archana. >> >> >> On Aug 31, 2017 12:29 PM, "Muhammad Yaseen" <yaseenaftab...@gmail.com> >> wrote: >> >> Hello, >> >> I am using an http source for my Flume agent and would like to know if >> there a size limit (explicit or implicit) on POST request body/data >> (content length) for Flume's HTTP Source ? >> >> I have a handler which is throwing* "MalformedJsonException: >> Unterminated string at line 1 column 966657*" (number varies) . I know >> for a fact that the JSON is correct, for some reason the Handler is >> dropping the POST body. This leads to *"HTTPBadRequestException: Request >> has invalid JSON Syntax."* >> >> The source receives a huge request with an array of JSON objects as >> events (10000-30000 events per request body). >> This problem occurs when the content length ( request.getContentLength() >> ) is large e.g. 2142675 but works fine when the length is smaller, >> e.g. 38320. >> >> Has anyone experienced this ? >> >> I am interested in the reason causing this problem. >> >> Regards, >> Muhammad Yaseen >> >> >> >
