Alberne opened a new issue, #9476: URL: https://github.com/apache/seatunnel/issues/9476
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What happened In some scenarios, I need to set the Content-Type header to "application/json;utf-8". Therefore, I manually added it in the configuration file's headers. Consequently, the code logic should first check whether it already exists to avoid duplicate additions. bug code snippet: we should check if already exists HTTP.CONTENT_TYPE to avoid duplicate additions. ``` .... } else { // we should check if already exists HTTP.CONTENT_TYPE to avoid duplicate additions. request.addHeader(HTTP.CONTENT_TYPE, APPLICATION_JSON); StringEntity entity = new StringEntity(JsonUtils.toJsonString(body), ContentType.APPLICATION_JSON); entity.setContentEncoding(new BasicHeader(HTTP.CONTENT_TYPE, APPLICATION_JSON)); request.setEntity(entity); } ..... ``` complete code snippet： ``` private void addBody(HttpEntityEnclosingRequestBase request, Map<String, Object> body) throws UnsupportedEncodingException { if (MapUtils.isEmpty(body)) { body = new HashMap<>(); } boolean isFormSubmit = request.getHeaders(HTTP.CONTENT_TYPE) != null && request.getHeaders(HTTP.CONTENT_TYPE).length > 0 && APPLICATION_FORM.equalsIgnoreCase( request.getHeaders(HTTP.CONTENT_TYPE)[0].getValue()); if (isFormSubmit) { if (MapUtils.isNotEmpty(body)) { List<NameValuePair> parameters = new ArrayList<>(); Set<Map.Entry<String, Object>> entrySet = body.entrySet(); for (Map.Entry<String, Object> e : entrySet) { String name = e.getKey(); String value = e.getValue().toString(); NameValuePair pair = new BasicNameValuePair(name, value); parameters.add(pair); } // Set to the request's http object request.setEntity(new UrlEncodedFormEntity(parameters, ENCODING)); } } else { request.addHeader(HTTP.CONTENT_TYPE, APPLICATION_JSON); StringEntity entity = new StringEntity(JsonUtils.toJsonString(body), ContentType.APPLICATION_JSON); entity.setContentEncoding(new BasicHeader(HTTP.CONTENT_TYPE, APPLICATION_JSON)); request.setEntity(entity); } } ``` the code detail: https://github.com/apache/seatunnel/blob/a7675d1782d82db5424c4f0db362663cbc1e3ff3/seatunnel-connectors-v2/connector-http/connector-http-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/http/client/HttpClientProvider.java#L492C13-L492C68 ### SeaTunnel Version 2.3.11 ### SeaTunnel Config ```conf env { parallelism = 1 job.mode = "BATCH" } source { Http { plugin_output = "http" url = "http://127.0.0.1:5000/api/cursor_data"; method = "GET" format = "json" content_field = "$.data.*" keep_page_param_as_http_param = true pageing = { page_type="Cursor" cursor_field ="cursor" cursor_response_field="$.paging.cursors.next" } header ={ Content-Type=application/json;utf-8 } schema = { fields { content=string id=int name=string } } json_field = { content = "$.data[*].content" id = "$.data[*].id" name = "$.data[*].name" } } } # Console printing of the read Http data sink { Console { parallelism = 1 } } ``` ### Running Command ```shell Assuming config file is `http_paging.config` ${SEATUNNEL_HOME}/bin/seatunnel.sh --deploy-mode local -c http_paging.config ``` ### Error Exception ```log Although no explicit errors occur, certain APIs require the exact Content-Type: application/json;charset=utf-8 header. Since the HTTP signature validation also depends on this specific value, I must configure it directly in the settings file. ``` ### Zeta or Flink or Spark Version _No response_ ### Java or Scala Version _No response_ ### Screenshots _No response_ ### Are you willing to submit PR? - [x] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
