Zhang Hechuan created FLINK-37265:
-
Summary: document of python elasticsearch connector exeample has a
mistake
Key: FLINK-37265
URL: https://issues.apache.org/jira/browse/FLINK-37265
Project: Flink
Ahmed Hamdy created FLINK-36651:
---
Summary: Fix CI tests for ElasticSearch Connector
Key: FLINK-36651
URL: https://issues.apache.org/jira/browse/FLINK-36651
Project: Flink
Issue Type: Bug
Mingliang Liu created FLINK-35424:
-
Summary: Elasticsearch connector 8 supports SSL context
Key: FLINK-35424
URL: https://issues.apache.org/jira/browse/FLINK-35424
Project: Flink
Issue Type
Weijie Guo created FLINK-35416:
--
Summary: Weekly CI for ElasticSearch connector failed to compile
Key: FLINK-35416
URL: https://issues.apache.org/jira/browse/FLINK-35416
Project: Flink
Issue
Mingliang Liu created FLINK-35287:
-
Summary: Builder builds NetworkConfig for Elasticsearch connector 8
Key: FLINK-35287
URL: https://issues.apache.org/jira/browse/FLINK-35287
Project: Flink
Zhongqiang Gong created FLINK-35181:
---
Summary: CLONE - Bump org.apache.commons:commons-compress from
1.24.0 to 1.26.1 for Flink Elasticsearch connector
Key: FLINK-35181
URL: https://issues.apache.org/jira
Mingliang Liu created FLINK-34369:
-
Summary: Elasticsearch connector supports SSL provider
Key: FLINK-34369
URL: https://issues.apache.org/jira/browse/FLINK-34369
Project: Flink
Issue Type
Weijie Guo created FLINK-33635:
--
Summary: ElasticSearch connector main branch can not compile
Key: FLINK-33635
URL: https://issues.apache.org/jira/browse/FLINK-33635
Project: Flink
Issue Type
Yuxin Tan created FLINK-33493:
-
Summary: Elasticsearch connector ElasticsearchWriterITCase test
failed
Key: FLINK-33493
URL: https://issues.apache.org/jira/browse/FLINK-33493
Project: Flink
Jing Ge created FLINK-33195:
---
Summary: ElasticSearch Connector should directly depend on
3rd-party libs instead of flink-shaded repo
Key: FLINK-33195
URL: https://issues.apache.org/jira/browse/FLINK-33195
Yuxin Tan created FLINK-32194:
-
Summary: Elasticsearch connector should remove the dependency on
flink-shaded
Key: FLINK-32194
URL: https://issues.apache.org/jira/browse/FLINK-32194
Project: Flink
Fang Yong created FLINK-31801:
-
Summary: Missing elasticsearch connector on maven central
repository
Key: FLINK-31801
URL: https://issues.apache.org/jira/browse/FLINK-31801
Project: Flink
Weijie Guo created FLINK-31263:
--
Summary: Add smoke test for ElasticSearch connector
Key: FLINK-31263
URL: https://issues.apache.org/jira/browse/FLINK-31263
Project: Flink
Issue Type: Sub-task
nvolved!
>
> On Thu, Nov 10, 2022 at 12:51 PM Chesnay Schepler
> wrote:
>
>> The Apache Flink community is very happy to announce the release of
>> Apache Flink Elasticsearch Connector 3.0.0.
>>
>> Apache Flink® is an open-source stream processing framewo
with the first externalized connector for Flink. Thanks a lot to
> all of you involved!
>
> On Thu, Nov 10, 2022 at 12:51 PM Chesnay Schepler <mailto:ches...@apache.org>> wrote:
> The Apache Flink community is very happy to announce the release of
> Apache Flink Elasticse
Really happy with the first externalized connector for Flink. Thanks a lot
to all of you involved!
On Thu, Nov 10, 2022 at 12:51 PM Chesnay Schepler
wrote:
> The Apache Flink community is very happy to announce the release of
> Apache Flink Elasticsearch Connector 3.0.0.
>
> Apache
The Apache Flink community is very happy to announce the release of
Apache Flink Elasticsearch Connector 3.0.0.
Apache Flink® is an open-source stream processing framework for
distributed, high-performing, always-available, and accurate data
streaming applications.
The release is available
Chesnay Schepler created FLINK-29972:
Summary: Pin Flink docs to Elasticsearch Connector 3.0.0
Key: FLINK-29972
URL: https://issues.apache.org/jira/browse/FLINK-29972
Project: Flink
I'm happy to announce that we have unanimously approved this release.
There are 3 approving votes, 3 of which are binding:
* Danny
* Martijn
* Chesnay
There are no disapproving votes.
Thanks everyone!
On 08/11/2022 11:09, Martijn Visser wrote:
+1 (binding)
- Downloaded artifacts
- Checked ha
+1 (binding)
- Downloaded artifacts
- Checked hash and signature
- No binaries in source archive found
- Verified NOTICE files
- Build from source code
- Spinned up a Flink 1.16 cluster with Kafka and Elasticsearch7 and
used flink-sql-connector-elasticsearch7-3.0.0.jar to successfully sent data
fr
+1 (non-binding)
- checked signatures
- checked tag
- checked LICENSE/NOTICE files
- no binaries in source
- checked versions
On Mon, Nov 7, 2022 at 2:32 PM Danny Cranmer
wrote:
> +1 (binding)
>
> - Source release does not contain binaries
> - Source release builds and tests pass (when using
+1 (binding)
- Source release does not contain binaries
- Source release builds and tests pass (when using CI log4j config)
- Verified signature of source archive
- Source code tag exists
- pom versions look correct in the source release
- README is ok
- The staging repository [1] contains expecte
+1 (binding)
- correct Flink version in source release
- no -SNAPSHOT version in source release
- source release compiles and tests are passing
- (mvn verify
-Dlog4j.configurationFile=file://$(pwd)/tools/ci/log4j.properties)
- clean source release
- tag exists and points to correct commit
-
I remembered now; the test failure you ran into is because this
particular fails if you don't have logging enabled.
See FLINK-292789.
On 03/11/2022 21:25, Danny Cranmer wrote:
Hello Chesnay,
-1 (binding)
I do not approve the release due to the source release [1] containing
binaries, the full
I updated the source release on dist.apache.org.
On 03/11/2022 22:03, Chesnay Schepler wrote:
Good findings!
I thought the new scripts ensured a pristine source release, will look
into that tomorrow.
CI passed for the RC so I'd currently assume it's a local issue. But I
remember seeing that
Good findings!
I thought the new scripts ensured a pristine source release, will look
into that tomorrow.
CI passed for the RC so I'd currently assume it's a local issue. But I
remember seeing that particular error message on CI once in a while.
The archunit stored.rules files should not ha
Hello Chesnay,
-1 (binding)
I do not approve the release due to the source release [1] containing
binaries, the full /target folder is present for each module. Assuming we
missed a "mvn clean"
Other concerns/questions:
- I performed a fresh clone of the source and checked out the tag. The
build
Note that this _shouldn't_ be a problem since connector-base is part of
the distribution and should usually be loaded first than the version
bundled by the connector.
On 03/11/2022 11:28, Chesnay Schepler wrote:
* rely on binary compatibility between Flink /*patch */versions
On 03/11/2022 11:
* rely on binary compatibility between Flink /*patch */versions
On 03/11/2022 11:25, Chesnay Schepler wrote:
I've stumbled in an issue regarding flink-connector-base.
Since the externalized connectors rely on binary compatibility between
Flink minor versions, connector-base should never be bun
I've stumbled in an issue regarding flink-connector-base.
Since the externalized connectors rely on binary compatibility between
Flink minor versions, connector-base should never be bundled by an
externalized connector since connector-base may rely on internal APIs
(which ti is allowed to use
Hi everyone,
Please review and vote on the release candidate #2 for the version
3.0.0, as follows:
[ ] +1, Approve the release
[ ] -1, Do not approve the release (please provide specific comments)
The complete staging area is available for your review, which includes:
* JIRA release notes [1],
currently elasticsearch connector depends on jackson-bom 2.13.2.20220328
which has 2 CVEs CVE-2022-42003[1] CVE-2022-42004[2] fixed in
2.13.4.20221013 [3]
Does it make sense to include it in this version?
[1] https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-42003
[2] https://cve.mitre.org
Chesnay Schepler wrote:
> Yeah we should bump that to be closer to the connector version released
> with 1.16.0.
>
> On 02/11/2022 15:53, Sergey Nuyanzin wrote:
> > still checking
> > however there is at least one finding I would like to highlight
> > currently elas
Yeah we should bump that to be closer to the connector version released
with 1.16.0.
On 02/11/2022 15:53, Sergey Nuyanzin wrote:
still checking
however there is at least one finding I would like to highlight
currently elasticsearch connector depends on jackson-bom 2.13.2.20220328
which has 2
still checking
however there is at least one finding I would like to highlight
currently elasticsearch connector depends on jackson-bom 2.13.2.20220328
which has 2 CVEs CVE-2022-42003[1] CVE-2022-42004[2] fixed in
2.13.4.20221013 [3]
Does it make sense to include it in this version?
[1] https
Hi everyone,
Please review and vote on the release candidate #1 for the version
3.0.0, as follows:
[ ] +1, Approve the release
[ ] -1, Do not approve the release (please provide specific comments)
The complete staging area is available for your review, which includes:
* JIRA release notes [1],
lkhan Dadashov
wrote:
> Hello all,
>
> Want to help upgrade the Flink ElasticSearch connector to 8.X.X version.
>
> `elasticsearch-7` connector uses ElasticSearch `7.10.2` version.
>
>
> https://github.com/apache/flink-connector-elasticsearch/blob/main/flink-connector-elasticsea
Hello all,
Want to help upgrade the Flink ElasticSearch connector to 8.X.X version.
`elasticsearch-7` connector uses ElasticSearch `7.10.2` version.
https://github.com/apache/flink-connector-elasticsearch/blob/main/flink-connector-elasticsearch7/pom.xml#L39
When the ElasticSearch server side
Chesnay Schepler created FLINK-29263:
Summary: Remove ElasticSearch connector from apache/flink repo
Key: FLINK-29263
URL: https://issues.apache.org/jira/browse/FLINK-29263
Project: Flink
Niels Basjes created FLINK-28911:
Summary: Elasticsearch connector fails build
Key: FLINK-28911
URL: https://issues.apache.org/jira/browse/FLINK-28911
Project: Flink
Issue Type: Bug
Deepika Sahu created FLINK-28116:
Summary: Elasticsearch connector doesn't provide security through
token
Key: FLINK-28116
URL: https://issues.apache.org/jira/browse/FLINK-28116
Project:
Martijn Visser created FLINK-27757:
--
Summary: Elasticsearch connector should not use
flink-table-planner but flink-table-planner-loader
Key: FLINK-27757
URL: https://issues.apache.org/jira/browse/FLINK-27757
> Cheers,
> >
> > Konstantin
> >
> >
> >
> >
> > Am Mi., 11. Mai 2022 um 20:10 Uhr schrieb Martijn Visser <
> > martijnvis...@apache.org>:
> >
> >> Hi everyone,
> >>
> >> As mentioned in the previous update [1] we w
ioned in the previous update [1] we were working on the final step
for moving out the Elasticsearch connector, which was related to the
documentation integration. This has now been completed: the documentation
you see on
https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/datastr
isser <
martijnvis...@apache.org>:
> Hi everyone,
>
> As mentioned in the previous update [1] we were working on the final step
> for moving out the Elasticsearch connector, which was related to the
> documentation integration. This has now been completed: the documentat
Hi everyone,
As mentioned in the previous update [1] we were working on the final step
for moving out the Elasticsearch connector, which was related to the
documentation integration. This has now been completed: the documentation
you see on
https://nightlies.apache.org/flink/flink-docs-master
Martijn Visser created FLINK-27393:
--
Summary: Move Elasticsearch documentation from Flink repository to
the Flink Elasticsearch connector repository
Key: FLINK-27393
URL: https://issues.apache.org/jira/browse
Alexander Fedulov created FLINK-27142:
-
Summary: Remove bash tests dependencies on the Elasticsearch
connector.
Key: FLINK-27142
URL: https://issues.apache.org/jira/browse/FLINK-27142
Project
Jing Ge created FLINK-27061:
---
Summary: Improve the ArchUnit test infra in the external
elasticsearch connector
Key: FLINK-27061
URL: https://issues.apache.org/jira/browse/FLINK-27061
Project: Flink
Jing Ge created FLINK-27057:
---
Summary: Connector to external repo migration - Elasticsearch
Connector
Key: FLINK-27057
URL: https://issues.apache.org/jira/browse/FLINK-27057
Project: Flink
Issue
Jing Ge created FLINK-26884:
---
Summary: Move Elasticsearch connector to external connector
repository
Key: FLINK-26884
URL: https://issues.apache.org/jira/browse/FLINK-26884
Project: Flink
Issue
Alexander Preuss created FLINK-26634:
Summary: Update Chinese version of Elasticsearch connector docs
Key: FLINK-26634
URL: https://issues.apache.org/jira/browse/FLINK-26634
Project: Flink
Fabian Paul created FLINK-26633:
---
Summary: Elasticsearch connector does not report recordsSend metric
Key: FLINK-26633
URL: https://issues.apache.org/jira/browse/FLINK-26633
Project: Flink
Alexander Preuss created FLINK-26281:
Summary: Test Elasticsearch connector End2End
Key: FLINK-26281
URL: https://issues.apache.org/jira/browse/FLINK-26281
Project: Flink
Issue Type
Alexander Preuss created FLINK-25589:
Summary: Update Chinese version of Elasticsearch connector docs
Key: FLINK-25589
URL: https://issues.apache.org/jira/browse/FLINK-25589
Project: Flink
Yubin Li created FLINK-25563:
Summary: Support sink parallelism configuration to Elasticsearch
connector
Key: FLINK-25563
URL: https://issues.apache.org/jira/browse/FLINK-25563
Project: Flink
Adam Roberts created FLINK-25211:
Summary: Elasticsearch connector version compatibility
Key: FLINK-25211
URL: https://issues.apache.org/jira/browse/FLINK-25211
Project: Flink
Issue Type
Ada Wong created FLINK-25141:
Summary: Elasticsearch connector customize sink parallelism
Key: FLINK-25141
URL: https://issues.apache.org/jira/browse/FLINK-25141
Project: Flink
Issue Type
jacky jia created FLINK-24400:
-
Summary: does elasticsearch connector can suport
UpdateByQueryRequest?
Key: FLINK-24400
URL: https://issues.apache.org/jira/browse/FLINK-24400
Project: Flink
jinfeng created FLINK-24072:
---
Summary: Add support for setting default headers in elasticsearch
connector
Key: FLINK-24072
URL: https://issues.apache.org/jira/browse/FLINK-24072
Project: Flink
huxixiang created FLINK-23423:
-
Summary: Translate the page of "Elasticsearch Connector" into
Chinese
Key: FLINK-23423
URL: https://issues.apache.org/jira/browse/FLINK-23423
Proj
Xiang Yang created FLINK-22723:
--
Summary: multi-field-dynamic index is not supported by
elasticsearch connector
Key: FLINK-22723
URL: https://issues.apache.org/jira/browse/FLINK-22723
Project: Flink
jinfeng created FLINK-21068:
---
Summary: Add new timeout options for Elasticsearch connector
Key: FLINK-21068
URL: https://issues.apache.org/jira/browse/FLINK-21068
Project: Flink
Issue Type
xiaozilong created FLINK-21062:
--
Summary: Meeting NPE when using the dynamic Index in elasticsearch
connector
Key: FLINK-21062
URL: https://issues.apache.org/jira/browse/FLINK-21062
Project: Flink
wangsan created FLINK-20768:
---
Summary: Support routing field for Elasticsearch connector
Key: FLINK-20768
URL: https://issues.apache.org/jira/browse/FLINK-20768
Project: Flink
Issue Type
Peidian Li created FLINK-20564:
--
Summary: Add metrics for ElasticSearch connector
Key: FLINK-20564
URL: https://issues.apache.org/jira/browse/FLINK-20564
Project: Flink
Issue Type: Improvement
Oneway Chen created FLINK-20210:
---
Summary: "RequestFactory" interface in flink elasticsearch
connector doesn't support "routing" param
Key: FLINK-20210
URL: https://issues.apache.or
CloseRiver created FLINK-19943:
--
Summary: Support sink parallelism configuration to ElasticSearch
connector
Key: FLINK-19943
URL: https://issues.apache.org/jira/browse/FLINK-19943
Project: Flink
weizheng created FLINK-19428:
Summary: Translate "Elasticsearch Connector" page of "DataStream
Connectors" into Chinese
Key: FLINK-19428
URL: https://issues.apache.org/jira/browse/FLINK-19428
魏旭斌 created FLINK-19407:
---
Summary: Translate "Elasticsearch Connector" page of "Connectors"
into Chinese
Key: FLINK-19407
URL: https://issues.apache.org/jira/browse/FLINK-19407
Project: F
Shuiqiang Chen created FLINK-1:
--
Summary: Add ElasticSearch connector for Python DataStream API
Key: FLINK-1
URL: https://issues.apache.org/jira/browse/FLINK-1
Project: Flink
Shuiqiang Chen created FLINK-18887:
--
Summary: Add ElasticSearch connector for Python DataStream API
Key: FLINK-18887
URL: https://issues.apache.org/jira/browse/FLINK-18887
Project: Flink
Jun Qin created FLINK-18702:
---
Summary: Flink elasticsearch connector leaks threads and
classloaders thereof
Key: FLINK-18702
URL: https://issues.apache.org/jira/browse/FLINK-18702
Project: Flink
DeFOX created FLINK-18535:
-
Summary: Elasticsearch connector hangs for threads deadlocked
Key: FLINK-18535
URL: https://issues.apache.org/jira/browse/FLINK-18535
Project: Flink
Issue Type: Bug
Yangze Guo created FLINK-18361:
--
Summary: Support username and password options for new
Elasticsearch connector
Key: FLINK-18361
URL: https://issues.apache.org/jira/browse/FLINK-18361
Project: Flink
Hi Jacky Lau,
1) yes I saw that
2) I saw sources like IntegerSource which are bounded and which extend
RichParallelSourceFunction. This is why I mentioned it.
3) True, there is an hadoop ES connector in Beam but it is more of a
side connector, the main one is ElasticsearchIO here:
https://g
hi Jacky Lau :
I agree with jark's point of view, the use of es is not just to read data,
more use is to group query, aggregate these.
Best,
Forward
Jacky Lau 于2020年6月5日周五 下午2:47写道：
> hi Etienne Chauchot:
> you can read here https://www.jianshu.com/p/d32e17dab90c, which is
> chinese.But you ca
hi Etienne Chauchot:
you can read here https://www.jianshu.com/p/d32e17dab90c, which is
chinese.But you can konw that slice api has poor performance in es-hadoop
project .
And i found that es-hadoop has removed this and disable sliced scrolls by
default. you can see below, which i found in the las
> 5. DataStream source
> It would be nicer to expose a DataStream source too, and share
> implementations as much as possible.
>
>
> Best,
> Jark
>
>
> On Thu, 4 Jun 2020 at 22:07, Etienne Chauchot <
> echauchot@
> > wrote:
>
>> Hi,
>>
&g
hi Etienne Chauchot:
thanks for your discussion.
for 1) we do not supprt es unbouded source currently
for 2) RichParallelSourceFunction is used for streaming ,InputFormat is for
batch
for 3) i downloaded beam just now. and the beam es connector is also using
es-hadoop. i have read the code of e
wrote:
> Hi,
>
> I made the Elasticsearch connector of Apache Beam and I was thinking
> about doing the same for Flink when I came by this discussion. I have
> some comments regarding the design doc:
>
> 1. Streaming source:
>
> ES has data streams features but only for time
Hi,
I made the Elasticsearch connector of Apache Beam and I was thinking
about doing the same for Flink when I came by this discussion. I have
some comments regarding the design doc:
1. Streaming source:
ES has data streams features but only for time series data; the aim of
this source is
Hi, Jackey.
Thanks for driving this discussion. I think this proposal should be a
FLIP[1] since it impacts the public interface. However, as we have
only some preliminary discussions atm, a design draft would be ok. But
it would be better to organize your document according to [2].
I've two basic
Hi Robert Metzger:
Thanks for your response. could you please read this docs.
https://www.yuque.com/jackylau-sc7w6/bve18l/14a2ad5b7f86998433de83dd0f8ec067
. Any Is it any problem here? we are worried about
we do not think throughly. thanks.
--
Sent from: http://apache-flink-mailing-list-arc
Hi Jacky,
thanks a lot for starting the discussion. I have no objections to adding
support for reading data from ElasticSearch as well, as long as we clearly
state the performance and correctness implications / guarantees in the
Flink documentation.
On Tue, Jun 2, 2020 at 9:52 AM Jacky Lau wrote:
Hi all!
We have started some preliminary work on the flink elasticsearch integration
(es connector for es version7) at hikvision research institute.
It seems that the integration should think throughly. And we want to
contribute our code for the conmunity.
So I think I should open a discussion t
Danny Chen created FLINK-17841:
--
Summary: Add document for new ElasticSearch connector
Key: FLINK-17841
URL: https://issues.apache.org/jira/browse/FLINK-17841
Project: Flink
Issue Type: Sub
Jark Wu created FLINK-17832:
---
Summary: Add documentation for the new Elasticsearch connector
Key: FLINK-17832
URL: https://issues.apache.org/jira/browse/FLINK-17832
Project: Flink
Issue Type: Sub
Jark Wu created FLINK-17027:
---
Summary: Introduce a new Elasticsearch connector with new property
keys
Key: FLINK-17027
URL: https://issues.apache.org/jira/browse/FLINK-17027
Project: Flink
Issue
zhisheng created FLINK-16788:
Summary: ElasticSearch Connector SQL DDL add optional config (eg:
enable-auth/username/password)
Key: FLINK-16788
URL: https://issues.apache.org/jira/browse/FLINK-16788
Jiaqi Li created FLINK-16280:
Summary: Fix sample code errors in the documentation about
elasticsearch connector.
Key: FLINK-16280
URL: https://issues.apache.org/jira/browse/FLINK-16280
Project: Flink
hehuiyuan created FLINK-13810:
-
Summary: Update `Elasticsearch Connector` label
Key: FLINK-13810
URL: https://issues.apache.org/jira/browse/FLINK-13810
Project: Flink
Issue Type: Wish
Alberto Romero created FLINK-13519:
--
Summary: Elasticsearch Connector sample code for Scala on version
6.x will not work
Key: FLINK-13519
URL: https://issues.apache.org/jira/browse/FLINK-13519
Jark Wu created FLINK-12942:
---
Summary: Translate "Elasticsearch Connector" page into Chinese
Key: FLINK-12942
URL: https://issues.apache.org/jira/browse/FLINK-12942
Project: Flink
Issue
Hi,
I'm afraid you cannot write to different indices using the table API
ElasticSearch connector. Now I know why you wanted to go through
datastream API.
What you could do to transform from Row to JSON is to use
org.apache.flink.formats.json.JsonRowSerializationSchema from
flink-json. You
the indices here.
> > Is there a way to come around this ??
> >
> > Regards,
> > ~Ramya.
> >
> > On Thu, Jan 10, 2019 at 2:04 PM Dawid Wysakowicz >
> > wrote:
> >
> > > Hi Ramya,
> > >
> > > Have you tried writing to ES di
gt; >
> > Have you tried writing to ES directly from table API? You can check the
> > ES connector for table API here:
> >
> >
> https://ci.apache.org/projects/flink/flink-docs-release-1.7/dev/table/connect.html#elasticsearch-connector
> >
> > Best,
> &g
e API? You can check the
> ES connector for table API here:
>
> https://ci.apache.org/projects/flink/flink-docs-release-1.7/dev/table/connect.html#elasticsearch-connector
>
> Best,
>
> Dawid
>
> On 10/01/2019 09:21, Ramya Ramamurthy wrote:
> > Hi,
> >
> > I
Hi Ramya,
Have you tried writing to ES directly from table API? You can check the
ES connector for table API here:
https://ci.apache.org/projects/flink/flink-docs-release-1.7/dev/table/connect.html#elasticsearch-connector
Best,
Dawid
On 10/01/2019 09:21, Ramya Ramamurthy wrote:
> Hi,
>
Hi,
I am learning to Flink. With Flink 1.7.1, trying to read from Kafka and
insert to ElasticSearch. I have a kafka connector convert the data to a
Flink table. In order to insert into Elasticsearch, I have converted this
table to a datastream, in order to be able to use the ElasticSearchSink.
But
1 - 100 of 115 matches
Mail list logo
