[
https://issues.apache.org/jira/browse/IGNITE-27311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Tupitsyn updated IGNITE-27311:
------------------------------------
Description:
DataStreamer assumes that key columns come before value columns in the schema,
which is not always true.
Streaming into a table like this will mix up some column values:
{code:sql}
CREATE TABLE REPRODUCE_BUG (
name VARCHAR,
data VARCHAR,
uniqueId VARCHAR,
PRIMARY KEY (name, uniqueId)
)
{code}
Reproducer:
{code:java}
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
@Builder
public static class BugKey {
String uniqueId;
String name;
}
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
@Builder
public static class BugValue {
String data;
}
@Test
@SneakyThrows
void reproduceBug() {
SubmissionPublisher<DataStreamerItem<Map.Entry<BugKey, BugValue>>>
publisher = new SubmissionPublisher<>();
CompletableFuture<Void> streamFeature;
var kv = igniteClient
.tables()
.table("REPRODUCE_BUG")
.keyValueView(BugKey.class, BugValue.class);
var streaming = List.of(
Map.entry(BugKey.builder().name("name1").uniqueId("id1").build(),
BugValue.builder().data("data1").build()),
Map.entry(BugKey.builder().name("name2").uniqueId("id2").build(),
BugValue.builder().data("data2").build())
);
try (publisher) {
streamFeature = kv.streamData(publisher, new
DataStreamerOptions.Builder().pageSize(1000).build());
streaming.forEach(entry ->
publisher.submit(DataStreamerItem.of(entry)));
}
streamFeature.join();
}
{code}
was:
DataStreamer assumes that key columns come before value columns in the schema,
which is not always true.
Streaming into a table like this will mix up some column values:
{code:sql}
CREATE TABLE REPRODUCE_BUG (
name VARCHAR,
data VARCHAR,
uniqueId VARCHAR,
PRIMARY KEY (name, uniqueId)
)
{code}
> KeyValueView DataStreamer mixes up column values
> ------------------------------------------------
>
> Key: IGNITE-27311
> URL: https://issues.apache.org/jira/browse/IGNITE-27311
> Project: Ignite
> Issue Type: Bug
> Components: data streamer ai3
> Reporter: Pavel Tupitsyn
> Assignee: Pavel Tupitsyn
> Priority: Critical
> Labels: ignite-3
> Fix For: 3.2
>
>
> DataStreamer assumes that key columns come before value columns in the
> schema, which is not always true.
> Streaming into a table like this will mix up some column values:
> {code:sql}
> CREATE TABLE REPRODUCE_BUG (
> name VARCHAR,
> data VARCHAR,
> uniqueId VARCHAR,
> PRIMARY KEY (name, uniqueId)
> )
> {code}
> Reproducer:
> {code:java}
> @Getter
> @Setter
> @NoArgsConstructor
> @AllArgsConstructor
> @Builder
> public static class BugKey {
> String uniqueId;
> String name;
> }
> @Getter
> @Setter
> @NoArgsConstructor
> @AllArgsConstructor
> @Builder
> public static class BugValue {
> String data;
> }
> @Test
> @SneakyThrows
> void reproduceBug() {
> SubmissionPublisher<DataStreamerItem<Map.Entry<BugKey, BugValue>>>
> publisher = new SubmissionPublisher<>();
> CompletableFuture<Void> streamFeature;
> var kv = igniteClient
> .tables()
> .table("REPRODUCE_BUG")
> .keyValueView(BugKey.class, BugValue.class);
> var streaming = List.of(
> Map.entry(BugKey.builder().name("name1").uniqueId("id1").build(),
> BugValue.builder().data("data1").build()),
> Map.entry(BugKey.builder().name("name2").uniqueId("id2").build(),
> BugValue.builder().data("data2").build())
> );
> try (publisher) {
> streamFeature = kv.streamData(publisher, new
> DataStreamerOptions.Builder().pageSize(1000).build());
> streaming.forEach(entry ->
> publisher.submit(DataStreamerItem.of(entry)));
> }
> streamFeature.join();
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
