I've not used the BigTable writer, but It should be just creating a
function that takes a MyClass and converts it to an
Iterable<bigtable.v2.Mutation>,
then calling that from a DoFn you create and outputting the resulting
Iterable to the OutputReceiver. Then your pipeline would be something like
pipeline.apply(CassandraIO.read().withTable("MyClass").withBlahblah)
.apply(new CustomDoFnToConvert())
.apply(BigTable.write())
*~Vincent*
On Tue, Feb 7, 2023 at 11:11 AM Adam Scott <adam.c.sc...@gmail.com> wrote:
> Thank you Vincent! Thanks for the quick response!
>
> The goal is to read from a Cassandra query and write to a Bigtable table.
>
> I've tried combining this technique, below, with the test examples you
> pointed out. But I am afraid my efforts were wholly naive.
>
> https://github.com/GoogleCloudPlatform/DataflowTemplates/blob/main/v1/src/main/java/com/google/cloud/teleport/bigtable/CassandraToBigtable.java
>
> It seems I would want, if I were dreaming, instead of a
> BeamRowToBigtableFn (
> https://github.com/GoogleCloudPlatform/DataflowTemplates/blob/cc799861a458eef329aa5a15189a8045109e43e0/v1/src/main/java/com/google/cloud/teleport/bigtable/CassandraToBigtable.java#L220)
> , a Beam<myClasss>toBigtableFn.
>
> Any hints on how to add a write/output to Bigtable?
>
> TIA,
> Adam
>
>
>
> On Fri, Feb 3, 2023 at 1:25 PM Vincent Marquez <vincent.marq...@gmail.com>
> wrote:
>
>> There are some examples in the test code that should be easy enough to
>> follow.
>>
>> Here is an example of just querying the entire table:
>>
>>
>> https://github.com/apache/beam/blob/master/sdks/java/io/cassandra/src/test/java/org/apache/beam/sdk/io/cassandra/CassandraIOTest.java#L460
>>
>> Here's an example of using readAll to only pull certain keys:
>>
>>
>> https://github.com/apache/beam/blob/master/sdks/java/io/cassandra/src/test/java/org/apache/beam/sdk/io/cassandra/CassandraIOTest.java#L381
>>
>>
>>
>>
>> *~Vincent*
>>
>>
>> On Fri, Feb 3, 2023 at 12:59 PM Adam Scott <adam.c.sc...@gmail.com>
>> wrote:
>>
>>> HI All,
>>>
>>> does anyone have an example of using CassandraIO to query a table?
>>>
>>> The following mentions "Alternatively, one may use
>>> CassandraIO.<Person>readAll()
>>> .withCoder(SerializableCoder.of(Person.class)) to query a subset of the
>>> Cassandra database by creating a PCollection of CassandraIO.Read<Person>
>>> each with their own query or RingRange."
>>>
>>> https://beam.apache.org/releases/javadoc/current/org/apache/beam/sdk/io/cassandra/CassandraIO.html
>>>
>>> Hoping there would be an example of this.
>>>
>>> TIA,
>>> Adam
>>>
>>>
>>>
>>>
