On Tue, Jul 26, 2022 at 6:12 PM Chamikara Jayalath via dev < dev@beam.apache.org> wrote:
> > > On Mon, Jul 25, 2022 at 12:53 PM Lina Mårtensson via dev < > dev@beam.apache.org> wrote: > >> Hi dev, >> >> We're starting to incorporate BigTable in our stack and I've delighted >> my co-workers with how easy it was to create some BigTables with >> Beam... but there doesn't appear to be a reader for BigTable in >> Python. >> >> First off, is there a good reason why not/any reason why it would be >> difficult? >> > > There's was a previous effort to implement a Python BT source but that was > not completed: > https://github.com/apache/beam/pull/11295#issuecomment-646378304 > > >> >> I could write one, but before I start, I'd love some input to make it >> easier. >> >> It appears that there would be two options: either write one in >> Python, or try to set one up with x-language from Java which I see is >> done e.g. with the Spanner IO Connector. >> Any recommendation on which one to pick or potential pitfalls in either >> choice? >> >> If I write one in Python, what should I think about? >> It is not obvious to me how to achieve parallelization, so any tips >> here would be welcome. >> > > I would strongly prefer developing a Python wrapper for the existing Java > BT source using Beam's Multi-language Pipelines framework over developing a > new Python source. > > https://beam.apache.org/documentation/programming-guide/#multi-language-pipelines > This is the way. > > <https://beam.apache.org/documentation/programming-guide/#multi-language-pipelines> > > Thanks, > Cham > > > >> >> Thanks! >> -Lina >> >
