Initial Shim library up
https://github.com/apache/hadoop-api-shim
much of this relates to testing; we need a module to test a common set of
tests on every supported hadoop build, with validation that expected
features are present, and that the same operations work when both
reflection and fall bac
On Thu, 1 Jun 2023 at 18:46, Owen O'Malley wrote:
> For the case in question, ORC can extend its shim module to do exactly
> this. Let me take a look at the async ORC patch to see what needs to
> happen.
>
> BTW, Steve I'll be at Buzzwords for your talk.
>
ooh, cool. it's always good to have
For the case in question, ORC can extend its shim module to do exactly
this. Let me take a look at the async ORC patch to see what needs to
happen. BTW, Steve I'll be at Buzzwords for your talk.
It may make sense to have such a library, but ORC already has that
mechanism.
.. Owen
On Thu, Jun 1,
That sounds good
On Thu, 1 Jun 2023 at 18:09, Steve Loughran
wrote:
> hadoop-api-shim ?
>
> On Thu, 1 Jun 2023 at 04:07, Ayush Saxena wrote:
>
> > +1, for the new repo.
> >
> > The name sounds fine, but good if we have scope of having “hadoop-”
> > prefix, we have that for almost all of the sub
hadoop-api-shim ?
On Thu, 1 Jun 2023 at 04:07, Ayush Saxena wrote:
> +1, for the new repo.
>
> The name sounds fine, but good if we have scope of having “hadoop-”
> prefix, we have that for almost all of the subprojects/modules
>
> Can hadoop-shims or hadoop-shims-api or something on similar lin
+1, for the new repo.
The name sounds fine, but good if we have scope of having “hadoop-”
prefix, we have that for almost all of the subprojects/modules
Can hadoop-shims or hadoop-shims-api or something on similar lines work?
-Ayush
> On 01-Jun-2023, at 1:18 AM, Steve Loughran
> wrote:
>
> I
I want to create a new repository to put a shim library to allow previous
releases to access the more recent hadoop filesystem APIs -currently the
open source implementations of parquet, ORC can't use vectored io, in
particular, even though we can in Cloudera. Providing a shim opens them up
to all
