That's great. Any feedback (whether feature needs for your use cases / security / configurability) would be useful as more development time in spent on Flight this year
On Mon, Jan 7, 2019 at 6:24 PM Tim Bisson <biss...@gmail.com> wrote: > > I was finally able to build and run the flight-test-server/flight-test > binaries by hacking the arrow/flight/CMakeLists.txt to manually satisfy the > dependencies. > > I think part of my issue was trying to use a separate local grpc > installation to satisfy dependencies. > > I can now work through the test code to understand flight internals and > also work backward to get to an automated arrow flight build state. > > Thanks, > Tim > > On Mon, Jan 7, 2019 at 9:09 AM Wes McKinney <wesmck...@gmail.com> wrote: > > > Did you build gRPC using CMake or autotools (I was using CMake, so > > things are specialized to that for the moment)? As I said the current > > state of the toolchain configuration for gRPC was temporary to enable > > me to develop the initial unit tests and benchmarks. It'll be some > > time before this is production ready, so there's some > > blood/sweat/tears required here. There will hopefully be some progress > > on this in the first half of this year so keep an eye out. > > > > On Fri, Jan 4, 2019 at 12:26 PM Tim Bisson <biss...@gmail.com> wrote: > > > > > > Thanks Wes, > > > > > > Yea, we might be trying to play with the code a bit too early. > > > > > > I made a little more progress, but became stuck during make. I might be > > at > > > the same place as kszucs: > > > https://github.com/apache/arrow/pull/2547#issuecomment-425744800 > > > > > > This could be completely incorrect, but I made it past cmake by > > hard-coding > > > environment variables (such as GPR_STATIC_LIB) in > > ThirdPartyToolChain.cmake > > > because LOCATION wasn't defined. > > > - get_property(GPR_STATIC_LIB TARGET gRPC::gpr PROPERTY LOCATION) > > > + set(GPR_STATIC_LIB "/usr/local/lib/libgpr.a") > > > > > > However, now I fail on protobuf dependency: > > > [ 95%] Built target arrow-stream-to-file > > > make -f src/arrow/flight/CMakeFiles/arrow_flight_objlib.dir/build.make > > > src/arrow/flight/CMakeFiles/arrow_flight_objlib.dir/depend > > > make[2]: Entering directory '/home/t.bisson/git/arrow/cpp/debug' > > > make[2]: *** No rule to make target '../src/arrow/flight/protobuf', > > needed > > > by 'src/arrow/flight/Flight.pb.cc'. Stop. > > > make[2]: Leaving directory '/home/t.bisson/git/arrow/cpp/debug' > > > CMakeFiles/Makefile2:1450: recipe for target > > > 'src/arrow/flight/CMakeFiles/arrow_flight_objlib.dir/all' failed > > > make[1]: *** [src/arrow/flight/CMakeFiles/arrow_flight_objlib.dir/all] > > > Error 2 > > > make[1]: Leaving directory '/home/t.bisson/git/arrow/cpp/debug' > > > Makefile:143: recipe for target 'all' failed > > > > > > Google indirectly suggests header/c code generation might not be working. > > > However, I was able to generate the the files manually into their correct > > > directories. > > > > > > Any additional suggestions are appreciated. > > > > > > Thanks, > > > Tim > > > > > > > > > > > > > > > On Wed, Jan 2, 2019 at 7:28 PM Wes McKinney <wesmck...@gmail.com> wrote: > > > > > > > Hi Tim, > > > > > > > > This part of the codebase is a good ways from production-ready -- we > > aren't > > > > running the tests in CI for example. I put in enough hacks to be able > > to > > > > build using a local gRPC install that used CMake. Almost certainly a > > number > > > > of changes to FindGRPC.cmake will need to be made to get things > > working for > > > > more general installs of the libraries. > > > > > > > > The gRPC build system is a bit of a pain because it may install all of > > its > > > > third-party dependencies, possibly causing conflicts with other parts > > of > > > > the Arrow build system. > > > > > > > > The Flight RPC project is definitely a priority for 2019 so would be > > glad > > > > to have you involved, and we can try to help resolve the immediate > > build > > > > issues. > > > > > > > > Wes > > > > > > > > On Wed, Jan 2, 2019, 8:01 PM Tim Bisson <biss...@gmail.com wrote: > > > > > > > > > Hello All, > > > > > > > > > > Apologies in advance for the beginner question as I'm not versed in > > > > cmake. > > > > > > > > > > I'm having difficulties getting Arrow dependencies satisfied with > > the C++ > > > > > version (cmake -DARROW_FLIGHT=ON ..). > > > > > > > > > > Does anyone have any tips or suggestions that enable them to compile > > > > arrow > > > > > with flight on? > > > > > > > > > > I've compiled and installed grpc (and protobuf as a submodule under > > > > grpc). > > > > > I also believe I followed the directions under flight correctly: > > > > > export PROTOBUF_HOME=/home/t.bisson/git/grpc/third_party/protobuf > > > > > export LD_LIBRARY_PATH=$PROTOBUF_HOME:/usr/local/lib > > > > > cmake -DARROW_FLIGHT=ON .. > > > > > > > > > > All the libraries are under /usr/local/lib, like libgpr.a. > > > > > > > > > > Thanks, > > > > > Tim > > > > > > > > > > > > > > > The error I'm receiving is this: > > > > > Added static library dependency zstd_static: > > > > > /home/t.bisson/git/a/cpp/debug/zstd_ep-install/lib/libzstd.a > > > > > -- PROTOBUF_HOME: /home/t.bisson/git/grpc/third_party/protobuf > > > > > -- Found the Protobuf headers: > > > > > /home/t.bisson/git/grpc/third_party/protobuf/src > > > > > -- Found the Protobuf shared library: /usr/local/lib/libprotobuf.so > > > > > -- Found the Protobuf library: /usr/local/lib/libprotobuf.a > > > > > -- Found the Protoc shared library: /usr/local/lib/libprotoc.so > > > > > -- Found the Protoc library: /usr/local/lib/libprotoc.a > > > > > -- Found the Protoc executable: /usr/local/bin/protoc > > > > > Added static library dependency protobuf_static: > > > > > /usr/local/lib/libprotobuf.a > > > > > CMake Error at cmake_modules/ThirdpartyToolchain.cmake:1252 > > > > (get_property): > > > > > get_property could not find TARGET gRPC::gpr. Perhaps it has not > > yet > > > > > been > > > > > created. > > > > > Call Stack (most recent call first): > > > > > CMakeLists.txt:523 (include) > > > > > > > > > > > > > > > CMake Error at cmake_modules/BuildUtils.cmake:86 (message): > > > > > No static or shared library provided for grpc_gpr > > > > > Call Stack (most recent call first): > > > > > cmake_modules/ThirdpartyToolchain.cmake:1253 (ADD_THIRDPARTY_LIB) > > > > > CMakeLists.txt:523 (include) > > > > > > > > > > > > > > > > > > > > > > > > > > ---------------------------------------------------------------------------------------- > > > > > > > > > > The line in ThirdpartToolchain.cmake is: > > > > > > > > > > get_property(GPR_STATIC_LIB TARGET gRPC::gpr PROPERTY LOCATION) > > > > > ADD_THIRDPARTY_LIB(grpc_gpr > > > > > STATIC_LIB ${GPR_STATIC_LIB}) > > > > > > > > > > However, setting GRPC_HOME brought this set of errors: > > > > > > > > > > Added static library dependency protobuf_static: > > > > > /usr/local/lib/libprotobuf.a > > > > > CMake Error at cmake_modules/ThirdpartyToolchain.cmake:1248 > > > > (find_package): > > > > > Could not find a package configuration file provided by "gRPC" > > with any > > > > > of > > > > > the following names: > > > > > > > > > > gRPCConfig.cmake > > > > > grpc-config.cmake > > > > > > > > > > Add the installation prefix of "gRPC" to CMAKE_PREFIX_PATH or set > > > > > "gRPC_DIR" to a directory containing one of the above files. If > > "gRPC" > > > > > provides a separate development package or SDK, be sure it has been > > > > > installed. > > > > > Call Stack (most recent call first): > > > > > CMakeLists.txt:523 (include) > > > > > > > > > > Do you think I should continue down this path of trying to satisfy > > the > > > > > dependency? > > > > > ---------- > > > > > > > > > > contents of /usr/local/lib: > > > > > $ ls /usr/local/lib > > > > > libaddress_sorting.a libgrpc.a > > > > > libgrpc_cronet.so.7 libgrpcpp_channelz.so.1 > > > > > libgrpc++.so.1 libgrpc++_unsecure.so.1 > > > > > libprotobuf-lite.la libprotoc.la > > > > > libaddress_sorting.so libgrpc++.a > > > > > libgrpc_cronet.so.7.0.0-dev > > libgrpcpp_channelz.so.1.18.0-dev > > > > > libgrpc++.so.1.18.0-dev libgrpc++_unsecure.so.1.18.0-dev > > > > > libprotobuf-lite.so libprotoc.so > > > > > libaddress_sorting.so.7 libgrpc_cronet.a > > > > > libgrpc++_error_details.a libgrpc++_reflection.a > > > > > libgrpc.so.7 libgrpc_unsecure.so.7 > > > > > libprotobuf-lite.so.17 libprotoc.so.17 > > > > > libaddress_sorting.so.7.0.0-dev libgrpc++_cronet.a > > > > > libgrpc++_error_details.so libgrpc++_reflection.so > > > > > libgrpc.so.7.0.0-dev libgrpc_unsecure.so.7.0.0-dev > > > > > libprotobuf-lite.so.17.0.0 libprotoc.so.17.0.0 > > > > > libgpr.a libgrpc_cronet.so > > > > > libgrpc++_error_details.so.1 libgrpc++_reflection.so.1 > > > > > libgrpc_unsecure.a liblogdevice.so > > > > libprotobuf.so > > > > > pkgconfig > > > > > libgpr.so libgrpc++_cronet.so > > > > > libgrpc++_error_details.so.1.18.0-dev > > > > libgrpc++_reflection.so.1.18.0-dev > > > > > libgrpc++_unsecure.a libprotobuf.a > > > > > libprotobuf.so.17 python2.7 > > > > > libgpr.so.7 libgrpc++_cronet.so.1 > > > > > libgrpcpp_channelz.a libgrpc.so > > > > > libgrpc_unsecure.so libprotobuf.la > > > > > libprotobuf.so.17.0.0 python3.6 > > > > > libgpr.so.7.0.0-dev libgrpc++_cronet.so.1.18.0-dev > > > > > libgrpcpp_channelz.so libgrpc++.so > > > > > libgrpc++_unsecure.so libprotobuf-lite.a > > libprotoc.a > > > > > > > > > > Thanks, > > > > > Tim > > > > > > > > > > >
