Fwd: Companies and Organizations listing for datapipelines.com

2020-08-12 Thread roland
Hi there, Just following up on my previous email since I haven't heard back. Some more info about the project: - recently upgraded to use spark 3.0.0 - uses spark-core and spark-sql modules - AWS S3 support - JDBC support Short description: Create and schedule data pipelines step-by-step using

Re: SPIP: Catalog API for view metadata

2020-08-12 Thread John Zhuge
Hi Wenchen, Thanks for the feedback! 1. Add a new View API. How to avoid name conflicts between table and view? > When resolving relation, shall we lookup table catalog first or view > catalog? See clarification in SPIP section "Proposed Changes - Namespace": - The proposed new view substi

Re: SPIP: Catalog API for view metadata

2020-08-12 Thread Wenchen Fan
Hi John, Thanks for working on this! View support is very important to the catalog plugin API. After reading your doc, I have one high-level question: should view be a separated API or it's just a special type of table? AFAIK in most databases, tables and views share the same namespace. You can'