Fwd: Companies and Organizations listing for datapipelines.com

2020-08-12 Thread roland
using a GUI, without writing code. Url:https://www.datapipelines.com/ Kind regards, Roland Pogonyi Data Pipelines > -- Original Message -- > From: rol...@datapipelines.com > To: "dev@spark.apache.org" > Date: 8 June 2020 10:25 &

Companies and Organizations listing for datapipelines.com

2020-06-08 Thread roland
if it would be possible to list it under the "Companies and Organizations" on the Spark website ( https://spark.apache.org/powered-by.html). Many thanks. Best regards, Roland Pogonyi datapipelines.com

Re: How to load Python Pickle File in Spark Data frame

2019-08-26 Thread Roland Johann
Best Regards Roland Johann Software Developer/Data Engineer phenetic GmbH Lütticher Straße 10, 50674 Köln, Germany Mobil: +49 172 365 26 46 Mail: roland.joh...@phenetic.io Web: phenetic.io Handelsregister: Amtsgericht Köln (HRB 92595) Geschäftsführer: Roland Johann, Uwe Reimann > Am 26.08.2

Re: Please help view the problem of spark dynamic partition

2019-08-23 Thread Roland Johann
ed by year/month/day INSERT OVERWRITE TABLE bsl12.email_edge_lyh_partitioned2 PARTITION (year, month, day) SELECT year(link_crtd_date, '-MM-dd') as year, month(link_crtd_date, '-MM-dd') as month, day(link_crtd_date, '-MM-dd') as day, * FROM bsl1

Re: [K8S] properties file via SPARK_CONF_DIR and --properties-file prevents definition of own properties via secrets/own mounts

2019-08-19 Thread Roland Johann
I would like to discuss if we can introduce a change here - maybe it’s also possible to get that into 2.4.4? I’m keen to contribute that change. Please let me know your thoughts on this. King Regards Roland Johann Software Developer/Data Engineer phenetic GmbH Lütticher Straße 10, 50674 Köln

[K8S] properties file via SPARK_CONF_DIR and --properties-file prevents definition of own properties via secrets/own mounts

2019-08-13 Thread Roland Johann
properties, such as from secrets - currently it seems to be the only possibility to define spark config properties from k8s secrets. Is the current implementation intent? If yes, what are the reasons behind that decision? Thanks and kind regards Roland Johann Software Developer/Data Engineer

Re: Expression/LogicalPlan dichotomy in Spark SQL Catalyst

2015-12-23 Thread Roland Reumerman
can make magic happen. P.S. I've been informed by someone on the mailing list that some other Spark developers are working on a similar concept, called the push-down of everything: https://issues.apache.org/jira/browse/SPARK-12449 Roland From: Michael A

Expression/LogicalPlan dichotomy in Spark SQL Catalyst

2015-12-21 Thread Roland Reumerman
ode/BinaryExpression. Much obliged, Roland Reumerman mendix.com