Re:run spark apps in linux crontab

2016-07-20 Thread focus
Hi, I just meet this problem, too! The reason is crontab runtime doesn't have the variables you defined, such as $SPARK_HOME. I defined the $SPARK_HOME and other variables in /etc/profile like this: export $MYSCRIPTS=/opt/myscripts export $SPARK_HOME=/opt/spark then, in my crontab job script d

?????? Does Spark Streaming support streaming from a database table?

2015-07-14 Thread focus
Hi In our case, we have some data stored in a Oracle database table, and new records will be added into this table. We need to analyse new records to calculate some values continuesly, then we write a program to monitor the table every minute. Because every record has a increased unique ID num