Re: I want to apply external function on streaming data

I have external function scaler1 = MinMaxScaler(feature_range=(-1, 1)) def difference(dataset, interval=1): diff = list() for i in range(interval, len(dataset)): value = dataset[i] - dataset[i - interval] diff.append(value) return Series(diff)

Re: I want to apply external function on streaming data

Can any one help me please -- Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/ - To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

I want to apply external function on streaming data

I have 3 external function I want to apply external function on streaming data, I use sliding window to get streaming data but I can not apply external function on streaming data I write code by python -- Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/ --

why y.size is 65536 but y size in new dataset is 1000

please help me, code write in spark by python error is Caused by: java.lang.IllegalArgumentException: requirement failed: BLAS.dot(x: Vector, y:Vector) was given Vectors with non-matching sizes: x.size = 1000, y.size = 65536 why y.size is 65536 but y size in new dataset is 1000 1-I train model o

I want read text cluomn from kafka topic pyspark

I write code to read data from twitter and send data to kafka topic and I write anther code to read data from kafka topic I want to return just text column from data -- Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/ ---

how can solve this error

I write code to connect kafka with spark using python and I run code on jupyer my code import os #os.environ['PYSPARK_SUBMIT_ARGS'] = '--jars /home/hadoop/Desktop/spark-program/kafka/spark-streaming-kafka-0-8-assembly_2.10-2.0.0-preview.jar pyspark-shell' os.environ['PYSPARK_SUBMIT_ARGS'] = "--pack