I see. You might try this, create a pipeline of just your feature
transformers, then call fit() on the complete dataset to get a model.
Finally make second pipeline and add this model and the decision tree as
stages.
On Aug 30, 2016 8:19 PM, "Bahubali Jain" wrote:
> Hi Bryan,
> Thanks for the r
Hi Bryan,
Thanks for the reply.
I am indexing 5 columns ,then using these indexed columns to generate the
"feature" column thru vector assembler.
Which essentially means that I cannot use *fit()* directly on
"completeDataset" dataframe since it will neither have the "feature" column
and nor the 5 i
You need to first fit just the VectorIndexer which returns the model, then
add the model to the pipeline where it will only transform.
val featureVectorIndexer = new VectorIndexer()
.setInputCol("feature")
.setOutputCol("indexedfeature")
.setMaxCategories(180)
.fit(completeDataset)
Hi,
I had run into similar exception " java.util.NoSuchElementException: key
not found: " .
After further investigation I realized it is happening due to vectorindexer
being executed on training dataset and not on entire dataset.
In the dataframe I have 5 categories , each of these have to go thru
Hi Rich,
I looked at the notebook and it seems like you are fitting the
StringIndexer and VectorIndexer to only the training data, and it should
the the entire data set. So if the training data does not include all of
the labels and an unknown label appears in the test data during evaluation,
the
Hi Bryan.
Thanks for your continued help.
Here is the code shown in a Jupyter notebook. I figured this was easier
that cutting and pasting the code into an email. If you would like me to
send you the code in a different format let, me know. The necessary data is
all downloaded within the noteboo
Are you fitting the VectorIndexer to the entire data set and not just
training or test data? If you are able to post your code and some data to
reproduce, that would help in troubleshooting.
On Tue, Jun 28, 2016 at 4:40 PM, Rich Tarro wrote:
> Thanks for the response, but in my case I reversed
Thanks for the response, but in my case I reversed the meaning of
"prediction" and "predictedLabel". It seemed to make more sense to me that
way, but in retrospect, it probably only causes confusion to anyone else
looking at this. I reran the code with all the pipeline stage inputs and
outputs name
The problem might be that you are evaluating with "predictionLabel" instead
of "prediction", where predictionLabel is the prediction index mapped to
the original label strings - at least according to the
RandomForestClassifierExample, not sure if your code is exactly the same.
On Tue, Jun 28, 2016
