Spark ml how to extract split points from trained decision tree mode

2020-06-11 Thread AaronLee
I am following official spark 2.4.3 tutorial trained a decision tree model. How to extract split points from the trained model? // model val dt = new DecisionTreeClassifier() .setLabelCol("indexe

Re: Spark ml how to extract split points from trained decision tree mode

2020-06-11 Thread AaronLee
Thanks srowen. I also checked https://www.programcreek.com/scala/org.apache.spark.ml.tree.InternalNode. Splits are available via "InternalNode" ".split" attribute. But "dtm.rootNode" belongs to "LeafNode". ``` scala> dtm.rootNode res9: org.apache.spark.ml.tree.Node = LeafNode(prediction = 0.0, i

Re: Spark ml how to extract split points from trained decision tree mode

2020-06-11 Thread AaronLee
@srowen. You are totally right, the model was not trained correctly. But it is weird as the dataset I used actually has 50m rows. It has binary label with 20% positive, and 1 feature in feature vector. Do not understand why it does not trained correctly ``` scala> df2.count res56: Long = 4817485

Re: Spark ml how to extract split points from trained decision tree mode

2020-06-12 Thread AaronLee
instead continue explore and debug, switch to sklearn decision tree in the end ... lol -- Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ - To unsubscribe e-mail: user-unsubscr...@spark.apache.org