[
https://issues.apache.org/jira/browse/SPARK-19234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15824577#comment-15824577
]
Sean Owen commented on SPARK-19234:
-----------------------------------
I am not so familiar with AFT, but is it valid to have failure time = 0 in this
model? AFAIK it's regression the log of failure time
> AFTSurvivalRegression chokes silently or with confusing errors when any
> labels are zero
> ---------------------------------------------------------------------------------------
>
> Key: SPARK-19234
> URL: https://issues.apache.org/jira/browse/SPARK-19234
> Project: Spark
> Issue Type: Bug
> Components: ML
> Affects Versions: 2.1.0
> Environment: spark-shell or pyspark
> Reporter: Andrew MacKinlay
> Attachments: spark-aft-failure.txt
>
>
> If you try and use AFTSurvivalRegression and any label in your input data is
> 0.0, you get coefficients of 0.0 returned, and in many cases, errors like
> this:
> {{17/01/16 15:10:50 ERROR StrongWolfeLineSearch: Encountered bad values in
> function evaluation. Decreasing step size to NaN}}
> Zero should, I think, be an allowed value for survival analysis. I don't know
> if this is a pathological case for AFT specifically as I don't know enough
> about it, but this behaviour is clearly undesirable. If you have any labels
> of 0.0, you get either a) obscure error messages, with no knowledge of the
> cause and coefficients which are all zero or b) no errors messages at all and
> coefficients of zero (arguably worse, since you don't even have console
> output to tell you something's gone awry). If AFT doesn't work with
> zero-valued labels, Spark should fail fast and let the developer know why. If
> it does, we should get results here.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
