-
Notifications
You must be signed in to change notification settings - Fork 3.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Classifier predict numerical precision issue with large raw_score #6405
Comments
Thanks for using LightGBM. Are you able to share some minimal code showing precisely what you mean? I'm unsure how to interpret some of these statements like "a very large number with value 0 or value 1". |
I cannot share much in the way of specifics, but here is the workflow:
If instead I look at:
Then As I mentioned, a previous version of lightgbm did not behave in the current way, avoiding this problem. |
Ok, so to clarify:
|
Apologies, yea I am using |
Description
I have a trained model with a binary objective using n_estimators=1000. The model performance (AUC) is quite good. I need the raw probabilities for selection by ranking. The probabilities provided by
presict_proba
orpredict
however have a very large number with value 0 or value 1 and an odd bowl shaped distribution.When I use
raw_score=True
, I get scores from -11k to +135k without a large number being squashed to the min or max, and an expected distribution. Applying a simple sigmoid to these raw scores regains the non-raw scores. This clearly shows that the numerical precision is insufficient to distinguish the very low and very high values, so they get flattened to 0 and 1 respectively.I believe the raw_score values should be normalized in some way first so as to avoid this problem. I have an old version of the model using v2 of lightgbm that does not have this issue (with training data and parameters nearly identical). Perhaps the old version averaged rather than summed the random Forrest raw scores, to avoid a number-of-trees dependence, before applying the sigmoid function? These seems like the right approach.
Environment info
LightGBM version or commit hash: 4.3.0
Running on python 3.11 in AWS (sagemaker)
The text was updated successfully, but these errors were encountered: