-
-
Notifications
You must be signed in to change notification settings - Fork 5.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
enhance freqai to accept categorical_features #8785
Comments
is it possible to enhance freqai and add categorical features in feature engineering with column name like "%-categ-" and by adding in model.fit()
|
Hello, Yes I recognize this because I identified this problem during a discord conversation with you. If you remember, i asked you to begin using the new pipeline PR #8692 so you could take full control over the normalization range, switching to 0,1 from the present -1,1. Thus avoiding the negative values entirely. As I mentioned to you in the discord, your pipeline would be: import datasieve.transforms as ds
def define_data_pipeline(self) -> Pipeline:
"""
User defines their custom feature pipeline here (if they wish)
"""
feature_pipeline = Pipeline([
('const', ds.VarianceThreshold()),
('qt', ds.SKLearnWrapper(MaxMinScaler(feature_range=(0,1))),
('di', ds.DissimilarityIndex(di_threshold=1)
])
return feature_pipeline while I agree it’s not the final solution to your problem, it is certainly the step in the correct direction. It makes no sense to spend time modifying the stable branch pipeline now since it’s being torn out and replaced as shown in #8692 . As for automating something and using a keyword like As for your current problem, please use the aforementioned code with #8692 (make sure to reinstall as the PR has new dependencies) to ensure you don’t have any negative values, then please let us know if it fixes your problem or not. |
thanks @robcaulk haha didnt know it was you . thanks for always helping me out... will check the pipeline.... |
Also, I should add, if you aren’t making use of SVM, DI, or PCA, then you actually don’t need the normalization or the pipeline at all. Decision trees do not require normalized data, it is these other methods that do. You can determine if you are using those by looking at your config and seeing if they are set to true or false. |
Describe your environment
(if applicable)
Describe the enhancement
im using categorical feature however they turn into negative values and lightgbm converts them to Nan
freqtrade | [LightGBM] [Warning] Met negative value in categorical features, will convert it to NaN
freqtrade | [LightGBM] [Warning] Met negative value in categorical features, will convert it to NaN
freqtrade | [LightGBM] [Warning] Met negative value in categorical features, will convert it to NaN
freqtrade | [LightGBM] [Warning] Met negative value in categorical features, will convert it to NaN
freqtrade | [LightGBM] [Warning] Met negative value in categorical features, will convert it to NaN
freqtrade | [LightGBM] [Warning] Met negative value in categorical features, will convert it to NaN
freqtrade | [LightGBM] [Warning] Met negative value in categorical features, will convert it to NaN
freqtrade | [LightGBM] [Warning] Met negative value in categorical features, will convert it to NaN
freqtrade | [LightGBM] [Warning] Met negative value in categorical features, will convert it to NaN
freqtrade | /home/ftuser/.local/lib/python3.10/site-packages/lightgbm/basic.py:1780: UserWarning: Overriding the parameters from Reference Dataset.
freqtrade | _log_warning('Overriding the parameters from Reference Dataset.')
freqtrade | /home/ftuser/.local/lib/python3.10/site-packages/lightgbm/basic.py:1513: UserWarning: categorical_column in param dict is overridden.
freqtrade | _log_warning(f'{cat_alias} in param dict is overridden.')
the expectation to categorical features should be (0,1,2)
Explain the enhancement you would like
is there a way to exclude categorical features normalization in the training_features?
below is the model used to let lightgbm recognize categorical_features
The text was updated successfully, but these errors were encountered: