Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

immutable feature changes when using #1022

Open
Berlyli866 opened this issue Oct 19, 2024 · 2 comments
Open

immutable feature changes when using #1022

Berlyli866 opened this issue Oct 19, 2024 · 2 comments

Comments

@Berlyli866
Copy link

Berlyli866 commented Oct 19, 2024

Hi team,
First of all, thanks to the team for working on building such a good package for us to use.

I follow the example Counterfactual with Reinforcement Learning (CFRL) on Adult Census to build my own CFL.

I have a data set that is a mix of numerical, binary, and category features.
I trained a random forest classification model as the predictor model and ran counterfactualtabluer to generate the counterfactual for features that I am interested in. Below is part of the code on how i specify the candidate features and immutable feature


ranges = {'num_image': [1, 16], 
         'num_alternative_image': [0,6],
          'num_market_bullets':[5,19]
         }

from alibi.explainers import CounterfactualRLTabular
explainer = CounterfactualRLTabular(predictor=predictor,
                                    encoder=heae.encoder,
                                    decoder=heae.decoder,
                                    latent_dim=LATENT_DIM,
                                    encoder_preprocessor=heae_preprocessor,
                                    decoder_inv_preprocessor=heae_inv_preprocessor,
                                    coeff_sparsity=COEFF_SPARSITY,
                                    coeff_consistency=COEFF_CONSISTENCY,
                                    category_map=cate_map,
                                    feature_names=model_attr,
                                    #ranges=ranges,
                                    immutable_features=immutable_features,
                                    train_steps=TRAIN_STEPS,
                                    batch_size=BATCH_SIZE,
                                    backend="tensorflow")


explainer = explainer.fit(X=X_train.to_numpy())

X_positive = X_test[np.argmax(predictor(X_test), axis=1) == 1]
X = X_positive[:1000]
Y_t = np.array([0])
#index 20 num_image, 21 num_alternative_image, 22 num_market_bullets. if i put feature name i will get error somehow. 
C = [{20: [1, 10],21:[0,6], 22: [5, 10]}]
explanation = explainer.explain(X, Y_t, C)

after I get the counterfactual df I compared it with original df and got the difference columns below. The avg_delivery_days is immutable but also changes though very tiny change, for 'num_image', 'num_alternative_image' , 'num_market_bullets' the change is also minimal. Can I see the changed features play an important role in predicting the label (>0.4 or <=0.4) since a small change and flip the label ? Did i use the right counterfactual function for my use case? :
Screenshot 2024-10-19 at 18 19 57

For tabluar data , do i always need encoder and decoder? if its already binary should i put binary feature in category_map in below function ?

heae_preprocessor, heae_inv_preprocessor = get_he_preprocessor(X=X_train, feature_names=model_attr, category_map=cate_map, feature_types=feature_types)

Another question I have is what function I can use for the environment models, such as boost regression or a regression type of black box model?
If I tried to use

explainer = CounterfactualRLTabular(predictor=predictor,
                                    encoder=heae.encoder,
                                    decoder=heae.decoder,
                                    latent_dim=LATENT_DIM,
                                    encoder_preprocessor=heae_preprocessor,
                                    decoder_inv_preprocessor=heae_inv_preprocessor,
                                    coeff_sparsity=COEFF_SPARSITY,
                                    coeff_consistency=COEFF_CONSISTENCY,
                                    category_map=cate_map,
                                    feature_names=model_attr,
                                    #ranges=ranges,
                                    immutable_features=immutable_features,
                                    train_steps=TRAIN_STEPS,
                                    batch_size=BATCH_SIZE,
                                    backend="tensorflow")

but replace predictor as the boost regression model. What other changes do I need to make since the regression model, the prediction is continuous, how can i customize the reward function?

sorry for all these questions, as i am a starter in RL and is still learning everthing so forgive me if my questions sounds dump.

thanks for your time and help

@RobertSamoilescu
Copy link
Collaborator

Hi @Berlyli866,

From what I can see in the code snippet above, you are not really following the conventions presented in the documentation. Please also have a look at the docs here, besides the Adult Census example we provide.

Some issues that I can see in the code above are:

  • ranges - are incorrectly specified.
    An example of correctly specified feature in your case would be:
ranges = {
   'num_image': [0.0, 1.0], 
   'num_alternative_image': [-1.0, 0.0],
   'num_market_bullets': [-1.0, 1.0]
}

This means that when generating the counterfactual for a given instance num_image can only increase, num_alternative_image can only decrease, num_market_bulltets can increase or decrease (you can actually omit specifying the feature which can increase or decrease).

  • feature_names - should be a list of strings containing the column names of your data frame. The order of the features within feature_names is important. They should match exactly the order of your columns.

  • immutable_features - should be a list of strings and contain column names that are present in the feature_names list. I can't tell exactly what you passed, but I think it is worth mentioning.

  • C - should be a list of dictionaries with strings as keys, where each key is a name of a column. From what I see, you are providing column indices as keys. Note that for numerical features, the conditioning represents a delta change from the original value and 0 must be included in the interval you specified. For example C=[{"num_image": [0, 5]}] means that num_image value will be changed by a maximum of 5 units.

You don't necessarily need to train an autoencoder. You can see an example here how to do that. Please read the final paragraph in this comment.

You should provide a complete description of category_map. So, yes, binary features should be specified as well.

Although we didn't test explicitly for regression models, I think the implementation still supports it for a single target. It is really up to you how to design the reward function. You can give a sparse or continuous reward based on the distance from your target. The reward function can be specified through the parameter reward_func (see docs here)

@Berlyli866
Copy link
Author

Berlyli866 commented Oct 28, 2024

Hi Robert,
Thanks for your response and help here !

For C if i use the string as keys, i will have error
``

but if i use the indices key, it will run through. In my code the C is

index map : 
26 num_image
27 num_alternative_image
28 num_market_bullets
2 salient_bullet
12 image_life_style
39 return_policy_90-Day

C = [{26: [4, 17],27:[0,6], 28: [5, 22]}]


which means I want the num_image changes from 4 to 17 , num_alternative_image changes from 0 to 6, and num_market_bullets change from 5 to 22, sailent_bullets can only be 1 , image_life_style can only be 1, return_policy_90-Day can only be 1

I actually have other features in C but the output after i ran explanation = explainer.explain(X, Y_t, C) i did not see any change for the feature I specify in C and not in the immutable_features list. Do you know what the reason may be?
below are my change features and their range :

change_features=['num_image',
 'num_alternative_image',
 'num_market_bullets','image_life_style','return_policy_90-Day','salient_bullet']

ranges = {'image_life_style'::[0.0, 1.0]
          'return_policy_90-Day':[0.0, 1.0]
          'salient_bullet':[0.0, 1.0]
         }

image_life_style, return_policy_90-Day,salient_bullet are all binary features. I want them to change only from 0 to 1. If it's 1, don't change it. Do I specify the range right? For other changeable features, they can increase or decrease, so I omit them here.

the category_map=cate_map in my code is all binary features index and their value [0,1] so something like :

{0: [0, 1],
 2: [1, 0],
 6: [1, 0],
 7: [0, 1]}

0,2,6,7 is the feature index. Now that I know if they are binary, we also need to specify. thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants