Skip to content

josyulakrishna/GraphNN-For-Learning-Dynamics-and-Generating-Policies-with-Explanations-using-Decision-Trees

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

20 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Introduction

Based on the original Repository at https://github.com/fxia22/gn.pytorch/

  • The project learns the dynamics of the agent by using the Graph Neural Networks Graph networks as learnable physics engines for inference and control.

  • The learned GNN Model is integrated into the Deep Mind Control Suite to learn a policy for reaching the goal, please replace your original dm_control package with the files from dm_control_mod

  • The code for explanation is contained in the explanations folder the files contain instructions for generating the decision trees for policy. Please look into "explainPolicy.ipynb" and "explainDynamics.ipynb"

  • Models and files are named _swimmer3 for all swimmer3 trained policies and gnn models and the data sets

Dependencies

Please install the following Dependencies

Generate data

Generate data with gen_data_new.py script, for the 3-link swimmers states. A pregenerated data file is given here "swimmer3.npy" and "swimmer3_eval.npy"

Normalise the data using "normalizer.py" a pre-generated file is given here "normalize3.pth"

Train GN

Change line 12 in train_gn.py to be from dataset import SwimmerDataset

python train_gn.py to train the model. The learning rate schedule corresponds to "fast training" in original paper.

Evaluate GN

Change line 14 in evaluate_gn.py to be from dataset import SwimmerDataset

python evaluate_gn.py <model path>

Predicted and Actual dynamics of swimmer3

  • GNN loss for swimmer3 gnn loss for swimmer3

  • GNN predicted vs actual state gnn loss for swimmer3

Swimmer3 learned policy

TD3 has worked the best for us with an mean reward of 218 TD3 reward for swimmer3

Demo of swimmer3 moving

TD3 demo

Explanations

Using a set of observations we predict the angle the angle of the head using the decision trees predictions.

Change line 12 in train_gn.py to be from testdataset import SwimmerDataset Change line 14 in evaluate_gn.py to be from testdataset import SwimmerDataset

The policy explanation is attempted by building a decision tree of optimal policy and model policy. The details are given in "explainDynamics.ipynb" and "explainPolicy.ipynb"

decision tree for model predictions

decision tree for optimal actions

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published