From 1eda3389e99648b8077a0e749ec567290b716bb4 Mon Sep 17 00:00:00 2001 From: corentinlger Date: Thu, 2 May 2024 19:55:47 +0200 Subject: [PATCH] Update end of the tutorial --- notebooks/simulator_tutorial.ipynb | 146 +++++++++++++++++------------ 1 file changed, 85 insertions(+), 61 deletions(-) diff --git a/notebooks/simulator_tutorial.ipynb b/notebooks/simulator_tutorial.ipynb index 34dbe9d..a9d67f9 100644 --- a/notebooks/simulator_tutorial.ipynb +++ b/notebooks/simulator_tutorial.ipynb @@ -2,7 +2,7 @@ "cells": [ { "cell_type": "code", - "execution_count": 3, + "execution_count": 1, "metadata": {}, "outputs": [], "source": [ @@ -22,12 +22,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Rendering functions" + "### Rendering functions\n", + "\n", + "We first define some custom rendering functions to visualize the state of the simulation, you don't have to understand them to follow the rest of the tutorial." ] }, { "cell_type": "code", - "execution_count": 4, + "execution_count": 2, "metadata": {}, "outputs": [], "source": [ @@ -211,14 +213,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Create simulator state\n", + "### 1 - Create the simulator state\n", "\n", "The Simulator State encapsulates information about the simulation environment, including its size, the maximum number of objects and agents, physical parameters for collisions, and other relevant settings. It also stores parameters related to the simulation's execution, such as whether to JIT-compile the update functions and the use of fori_loop. Additionally, it contains data for any connected client, like the simulation speed and interface update frequency." ] }, { "cell_type": "code", - "execution_count": 5, + "execution_count": 3, "metadata": {}, "outputs": [ { @@ -263,14 +265,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Create entities state\n", + "### 2 - Create the entities state\n", "\n", "Contains all physical parameters (position, mass, momentum, friction ...) about all entities of the system (agents, objects). We store all the data in jax arrays of length n_entities (max_agents + max_objects). Because those arrays encompass information about agents but also objects, we define an array entity_types to know the type of each entity. We also have an array called exists, that defines which entity exist or not. This is because we have to keep arrays of same shape with jax, so we can't just add new agents to a list for example (that is also why we need to define max_agents / objects in advance)." ] }, { "cell_type": "code", - "execution_count": 6, + "execution_count": 4, "metadata": {}, "outputs": [], "source": [ @@ -328,7 +330,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Create agents state\n", + "### 3 - Create the agents state\n", "\n", "All information about the agents that are not shared by other entities. At the moment, the agents are braitenberg vehicles, that is why they have behaviors, wheels, motors, proximeter ... \n", "\n", @@ -337,7 +339,7 @@ }, { "cell_type": "code", - "execution_count": 7, + "execution_count": 5, "metadata": {}, "outputs": [], "source": [ @@ -375,14 +377,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Create objects state\n", + "### 4 - Create objects state\n", "\n", "Do the same for objects, here we only specify their color and their corresponding index in the entities dataclass." ] }, { "cell_type": "code", - "execution_count": 8, + "execution_count": 6, "metadata": {}, "outputs": [], "source": [ @@ -404,14 +406,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Create a state\n", + "### 5 - Create the state\n", "\n", "The state contains all the information about simulator, entities, agents and objects states." ] }, { "cell_type": "code", - "execution_count": 9, + "execution_count": 7, "metadata": {}, "outputs": [], "source": [ @@ -436,7 +438,7 @@ }, { "cell_type": "code", - "execution_count": 10, + "execution_count": 8, "metadata": {}, "outputs": [ { @@ -458,20 +460,22 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Wrap the state inside a simulator" + "### Wrap the state inside a simulator\n", + "\n", + "The simulator needs 3 arguments to be initialized, the initial state (we just created it), a bank of behaviors for the agents and a function to compute the physcis of the simulation (we import these two elements in the next cell)." ] }, { "cell_type": "code", - "execution_count": 11, + "execution_count": 9, "metadata": {}, "outputs": [], "source": [ - "from vivarium.simulator import behaviors\n", + "from vivarium.simulator.behaviors import behavior_bank\n", "from vivarium.simulator.simulator import Simulator\n", "from vivarium.simulator.physics_engine import dynamics_rigid\n", "\n", - "simulator = Simulator(state, behaviors.behavior_bank, dynamics_rigid)" + "simulator = Simulator(state, behavior_bank, dynamics_rigid)" ] }, { @@ -501,12 +505,12 @@ "source": [ "### Run a simulation and save it\n", "\n", - "At the moment the simulation will run with the default behavior assigned to the agents. They are in agressive mode, which means they will rush toward the closest entity in their field of view. The simulation also includes physics elements such as collisions between entities etc. All the functions used to compute the next state of the simulation are currently located in the [physcis engine](../vivarium/simulator/physics_engine.py) file. " + "state initial + run sim = modifier cet etat ... At the moment the simulation will run with the default behavior assigned to the agents. They are in agressive mode, which means they will rush toward the closest entity in their field of view. The simulation also includes physics elements such as collisions between entities etc. All the functions used to compute the next state of the simulation are currently located in the [physcis engine](../vivarium/simulator/physics_engine.py) file. " ] }, { "cell_type": "code", - "execution_count": 12, + "execution_count": 10, "metadata": {}, "outputs": [], "source": [ @@ -515,7 +519,7 @@ }, { "cell_type": "code", - "execution_count": 13, + "execution_count": 11, "metadata": {}, "outputs": [], "source": [ @@ -540,7 +544,7 @@ }, { "cell_type": "code", - "execution_count": 14, + "execution_count": 12, "metadata": {}, "outputs": [ { @@ -570,7 +574,7 @@ }, { "cell_type": "code", - "execution_count": 15, + "execution_count": 13, "metadata": {}, "outputs": [], "source": [ @@ -587,7 +591,7 @@ }, { "cell_type": "code", - "execution_count": 16, + "execution_count": 14, "metadata": {}, "outputs": [ { @@ -620,7 +624,7 @@ }, { "cell_type": "code", - "execution_count": 17, + "execution_count": 15, "metadata": {}, "outputs": [], "source": [ @@ -649,7 +653,7 @@ }, { "cell_type": "code", - "execution_count": 18, + "execution_count": 16, "metadata": {}, "outputs": [], "source": [ @@ -666,7 +670,7 @@ }, { "cell_type": "code", - "execution_count": 19, + "execution_count": 17, "metadata": {}, "outputs": [ { @@ -695,12 +699,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Because we use jax and need to have fixed arrays shape, we have arrays fo size max_agents, max_objects ... But in some situations we want these agents or objects not to exist. To do so, we just need to change the exist field of entity_state (set the value of the entities either to 1 or 0). This will \"remove\" the non existing entities from the simulation and they won't interact anymore with the other ones (no more collisions, no more motor activations for agents ...). " + "In our JAX simulation, we work with fixed-size arrays for agents and objects. However, there might be situations where you want some agents or objects not to exist. To achieve this, you can manipulate the exist field of the entity_state.\n", + "\n", + "To \"remove\" an entity from the simulation, simply set its corresponding value in the exist field to 0. Conversely, to add an entity, set the value to 1. By doing so, the non-existent entities will no longer interact with other entities in the simulation, meaning there will be no collisions or motor activations for agents. This approach allows you to effectively manage entities within the fixed-size array constraints of the JAX simulation." ] }, { "cell_type": "code", - "execution_count": 20, + "execution_count": 18, "metadata": {}, "outputs": [], "source": [ @@ -721,7 +727,7 @@ }, { "cell_type": "code", - "execution_count": 21, + "execution_count": 19, "metadata": {}, "outputs": [ { @@ -753,19 +759,19 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "And now we indeed only have 2 FEAR agents and 1 object left. If we want to add these entities back, we can simply set their exist value back to 1 as follows :" + "And now we indeed only have 2 FEAR agents and 1 object left. If we want to add these entities back, we can simply set their value in the exist attribute back to 1 as follows :" ] }, { "cell_type": "code", - "execution_count": 22, + "execution_count": 20, "metadata": {}, "outputs": [], "source": [ "state = simulator.state\n", "entities_exist = state.entity_state.exists\n", - "entities_exist = entities_exist.at[agents_idx[:3]].set(jnp.array([1])) # Set to 1\n", - "entities_exist = entities_exist.at[objects_idx[:1]].set(jnp.array([1])) # Set to 1\n", + "entities_exist = entities_exist.at[agents_idx[:3]].set(jnp.array([1])) # Set back to 1\n", + "entities_exist = entities_exist.at[objects_idx[:1]].set(jnp.array([1])) # Set back to 1\n", "\n", "entity_state = state.entity_state.set(exists=entities_exist)\n", "state = state.set(entity_state=entity_state)\n", @@ -778,12 +784,18 @@ "source": [ "### Implement a custom behavior\n", "\n", - "To implement behaviors of agents, you first need to access their observation of the environment in order to compute their action (activation of their motors). To do so, we can access three type of information for each agent : their proximeter activations, and their relative distance and angles with their neihbor entities;" + "To create custom agent behaviors, you need to access their environmental observations to compute their actions (motor activations). You can access three types of information for each agent:\n", + "\n", + "- Proximeter activations\n", + "- Relative distances to neighboring entities\n", + "- Relative angles with neighboring entities\n", + "\n", + "Using this information, you can design and implement custom behaviors for your agents." ] }, { "cell_type": "code", - "execution_count": 23, + "execution_count": 21, "metadata": {}, "outputs": [], "source": [ @@ -793,6 +805,7 @@ "proximter_activations = state.agent_state.prox\n", "proximity_map_dist = state.agent_state.proximity_map_dist\n", "proximity_map_theta = state.agent_state.proximity_map_theta % (2 * jnp.pi)\n", + "# Also access the motor values of the agents\n", "motors_activations = state.agent_state.motor" ] }, @@ -805,11 +818,10 @@ }, { "cell_type": "code", - "execution_count": 24, + "execution_count": 22, "metadata": {}, "outputs": [], "source": [ - "\n", "def change_agent_color(state, ag_idx, color):\n", " updated_agent_color = state.agent_state.color.at[ag_idx].set(jnp.array(color))\n", " agent_state = state.agent_state.set(behavior=updated_agent_behavior, color=updated_agent_color)\n", @@ -824,7 +836,7 @@ }, { "cell_type": "code", - "execution_count": 25, + "execution_count": 23, "metadata": {}, "outputs": [ { @@ -877,18 +889,43 @@ "print(f\"source agent motor activations (left, right): \\n{motors_activations[source_ag_idx]}\")\n" ] }, + { + "cell_type": "code", + "execution_count": 24, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Array(-0.00973955, dtype=float32)" + ] + }, + "execution_count": 24, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "jnp.sin(proximity_map_theta[source_ag_idx][target_ag_idx])" + ] + }, { "cell_type": "markdown", "metadata": {}, "source": [ - "We can indeed see that the target agent (2) is the closest to the source agent (4), and its relative angle is almost 2*pi (the source agent is pointing in his direction). Because the target agent is the closest, it is the one activating the proximeters of the source agent. ... Understand why left prox is activated and not right one here .... bc angle < 2 * pi so it should be the right proximter activated right ?\n", - "\n", - "To define the classical behaviors of braitenberg agents, we use these kind of fns (not exactly because no jax here) where you map the proximeters values of an agent to its motor activations." + "We can indeed observe that the target agent (2) is the closest to the source agent (4), with a relative angle nearly equal to 2*pi (meaning the source agent is facing its direction). Consequently, the target agent is the one triggering the proximeter activations of the source agent." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "To define the classical behaviors of braitenberg agents, we use these kind of functions (not exactly because these aren't jax functions) where you map the proximeters values of an agent to its motor activations." ] }, { "cell_type": "code", - "execution_count": 26, + "execution_count": 25, "metadata": {}, "outputs": [], "source": [ @@ -909,12 +946,12 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "If we wanna attribute custom behavior to an agent, set it behavior to manual and implement a custom behavior. For example, we will set the left motor of the black agent to 1 and the left to 0, so it will just keep turning to the left." + "If we want to attribute a custom behavior to an agent, we need to set its behavior attribute to manual and implement a custom behavior. For example, we will set the left motor of the black agent to 1 and the left to 0, so it will just keep turning to the left." ] }, { "cell_type": "code", - "execution_count": 39, + "execution_count": 26, "metadata": {}, "outputs": [], "source": [ @@ -926,7 +963,7 @@ }, { "cell_type": "code", - "execution_count": 36, + "execution_count": 27, "metadata": {}, "outputs": [], "source": [ @@ -943,7 +980,7 @@ }, { "cell_type": "code", - "execution_count": 38, + "execution_count": 28, "metadata": {}, "outputs": [ { @@ -965,20 +1002,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Now, you can even implement more complex behaviors with all the info you have ..." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Refactor ideas (just notes for me, will remove them from notebook)\n", - "\n", - "--> Should also remove all the information about proximeters from physics engine (i.e physics engine only an util function to provide help for collisions ...). Think the best thing for an user would surely be to have a kind of file to define an environment (e.g EvoJax, JaxMARL) and import physics fn from physics engine (prox ... would be computed here, or user could define a different type of observation for tis agents). Can be wrapped within a simulator (doesn't wrap a state but an env that can step ...) Then in the simulator use run fns ... that call this step, save the simulation ...\n", - "The idea behind this is that wanna do RL / neurevolution only needs a step method, not the whole simulator class (but it is way more convenient to use the interface).\n", - "\n", - "--> Also something more general about agents observation, in braitenberg case they are sensors (could be vision ... right ?) Then we could have agents that have an obs field and \n", - "--> I think what would be interesting is having a field obs (like RL observation for agents, i.e the neighbor map the agents have). I think it is easier to rename it observation so it becomes pretty clear what is is about. " + "Now that you understand the basics of the simulator, you can implement your own custom behaviors." ] } ],