Skip to content

Commit

Permalink
Merge pull request #10 from tum-pbs/update-guide-for-mouse-interaction
Browse files Browse the repository at this point in the history 
Update to guide.md
  • Loading branch information
@wi-re
wi-re authored Nov 6, 2024
2 parents e350370 + 6f11752 commit 584360b
Show file tree
Hide file tree
Showing 2 changed files with 92 additions and 108 deletions.
3 changes: 3 additions & 0 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -124,3 +124,6 @@ public:
1. Add the source file to the [Scenes](Scenes) directory.
2. Run `cmake . -B build` again before building the project.
## Further reading:
If you would like to know more, or work through a demo example look at our guide in this repository, which you can find in the [guide.md file](https://github.com/tum-pbs/game-physics-template/blob/main/guide.md). This guide works through input handling, extending scenes and further details like adaptive timestepping and will be updated over time too.
197 changes: 89 additions & 108 deletions guide.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -394,7 +394,7 @@ void Scene1::simulateStep(){
To remove particles after 1 second we add this call to the end of the simulateStep function where we remove particles if a conditional is true for the respective entry:
```cpp
particles.erase(std::remove_if(particles.begin(), particles.end(), [](const Particle& particle){
return particle.lifetime > 1.f;
return particle.lifetime > 1.5f;
}), particles.end());
```
All of this gives us this interactive physics simulation:
Expand All @@ -406,7 +406,7 @@ All of this gives us this interactive physics simulation:
If we want to add gravity we can simply add a gravitational field in the simulateStep function and update the velocity accordingly:

```cpp
glm::vec3 gravityAccel = glm::vec3(0, -9.81f, 0);
glm::vec3 gravityAccel = glm::vec3(0, 0, -9.81f);

for (auto& particle : particles){
particle.position += 0.01f * particle.velocity;
Expand All @@ -415,9 +415,11 @@ for (auto& particle : particles){
}
```

In this case we set the gravity to be pointing down in the z-direction. Note that some physics engines assign the "down" direction to be the negative y-coordinate (this is an interesting legacy development, e.g., consider 2D games where the down direction is naturally y). Within this framework we always set the z-direction to be up and down and chose the gravity accordingly.

Which gives us our final physics system:

![alt text](https://github.com/user-attachments/assets/34273fe0-48a0-46e9-b9e3-f3da85563a85)
![alt text](https://github.com/user-attachments/assets/80a47bc6-8266-4144-9ec2-74f4447f74cf)

## Keyboard input

Expand Down Expand Up @@ -483,7 +485,7 @@ The first two states are self-explanatory and describe the state of a key after
A key being pressed and released is an _event_, i.e., it is a change of state of the keyboard and if we only look at the _state_ of a key then we cannot detect this single event, however, we can utilize our operating system or window system to send us a notification if this _event_ occured and we have to write functions that react to events instead of checking the state of keys. This has some other convenient advantages, e.g., if our game is running at 1 fps then we have to hold a key for an entire second to make sure the key is being pressed while _some_ function checks for input and there is no way to queue events, whereas for an event based system we only need to make sure that the event is sent at _some_ point which is then queued up for processing. While this may be an extreme example, there can be weird interactions or situations where two functions check for key state during a frame and one sees a key as being held and one sees a key as not being held, leading to very subtle and hard to reproduce bugs. On the other hand, for an event based system a common bug is someone _tabbing_ out of the game while holding a key such that the key _release_ event is never sent. Can you think of other issues?
If you want to use input during any submission __DO NOT USE THE GLFW VARIANT__. The GLFW variant requires changes to files outside of the Scene folder which are not part of the code you submit. (Note that this will only mean that the tutor grading your exercise wont be able to press buttons for any interactive element you chose to implement, and you will not be able to use this during the exam)
Implementing an event based input, e.g., using GLFW key callbacks or ImGUI events, requires changes to files outside of the Scene folder which are not part of the code you submit. (Note that this will mean that the tutor grading your exercise wont be able to press buttons for any interactive element you chose to implement, and you will not be able to use this during the exam so we stronlgy recommend using immediate inputs).
### ImGUI Immediate mode inputs
Expand All @@ -496,7 +498,7 @@ void Scene1::simulateStep(){
// roll += roll_increment;
// yaw += yaw_increment;
glm::vec3 gravityAccel = glm::vec3(0, -9.81f, 0);
glm::vec3 gravityAccel = glm::vec3(0, 0, -9.81f);
for (auto& particle : particles){
particle.position += 0.01f * particle.velocity;
Expand All @@ -505,7 +507,7 @@ void Scene1::simulateStep(){
}
particles.erase(std::remove_if(particles.begin(), particles.end(), [](const Particle& particle){
return particle.lifetime > 1.f;
return particle.lifetime > 1.5f;
}), particles.end());
Expand All @@ -527,142 +529,121 @@ void Scene1::simulateStep(){
}
```

Which are all necessary changes. You may still want to try the GLFW based variant as it shows some further C++ features and might be useful in general.
Which are all necessary changes. If you are very curious about input handling then implementing a GLFW event based version can be quite a challenge as it requires changes to multiple files, even outside of the Scenes folder, and potentially some manual tracking of inputs, however, this is not necessary as immediate mode inputs will suffice for all features for this course.

![Example image of the final result showing a spray of many spheres](https://github.com/user-attachments/assets/fb85a57d-fec7-451e-8d85-3baa0d8e38a3)

## Real Time Integration

### GLFW Event based Input
So far we only used a fixed hardcoded timestep, but this may not be the most appropriate solution for your game. A common way to do physics is to instead implement an adaptive timestepping that takes the framerate of the game into account, which you can simply get via ImGui using `ImGui::GetIO().DeltaTime`. Implementing this in our demo code then is straight forward:

Adding the GLFW event based input is significantly more involved as we first need to tell GLFW to send all key inputs to a custom function instead of letting ImGUI handle everything. To do this we need to make changes in the src folder (reminder: you do not submit these files!):
```cpp
// src/Simulator.cpp
void Simulator::init(){
//...
glfwSetWindowUserPointer(renderer.getWindow(), this);
// Scene1.cpp

glfwSetKeyCallback(renderer.getWindow(), [](GLFWwindow *window, int key, int scancode, int action, int mods) {
auto simulator = static_cast<Simulator *>(glfwGetWindowUserPointer(window));
simulator->onKeyInput(window, key, scancode, action, mods);
});
void Scene1::simulateStep(){
float realtimeDt = ImGui::GetIO().DeltaTime;
// ...
for (auto& particle : particles){
particle.position += realtimeDt * particle.velocity;
particle.lifetime += realtimeDt;
particle.velocity += gravityAccel * realtimeDt;
}
// ...
}
```

What we do here is to first set a _user_ pointer in the GLFW window state, which is useful as the event function cannot otherwise access the simulator instance. Note that we could also do this by using a Singleton design pattern for the simulator, which may be something you want to do in your own code. Also note that this requires some _C_ wizardry via void-pointers. Please try to avoid this as much as possible in your own code as casting things to and from void-pointers is very error prone with unpredictable and unexpected behavior, e.g., if things go wrong your code might order some pizza online.
## Mouse Input

We also need to actually add this onKeyInput function to the parent scene class

Something that may come in handy is having the ability to have a user click and drag on the screen to apply a force to the objects in the scene, even though it is not that useful for this demo example.

This can be done as well in our framework but is a bit more involved as this requires camera information that is not readily available during the simulator (a simulation does not have the concept of a camera in most cases). Accordingly we need to save all relevant information during a part of the render process where it is available to make it available during the simulation.

To do this we add a few members to our _Scene_ class:
```cpp
//Scene.h
// Scene.h
class Scene{
//...
virtual void onKeyInput(GLFWwindow* window, int key, int scancode, int action, int mods){}
glm::mat4 cameraMatrix = glm::mat4(1);
glm::vec3 fwd = glm::vec3(1, 0, 0);
glm::vec3 right = glm::vec3(0, 1, 0);
glm::vec3 up = glm::vec3(0, 0, 1);
}
```
To do this we need to override the onKeyInput function of the actual scene and create a structure to keep track of the currently pressed inputs (more on this later):
Which are then used to store all relevant information. In these members we will store the camera matrix and from this also derive the forward (the direction _into_ the screen) and the right and up vectors (relative to the camera coordinate system). The forward direction could also be used to _shoot_ objects into the scene, whereas the right and up will be used for a dragging interaction.
To capture this information we need to modify the _onDraw_ function (make sure to do this in the child scenes if they override the parent function too!):
```cpp
// Scene1.h
class Scene1 : public Scene{
//...
struct inputState{
bool space = false;
bool w = false, a = false, s = false, d = false;
bool e = false, q = false;
};
inputState keyState;
virtual void onKeyInput(GLFWwindow *window, int key, int scancode, int action, int mods) override;
// Scene.cpp
void onDraw(Renderer& renderer){
//...
cameraMatrix = renderer.camera.viewMatrix;
fwd = inverse(cameraMatrix) * glm::vec4(0, 0, 1, 0);
right = inverse(cameraMatrix) * glm::vec4(1, 0, 0, 0);
up = inverse(cameraMatrix) * glm::vec4(0, 1, 0, 0);
}
```

Key input in GLFW works by sending events whenever a key is pressed. To find out which key is being pressed we can look at the _key_ argument, e.g., if we press spacebar then key is equal to `GLFW_KEY_SPACE`. _mods_ contains any modifiers, i.e., shift, alt and control, whereas scancode is an internal value that we do not need to worry about too much. The important part for us now is the action which can have three values:

- GLFW_KEY_PRESS
- GLFW_KEY_HELD
- GLFW_KEY_REPEAT
To then add a dragging interaction for the right mouse button (the left is used for camera interactivity) we can check if the mouse button was released before the current frame using
```cpp
ImGui::IsMouseReleased(ImGuiMouseButton_Right)
```
The first value is sent on the initial press of the button and the repeat value is sent once we release the button, note that if a user tabs out of the program and releases the key without the games window being in focus, no release event will be sent and the input is _stuck_. We could naively use the held event to shoot spheres but this would not be fun on most systems as there is a multiple second delay before this event is triggered. In most applications this is beneficial (try pressing a key on your keyboard in an editor and notice the delay), but for us this is not ideal.
Which returns true in this case (we could also add interactivity where the button is _held_ down instead of released but this is left to the reader as an exercise). We can then get the amount of _pixels_ the mouse was dragged by using
```cpp
auto drag = ImGui::GetMouseDragDelta(1);
```
Note the 1 in the argument to get the drag delta for the _right_ mouse button. If the drag distance was zero we skip adding a force to an object, e.g., if this is done during a loop over all objects:
```cpp
if(ImGui::IsMouseReleased(ImGuiMouseButton_Right)){
auto drag = ImGui::GetMouseDragDelta(1);
if(drag.x == 0 && drag.y == 0)
continue;
//...
}
```

Instead we will key track of which key is currently held manually:
To now calculate the mouse force we multiply the drag in the x-direction with the _right_ vector of the camera and the drag in y-direction with the _up_ vector of the camera. Due to conventions and preferences some users may prefer inverting certain components, e.g., should the mouse being moved left move the object to the left (this makes a lot of sense for dragging interactions) or should we use a _spring_ like interaction where we move the mouse to the right and on releasing the button the distance we moved influences the force applied to the object in the opposite direction (this makes a lot of sense for on release interactions). In a full game engine it makes sense to make this configurable for the user but here we invert the y-coordinate (due to screen coordinate systems) and keep the x as is:
```cpp
// Scene1.cpp
void Scene1::onKeyInput(GLFWwindow* window, int key, int scancode, int action, int mods){
if(action == GLFW_PRESS){
if(key == GLFW_KEY_W)
keyState.w = true;
if(key == GLFW_KEY_A)
keyState.a = true;
if(key == GLFW_KEY_S)
keyState.s = true;
if(key == GLFW_KEY_D)
keyState.d = true;
if(key == GLFW_KEY_E)
keyState.e = true;
if(key == GLFW_KEY_Q)
keyState.q = true;
if(key == GLFW_KEY_SPACE)
keyState.space = true;
}
if(action == GLFW_RELEASE){
if(key == GLFW_KEY_W)
keyState.w = false;
if(key == GLFW_KEY_A)
keyState.a = false;
if(key == GLFW_KEY_S)
keyState.s = false;
if(key == GLFW_KEY_D)
keyState.d = false;
if(key == GLFW_KEY_E)
keyState.e = false;
if(key == GLFW_KEY_Q)
keyState.q = false;
if(key == GLFW_KEY_SPACE)
keyState.space = false;
}
if(ImGui::IsMouseReleased(ImGuiMouseButton_Right)){
auto drag = ImGui::GetMouseDragDelta(1);
if(drag.x == 0 && drag.y == 0)
continue;
auto dx = drag.x * right;
auto dy = -drag.y * up;
object.force += dx + dy;
}
```

Which is repetetive and error prone so try avoiding code like this if possible. We then modify our simulateStep function to no longer automatically rotate the cube and shoot a sphere when the spacebar is held down:
You might also want to scale this force or precompute the value once outside of looping over all objects to avoid recomputing the same values over and over again. Also make sure to apply this force in every substep of multistep integrators!

Applying this to our particle shooter is straight forward by modifying the simulateStep function (after including the changes to the Scene1 class and the onDraw function):
```cpp
// Scene1.cpp
void Scene1::simulateStep(){
// pitch += pitch_increment;
// roll += roll_increment;
// yaw += yaw_increment;
// ...
if(ImGui::IsMouseReleased(ImGuiMouseButton_Right)){
auto drag = ImGui::GetMouseDragDelta(1);
if(!(drag.x == 0 && drag.y == 0)){
auto dx = drag.x * right;
auto dy = -drag.y * up;
for (auto& particle : particles){
particle.velocity += (dx + dy) * 0.01f;
}
}
}
}
```

glm::vec3 gravityAccel = glm::vec3(0, -9.81f, 0);
And an example of the behavior this leads to after dragging particles to the right:

for (auto& particle : particles){
particle.position += 0.01f * particle.velocity;
particle.lifetime += 0.01f;
particle.velocity += gravityAccel * 0.01f;
}
particles.erase(std::remove_if(particles.begin(), particles.end(), [](const Particle& particle){
return particle.lifetime > 1.f;
}), particles.end());
![Image showing mouse interaciton](https://github.com/user-attachments/assets/2f770c5c-f35d-4ff2-86c7-fcab37bfa1e0)

if(keyState.space)
launchSphere();
if(keyState.w)
pitch += pitch_increment;
if(keyState.s)
pitch -= pitch_increment;
if(keyState.a)
roll += roll_increment;
if(keyState.d)
roll -= roll_increment;
if(keyState.q)
yaw += yaw_increment;
if(keyState.e)
yaw -= yaw_increment;
lastLaunch++;
}
While this interaction is not _physical_ it allows us to add some fun interactivity. If you make these changes you may also want to change how long particles exist as they vanish very quickly which you can do by updating
```cpp
particles.erase(std::remove_if(particles.begin(), particles.end(), [](const Particle& particle){
return particle.lifetime > 1.5f;
}), particles.end());
```

Which gives us our final interactive program that covers all relevant aspects of the framework!

![Example image of the final result showing a spray of many spheres](https://github.com/user-attachments/assets/fb85a57d-fec7-451e-8d85-3baa0d8e38a3)
You may also want to consider making the lifetime a GUI element or updating it based on the interactions, e.g., a particle that was dragged around has the lifetime counter reset.

0 comments on commit 584360b

Please sign in to comment.