A simple HTN planner based around the principles of the Builder Pattern, Original ported into C++ Version, used to create a Godot template.

This is a work in progress! I have not created an example to test the success of this design yet.

Disclaimer

Due to the level of abstraction and high customization of Fluid HTN, it is impossible to port into Godot. The best solution is to develop the AI system in C++, but develop the sensors and operators in GdScript and have the C++ code call those functions.

Since GdScript is duck typed and does not have interfaces, porting this to GdScript would be challenging since this project relies on the Builder Pattern, Queues, etc.

Usage

For step by step instructions on how to use this template see : Project Read Me

How it Works

Agents are fully designed in the C++ extension, operators and sensors are done in GdScript. Once you have fully designed your Agent(s) knowledge/context, domain int c++ and in gdextension call all your operators from gdscript functions, you can then compile the plugin and include it's binaries in your project where Godot will call the binaries of your built extension. For more information on building see : Project ReadMe

1. Context
   1. Knowledge system, the brain of the Agent
   2. Enums for what it knows in the moment
      1. i.e. EnemyInRange
   3. Lists for data it needs to know about
      1. i.e. KnownEnemies
2. Domain
   1. How the Agent knows what to do and if they can do what
   2. Priority
      1. Items higher in the Domain are chosen if they meet their preconditions
3. Effects
   1. If an Action succeeds how does it change the Agents Knowledge
   2. Effects is our prediction of how state will change for performing said action
4. Conditions
   1. What must be true for an Action to be chosen
5. Operators (A.K.A. ActionsAPI)
   1. Agent Actions can be fully built in Godot/GdScript or C++.
   2. If GdScript is chosen, we can define operators in GdScript and execute them from C++
      1. i.e. Execute : MoveTo(enemyX14)
         1. Where enemyX14 was chosen by the Planner based on the Knowledge of the AI's Context
      2. The Planner needs to know the success or failure of an operator
   3. If C++ is chosen, updating the Agent will require passing in necessary information like delta, position, animations, etc.
6. Sensors (A.K.A. StimuliAPI)
   1. Sensors can be fully built in Godot/GdScript and then encoded and passed to the AI
   2. i.e. Collisions shape on Agent collides and detects using Raycast that enemy is in front of it. Update the Agent's world state by sending the sensor data of 'EnemyInRange' to GDExtension.

The Process of Agent System Creation

The steps necessary to understand and design an Agent System.

Step 1 : Define the Problem

Start by writing out what you want the Agent to be able to do and learn about.

i.e. I want an Agent that can pick up apples, see enemies, shoot enemies, etc.

Step 2 : Define When & What

When can the Agent do the things you want it to do?

i.e. They can shoot when they see an enemy, they can pick up apples that are within range, etc.

What happens when the Agent does the things you want it to do?

i.e. After shooting enemy bullets decrease, after picking up an apple there are less apples on the ground, etc.

Step 2 : Create HTN Notation from the Problem

Now that we know what we want to accomplish. We can then break it down into its parts.

We do this by defining the ecosystem of the Agent(s) in HTN Notation. We can call this "Agent Notation".

The following describes how to create and write your Agent logic before you even begin to code:

Select Task [TaskName(term1, term2,...)]
    Preconditions [Condition1, Condition2,...]
        Subtasks [task1(term1, term2,...). task2(term1, term2,...),...]
    Splice ( subdomain1 )
    Preconditions [Condition1, Condition2,...]
        Subtasks [task1(term1, term2,...). task2(term1, term2,...),...]
PausePlan()
Sequence Task [TaskName(term1, term2,...)]
    Slot( slotID )
    Preconditions [Condition1, Condition2,...]
        Subtasks [task1(term1, term2,...). task2(term1, term2,...),...]

Primitive Task 1 [task1()]
    Preconditions [<conditions>] //Optional
    Operator [task1Operator(<parameters>)]
        Effects [<effect>] // Optional
Primitive Task 2 [task2()]
    Preconditions [<conditions] //Optional
    Operator [task2Operator(<parameters>)]
        Effects [<effect>] // Optional

HTN Notation Adapted From Chapter : 12.3

The goal is to split up the Agents features into Awareness, Knowledge, Actions, and Tasks. Take everything you decided about your Agent and split it up into the following using HTN Notation:

1. Choices
   1. The top successful/valid option is the decision that Agent will make
2. Awareness
   1. Stimuli from the Agents environment
   2. Sensors that collect information and convert into Knowledge the AI can understand
3. Knowledge
   1. You can have as many of these as you need
   2. i.e. AgentWorldKnowledge
      1. i.e. EnemyInRange | Agent can see an enemy
      2. i.e. AgentHealth
4. Tasks
   1. What the Agent can decide to do
   2. Your Selector is your IF/ELSE chain, and your Sequence is your AND chain
   3. Selector - Complete a Single Task
      1. i.e. Can Shoot a target
         1. Pre-Conditions
            1. i.e. If EnemyInRange
   4. Sequence - Complete all sub-tasks successfully
      1. Will require that ALL sub-tasks are valid and will branch down each one in order.
      2. i.e. FindEnemy(), MoveToEnemy(), AttackEnemy()
5. Actions/Primitives (2 Separate things in Code but same thing in idea)
   1. What the Agent can do
   2. i.e. Shoot Target
      1. Effects
         1. i.e. AgentAttackScore += 1

Agent Notation Example :

Select Task [ KillEnemy() ]
    Preconditions [ (EnemyRange, DashBackRange, LessOrEqual), (EnemyVisible, True) ]
        Primitive(DashBack)
    Preconditions [ (EnemyRange, LessOrEqual), (EnemyVisible, True) ]
        Primitive(MoveAwayFromEnemy)
        Primitive(ShootAtEnemy)
    Preconditions [ (Location, NavigationNearEnemy, NotEqual), (EnemyRange, GreaterOrEqual), (EnemyVisible, True), (PathBlocked, True)]
        Primitive(MoveTo)
            Operator [ MoveTo() ]
        Primitive(NavigationIdle)
            Operator [ NavigationIdle() ]
    Precondition(EnemyVisible, False)
        Select Task [EnemyLost() ]
Select Task [ EnemyLost() ]
    ...
Select Task [ RushEnemy() ]
    PrimitiveTask [ (RunTowardsCover) ]
    PrimitiveTask [ (EnterCover) ]
    PrimitiveTask [ (ThrowGrenade) ]
    PrimitiveTask [ (ShootAtEnemy) ]
PrimitiveTask [ (ShootAtEnemy) ]
    Preconditions [ (EnemyVisible, True) ]
        Operator[ ShootAtEnemyOperator() ]
            Effects [ (RotateToFocus), (Vulnerability, 1), (Frustrated, 0) ]
PrimitiveTask [ (RunTowardsCover) ]
    ...
PrimitiveTask [ (EnterCover) ]
    ...
PrimitiveTask [ (ThrowGrenade) ]
    ...
PrimitiveTask [ (ShootAtEnemy) ]
    ...
PrimitiveTask [ (ScanPatrolPoint) ]
    Preconditions [ (AlertLevel, IGNORANT), (EnemyVisible, False) ]
        Operator[ ScanPatrolPointOperator() ]
        Effects [ (AreaClear, True) ]
Primitive(MeleeEnemy)
    Preconditions [ (EnemyInRange, True) ]
        Operator [ MeleeEnemyOperator() ]
            Effects [ (Damaged, False),(IsInPain, NO_PAIN), (Balanced, True) ]
Primitive(MoveTo(destination))
Primitive(NavigationIdle)

Step 3 : Convert your HTN Notation to Fluid HTN

Now that you have taken the time to really think out what you want your Agent(s) to do, it should be trivial how to convert it to code. Take your HTN Notation (which is a written version of the Fluid HTN Domain) & implement everything you need for the Agent to plan, come alive.

The following attempts to illustrate the architecture of the CPP version of Fluid HTN:

Based on your Agent Notation:

1. Implement : Sensors
   1. Awareness inputs of things the Agent can consume
   2. Define Simple systems that create Knowledge/Enum-Values
2. Implement : World State
   1. Knowledge : WorldStates you need and what their values will be.
   2. Define the Enums
3. Implement : Actions
   1. Primitive task that can contain conditions, an operator and effects.
   2. Defined in the Domain
4. Implement : Operators
   1. Executions of Primitive Tasks
   2. Define IOperator for each primitive task
   3. Must return success or failure
5. Implement : Pre-conditions
   1. Requirements for choosing a task
   2. Conditions(string name, Func<T, bool> condition)
   3. Must return a boolean that represents a success or failure
6. Implement : Effects
   1. State Changes if a task is successful
   2. Effect(string name, EffectType effectType, Action<T, EffectType> action)
7. Implement : Domain
   1. The Domain definition : term to describe the entire task hierarchy

Using the example from Fluid HTN Troll Bridge Example the pre-conditions, effects, actions, & end(); are all consumed into the AIBuilder.

This simplifies the Domain to look like in the example:

return new AIDomainBuilder("Troll")
            .Select("Received damage")
                .ReceivedDamage()
            .End()
            .Sequence("Bridge patrol")
                .HasState(AIWorldState.HasBridgesInSight)
                .FindBridge()
                .MoveToBridge()
                .Wait(2f)
            .End()
            .Build();

Where .ReceivedDamage() includes, action, pre-condition, operator, effect, and End().

The condition .HasState(AIWorldState.HasBridgesInSight) is not inside a primitive because it must be validated before going into the FindBridge() primitive.

Step 4 : Code

Assuming you want to write the Operators & Sensors in GdScript.

Implement the following Interfaces:

1. BaseContext | i.e. AgentA : BaseContext
   1. Brain of the Agent
   2. Enums for what it knows in the moment
      1. i.e. EnemyInRange
   3. Lists for data it needs to know about
      1. i.e. KnownEnemies
2. BaseDomainBuilder
   1. Contains Operators
3. IOperator
   1. Create Operators for all possible Primitives
   2. This simply calls the functions from GdScript
4. Domain
   1. The HTN of your Agent Type
   2. This can include subdomains, slots, and splices
5. ISensors
   1. Functions that consume World State and update Agents Knowledge

Implement the following in Godot:

1. Sensors
   1. i.e. Enemy Detection -> EnemyInRange
   2. Send data converted to WorldState values to the C++ code
2. Operators
   1. Agents ActionAPI

Step 5 : GDExtension

Connect up your code to Godot using GDExtension by exposing the necessary functions & signals.

Fluid HTN Overview

Each Agent has a brain of knowledge, a sensory system, an actions system, and a decisions system.

The goal of HTN is to go through the decomposition and find a plan that succeeds. Going through the designed Choices, Conditions, Effects, Operators and World State.

A link to a simple example

Parts of this Read Me are pulled from : Fluid HTN Ref

Sensory System

The agent needs a way to know what is going on around it to be able to make decisions.

Sensors are ways in which the Agent knows what it can and cannot do.

Knowledge System

The agent needs a way to store information about the world around it to be able to understand it. You have to define what the data means to the AI, and you also have to both collect it (sensors) and translate it into the context (brain) that the AI understands.

Knowledge in an HTN is encoded by using an ENUM that defines the world state.

"The world state is essentially a vector of properties that describe what our HTN is going to reason about." Chapter : 12.2.1 "...In the case of WsIsTired, the byte can represent the Boolean values zero and one. With WsEnemyRange, the values in the enum EEnemyRange are used. It’s important to note that the world state only needs to represent what is needed for the HTN to make decisions. That’s why WsEnemyRange is represented by abstract values, instead of the actual range. The goal of the world state isn’t to represent every possible state of every possible object in the game. It only needs to represent the problem space that our planner needs to make decisions."

    enum EHtnWorldStateProperties
    {
        WsEnemyRange,
        WsHealth,
        WsIsTired,
        …
    }

    enum EEnemyRange
    {
        MeleeRange,
        ViewRange,
        OutOfRange,
        …
    }

    vector<byte> CurrentWorldState;
    EEnemyRange currentRange = CurrentWorldState[WsEnemyRange];
    CurrentWorldState[WsEnemyRange] = MeleeRange;

Decisions System

HTN outputs a plan, which is a sequence of tasks. Decisions are defined by creating Primitives and Compound Tasks. Where Primitives are single step and compound are a set of primitives.

Primitive Tasks

Primitive Tasks are comprised of an operator and sets of effects & conditions.

1. Conditions
   1. In order for a primitive to execute conditions must be valid
   2. Executing Conditions
      1. Evaluated every planner tick
      2. Used for cases where you need to re-evaluate the validity of your conditions after planning and after setting a new task as current task during execution
         1. (e.g. the operator of the task will return continue).
         2. Here, the operator returns continue and the condition becomes "invalid", the planner will not automatically know.
         3. Executing Conditions is one way to ensure that the planner realize that the task is now invalid.
      3. Another way is to put this logic inside the Operator and have it return Failure, which should yield the same result in practice (it triggers a replan)
2. Effects
   1. Effects is our prediction of how state will change for performing said action
   2. If task is successful, effects will apply to Agents world state and make the current plan we are evaluating continue to formulate
   3. PlanOnly : Effect
      1. Temporarily changes the world state during planning, used as prediction about the future.
      2. Its change on the world state is removed before plan execution.
      3. This can be useful when we need other systems to set the world state during execution.
   4. PlanAndExecute : Effect
      1. Only that during execution
      2. When the task they represent complete its execution successfully, the effect is re-applied
      3. This is useful in the cases where you don't have other systems to set the world state during execution.
   5. Permanent : Effect
      1. Applied during planning, but not removed from the world state before execution
      2. This can be very useful when there's some state we change only during planning
      3. e.g. do this thing three times then do this other thing.
      4. It can also be useful when we want to update world state in our runtime code, where we definitely want our changes to be permanent.
3. Operator
   1. Atomic Action the Agent does

"As an example, let’s take the two tasks SprintToEnemy and WalkToNextBridge. Both of these tasks use the MoveTo operator, but the two tasks change the state of our NPC in different ways. On the successful completion of SprintToEnemy, our NPC will be at the enemy and tired, specified by the task’s effects. WalkToNextBridge task’s effects would set the NPC’s location to the bridge and he’d be a little more bored." Chapter : 12.2.3

Compound Tasks

Can be understood as high level tasks that have multiple ways of being accomplished.

"Using the Trunk Thumper as an example, he may have the task AttackEnemy. Our Thumper may have different ways of accomplishing this task. If he has access to a tree trunk, he may run to his target and use it as a melee weapon to “thump” his enemy. If no tree trunks are available, he can pull large boulders from the ground and toss them at our enemy. He may have a multitude of other approaches if the conditions are right." Chapter : 12.2.4

In Fluid HTN, Compound tasks are split between "Selectors" and "Sequencers".

1. Selectors
   1. Must be able to decompose a single sub-task
2. Sequencers
   1. Must be able to decompose all its sub-tasks successfully for itself to have decomposed successfully
3. Random Select
   1. Extensions Project : https://github.com/ptrefall/fluid-hierarchical-task-network-ext
4. Utility Select
   1. Extensions Project : https://github.com/ptrefall/fluid-hierarchical-task-network-ext

Actions System

Actions are defined as Operators. The things the AI can actually do.

Every time an operator updates, it returns a status whether it succeeded, failed or need to continue next tick.

Partial Planning

Used only inside a Sequence in the Domain. It allows to only plan up to that point and continue when the partial part of the plan has completed.

Splicing

Splicing lets us inject sub-domains for code-reuse.

Is a selector, so the root of a sub domain will always be encapsulated in a slot selector of the parent domain

Slots

Slots allows us to inject subdomains at runtime.

Ability to setup compound tasks specially designed for specialized behavior that are added into the hierarchy at runtime. Typically this would be smart objects, scripted events or meeting needs to add or remove new behavior at runtime, like binding abilities to weapons or spells or otherwise.

You can setup pre-determined slots that are filled later dynamically.

Interaction with smart objects only if they are in range or available.

Planner Execution

The execution will check conditions of a task when it starts, run execution conditions every tick, run the task operator every tick and apply "executing" effects upon task success.