Optimal Clustering of Substations for Topology Optimization Using the Louvain Algorithm

[BamunugeDR99]

Related Problem & Feature Request

Related Problem

In the official Grid2Op repo; a Multi-Agent Reinforcement Learning(MARL) example is currently being developed under the branch dev_multiagent .

In the current implementation, when creating the multi-agent environment a parameter called ACTION_DOMAINS is passed. This represents a dictionary that maps agent names to a list of substations they control. This approach aims to cluster the grid into smaller subgrids, each managed by an individual agent.

However, the primary issue with the current implementation is the lack of an optimal clustering methodology. Currently; as shown below, the agents and the substations they control are hard-coded (i.e. the substation ids are manually assigned by hand) for the l2rpn_case14_sandbox environment.

ACTION_DOMAINS = {
        'agent_0' : [0, 1, 2, 3, 4],
        'agent_1' : [5, 6, 7, 8, 9, 10, 11, 12, 13]
    }

This hardcoded clustering approach is not ideal because:

1. Lack of Flexibility: Hardcoding limits the adaptability of the solution to different environments and scenarios.

2. Suboptimal Clustering: Without a systematic method for clustering, the current setup may not effectively optimize the agents' performance.

Feature Request

Due to the limitations of the current hardcoded clustering, our team @rootcodelabs is exploring methods to perform this clustering more optimally. Currently, manually assigning IDs to agents and clustering the grid does not adequately consider factors such as connectivity and the inherent relationships between different sections of the grid. This can lead to sub-optimal performance of the agent. To address this, we are utilizing a scalable clustering algorithm that can better partition the grid. This algorithm will ensure optimal clustering, with subgrids sharing more internal connectivity. This approach aims to improve overall performance and scalability by creating clusters that are more logically connected and efficient.

By addressing this issue, we aim to strengthen the current MARL approach implemented within Grid2Op.

Detailed feature request is available at (#613)

Solution: Clustering Substations using the Louvain Algorithm

To improve substation clustering in the Grid2Op environment, our team explored various graph clustering algorithms. The basis for exploring these algorithms lies in their ability to analyze and partition the grid based on connectivity. Graph clustering algorithms help identify communities or clusters within a network, ensuring that closely connected substations are grouped together. This results in more cohesive subgrids and optimized performance. After evaluating multiple such algorithms, we selected the Louvain Algorithm for its superior performance in detecting community structures within large networks and its ability to maximize modularity. This ensures that the resulting clusters have dense internal connections and sparser connections between them, making it particularly suitable for our needs.

Solution Implementation

• The solution is implemented as a utility function and can be viewed in utils.py

from grid2op.Environment import Environment
import numpy as np
from sknetwork.clustering import Louvain
from scipy.sparse import csr_matrix

class ClusterUtils:
    """
    Outputs clustered substation based on the Louvain graph clustering method.
    """
    
    # Create connectivity matrix
    @staticmethod
    def create_connectivity_matrix(env:Environment):
        """
        Creates a connectivity matrix for the given grid environment.

        The connectivity matrix is a 2D NumPy array where the element at position (i, j) is 1 if there is a direct 
        connection between substation i and substation j, and 0 otherwise. The diagonal elements are set to 1
        to indicate self-connections.

        Args:
            env (grid2op.Environment): The grid environment for which the connectivity matrix is to be created.

        Returns:
            connectivity_matrix: A 2D Numpy array of dimension (env.n_sub, env.n_sub) representing the 
            substation connectivity of the grid environment.
        """
        connectivity_matrix = np.zeros((env.n_sub, env.n_sub))
        for line_id in range(env.n_line):
            orig_sub = env.line_or_to_subid[line_id]
            extrem_sub = env.line_ex_to_subid[line_id]
            connectivity_matrix[orig_sub, extrem_sub] = 1
            connectivity_matrix[extrem_sub, orig_sub] = 1
        return connectivity_matrix + np.eye(env.n_sub)

    
       
    # Cluster substations
    @staticmethod
    def cluster_substations(env:Environment):
        """
        Clusters substations in a power grid environment using the Louvain community detection algorithm.

       This function generates a connectivity matrix representing the connections between substations in the given
       environment; and applies the Louvain algorithm to cluster the substations into communities. The resulting 
       clusters are formatted into a dictionary where each key corresponds to an agent and the value is a list of 
       substations assigned to that agent.

        Args:
            env (grid2op.Environment): The grid environment for which the connectivity matrix is to be created.
            
        Returns:
                (MADict):
                    - keys : agents' names 
                    - values : list of substations' id under the control of the agent.
        """

        # Generate the connectivity matrix
        matrix = ClusterUtils.create_connectivity_matrix(env)

        # Perform clustering using Louvain algorithm
        louvain = Louvain()
        adjacency = csr_matrix(matrix)
        labels = louvain.fit_predict(adjacency)

        # Group substations into clusters
        clusters = {}
        for node, label in enumerate(labels):
            if label not in clusters:
                clusters[label] = []
            clusters[label].append(node)

        # Format the clusters
        formatted_clusters = {f'agent_{i}': nodes for i, nodes in enumerate(clusters.values())}
        
        return formatted_clusters

Sample Execution & Output

• Sample execution can be viewed in ray_example3.py

import grid2op
from lightsim2grid import LightSimBackend
from grid2op.multi_agent import ClusterUtils


ENV_NAME = "l2rpn_case14_sandbox"
env = grid2op.make(ENV_NAME, backend=LightSimBackend())

# Get ACTION_DOMAINS by clustering the substations
ACTION_DOMAINS = ClusterUtils.cluster_substations(env)

Output

{
'agent_0': [0, 1, 2, 3, 4],
'agent_1': [5, 11, 12], 
'agent_2': [6, 7, 8, 13], 
'agent_3': [9, 10]
}

[BDonnot]

Hello,

Some tests does not appear to pass, but i'm not sure it's due to your PR.

As soon as I can i'll fix the "dev_multiagent" branch and tests should pass (hopefully) here too.

Sorry for the delay :-/