Segments distribution view

[Archalbc]

Hi !

What about adding a treemap showing the segments distribution among nodes ? The goal is to get a better view of the current situation and help understand why some querynodes are overloaded. Having the possibility directly move segment is a bonus.

Currently I'm able to do that using a Python script calling pymilvus utility.get_query_segment_info() and collection.get_replicas(), and generate a treemap view with plotly.express. I modified some javascript stuff inside to allow copy/paste the segment ID by right-click and call next manually call a move-segment.py script.

Treemap are interactive and use this logic :

• Treemap by Node: 'All', 'Node', 'Collection', 'Segment_Type', 'Segment_ID',
• Treemap by Collection: 'All', 'Collection', 'Group_ID', 'Node', 'Segment_ID'

The treemap by Collection add the "Replica_group" granularity to avoid error while trying to move segment on another node with a different replicas group.

"embedding_coll_x" are my anonymized collections ;)

by Collection

by Node

This is more or less something like the Elasticsearch Cerebro UI

You can have something similar with grafana, but it's not very usable.

Next I would like to add the memory usage per node and their total number of sealed segments to help me choose the right node to move the segment without switching with grafana.

[shanghaikid]

This is cool, I would like to add this later.

[Archalbc]

Hi,
new version of my internal homemade tool so far:

• add the association between node-id and pod_name
• the number of sealed/growing segments
• the memory usage
• Treemap by collection is sized using Entites number
• Treemap by node is sized by Memory usage
• a quick homemade favicon for the flavor x)
• a little helper in the nav bar to pre-fill a script to move segments using pymilvus (currently I have to run it using my terminal but later, my tool will probably allow to do it by itself). The field is automatically filled by a right-click on an element in the treemap.

Using the :9091/metrics endpoint of querynode and kubernetes API in python.

[shanghaikid]

Hi, new version of my internal homemade tool so far:

• add the association between node-id and pod_name
• the number of sealed/growing segments
• the memory usage
• Treemap by collection is sized using Entites number
• Treemap by node is sized by Memory usage
• a quick homemade favicon for the flavor x)
• a little helper in the nav bar to pre-fill a script to move segments using pymilvus (currently I have to run it using my terminal but later, my tool will probably allow to do it by itself). The field is automatically filled by a right-click on an element in the treemap.

Using the :9091/metrics endpoint of querynode and kubernetes API in python.

This looks cool, can you share your code?

[Archalbc]

Hi ! Thank you for your feedback, I'm asking internally if I can share the code somehow ;)

[Archalbc]

Hey @shanghaikid I sent you an email with the code :)

[shanghaikid]

Thank you so much, I will look at this soon.

[douglarek]

so cool.