Does Stream Per Subject have any performance benifits over multiple subjects within same stream #6100
-
If i have 100 subjects, is it ideal to create 100 streams, as stream in jetstream is the main queue which forwards messages to consumers, or is it better to have 1 stream with 100 subjects with subject filters? I have tested that in a 1 stream & 100 subjects configuration, even if 1 consumer goes down, it does not block the other unique messages arriving on diffrent subject filters. So which scnario is optimal? Could someone shed some light on this aspect. |
Beta Was this translation helpful? Give feedback.
Replies: 3 comments 1 reply
-
Many Consumers per single Stream is usually the simple and good pattern as a starting point. Each Stream, if replicated (3 or more replicas) will have some overhead for maintaing its own RAFT group. Consumers can also scale on Stream - as each consumer leader (if replicated) can live on different Stream replica, balancing the load. |
Beta Was this translation helpful? Give feedback.
-
@Jarema could you pls provide some documentation if available where i can understand the internals of how jetstream manages consumers in single stream & many subject scenario |
Beta Was this translation helpful? Give feedback.
-
Thank you so much @Jarema |
Beta Was this translation helpful? Give feedback.
Many Consumers per single Stream is usually the simple and good pattern as a starting point.
It's rarely a good idea to have stream per subject.
Each Stream, if replicated (3 or more replicas) will have some overhead for maintaing its own RAFT group.
NATS with it's unique subject wildcards and filters allows for efficient filtering messages from the stream.
I would stick to one stream, unless you need different retention policies for some subjects for example.
Consumers can also scale on Stream - as each consumer leader (if replicated) can live on different Stream replica, balancing the load.