What is the idea behind, and the implications of, forwarding all events to a single queue?

[michielvoo]

I noticed that, with the Azure Service Bus transport, when subscribing to several topics, the subscriptions in Azure Service Bus are all configured to forward messages to the same queue, the input queue that is specified during configuration of Rebus in normal mode.

To be honest, I am not really sure what the implications are for message delivery. If the application would receive messages from multiple subscriptions instead of the single queue, would that change the order in which messages are delivered to the application? Consider this scenario: subscriptions A and B each have 1000 messages, and subscriptions C is empty. Then a single message appears in subscriptions C.

With forwarding, this single message will be number 2001 in line to be processed. Without forwarding, is there any reason to believe that the single message in subscription C is delivered to the application earlier by Azure Service Bus? Perhaps delivery of messages to a single application from multiple subscriptions works such that the single message in subscription C will be number 3 in line to be processed.

This is currently my assumption, that forwarding may simplify the way Rebus is configured (just specify a single input queue), but will result in delayed delivery of messages from low(er)-volume subscriptions.

[mookid8000]

You are absolutely right in your observation that Rebus (1 instance of it) will always receive messages from one single input queue, and so the act of subscribing to topics is done by "binding" topics to the input queue, thus creating a topology similar to how the "topic exchange" works with RabbitMQ/AMQP.

Since the "forware-to-queue"-functionality is configure on the subscription, I am pretty sure that messages published to the topic get dispatched quickly to the owned subscriptions and thus delivered to any queues configured as forwarding targets, so I would be very surprised if an event could "overtake" any other events.

To translate it into the scenario you're describing: Subscriptions A and B would probably NEVER have 1000 messages each, because the messages would have been forwarded to the target queue immediately when received in the subscription, so the messages would simply end up being processed in the order they're in in the queue(*). And then, when the single message gets delivered to C it will simply be number 2001 in the queue if none of the other messages have been processed yet.

I hope that made sense 🙂

---

(*) With the ever-lasting caveat that messages can end up being processed out-of-order because of parallel processing and/or having resided in dead-letter queues for a while. Never rely on message ordering when using queues, because messages in queues are meant for ATOMIC bits of work and not for STREAMS of things.

[michielvoo]

Indeed, what I meant to say is that subscriptions A and B received a 1000 messages each, and those messages would have already been forwarded to the target queue by Azure Service Bus.

Thanks for explaining the part about the topology being similar to that of RabbitMQ, I think it's important to know for developers that are not using RabbitMQ.

I guess then to change the topology through code would require configuration of multiple Rebus instances. For example, if I know subscription C is low volume, and I don't want messages of type C to queue behind those from subscriptions A and B, I should set up a second Rebus instance, with its own target queue, and use it to subscribe to messages of type C.