This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/ACCESS.2020.3046503, IEEE Access
Guo Fu et al.: A Fair Comparison of Message Queuing Systems
ent. According to whether the message protocols used are
open to industry, they can be classified into open protocols
and private protocols. The widely used open message pro-
tocols include AMQP [17], XMPP [18], REST [19], and
STOMP [20]. Some systems extend basic protocols accord-
ing to their own requirements, while some use customized
messaging protocols. These protocols are called private pro-
tocols.
Consumption Mode. There are two modes of consump-
tion: Push and Pull. (1) Push mode requires that the mes-
sage system actively pushes messages to the consumer. This
mode has better real-time performance, but requires a certain
flow control mechanism. Once the system pushes a large
number of messages to the consumer, it will incur heavy
load to consumer to make it slow or even crash the service.
(2) Pull mode refers to that the consumer actively requests
pull messages from the message system. In this mode, the
consumer can consume message data according to its own
capacity and send pull request to the system every other time
interval. However, this requires consumer to set a reasonable
message pull interval, which is not trivial. Too frequent pull
requests will incur significant heavy burden to the message
system, but if the pull requests are not frequent enough it
would inevitably cause data latency problem.
System Architecture. Distributed message queuing sys-
tems typically have two architectures: master-slave and peer-
to-peer. In master-slave architecture, Brokers are divided into
a master and multiple slaves. The master provides service
and the slaves serve as its backup. Once the master goes
down, the slaves can take over related service. In peer-to-
peer architecture, all Brokers have the same status, and each
message is backed up in multiple Brokers to be resistant to
message loss and machine failure.
Message Queuing Model. There are two message queu-
ing models in message service: point-to-point and publish-
subscribe. In point-to-point model, every message will be
sent to a specific Queue, only one corresponding consumer
can obtain this message. In publish-subscribe model, every
message has a category called Topic, a subscriber to a Topic
can consume all of its messages, and a Topic can be sub-
scribed by multiple consumers.
Usability. The usability of a system is also an important
production feature, which helps them get start easily. The
usability can be evaluated in several aspects, such as the
easiness of installation, the completeness of documentation,
the functionalities of management and monitoring, etc. A
system with high usability can attract more users by saving
their time on learning to use and maintain it.
Compatibility. Message queuing system can play the role
of message middleware for many-task computing on a big-
data platform [21]. Therefore, compatibility with other tools
such as storage and processing is also an important feature
that should be considered. We mainly analyze the compatibil-
ity of message queuing systems with the mainstream storage
system HDFS [1] and the big data processing system Spark
[22] and Flink [23].
B. QUALITY-OF-SERVICE GUARANTEE FEATURES
As a middleware system providing message buffering ser-
vice, the message systems are defined by several required
and desired guarantees. Especially for distributed message
systems, these quality-of-service guarantees become more
important in production usage.
Delivery Guarantee. To ensure that messages are trans-
mitted between producers and consumers, there are generally
three delivery guarantees : (1) at-most-once: Messages are
transmitted at most once, they may be lost, but will never
be transmitted repeatedly. (2) at-least-once: Messages are
transmitted at least once, they will never be lost, but may be
repeated. (3) exactly-once: Every message must be transmit-
ted once and only once. For most message queuing systems,
only at-most-once guarantee or at-least-once guarantee is
provided [24]. It is difficult to realize exactly-once guarantee.
Ordering Guarantee. In a message system, it is very
important to ensure that messages are sent and consumed
in the same order. In some application scenarios, the order
of messages is very strict. There are three ordering guar-
antee variants: (1) no-ordering: This is an ideal case for
performance. Because no ordering guarantee is provided, no
additional resources are used in this mode. (2) partition(or
queue)-ordering : In some message queuing systems, a par-
tition is the basic unit for parallel operations. In this mode,
messages are guaranteed to be ordered within a partition,
but it is not guaranteed across partitions. Though requiring
more resources than the previous mode, it can also have high
performance. (3) global-ordering: In order to provide a global
ordering guarantee across different channels, it requires lots
of resources to keep synchronization between producers and
consumers, which significantly degrades the performance of
message systems.
Reliability. Message queuing systems should be strong
against software and hardware failures. If one or more ma-
chines are down, other machines can be used without affect-
ing the system. This relies on the synchronization and repli-
cation of messages, which will take up additional hardware
and software resources.
Scalability. The scalability means that a message queu-
ing system can continue to scale as demand increases. For
instance, the processing ability can be easily changed by
adjusting the number of messages, partitions, or produc-
ers/consumers. This can be achieved by adding new machines
(or other hardware resources) to the system.
Transaction Support. The transaction requirement re-
quires that: either a complete sequence of messages is sent
(received), or none of them is. For instance, the transaction
can be successfully committed only after the consumer has
consumed the entire message queue.
C. PERFORMANCE FEATURES
Regarding the performance features, two key metrics are the
latency (or response time) and the throughput [15].
Latency. Latency measures how long it takes for a mes-
sage to be transmitted between endpoints. In message sys-
VOLUME 4, 2016 3