Full-service Internet Marketing & Web Development
Recent Posts
Sponsors
 |
RabbitMQ Message QueueMike Peters, 04-16-2010 |
RabbitMQ is an open source messaging server, allowing you to scale by queuing jobs to be processed offline.
If you're looking to scale your service, integrating message-queues into your application is a MUST. Without offline processing, you'll quickly hit a glass ceiling on maximum throughput.
In the world of message-queues, there are lots of different solutions. From the over-simplistic PHP Dropr, through RabbitMQ, ActiveMQ, Fuse, ZeroMQ and commercial enterprise queue systems.
There's a great post by the makers of SecondLife, sharing their evaluation of 9 popular message-queue systems here. Take the time to review available message-queues and test which one best suits your needs.
Here at SPI, our original implementation of distributed message-queues was MySQL based. Two master-master databases, sharing a queue table. While it did work for our needs, databases suck for message queues and we needed a better solution.
We had a short run with ActiveMQ -- another popular message-queue solution, but were not happy with the test results. RabbitMQ is a lot faster (x40 times), supports AMQP and is better suited for distributed clusters.
Let's cover the steps to install RabbitMQ on your server.
1. Getting started
Download the latest version of RabbitMQ from this page.
Use the Bin version. No need to recompile.
2. Download JRE
Download and install the version of JRE matching your system (32bit or 64bit) from the Java SE Download page.
3. Install Erlang
If you're on FreeBSD, simply do:
Otherwise, Download Erlang.
4. Start RabbitMQ
Changedir to the path where you installed RabbitMQ and start it with:
Simple right? You now have the RabbitMQ up and running (in the background), ready to relay messages.
It's important to understand the terminology and building blocks (Queues, Exchanges, Bindings) of RabbitMQ.
Take the time to review the Getting Started Slides and FAQ.
5. Communicating with RabbitMQ
Now that we have RabbitMQ up and running we can start the fun part of writing and reading messages.
For the purpose of this guide, we'll use PHP to communicate with RabbitMQ.
There are several PHP-wrappers for RabbitMQ. Our favorite one is the clean and simple PHP amqp lib.
The code below uses php-amqplib to connect and send messages to a RabbitMQ server.
The publisher (writes messages to queue):
require_once('amqp.inc');
$HOST = 'localhost';
$PORT = 5672;
$USER = 'guest';
$PASS = 'guest';
$VHOST = '/';
$EXCHANGE = 'router';
$QUEUE = 'msgs';
$conn = new AMQPConnection($HOST, $PORT, $USER, $PASS);
$ch = $conn->channel();
$ch->access_request($VHOST, false, false, true, true);
$msg_body = implode(' ', array_slice($argv, 1));
$msg = new AMQPMessage($msg_body, array('content_type' => 'text/plain'));
$ch->basic_publish($msg, $EXCHANGE);
$ch->close();
$conn->close();
The consumer (reads messages from queue):
require_once('amqp.inc');
$HOST = 'localhost';
$PORT = 5672;
$USER = 'guest';
$PASS = 'guest';
$VHOST = '/';
$EXCHANGE = 'router';
$QUEUE = 'msgs';
$CONSUMER_TAG = 'consumer';
$conn = new AMQPConnection($HOST, $PORT, $USER, $PASS);
$ch = $conn->channel();
$ch->access_request($VHOST, false, false, true, true);
$ch->queue_declare($QUEUE);
$ch->exchange_declare($EXCHANGE, 'direct', false, false, false);
$ch->queue_bind($QUEUE, $EXCHANGE);
function process_message($msg) {
global $ch, $CONSUMER_TAG;
echo "\n--------\n";
echo $msg->body;
echo "\n--------\n";
$ch->basic_ack($msg->delivery_info['delivery_tag']);
// Cancel callback
if ($msg->body === 'quit') {
$ch->basic_cancel($CONSUMER_TAG);
}
}
$ch->basic_consume($QUEUE, $CONSUMER_TAG, false, false, false, false, 'process_message');
// Loop as long as the channel has callbacks registered
while(count($ch->callbacks)) {
$ch->wait();
}
$ch->close();
$conn->close();
--
Further reading:
* Why Databases suck for messaging
* Introduction to Message Queues and how RabbitMQ works
* SecondLife (the game) evaluation of 9 message-queue solutions
* RabbitMQ Administration
* RabbitMQ Clustering
If you're looking to scale your service, integrating message-queues into your application is a MUST. Without offline processing, you'll quickly hit a glass ceiling on maximum throughput.
In the world of message-queues, there are lots of different solutions. From the over-simplistic PHP Dropr, through RabbitMQ, ActiveMQ, Fuse, ZeroMQ and commercial enterprise queue systems.
There's a great post by the makers of SecondLife, sharing their evaluation of 9 popular message-queue systems here. Take the time to review available message-queues and test which one best suits your needs.
Here at SPI, our original implementation of distributed message-queues was MySQL based. Two master-master databases, sharing a queue table. While it did work for our needs, databases suck for message queues and we needed a better solution.
We had a short run with ActiveMQ -- another popular message-queue solution, but were not happy with the test results. RabbitMQ is a lot faster (x40 times), supports AMQP and is better suited for distributed clusters.
Let's cover the steps to install RabbitMQ on your server.
1. Getting started
Download the latest version of RabbitMQ from this page.
Use the Bin version. No need to recompile.
unsetenv JAVA_HOME
mkdir /usr/tmp
fetch "http://www.rabbitmq.com/releases/rabbitmq-server/v1.7.2/rabbitmq-server-generic-unix-1.7.2.tar.gz"
tar xvfz rabbitmq-server-generic-unix-1.7.2.tar.gz
mkdir /usr/tmp
fetch "http://www.rabbitmq.com/releases/rabbitmq-server/v1.7.2/rabbitmq-server-generic-unix-1.7.2.tar.gz"
tar xvfz rabbitmq-server-generic-unix-1.7.2.tar.gz
2. Download JRE
Download and install the version of JRE matching your system (32bit or 64bit) from the Java SE Download page.
3. Install Erlang
If you're on FreeBSD, simply do:
cd /usr/ports/lang/erlang
make all
make install
make all
make install
Otherwise, Download Erlang.
4. Start RabbitMQ
Changedir to the path where you installed RabbitMQ and start it with:
cd bin
./rabbitmq-server -detached
./rabbitmq-server -detached
Simple right? You now have the RabbitMQ up and running (in the background), ready to relay messages.
It's important to understand the terminology and building blocks (Queues, Exchanges, Bindings) of RabbitMQ.
Take the time to review the Getting Started Slides and FAQ.
5. Communicating with RabbitMQ
Now that we have RabbitMQ up and running we can start the fun part of writing and reading messages.
For the purpose of this guide, we'll use PHP to communicate with RabbitMQ.
There are several PHP-wrappers for RabbitMQ. Our favorite one is the clean and simple PHP amqp lib.
The code below uses php-amqplib to connect and send messages to a RabbitMQ server.
The publisher (writes messages to queue):
require_once('amqp.inc');
$HOST = 'localhost';
$PORT = 5672;
$USER = 'guest';
$PASS = 'guest';
$VHOST = '/';
$EXCHANGE = 'router';
$QUEUE = 'msgs';
$conn = new AMQPConnection($HOST, $PORT, $USER, $PASS);
$ch = $conn->channel();
$ch->access_request($VHOST, false, false, true, true);
$msg_body = implode(' ', array_slice($argv, 1));
$msg = new AMQPMessage($msg_body, array('content_type' => 'text/plain'));
$ch->basic_publish($msg, $EXCHANGE);
$ch->close();
$conn->close();
The consumer (reads messages from queue):
require_once('amqp.inc');
$HOST = 'localhost';
$PORT = 5672;
$USER = 'guest';
$PASS = 'guest';
$VHOST = '/';
$EXCHANGE = 'router';
$QUEUE = 'msgs';
$CONSUMER_TAG = 'consumer';
$conn = new AMQPConnection($HOST, $PORT, $USER, $PASS);
$ch = $conn->channel();
$ch->access_request($VHOST, false, false, true, true);
$ch->queue_declare($QUEUE);
$ch->exchange_declare($EXCHANGE, 'direct', false, false, false);
$ch->queue_bind($QUEUE, $EXCHANGE);
function process_message($msg) {
global $ch, $CONSUMER_TAG;
echo "\n--------\n";
echo $msg->body;
echo "\n--------\n";
$ch->basic_ack($msg->delivery_info['delivery_tag']);
// Cancel callback
if ($msg->body === 'quit') {
$ch->basic_cancel($CONSUMER_TAG);
}
}
$ch->basic_consume($QUEUE, $CONSUMER_TAG, false, false, false, false, 'process_message');
// Loop as long as the channel has callbacks registered
while(count($ch->callbacks)) {
$ch->wait();
}
$ch->close();
$conn->close();
--
Further reading:
* Why Databases suck for messaging
* Introduction to Message Queues and how RabbitMQ works
* SecondLife (the game) evaluation of 9 message-queue solutions
* RabbitMQ Administration
* RabbitMQ Clustering
|
|
Subscribe Now to receive new posts via Email as soon as they come out.
Comments
Post your comments