Full-service Internet Marketing & Web Development
Recent Posts
Featured Posts
|
Mercurial Hook: PHP Syntax CheckMike Peters, June 16, 2010 |
In a previous post, we wrote about PHP real time syntax checking. A method to send email alerts when any critical scripts on your live servers fail to pass PHP syntax validation.
The problem with this approach is that it alerts you of a problem after-the-fact.
Someone uploads a bad PHP script which breaks a critical module, you are alerted, but by that time users are already affected.
Today we'll take it this concept one step further by writing a hook for the Mercurial version control system, preventing developers from checking-in scripts that don't pass syntax-checking.
Installing PHP Syntax Check Mercurial hook
Step 1: The shell script
Save this shell script under your Mercurial folder, calling it: php_syntax.sh
Step 2: Add hook to Mercurial config file
Update your Mercurial hgweb.config, adding the new hook under the 'hooks' section, like this:
[hooks]
pretxnchangegroup.syntax_check = /usr/home/mercurial/php_syntax.sh
--
Replace '/usr/home/mercurial' with the path where you saved php_syntax.sh
Attempting to push changes to the repository that don't pass php-syntax check, are now blocked.
Users will be presented with an error message and the transaction rolled back, until the user fixes the problem and pushes changes again.
View 5 Comment(s)
The problem with this approach is that it alerts you of a problem after-the-fact.
Someone uploads a bad PHP script which breaks a critical module, you are alerted, but by that time users are already affected.
Today we'll take it this concept one step further by writing a hook for the Mercurial version control system, preventing developers from checking-in scripts that don't pass syntax-checking.
Installing PHP Syntax Check Mercurial hook
Step 1: The shell script
Save this shell script under your Mercurial folder, calling it: php_syntax.sh
#!/usr/local/bin/bash
echo "STARTING PHP SYNTAX CHECK..."
# create a random temp file
temp_file=`/usr/bin/mktemp -t php_syntax_files`
# get all modified files and remove duplicate's
#note: use file_mods,file_adds instead
hg log -r $HG_NODE:tip --template "{files}\n" | sort | uniq > $temp_file
# Walk through each line
#for line in "$temp_file"; do
for line in $(< $temp_file); do
# Make sure it is a php file
if [ `echo $line | grep -E "\.(php)|(php4)|(php5)$"` ]
then
# create a random temp file
php_file=`/usr/bin/mktemp -t php_syntax_check`
# save the contents of this file (latest commit) to the temp file
hg cat -r tip $line > $php_file
# check the syntax
php_syntax_output=`/usr/local/bin/php-cgi -l -d display_errors=1 -d error_reporting=4 -d html_errors=0 < $php_file`;
# remove the temp file
rm -f $php_file;
test_syntax=`echo $php_syntax_output | grep "Parse error"`
if [ "$test_syntax" ];then
exit 1;
fi
fi
done
rm -f "$temp_file"
echo "STARTING PHP SYNTAX CHECK..."
# create a random temp file
temp_file=`/usr/bin/mktemp -t php_syntax_files`
# get all modified files and remove duplicate's
#note: use file_mods,file_adds instead
hg log -r $HG_NODE:tip --template "{files}\n" | sort | uniq > $temp_file
# Walk through each line
#for line in "$temp_file"; do
for line in $(< $temp_file); do
# Make sure it is a php file
if [ `echo $line | grep -E "\.(php)|(php4)|(php5)$"` ]
then
# create a random temp file
php_file=`/usr/bin/mktemp -t php_syntax_check`
# save the contents of this file (latest commit) to the temp file
hg cat -r tip $line > $php_file
# check the syntax
php_syntax_output=`/usr/local/bin/php-cgi -l -d display_errors=1 -d error_reporting=4 -d html_errors=0 < $php_file`;
# remove the temp file
rm -f $php_file;
test_syntax=`echo $php_syntax_output | grep "Parse error"`
if [ "$test_syntax" ];then
exit 1;
fi
fi
done
rm -f "$temp_file"
Step 2: Add hook to Mercurial config file
Update your Mercurial hgweb.config, adding the new hook under the 'hooks' section, like this:
[hooks]
pretxnchangegroup.syntax_check = /usr/home/mercurial/php_syntax.sh
--
Replace '/usr/home/mercurial' with the path where you saved php_syntax.sh
Attempting to push changes to the repository that don't pass php-syntax check, are now blocked.
Users will be presented with an error message and the transaction rolled back, until the user fixes the problem and pushes changes again.
View 5 Comment(s)
|
|
How to install APC on Linux / RedHatMike Peters, June 14, 2010 |
APC (Alternative PHP Cache) is an opcode cache for PHP that pre-compiles commonly used PHP scripts to machine-language, speeding up execution.
We previously used eAccelarator which is another opcode cache for PHP. We decided to switch from eAccelarator to APC due to stability issues and the fact that APC was chosen as the best opcode cache engine to be included with PHP6.
In this post, I'll cover the steps to install APC:
1. Type: 'pecl config-set preferred-state stable'
2. Type: 'pecl install apc'
3. When prompted type 'all' to change configuration
4. Type 'no' to change the option
5. Hit the enter key to run the installation script.
6. When complete, open /usr/local/lib/php.ini for editing
7. Comment out all eaccelerator lines if they exist
8. Add the following block:
extension=apc.so
apc.enabled="1"
apc.shm_segments="1"
apc.shm_size="100"
apc.ttl="5"
apc.user_ttl="5"
apc.gc_ttl="5"
apc.file_update_protection="2"
apc.enable_cli="1"
apc.max_file_size="1M"
apc.write-lock="1"
apc.report_autofilter="0"
apc.include_once_override="0"
Additional settings and explanations for each can be found at: http://www.php.net/manual/en/apc.configuration.php
9. Save the file and exit the editor
10. Change the system setting for shared memory as follows:
a. edit /etc/sysctl.conf
b. add this line:
c. the run the following command:
11. Restart FastCGI manager with:
View 1 Comment(s)
We previously used eAccelarator which is another opcode cache for PHP. We decided to switch from eAccelarator to APC due to stability issues and the fact that APC was chosen as the best opcode cache engine to be included with PHP6.
In this post, I'll cover the steps to install APC:
1. Type: 'pecl config-set preferred-state stable'
2. Type: 'pecl install apc'
3. When prompted type 'all' to change configuration
4. Type 'no' to change the option
5. Hit the enter key to run the installation script.
6. When complete, open /usr/local/lib/php.ini for editing
7. Comment out all eaccelerator lines if they exist
8. Add the following block:
extension=apc.so
apc.enabled="1"
apc.shm_segments="1"
apc.shm_size="100"
apc.ttl="5"
apc.user_ttl="5"
apc.gc_ttl="5"
apc.file_update_protection="2"
apc.enable_cli="1"
apc.max_file_size="1M"
apc.write-lock="1"
apc.report_autofilter="0"
apc.include_once_override="0"
Additional settings and explanations for each can be found at: http://www.php.net/manual/en/apc.configuration.php
9. Save the file and exit the editor
10. Change the system setting for shared memory as follows:
a. edit /etc/sysctl.conf
b. add this line:
kern.ipc.shmmax=134217728
c. the run the following command:
sysctl -w kern.ipc.shmmax=134217728
11. Restart FastCGI manager with:
/usr/local/bin/php-fpm restart
View 1 Comment(s)
|
|
Temporary File Names in CMike Peters, May 27, 2010 |
There's a lot of confusion among developers about the best way to generate a unique temporary file name in C.
If you're using C, the most suitable function that comes to mind is tmpnam():
While it seems like a great fit, you should never ever use tmpnam.
I'll repeat it again - avoid using tmpnam() at all cost.
The reason is - tmpnam() suffers from a race condition:
Since the temporary file never gets created, if you have multiple threads/processes calling tmpnam() at the same time, it is very possible for two instances to end up with the same exact temporary file name... And the consequences can be fatal.
The tmpnam() function should be deprecated. That's probably why it was never ported over to PHP.
In PHP you should use tempnam() or tmpfile(), both of which create the temporary file before returning the name, so you are guaranteed no two instances will ever end up with the same temporary file name.
Here's the correct way to get a temporary file name in C:
View 1 Comment(s)
If you're using C, the most suitable function that comes to mind is tmpnam():
char *tmpnam(char *str);
// Usage
printf ("My temporary file name is: %s\r\n", tmpnam("/usr/tmp"));
// Usage
printf ("My temporary file name is: %s\r\n", tmpnam("/usr/tmp"));
While it seems like a great fit, you should never ever use tmpnam.
I'll repeat it again - avoid using tmpnam() at all cost.
The reason is - tmpnam() suffers from a race condition:
Since the temporary file never gets created, if you have multiple threads/processes calling tmpnam() at the same time, it is very possible for two instances to end up with the same exact temporary file name... And the consequences can be fatal.
The tmpnam() function should be deprecated. That's probably why it was never ported over to PHP.
In PHP you should use tempnam() or tmpfile(), both of which create the temporary file before returning the name, so you are guaranteed no two instances will ever end up with the same temporary file name.
Here's the correct way to get a temporary file name in C:
char sTempfile[] = "/usr/tmp/mytmpfileXXXXXX"; // The X's are important
int tmp_handle;
if ( (tmp_handle=mkstemp(sTempfile)) < 1)
{
return 0;
}
close(tmp_handle);
// We now have the temporary filename in sTempfile
printf ("My temporary file name is: %s\r\n", sTempfile);
int tmp_handle;
if ( (tmp_handle=mkstemp(sTempfile)) < 1)
{
return 0;
}
close(tmp_handle);
// We now have the temporary filename in sTempfile
printf ("My temporary file name is: %s\r\n", sTempfile);
View 1 Comment(s)
|
|
MySQL InnoDB Deadlocks and Duplicate key errors (1213, 1205, 1062)Mike Peters, May 6, 2010 |
InnoDB is one of the best MySQL storage engines when you're looking for concurrent writes, transactions support and ACID reliability.
Switching from MyISAM to InnoDB is very straightforward (alter table MYTABLE engine=innodb), but there are a few pitfalls you should be prepared for.
Review the full list of InnoDB restrictions, understand them and make sure nothing collides with your existing design... Watch out for SELECT COUNT(*) on InnoDB - they require full table scans.
As part of this post I'd like to touch on three key issues with InnoDB that many developers fail to properly deal with:
#1062 Duplicate entry for key 'PRIMARY' error
This is bug 26316 that has been around for quite some time and is still alive and kicking as of version 5.1.43 of MySQL that we're using.
Not sure if it is really linked to triggers or not. Out of nowhere, MySQL will throw a 1062 duplicate key error on auto_increment keys. Something which by design is impossible.
To fix, you have to retry the transaction and it will go through on the second attempt.
#1213 Deadlock found when trying to get lock error
InnoDB locks rows and starts transactions internally as needed.
From time to time, particularly when concurrent threads are hitting the same rows, you're going to experience a deadlock.
Deadlocks happen when two transactions wait on each other to acquire a lock. For example:
Tx 1: lock A, then B
Tx 2: lock B, then A
Because InnoDB starts transactions on the internally, you -are- going to experience deadlocks.
No way of escaping it.
Fortunately when deadlocks do happen with this error 1213, all you have to do is retry the query until it goes through.
#1205 Lock wait timeout exceeded error
Similar to error 1213, this one will occur whether or not you are manually starting a transaction.
This error is more complicated to deal with -
If the transaction was triggered by InnoDB (atomic to the query you're running), you can simply retry the query and it will eventually go through.
If however this error shows up inside a transaction you started, it means any uncommitted inserts/updates are lost and will have to be resent in a new transaction block.
--
Below are two simple PHP MySQL wrapper functions you can use to gracefully handle the three issues I described, while logging all errors to a file.
function DBRead($query, $link_identifier=0)
{
// Read query (SELECT)
if ($link_identifier)
$result = mysql_query($query, $link_identifier);
else
$result = mysql_query($query);
// Return result
return $result;
}
function DBWrite($query, $link_identifier=0)
{
// If any of these error codes is returned by MySQL, we'll retry
$arr_need_to_retry_error_codes = array
(
1213, // Deadlock found when trying to get lock
1205 // Lock wait timeout exceeded
);
// Initialize
$cnt_retry = 0;
$error_str ="";
// Main loop
do
{
// Initialize 'flag_retry' indicating whether or not we need to retry this transaction
$flag_retry = 0;
// Write query (UPDATE, INSERT)
if ($link_identifier)
{
$result = mysql_query($query, $link_identifier);
$mysql_errno = mysql_errno($link_identifier);
$mysql_error = mysql_error($link_identifier);
}
else
{
$result = mysql_query($query);
$mysql_errno = mysql_errno();
$mysql_error = mysql_error();
}
// If failed,
if (!$result)
{
// Determine if we need to retry this transaction -
// If duplicate PRIMARY key error,
// or one of the errors in 'arr_need_to_retry_error_codes'
// then we need to retry
$flag_retry = (($mysql_errno==1062 &&
strpos($mysql_error,"for key 'PRIMARY'")!==false) ||
in_array($mysql_errno, $arr_need_to_retry_error_codes)) ;
// If this was error 1205, log it
if ($mysql_errno==1205)
DBLogError($query, "Error #1205 detected - review application logic", $link_identifier);
}
// If successful or failed but no need to retry
if ($result || empty($flag_retry))
{
// We're done
break;
}
// If we're up to here this means that -
// Error occured, wait 1 second before we try again
sleep(1);
$cnt_retry++;
// If we already retried 10 times, log error
if ($cnt_retry>=10)
{
$result = 0;
$error_str = "Retried $cnt_retry times due to error ".
$mysql_errno." and finally gave up";
break;
}
} while (1);
// If update query failed, log
if (!$result)
{
DBLogError($query, $error_str, $link_identifier);
}
// Return result
return $result;
}
function DBLogError($query="", $msg="", $link_identifier=0)
{
// Set these for easier access
if (!empty($link_identifier))
{
$mysql_errno = mysql_errno($link_identifier);
$mysql_error = mysql_error($link_identifier);
}
else
{
$mysql_errno = mysql_errno();
$mysql_error = mysql_error();
}
// Format 'msg' if we have it
if (!empty($msg)) $msg = "$msg";
else $msg = "Error #".$mysql_errno." $mysql_error";
// Log
$file = @fopen("/usr/tmp/dbwrite_error.log","a");
@fwrite($file, date("Y-m-d h:i:s")." ".$_SERVER['REQUEST_URI'].
": $msg; Query was: $query; ".
$_SERVER['SCRIPT_FILENAME'].")\r\n");
@fclose($file);
}
Switching from MyISAM to InnoDB is very straightforward (alter table MYTABLE engine=innodb), but there are a few pitfalls you should be prepared for.
Review the full list of InnoDB restrictions, understand them and make sure nothing collides with your existing design... Watch out for SELECT COUNT(*) on InnoDB - they require full table scans.
As part of this post I'd like to touch on three key issues with InnoDB that many developers fail to properly deal with:
#1062 Duplicate entry for key 'PRIMARY' error
This is bug 26316 that has been around for quite some time and is still alive and kicking as of version 5.1.43 of MySQL that we're using.
Not sure if it is really linked to triggers or not. Out of nowhere, MySQL will throw a 1062 duplicate key error on auto_increment keys. Something which by design is impossible.
To fix, you have to retry the transaction and it will go through on the second attempt.
#1213 Deadlock found when trying to get lock error
InnoDB locks rows and starts transactions internally as needed.
From time to time, particularly when concurrent threads are hitting the same rows, you're going to experience a deadlock.
Deadlocks happen when two transactions wait on each other to acquire a lock. For example:
Tx 1: lock A, then B
Tx 2: lock B, then A
Because InnoDB starts transactions on the internally, you -are- going to experience deadlocks.
No way of escaping it.
Fortunately when deadlocks do happen with this error 1213, all you have to do is retry the query until it goes through.
#1205 Lock wait timeout exceeded error
Similar to error 1213, this one will occur whether or not you are manually starting a transaction.
This error is more complicated to deal with -
If the transaction was triggered by InnoDB (atomic to the query you're running), you can simply retry the query and it will eventually go through.
If however this error shows up inside a transaction you started, it means any uncommitted inserts/updates are lost and will have to be resent in a new transaction block.
--
Below are two simple PHP MySQL wrapper functions you can use to gracefully handle the three issues I described, while logging all errors to a file.
function DBRead($query, $link_identifier=0)
{
// Read query (SELECT)
if ($link_identifier)
$result = mysql_query($query, $link_identifier);
else
$result = mysql_query($query);
// Return result
return $result;
}
function DBWrite($query, $link_identifier=0)
{
// If any of these error codes is returned by MySQL, we'll retry
$arr_need_to_retry_error_codes = array
(
1213, // Deadlock found when trying to get lock
1205 // Lock wait timeout exceeded
);
// Initialize
$cnt_retry = 0;
$error_str ="";
// Main loop
do
{
// Initialize 'flag_retry' indicating whether or not we need to retry this transaction
$flag_retry = 0;
// Write query (UPDATE, INSERT)
if ($link_identifier)
{
$result = mysql_query($query, $link_identifier);
$mysql_errno = mysql_errno($link_identifier);
$mysql_error = mysql_error($link_identifier);
}
else
{
$result = mysql_query($query);
$mysql_errno = mysql_errno();
$mysql_error = mysql_error();
}
// If failed,
if (!$result)
{
// Determine if we need to retry this transaction -
// If duplicate PRIMARY key error,
// or one of the errors in 'arr_need_to_retry_error_codes'
// then we need to retry
$flag_retry = (($mysql_errno==1062 &&
strpos($mysql_error,"for key 'PRIMARY'")!==false) ||
in_array($mysql_errno, $arr_need_to_retry_error_codes)) ;
// If this was error 1205, log it
if ($mysql_errno==1205)
DBLogError($query, "Error #1205 detected - review application logic", $link_identifier);
}
// If successful or failed but no need to retry
if ($result || empty($flag_retry))
{
// We're done
break;
}
// If we're up to here this means that -
// Error occured, wait 1 second before we try again
sleep(1);
$cnt_retry++;
// If we already retried 10 times, log error
if ($cnt_retry>=10)
{
$result = 0;
$error_str = "Retried $cnt_retry times due to error ".
$mysql_errno." and finally gave up";
break;
}
} while (1);
// If update query failed, log
if (!$result)
{
DBLogError($query, $error_str, $link_identifier);
}
// Return result
return $result;
}
function DBLogError($query="", $msg="", $link_identifier=0)
{
// Set these for easier access
if (!empty($link_identifier))
{
$mysql_errno = mysql_errno($link_identifier);
$mysql_error = mysql_error($link_identifier);
}
else
{
$mysql_errno = mysql_errno();
$mysql_error = mysql_error();
}
// Format 'msg' if we have it
if (!empty($msg)) $msg = "$msg";
else $msg = "Error #".$mysql_errno." $mysql_error";
// Log
$file = @fopen("/usr/tmp/dbwrite_error.log","a");
@fwrite($file, date("Y-m-d h:i:s")." ".$_SERVER['REQUEST_URI'].
": $msg; Query was: $query; ".
$_SERVER['SCRIPT_FILENAME'].")\r\n");
@fclose($file);
}
|
Thickbox - ColorBox - FancyBox - ShadowBox ComparisonBrett Batie, April 28, 2010 |
There are a large number of pop-up/dialog box libraries that are now available. Many of the pop-up dialog options out there support the same features. However, in our testing we found that some failed terribly.
As part of this post, I'd like to cover a quick comparison of what I found in hopes that it will help others in their search for the ideal pop-up library.
The popup controls I tested:
jQueryUI Dialog
DOM Window
Thickbox
fancybox
Colorbox
Shadowbox.js
At a first glance all of the above looked like they would work well. However we did find some major flaws with the above libraries. Here is what we found:
jQueryUI Dialog - This is by far our favorite pop-up dialog. It supports features such as modal, dragging it to other locations on the screen and allows the user to resize it. The features that jQuery offers makes it a top choice. However, the major flaw is it makes IE6 hang for about 10 seconds when the page is loading. That one flaw forced us to look at other options.
Dom window - For us this package is too basic. It does support the basic needs of a pop-up but when compared to some of the other libraries it was not nearly as good.
Thickbox - This library had all the features needed. We have used it in the past and have found it very easy to work with. The major pitfall with this library is it is no longer supported. The library works correctly today but there is no guarantee it will continue to work into the future and therefore is not a great choice.
FancyBox - This library has a good set of features unfortunately it loads very slowly in IE6 and can cause the browser to crash.
ColorBox - This library works very well, supports a good amount of features and does load correctly in all browsers including IE6. It would be nice if this library had all of the features of the jQueryUI Dialog and some may consider it a flaw that jQuery is required in order to use this library.
ShadowBox - This library works well in all of the major browsers and supports most features that are necessary for a nice pop-up. It does not have all of the features of jQueryUI Dialog. This library includes everything needed in one javascript and css file.
Based on my research I found the two best libraries to be colorbox and shadowbox.
These libraries offer very similar features and they both work well with all of the major browsers. For us ColorBox was the right choice because it has is under an MIT license, the size of the library is smaller than ShadowBox and it supports auto-resize-to-fit-content which ShadowBox didn't have.
View 7 Comment(s)
As part of this post, I'd like to cover a quick comparison of what I found in hopes that it will help others in their search for the ideal pop-up library.
The popup controls I tested:
At a first glance all of the above looked like they would work well. However we did find some major flaws with the above libraries. Here is what we found:
jQueryUI Dialog - This is by far our favorite pop-up dialog. It supports features such as modal, dragging it to other locations on the screen and allows the user to resize it. The features that jQuery offers makes it a top choice. However, the major flaw is it makes IE6 hang for about 10 seconds when the page is loading. That one flaw forced us to look at other options.
Dom window - For us this package is too basic. It does support the basic needs of a pop-up but when compared to some of the other libraries it was not nearly as good.
Thickbox - This library had all the features needed. We have used it in the past and have found it very easy to work with. The major pitfall with this library is it is no longer supported. The library works correctly today but there is no guarantee it will continue to work into the future and therefore is not a great choice.
FancyBox - This library has a good set of features unfortunately it loads very slowly in IE6 and can cause the browser to crash.
ColorBox - This library works very well, supports a good amount of features and does load correctly in all browsers including IE6. It would be nice if this library had all of the features of the jQueryUI Dialog and some may consider it a flaw that jQuery is required in order to use this library.
ShadowBox - This library works well in all of the major browsers and supports most features that are necessary for a nice pop-up. It does not have all of the features of jQueryUI Dialog. This library includes everything needed in one javascript and css file.
Based on my research I found the two best libraries to be colorbox and shadowbox.
These libraries offer very similar features and they both work well with all of the major browsers. For us ColorBox was the right choice because it has is under an MIT license, the size of the library is smaller than ShadowBox and it supports auto-resize-to-fit-content which ShadowBox didn't have.
View 7 Comment(s)
|
|
RabbitMQ Message QueueMike Peters, April 16, 2010 |
RabbitMQ is an open source messaging server, allowing you to scale by queuing jobs to be processed offline.
If you're looking to scale your service, integrating message-queues into your application is a MUST. Without offline processing, you'll quickly hit a glass ceiling on maximum throughput.
In the world of message-queues, there are lots of different solutions. From the over-simplistic PHP Dropr, through RabbitMQ, ActiveMQ, Fuse, ZeroMQ and commercial enterprise queue systems.
There's a great post by the makers of SecondLife, sharing their evaluation of 9 popular message-queue systems here. Take the time to review available message-queues and test which one best suits your needs.
Here at SPI, our original implementation of distributed message-queues was MySQL based. Two master-master databases, sharing a queue table. While it did work for our needs, databases suck for message queues and we needed a better solution.
We had a short run with ActiveMQ -- another popular message-queue solution, but were not happy with the test results. RabbitMQ is a lot faster (x40 times), supports AMQP and is better suited for distributed clusters.
Let's cover the steps to install RabbitMQ on your server.
1. Getting started
Download the latest version of RabbitMQ from this page.
Use the Bin version. No need to recompile.
2. Download JRE
Download and install the version of JRE matching your system (32bit or 64bit) from the Java SE Download page.
3. Install Erlang
If you're on FreeBSD, simply do:
Otherwise, Download Erlang.
4. Start RabbitMQ
Changedir to the path where you installed RabbitMQ and start it with:
Simple right? You now have the RabbitMQ up and running (in the background), ready to relay messages.
It's important to understand the terminology and building blocks (Queues, Exchanges, Bindings) of RabbitMQ.
Take the time to review the Getting Started Slides and FAQ.
5. Communicating with RabbitMQ
Now that we have RabbitMQ up and running we can start the fun part of writing and reading messages.
For the purpose of this guide, we'll use PHP to communicate with RabbitMQ.
There are several PHP-wrappers for RabbitMQ. Our favorite one is the clean and simple PHP amqp lib.
The code below uses php-amqplib to connect and send messages to a RabbitMQ server.
The publisher (writes messages to queue):
require_once('amqp.inc');
$HOST = 'localhost';
$PORT = 5672;
$USER = 'guest';
$PASS = 'guest';
$VHOST = '/';
$EXCHANGE = 'router';
$QUEUE = 'msgs';
$conn = new AMQPConnection($HOST, $PORT, $USER, $PASS);
$ch = $conn->channel();
$ch->access_request($VHOST, false, false, true, true);
$msg_body = implode(' ', array_slice($argv, 1));
$msg = new AMQPMessage($msg_body, array('content_type' => 'text/plain'));
$ch->basic_publish($msg, $EXCHANGE);
$ch->close();
$conn->close();
The consumer (reads messages from queue):
require_once('amqp.inc');
$HOST = 'localhost';
$PORT = 5672;
$USER = 'guest';
$PASS = 'guest';
$VHOST = '/';
$EXCHANGE = 'router';
$QUEUE = 'msgs';
$CONSUMER_TAG = 'consumer';
$conn = new AMQPConnection($HOST, $PORT, $USER, $PASS);
$ch = $conn->channel();
$ch->access_request($VHOST, false, false, true, true);
$ch->queue_declare($QUEUE);
$ch->exchange_declare($EXCHANGE, 'direct', false, false, false);
$ch->queue_bind($QUEUE, $EXCHANGE);
function process_message($msg) {
global $ch, $CONSUMER_TAG;
echo "\n--------\n";
echo $msg->body;
echo "\n--------\n";
$ch->basic_ack($msg->delivery_info['delivery_tag']);
// Cancel callback
if ($msg->body === 'quit') {
$ch->basic_cancel($CONSUMER_TAG);
}
}
$ch->basic_consume($QUEUE, $CONSUMER_TAG, false, false, false, false, 'process_message');
// Loop as long as the channel has callbacks registered
while(count($ch->callbacks)) {
$ch->wait();
}
$ch->close();
$conn->close();
--
Further reading:
* Why Databases suck for messaging
* Introduction to Message Queues and how RabbitMQ works
* SecondLife (the game) evaluation of 9 message-queue solutions
* RabbitMQ Administration
* RabbitMQ Clustering
If you're looking to scale your service, integrating message-queues into your application is a MUST. Without offline processing, you'll quickly hit a glass ceiling on maximum throughput.
In the world of message-queues, there are lots of different solutions. From the over-simplistic PHP Dropr, through RabbitMQ, ActiveMQ, Fuse, ZeroMQ and commercial enterprise queue systems.
There's a great post by the makers of SecondLife, sharing their evaluation of 9 popular message-queue systems here. Take the time to review available message-queues and test which one best suits your needs.
Here at SPI, our original implementation of distributed message-queues was MySQL based. Two master-master databases, sharing a queue table. While it did work for our needs, databases suck for message queues and we needed a better solution.
We had a short run with ActiveMQ -- another popular message-queue solution, but were not happy with the test results. RabbitMQ is a lot faster (x40 times), supports AMQP and is better suited for distributed clusters.
Let's cover the steps to install RabbitMQ on your server.
1. Getting started
Download the latest version of RabbitMQ from this page.
Use the Bin version. No need to recompile.
unsetenv JAVA_HOME
mkdir /usr/tmp
fetch "http://www.rabbitmq.com/releases/rabbitmq-server/v1.7.2/rabbitmq-server-generic-unix-1.7.2.tar.gz"
tar xvfz rabbitmq-server-generic-unix-1.7.2.tar.gz
mkdir /usr/tmp
fetch "http://www.rabbitmq.com/releases/rabbitmq-server/v1.7.2/rabbitmq-server-generic-unix-1.7.2.tar.gz"
tar xvfz rabbitmq-server-generic-unix-1.7.2.tar.gz
2. Download JRE
Download and install the version of JRE matching your system (32bit or 64bit) from the Java SE Download page.
3. Install Erlang
If you're on FreeBSD, simply do:
cd /usr/ports/lang/erlang
make all
make install
make all
make install
Otherwise, Download Erlang.
4. Start RabbitMQ
Changedir to the path where you installed RabbitMQ and start it with:
cd bin
./rabbitmq-server -detached
./rabbitmq-server -detached
Simple right? You now have the RabbitMQ up and running (in the background), ready to relay messages.
It's important to understand the terminology and building blocks (Queues, Exchanges, Bindings) of RabbitMQ.
Take the time to review the Getting Started Slides and FAQ.
5. Communicating with RabbitMQ
Now that we have RabbitMQ up and running we can start the fun part of writing and reading messages.
For the purpose of this guide, we'll use PHP to communicate with RabbitMQ.
There are several PHP-wrappers for RabbitMQ. Our favorite one is the clean and simple PHP amqp lib.
The code below uses php-amqplib to connect and send messages to a RabbitMQ server.
The publisher (writes messages to queue):
require_once('amqp.inc');
$HOST = 'localhost';
$PORT = 5672;
$USER = 'guest';
$PASS = 'guest';
$VHOST = '/';
$EXCHANGE = 'router';
$QUEUE = 'msgs';
$conn = new AMQPConnection($HOST, $PORT, $USER, $PASS);
$ch = $conn->channel();
$ch->access_request($VHOST, false, false, true, true);
$msg_body = implode(' ', array_slice($argv, 1));
$msg = new AMQPMessage($msg_body, array('content_type' => 'text/plain'));
$ch->basic_publish($msg, $EXCHANGE);
$ch->close();
$conn->close();
The consumer (reads messages from queue):
require_once('amqp.inc');
$HOST = 'localhost';
$PORT = 5672;
$USER = 'guest';
$PASS = 'guest';
$VHOST = '/';
$EXCHANGE = 'router';
$QUEUE = 'msgs';
$CONSUMER_TAG = 'consumer';
$conn = new AMQPConnection($HOST, $PORT, $USER, $PASS);
$ch = $conn->channel();
$ch->access_request($VHOST, false, false, true, true);
$ch->queue_declare($QUEUE);
$ch->exchange_declare($EXCHANGE, 'direct', false, false, false);
$ch->queue_bind($QUEUE, $EXCHANGE);
function process_message($msg) {
global $ch, $CONSUMER_TAG;
echo "\n--------\n";
echo $msg->body;
echo "\n--------\n";
$ch->basic_ack($msg->delivery_info['delivery_tag']);
// Cancel callback
if ($msg->body === 'quit') {
$ch->basic_cancel($CONSUMER_TAG);
}
}
$ch->basic_consume($QUEUE, $CONSUMER_TAG, false, false, false, false, 'process_message');
// Loop as long as the channel has callbacks registered
while(count($ch->callbacks)) {
$ch->wait();
}
$ch->close();
$conn->close();
--
Further reading:
* Why Databases suck for messaging
* Introduction to Message Queues and how RabbitMQ works
* SecondLife (the game) evaluation of 9 message-queue solutions
* RabbitMQ Administration
* RabbitMQ Clustering
| « Previous Posts | » Next Posts |
