Full-service Internet Marketing & Web Development
Recent Posts
Recommended Reads
|
How to install APC on Linux / RedHatMike Peters, June 14 -- Posted under Programming |
APC (Alternative PHP Cache) is an opcode cache for PHP that pre-compiles commonly used PHP scripts to machine-language, speeding up execution.
We previously used eAccelarator which is another opcode cache for PHP. We decided to switch from eAccelarator to APC due to stability issues and the fact that APC was chosen as the best opcode cache engine to be included with PHP6.
In this post, I'll cover the steps to install APC:
1. Type: 'pecl config-set preferred-state stable'
2. Type: 'pecl install apc'
3. When prompted type 'all' to change configuration
4. Type 'no' to change the option
5. Hit the enter key to run the installation script.
6. When complete, open /usr/local/lib/php.ini for editing
7. Comment out all eaccelerator lines if they exist
8. Add the following block:
extension=apc.so
apc.enabled="1"
apc.shm_segments="1"
apc.shm_size="100"
apc.ttl="5"
apc.user_ttl="5"
apc.gc_ttl="5"
apc.file_update_protection="2"
apc.enable_cli="1"
apc.max_file_size="1M"
apc.write-lock="1"
apc.report_autofilter="0"
apc.include_once_override="0"
Additional settings and explanations for each can be found at: http://www.php.net/manual/en/apc.configuration.php
9. Save the file and exit the editor
10. Change the system setting for shared memory as follows:
a. edit /etc/sysctl.conf
b. add this line:
c. the run the following command:
11. Restart FastCGI manager with:
View 1 Comment(s)
We previously used eAccelarator which is another opcode cache for PHP. We decided to switch from eAccelarator to APC due to stability issues and the fact that APC was chosen as the best opcode cache engine to be included with PHP6.
In this post, I'll cover the steps to install APC:
1. Type: 'pecl config-set preferred-state stable'
2. Type: 'pecl install apc'
3. When prompted type 'all' to change configuration
4. Type 'no' to change the option
5. Hit the enter key to run the installation script.
6. When complete, open /usr/local/lib/php.ini for editing
7. Comment out all eaccelerator lines if they exist
8. Add the following block:
extension=apc.so
apc.enabled="1"
apc.shm_segments="1"
apc.shm_size="100"
apc.ttl="5"
apc.user_ttl="5"
apc.gc_ttl="5"
apc.file_update_protection="2"
apc.enable_cli="1"
apc.max_file_size="1M"
apc.write-lock="1"
apc.report_autofilter="0"
apc.include_once_override="0"
Additional settings and explanations for each can be found at: http://www.php.net/manual/en/apc.configuration.php
9. Save the file and exit the editor
10. Change the system setting for shared memory as follows:
a. edit /etc/sysctl.conf
b. add this line:
kern.ipc.shmmax=134217728
c. the run the following command:
sysctl -w kern.ipc.shmmax=134217728
11. Restart FastCGI manager with:
/usr/local/bin/php-fpm restart
View 1 Comment(s)
|
|
Temporary File Names in CMike Peters, May 27 -- Posted under Programming |
There's a lot of confusion among developers about the best way to generate a unique temporary file name in C.
If you're using C, the most suitable function that comes to mind is tmpnam():
While it seems like a great fit, you should never ever use tmpnam.
I'll repeat it again - avoid using tmpnam() at all cost.
The reason is - tmpnam() suffers from a race condition:
Since the temporary file never gets created, if you have multiple threads/processes calling tmpnam() at the same time, it is very possible for two instances to end up with the same exact temporary file name... And the consequences can be fatal.
The tmpnam() function should be deprecated. That's probably why it was never ported over to PHP.
In PHP you should use tempnam() or tmpfile(), both of which create the temporary file before returning the name, so you are guaranteed no two instances will ever end up with the same temporary file name.
Here's the correct way to get a temporary file name in C:
View 1 Comment(s)
If you're using C, the most suitable function that comes to mind is tmpnam():
char *tmpnam(char *str);
// Usage
printf ("My temporary file name is: %s\r\n", tmpnam("/usr/tmp"));
// Usage
printf ("My temporary file name is: %s\r\n", tmpnam("/usr/tmp"));
While it seems like a great fit, you should never ever use tmpnam.
I'll repeat it again - avoid using tmpnam() at all cost.
The reason is - tmpnam() suffers from a race condition:
Since the temporary file never gets created, if you have multiple threads/processes calling tmpnam() at the same time, it is very possible for two instances to end up with the same exact temporary file name... And the consequences can be fatal.
The tmpnam() function should be deprecated. That's probably why it was never ported over to PHP.
In PHP you should use tempnam() or tmpfile(), both of which create the temporary file before returning the name, so you are guaranteed no two instances will ever end up with the same temporary file name.
Here's the correct way to get a temporary file name in C:
char sTempfile[] = "/usr/tmp/mytmpfileXXXXXX"; // The X's are important
int tmp_handle;
if ( (tmp_handle=mkstemp(sTempfile)) < 1)
{
return 0;
}
close(tmp_handle);
// We now have the temporary filename in sTempfile
printf ("My temporary file name is: %s\r\n", sTempfile);
int tmp_handle;
if ( (tmp_handle=mkstemp(sTempfile)) < 1)
{
return 0;
}
close(tmp_handle);
// We now have the temporary filename in sTempfile
printf ("My temporary file name is: %s\r\n", sTempfile);
View 1 Comment(s)
|
|
MySQL InnoDB Deadlocks and Duplicate key errors (1213, 1205, 1062)Mike Peters, May 5 -- Posted under Programming |
InnoDB is one of the best MySQL storage engines when you're looking for concurrent writes, transactions support and ACID reliability.
Switching from MyISAM to InnoDB is very straightforward (alter table MYTABLE engine=innodb), but there are a few pitfalls you should be prepared for.
Review the full list of InnoDB restrictions, understand them and make sure nothing collides with your existing design... Watch out for SELECT COUNT(*) on InnoDB - they require full table scans.
As part of this post I'd like to touch on three key issues with InnoDB that many developers fail to properly deal with:
#1062 Duplicate entry for key 'PRIMARY' error
This is bug 26316 that has been around for quite some time and is still alive and kicking as of version 5.1.43 of MySQL that we're using.
Not sure if it is really linked to triggers or not. Out of nowhere, MySQL will throw a 1062 duplicate key error on auto_increment keys. Something which by design is impossible.
To fix, you have to retry the transaction and it will go through on the second attempt.
#1213 Deadlock found when trying to get lock error
InnoDB locks rows and starts transactions internally as needed.
From time to time, particularly when concurrent threads are hitting the same rows, you're going to experience a deadlock.
Deadlocks happen when two transactions wait on each other to acquire a lock. For example:
Tx 1: lock A, then B
Tx 2: lock B, then A
Because InnoDB starts transactions on the internally, you -are- going to experience deadlocks.
No way of escaping it.
Fortunately when deadlocks do happen with this error 1213, all you have to do is retry the query until it goes through.
#1205 Lock wait timeout exceeded error
Similar to error 1213, this one will occur whether or not you are manually starting a transaction.
This error is more complicated to deal with -
If the transaction was triggered by InnoDB (atomic to the query you're running), you can simply retry the query and it will eventually go through.
If however this error shows up inside a transaction you started, it means any uncommitted inserts/updates are lost and will have to be resent in a new transaction block.
--
Below are two simple PHP MySQL wrapper functions you can use to gracefully handle the three issues I described, while logging all errors to a file.
function DBRead($query, $link_identifier=0)
{
// Read query (SELECT)
if ($link_identifier)
$result = mysql_query($query, $link_identifier);
else
$result = mysql_query($query);
// Return result
return $result;
}
function DBWrite($query, $link_identifier=0)
{
// If any of these error codes is returned by MySQL, we'll retry
$arr_need_to_retry_error_codes = array
(
1213, // Deadlock found when trying to get lock
1205 // Lock wait timeout exceeded
);
// Initialize
$cnt_retry = 0;
$error_str ="";
// Main loop
do
{
// Initialize 'flag_retry' indicating whether or not we need to retry this transaction
$flag_retry = 0;
// Write query (UPDATE, INSERT)
if ($link_identifier)
{
$result = mysql_query($query, $link_identifier);
$mysql_errno = mysql_errno($link_identifier);
$mysql_error = mysql_error($link_identifier);
}
else
{
$result = mysql_query($query);
$mysql_errno = mysql_errno();
$mysql_error = mysql_error();
}
// If failed,
if (!$result)
{
// Determine if we need to retry this transaction -
// If duplicate PRIMARY key error,
// or one of the errors in 'arr_need_to_retry_error_codes'
// then we need to retry
$flag_retry = (($mysql_errno==1062 &&
strpos($mysql_error,"for key 'PRIMARY'")!==false) ||
in_array($mysql_errno, $arr_need_to_retry_error_codes)) ;
// If this was error 1205, log it
if ($mysql_errno==1205)
DBLogError($query, "Error #1205 detected - review application logic", $link_identifier);
}
// If successful or failed but no need to retry
if ($result || empty($flag_retry))
{
// We're done
break;
}
// If we're up to here this means that -
// Error occured, wait 1 second before we try again
sleep(1);
$cnt_retry++;
// If we already retried 10 times, log error
if ($cnt_retry>=10)
{
$result = 0;
$error_str = "Retried $cnt_retry times due to error ".
$mysql_errno." and finally gave up";
break;
}
} while (1);
// If update query failed, log
if (!$result)
{
DBLogError($query, $error_str, $link_identifier);
}
// Return result
return $result;
}
function DBLogError($query="", $msg="", $link_identifier=0)
{
// Set these for easier access
if (!empty($link_identifier))
{
$mysql_errno = mysql_errno($link_identifier);
$mysql_error = mysql_error($link_identifier);
}
else
{
$mysql_errno = mysql_errno();
$mysql_error = mysql_error();
}
// Format 'msg' if we have it
if (!empty($msg)) $msg = "$msg";
else $msg = "Error #".$mysql_errno." $mysql_error";
// Log
$file = @fopen("/usr/tmp/dbwrite_error.log","a");
@fwrite($file, date("Y-m-d h:i:s")." ".$_SERVER['REQUEST_URI'].
": $msg; Query was: $query; ".
$_SERVER['SCRIPT_FILENAME'].")\r\n");
@fclose($file);
}
Switching from MyISAM to InnoDB is very straightforward (alter table MYTABLE engine=innodb), but there are a few pitfalls you should be prepared for.
Review the full list of InnoDB restrictions, understand them and make sure nothing collides with your existing design... Watch out for SELECT COUNT(*) on InnoDB - they require full table scans.
As part of this post I'd like to touch on three key issues with InnoDB that many developers fail to properly deal with:
#1062 Duplicate entry for key 'PRIMARY' error
This is bug 26316 that has been around for quite some time and is still alive and kicking as of version 5.1.43 of MySQL that we're using.
Not sure if it is really linked to triggers or not. Out of nowhere, MySQL will throw a 1062 duplicate key error on auto_increment keys. Something which by design is impossible.
To fix, you have to retry the transaction and it will go through on the second attempt.
#1213 Deadlock found when trying to get lock error
InnoDB locks rows and starts transactions internally as needed.
From time to time, particularly when concurrent threads are hitting the same rows, you're going to experience a deadlock.
Deadlocks happen when two transactions wait on each other to acquire a lock. For example:
Tx 1: lock A, then B
Tx 2: lock B, then A
Because InnoDB starts transactions on the internally, you -are- going to experience deadlocks.
No way of escaping it.
Fortunately when deadlocks do happen with this error 1213, all you have to do is retry the query until it goes through.
#1205 Lock wait timeout exceeded error
Similar to error 1213, this one will occur whether or not you are manually starting a transaction.
This error is more complicated to deal with -
If the transaction was triggered by InnoDB (atomic to the query you're running), you can simply retry the query and it will eventually go through.
If however this error shows up inside a transaction you started, it means any uncommitted inserts/updates are lost and will have to be resent in a new transaction block.
--
Below are two simple PHP MySQL wrapper functions you can use to gracefully handle the three issues I described, while logging all errors to a file.
function DBRead($query, $link_identifier=0)
{
// Read query (SELECT)
if ($link_identifier)
$result = mysql_query($query, $link_identifier);
else
$result = mysql_query($query);
// Return result
return $result;
}
function DBWrite($query, $link_identifier=0)
{
// If any of these error codes is returned by MySQL, we'll retry
$arr_need_to_retry_error_codes = array
(
1213, // Deadlock found when trying to get lock
1205 // Lock wait timeout exceeded
);
// Initialize
$cnt_retry = 0;
$error_str ="";
// Main loop
do
{
// Initialize 'flag_retry' indicating whether or not we need to retry this transaction
$flag_retry = 0;
// Write query (UPDATE, INSERT)
if ($link_identifier)
{
$result = mysql_query($query, $link_identifier);
$mysql_errno = mysql_errno($link_identifier);
$mysql_error = mysql_error($link_identifier);
}
else
{
$result = mysql_query($query);
$mysql_errno = mysql_errno();
$mysql_error = mysql_error();
}
// If failed,
if (!$result)
{
// Determine if we need to retry this transaction -
// If duplicate PRIMARY key error,
// or one of the errors in 'arr_need_to_retry_error_codes'
// then we need to retry
$flag_retry = (($mysql_errno==1062 &&
strpos($mysql_error,"for key 'PRIMARY'")!==false) ||
in_array($mysql_errno, $arr_need_to_retry_error_codes)) ;
// If this was error 1205, log it
if ($mysql_errno==1205)
DBLogError($query, "Error #1205 detected - review application logic", $link_identifier);
}
// If successful or failed but no need to retry
if ($result || empty($flag_retry))
{
// We're done
break;
}
// If we're up to here this means that -
// Error occured, wait 1 second before we try again
sleep(1);
$cnt_retry++;
// If we already retried 10 times, log error
if ($cnt_retry>=10)
{
$result = 0;
$error_str = "Retried $cnt_retry times due to error ".
$mysql_errno." and finally gave up";
break;
}
} while (1);
// If update query failed, log
if (!$result)
{
DBLogError($query, $error_str, $link_identifier);
}
// Return result
return $result;
}
function DBLogError($query="", $msg="", $link_identifier=0)
{
// Set these for easier access
if (!empty($link_identifier))
{
$mysql_errno = mysql_errno($link_identifier);
$mysql_error = mysql_error($link_identifier);
}
else
{
$mysql_errno = mysql_errno();
$mysql_error = mysql_error();
}
// Format 'msg' if we have it
if (!empty($msg)) $msg = "$msg";
else $msg = "Error #".$mysql_errno." $mysql_error";
// Log
$file = @fopen("/usr/tmp/dbwrite_error.log","a");
@fwrite($file, date("Y-m-d h:i:s")." ".$_SERVER['REQUEST_URI'].
": $msg; Query was: $query; ".
$_SERVER['SCRIPT_FILENAME'].")\r\n");
@fclose($file);
}
|
Thickbox - ColorBox - FancyBox - ShadowBox ComparisonBrett Batie, April 28 -- Posted under Programming |
There are a large number of pop-up/dialog box libraries that are now available. Many of the pop-up dialog options out there support the same features. However, in our testing we found that some failed terribly.
As part of this post, I'd like to cover a quick comparison of what I found in hopes that it will help others in their search for the ideal pop-up library.
The popup controls I tested:
jQueryUI Dialog
DOM Window
Thickbox
fancybox
Colorbox
Shadowbox.js
At a first glance all of the above looked like they would work well. However we did find some major flaws with the above libraries. Here is what we found:
jQueryUI Dialog - This is by far our favorite pop-up dialog. It supports features such as modal, dragging it to other locations on the screen and allows the user to resize it. The features that jQuery offers makes it a top choice. However, the major flaw is it makes IE6 hang for about 10 seconds when the page is loading. That one flaw forced us to look at other options.
Dom window - For us this package is too basic. It does support the basic needs of a pop-up but when compared to some of the other libraries it was not nearly as good.
Thickbox - This library had all the features needed. We have used it in the past and have found it very easy to work with. The major pitfall with this library is it is no longer supported. The library works correctly today but there is no guarantee it will continue to work into the future and therefore is not a great choice.
FancyBox - This library has a good set of features unfortunately it loads very slowly in IE6 and can cause the browser to crash.
ColorBox - This library works very well, supports a good amount of features and does load correctly in all browsers including IE6. It would be nice if this library had all of the features of the jQueryUI Dialog and some may consider it a flaw that jQuery is required in order to use this library.
ShadowBox - This library works well in all of the major browsers and supports most features that are necessary for a nice pop-up. It does not have all of the features of jQueryUI Dialog. This library includes everything needed in one javascript and css file.
Based on my research I found the two best libraries to be colorbox and shadowbox.
These libraries offer very similar features and they both work well with all of the major browsers. For us ColorBox was the right choice because it has is under an MIT license, the size of the library is smaller than ShadowBox and it supports auto-resize-to-fit-content which ShadowBox didn't have.
View 3 Comment(s)
As part of this post, I'd like to cover a quick comparison of what I found in hopes that it will help others in their search for the ideal pop-up library.
The popup controls I tested:
At a first glance all of the above looked like they would work well. However we did find some major flaws with the above libraries. Here is what we found:
jQueryUI Dialog - This is by far our favorite pop-up dialog. It supports features such as modal, dragging it to other locations on the screen and allows the user to resize it. The features that jQuery offers makes it a top choice. However, the major flaw is it makes IE6 hang for about 10 seconds when the page is loading. That one flaw forced us to look at other options.
Dom window - For us this package is too basic. It does support the basic needs of a pop-up but when compared to some of the other libraries it was not nearly as good.
Thickbox - This library had all the features needed. We have used it in the past and have found it very easy to work with. The major pitfall with this library is it is no longer supported. The library works correctly today but there is no guarantee it will continue to work into the future and therefore is not a great choice.
FancyBox - This library has a good set of features unfortunately it loads very slowly in IE6 and can cause the browser to crash.
ColorBox - This library works very well, supports a good amount of features and does load correctly in all browsers including IE6. It would be nice if this library had all of the features of the jQueryUI Dialog and some may consider it a flaw that jQuery is required in order to use this library.
ShadowBox - This library works well in all of the major browsers and supports most features that are necessary for a nice pop-up. It does not have all of the features of jQueryUI Dialog. This library includes everything needed in one javascript and css file.
Based on my research I found the two best libraries to be colorbox and shadowbox.
These libraries offer very similar features and they both work well with all of the major browsers. For us ColorBox was the right choice because it has is under an MIT license, the size of the library is smaller than ShadowBox and it supports auto-resize-to-fit-content which ShadowBox didn't have.
View 3 Comment(s)
|
|
RabbitMQ Message QueueMike Peters, April 16 -- Posted under Programming |
RabbitMQ is an open source messaging server, allowing you to scale by queuing jobs to be processed offline.
If you're looking to scale your service, integrating message-queues into your application is a MUST. Without offline processing, you'll quickly hit a glass ceiling on maximum throughput.
In the world of message-queues, there are lots of different solutions. From the over-simplistic PHP Dropr, through RabbitMQ, ActiveMQ, Fuse, ZeroMQ and commercial enterprise queue systems.
There's a great post by the makers of SecondLife, sharing their evaluation of 9 popular message-queue systems here. Take the time to review available message-queues and test which one best suits your needs.
Here at SPI, our original implementation of distributed message-queues was MySQL based. Two master-master databases, sharing a queue table. While it did work for our needs, databases suck for message queues and we needed a better solution.
We had a short run with ActiveMQ -- another popular message-queue solution, but were not happy with the test results. RabbitMQ is a lot faster (x40 times), supports AMQP and is better suited for distributed clusters.
Let's cover the steps to install RabbitMQ on your server.
1. Getting started
Download the latest version of RabbitMQ from this page.
Use the Bin version. No need to recompile.
2. Download JRE
Download and install the version of JRE matching your system (32bit or 64bit) from the Java SE Download page.
3. Install Erlang
If you're on FreeBSD, simply do:
Otherwise, Download Erlang.
4. Start RabbitMQ
Changedir to the path where you installed RabbitMQ and start it with:
Simple right? You now have the RabbitMQ up and running (in the background), ready to relay messages.
It's important to understand the terminology and building blocks (Queues, Exchanges, Bindings) of RabbitMQ.
Take the time to review the Getting Started Slides and FAQ.
5. Communicating with RabbitMQ
Now that we have RabbitMQ up and running we can start the fun part of writing and reading messages.
For the purpose of this guide, we'll use PHP to communicate with RabbitMQ.
There are several PHP-wrappers for RabbitMQ. Our favorite one is the clean and simple PHP amqp lib.
The code below uses php-amqplib to connect and send messages to a RabbitMQ server.
The publisher (writes messages to queue):
require_once('amqp.inc');
$HOST = 'localhost';
$PORT = 5672;
$USER = 'guest';
$PASS = 'guest';
$VHOST = '/';
$EXCHANGE = 'router';
$QUEUE = 'msgs';
$conn = new AMQPConnection($HOST, $PORT, $USER, $PASS);
$ch = $conn->channel();
$ch->access_request($VHOST, false, false, true, true);
$msg_body = implode(' ', array_slice($argv, 1));
$msg = new AMQPMessage($msg_body, array('content_type' => 'text/plain'));
$ch->basic_publish($msg, $EXCHANGE);
$ch->close();
$conn->close();
The consumer (reads messages from queue):
require_once('amqp.inc');
$HOST = 'localhost';
$PORT = 5672;
$USER = 'guest';
$PASS = 'guest';
$VHOST = '/';
$EXCHANGE = 'router';
$QUEUE = 'msgs';
$CONSUMER_TAG = 'consumer';
$conn = new AMQPConnection($HOST, $PORT, $USER, $PASS);
$ch = $conn->channel();
$ch->access_request($VHOST, false, false, true, true);
$ch->queue_declare($QUEUE);
$ch->exchange_declare($EXCHANGE, 'direct', false, false, false);
$ch->queue_bind($QUEUE, $EXCHANGE);
function process_message($msg) {
global $ch, $CONSUMER_TAG;
echo "\n--------\n";
echo $msg->body;
echo "\n--------\n";
$ch->basic_ack($msg->delivery_info['delivery_tag']);
// Cancel callback
if ($msg->body === 'quit') {
$ch->basic_cancel($CONSUMER_TAG);
}
}
$ch->basic_consume($QUEUE, $CONSUMER_TAG, false, false, false, false, 'process_message');
// Loop as long as the channel has callbacks registered
while(count($ch->callbacks)) {
$ch->wait();
}
$ch->close();
$conn->close();
--
Further reading:
* Why Databases suck for messaging
* Introduction to Message Queues and how RabbitMQ works
* SecondLife (the game) evaluation of 9 message-queue solutions
* RabbitMQ Administration
* RabbitMQ Clustering
If you're looking to scale your service, integrating message-queues into your application is a MUST. Without offline processing, you'll quickly hit a glass ceiling on maximum throughput.
In the world of message-queues, there are lots of different solutions. From the over-simplistic PHP Dropr, through RabbitMQ, ActiveMQ, Fuse, ZeroMQ and commercial enterprise queue systems.
There's a great post by the makers of SecondLife, sharing their evaluation of 9 popular message-queue systems here. Take the time to review available message-queues and test which one best suits your needs.
Here at SPI, our original implementation of distributed message-queues was MySQL based. Two master-master databases, sharing a queue table. While it did work for our needs, databases suck for message queues and we needed a better solution.
We had a short run with ActiveMQ -- another popular message-queue solution, but were not happy with the test results. RabbitMQ is a lot faster (x40 times), supports AMQP and is better suited for distributed clusters.
Let's cover the steps to install RabbitMQ on your server.
1. Getting started
Download the latest version of RabbitMQ from this page.
Use the Bin version. No need to recompile.
unsetenv JAVA_HOME
mkdir /usr/tmp
fetch "http://www.rabbitmq.com/releases/rabbitmq-server/v1.7.2/rabbitmq-server-generic-unix-1.7.2.tar.gz"
tar xvfz rabbitmq-server-generic-unix-1.7.2.tar.gz
mkdir /usr/tmp
fetch "http://www.rabbitmq.com/releases/rabbitmq-server/v1.7.2/rabbitmq-server-generic-unix-1.7.2.tar.gz"
tar xvfz rabbitmq-server-generic-unix-1.7.2.tar.gz
2. Download JRE
Download and install the version of JRE matching your system (32bit or 64bit) from the Java SE Download page.
3. Install Erlang
If you're on FreeBSD, simply do:
cd /usr/ports/lang/erlang
make all
make install
make all
make install
Otherwise, Download Erlang.
4. Start RabbitMQ
Changedir to the path where you installed RabbitMQ and start it with:
cd bin
./rabbitmq-server -detached
./rabbitmq-server -detached
Simple right? You now have the RabbitMQ up and running (in the background), ready to relay messages.
It's important to understand the terminology and building blocks (Queues, Exchanges, Bindings) of RabbitMQ.
Take the time to review the Getting Started Slides and FAQ.
5. Communicating with RabbitMQ
Now that we have RabbitMQ up and running we can start the fun part of writing and reading messages.
For the purpose of this guide, we'll use PHP to communicate with RabbitMQ.
There are several PHP-wrappers for RabbitMQ. Our favorite one is the clean and simple PHP amqp lib.
The code below uses php-amqplib to connect and send messages to a RabbitMQ server.
The publisher (writes messages to queue):
require_once('amqp.inc');
$HOST = 'localhost';
$PORT = 5672;
$USER = 'guest';
$PASS = 'guest';
$VHOST = '/';
$EXCHANGE = 'router';
$QUEUE = 'msgs';
$conn = new AMQPConnection($HOST, $PORT, $USER, $PASS);
$ch = $conn->channel();
$ch->access_request($VHOST, false, false, true, true);
$msg_body = implode(' ', array_slice($argv, 1));
$msg = new AMQPMessage($msg_body, array('content_type' => 'text/plain'));
$ch->basic_publish($msg, $EXCHANGE);
$ch->close();
$conn->close();
The consumer (reads messages from queue):
require_once('amqp.inc');
$HOST = 'localhost';
$PORT = 5672;
$USER = 'guest';
$PASS = 'guest';
$VHOST = '/';
$EXCHANGE = 'router';
$QUEUE = 'msgs';
$CONSUMER_TAG = 'consumer';
$conn = new AMQPConnection($HOST, $PORT, $USER, $PASS);
$ch = $conn->channel();
$ch->access_request($VHOST, false, false, true, true);
$ch->queue_declare($QUEUE);
$ch->exchange_declare($EXCHANGE, 'direct', false, false, false);
$ch->queue_bind($QUEUE, $EXCHANGE);
function process_message($msg) {
global $ch, $CONSUMER_TAG;
echo "\n--------\n";
echo $msg->body;
echo "\n--------\n";
$ch->basic_ack($msg->delivery_info['delivery_tag']);
// Cancel callback
if ($msg->body === 'quit') {
$ch->basic_cancel($CONSUMER_TAG);
}
}
$ch->basic_consume($QUEUE, $CONSUMER_TAG, false, false, false, false, 'process_message');
// Loop as long as the channel has callbacks registered
while(count($ch->callbacks)) {
$ch->wait();
}
$ch->close();
$conn->close();
--
Further reading:
* Why Databases suck for messaging
* Introduction to Message Queues and how RabbitMQ works
* SecondLife (the game) evaluation of 9 message-queue solutions
* RabbitMQ Administration
* RabbitMQ Clustering
|
|
Realtime PHP Syntax CheckingMike Peters, April 14 -- Posted under Programming |
If you love Agile Development and PHP as much as we do, you're probably going to have developers editing critical PHP scripts on live machines.
While many PHP editors include built-in syntax checking, it doesn't hurt to have a fail-safe in place. We wanted to come up with a way to apply php -l SCRIPT syntax checking on any PHP script as soon as it is updated.
At first the idea was to write a plugin for Mercurial - our version control system, so that whenever a changeset is pushed, the syntax will be checked.
Unfortunately this doesn't cover changes done via SSH or files uploaded via FTP by clients.
We came up with a shell script, executed by cron every minute, that iterates through all files changed within the last 3 days, applying "php -l" on those files and sending an email to the appropriate developer with the information about the error.
Regardless of how files are updated, we now have an automated way to syntax-check every single change to a PHP script.
The emails generated by the script follow this format:
CheckSyntax cronjob:
CheckSyntax.sh:
-
To install, save the two files (CheckSyntax and CheckSyntax.sh), chmod a+rx, update the email settings in CheckSyntax and add this line to your cronjob:
While many PHP editors include built-in syntax checking, it doesn't hurt to have a fail-safe in place. We wanted to come up with a way to apply php -l SCRIPT syntax checking on any PHP script as soon as it is updated.
At first the idea was to write a plugin for Mercurial - our version control system, so that whenever a changeset is pushed, the syntax will be checked.
Unfortunately this doesn't cover changes done via SSH or files uploaded via FTP by clients.
We came up with a shell script, executed by cron every minute, that iterates through all files changed within the last 3 days, applying "php -l" on those files and sending an email to the appropriate developer with the information about the error.
Regardless of how files are updated, we now have an automated way to syntax-check every single change to a PHP script.
The emails generated by the script follow this format:
Oops! Found a FATAL syntax error in a production PHP script...
DIRECTORY: /magnetic/
---------------------------------------------
FILE: ./affiliate/offline_credit_commission.php
---------------------------------------------
Parse error: syntax error, unexpected '<' in /offline_credit_commission.php on line 8
changeset: 2673:fe54a1e4dbe0
user: Doug
date: Wed Apr 14 00:29:26 2010 +0300
summary: Change offline credit affiliate commission
DIRECTORY: /magnetic/
---------------------------------------------
FILE: ./affiliate/offline_credit_commission.php
---------------------------------------------
Parse error: syntax error, unexpected '<' in /offline_credit_commission.php on line 8
changeset: 2673:fe54a1e4dbe0
user: Doug
date: Wed Apr 14 00:29:26 2010 +0300
summary: Change offline credit affiliate commission
CheckSyntax cronjob:
#!/bin/sh
# Change to source directory directory
cd $(dirname $0)
# Generate report
./CheckSyntax.sh >log.$$
a=$?
# Email report
# TODO: Update these with your settings
if [ $a = 1 ]
then
./sendEmail -f CheckSyntax@mycompany.com \
-t developers@mycompany.com \
-u "Check Syntax" \
-s mail.mymailserver.com \
-m < log.$$
fi
# Remove temporary file
rm -rf log.$$
# (eof)
# Change to source directory directory
cd $(dirname $0)
# Generate report
./CheckSyntax.sh >log.$$
a=$?
# Email report
# TODO: Update these with your settings
if [ $a = 1 ]
then
./sendEmail -f CheckSyntax@mycompany.com \
-t developers@mycompany.com \
-u "Check Syntax" \
-s mail.mymailserver.com \
-m < log.$$
fi
# Remove temporary file
rm -rf log.$$
# (eof)
CheckSyntax.sh:
#!/usr/local/bin/bash
##################################
# Config
##################################
#
# TODO: Update these with your settings
php="/usr/local/bin/php"
hg="/usr/local/bin/hg"
def="/magnetic"
#
##################################
# Directory passed in parameters?
if [ "$1" != "" ]
then
# Get the directory from parameters
dir=$1
else
# No parameter, use default
dir=$def
fi
# Change to the directory we need to check
cd $dir
# Prepare tmp file
tmp=/tmp/CheckSyntax.$$
# Header
echo "Oops! Found a FATAL syntax error in a production PHP script..."
echo ""
echo "DIRECTORY: $dir"
# Find PHP files
# Note: '-newerct' only workds under FreeBSD
find . -newerct '3 days ago' ! -path "*/temp/*" | grep "\.php$" >$tmp
# Loop files
cnt=0
err=0
tot=0
while read i
do
# If file isn't a file, skip
[ ! -f "$i" ] && continue;
((tot++))
# Syntax check
a=$($php -l $i | grep "^No syntax errors detected in")
# Is it OK?
if [ "$a" = "" ]
then
# Not OK, display details
echo ""
echo "----------------------------------------------------------------------------------------------"
echo "FILE: $i"
echo "----------------------------------------------------------------------------------------------"
a=$($php -d display_errors=1 -l $i | grep "Parse error")
echo $a | sed "s/<[a-zA-Z\/][^>]*>//g"
$hg log $i | head -n 5 | grep -v "^$"
((cnt++))
err=1
fi
done < $tmp
echo ""
echo "--"
echo ""
echo "$cnt BAD FILES"
echo "$tot TOTAL FILES"
echo ""
# Remove tmp file
rm -rf $tmp
# 0 - no error | 1 - error
exit $err
# (eof)
##################################
# Config
##################################
#
# TODO: Update these with your settings
php="/usr/local/bin/php"
hg="/usr/local/bin/hg"
def="/magnetic"
#
##################################
# Directory passed in parameters?
if [ "$1" != "" ]
then
# Get the directory from parameters
dir=$1
else
# No parameter, use default
dir=$def
fi
# Change to the directory we need to check
cd $dir
# Prepare tmp file
tmp=/tmp/CheckSyntax.$$
# Header
echo "Oops! Found a FATAL syntax error in a production PHP script..."
echo ""
echo "DIRECTORY: $dir"
# Find PHP files
# Note: '-newerct' only workds under FreeBSD
find . -newerct '3 days ago' ! -path "*/temp/*" | grep "\.php$" >$tmp
# Loop files
cnt=0
err=0
tot=0
while read i
do
# If file isn't a file, skip
[ ! -f "$i" ] && continue;
((tot++))
# Syntax check
a=$($php -l $i | grep "^No syntax errors detected in")
# Is it OK?
if [ "$a" = "" ]
then
# Not OK, display details
echo ""
echo "----------------------------------------------------------------------------------------------"
echo "FILE: $i"
echo "----------------------------------------------------------------------------------------------"
a=$($php -d display_errors=1 -l $i | grep "Parse error")
echo $a | sed "s/<[a-zA-Z\/][^>]*>//g"
$hg log $i | head -n 5 | grep -v "^$"
((cnt++))
err=1
fi
done < $tmp
echo ""
echo "--"
echo ""
echo "$cnt BAD FILES"
echo "$tot TOTAL FILES"
echo ""
# Remove tmp file
rm -rf $tmp
# 0 - no error | 1 - error
exit $err
# (eof)
-
To install, save the two files (CheckSyntax and CheckSyntax.sh), chmod a+rx, update the email settings in CheckSyntax and add this line to your cronjob:
# CheckSyntax
# TODO: Update path with the location of CheckSyntax
*/1 * * * * /home/CheckSyntax
# TODO: Update path with the location of CheckSyntax
*/1 * * * * /home/CheckSyntax
| « Previous Posts | » Next Posts |