Full-service Internet Marketing & Web Development

Call us Toll-Free:

1-800-218-1525

MySQL InnoDB Deadlocks and Duplicate key errors (1213, 1205, 1062)

Mike Peters, May 5

InnoDB is one of the best MySQL storage engines when you're looking for concurrent writes, transactions support and ACID reliability.

Switching from MyISAM to InnoDB is very straightforward (alter table MYTABLE engine=innodb), but there are a few pitfalls you should be prepared for.

Review the full list of InnoDB restrictions, understand them and make sure nothing collides with your existing design... Watch out for SELECT COUNT(*) on InnoDB - they require full table scans.

As part of this post I'd like to touch on three key issues with InnoDB that many developers fail to properly deal with:

#1062 Duplicate entry for key 'PRIMARY' error

This is bug 26316 that has been around for quite some time and is still alive and kicking as of version 5.1.43 of MySQL that we're using.

Not sure if it is really linked to triggers or not. Out of nowhere, MySQL will throw a 1062 duplicate key error on auto_increment keys. Something which by design is impossible.

To fix, you have to retry the transaction and it will go through on the second attempt.

#1213 Deadlock found when trying to get lock error

InnoDB locks rows and starts transactions internally as needed.

From time to time, particularly when concurrent threads are hitting the same rows, you're going to experience a deadlock.

Deadlocks happen when two transactions wait on each other to acquire a lock. For example:

Tx 1: lock A, then B
Tx 2: lock B, then A

Because InnoDB starts transactions on the internally, you -are- going to experience deadlocks.

No way of escaping it.

Fortunately when deadlocks do happen with this error 1213, all you have to do is retry the query until it goes through.

#1205 Lock wait timeout exceeded error

Similar to error 1213, this one will occur whether or not you are manually starting a transaction.

This error is more complicated to deal with -

If the transaction was triggered by InnoDB (atomic to the query you're running), you can simply retry the query and it will eventually go through.

If however this error shows up inside a transaction you started, it means any uncommitted inserts/updates are lost and will have to be resent in a new transaction block.

--

Below are two simple PHP MySQL wrapper functions you can use to gracefully handle the three issues I described, while logging all errors to a file.

Thickbox - ColorBox - FancyBox - ShadowBox Comparison

Brett Batie, April 28

There are a large number of pop-up/dialog box libraries that are now available. Many of the pop-up dialog options out there support the same features. However, in our testing we found that some failed terribly.

As part of this post, I'd like to cover a quick comparison of what I found in hopes that it will help others in their search for the ideal pop-up library.

The popup controls I tested:

jQueryUI Dialog

DOM Window

Thickbox

fancybox

Colorbox

Shadowbox.js

At a first glance all of the above looked like they would work well. However we did find some major flaws with the above libraries. Here is what we found:

jQueryUI Dialog - This is by far our favorite pop-up dialog. It supports features such as modal, dragging it to other locations on the screen and allows the user to resize it. The features that jQuery offers makes it a top choice. However, the major flaw is it makes IE6 hang for about 10 seconds when the page is loading. That one flaw forced us to look at other options.

Dom window - For us this package is too basic. It does support the basic needs of a pop-up but when compared to some of the other libraries it was not nearly as good.

Thickbox - This library had all the features needed. We have used it in the past and have found it very easy to work with. The major pitfall with this library is it is no longer supported. The library works correctly today but there is no guarantee it will continue to work into the future and therefore is not a great choice.

FancyBox - This library has a good set of features unfortunately it loads very slowly in IE6 and can cause the browser to crash.

ColorBox - This library works very well, supports a good amount of features and does load correctly in all browsers including IE6. It would be nice if this library had all of the features of the jQueryUI Dialog and some may consider it a flaw that jQuery is required in order to use this library.

ShadowBox - This library works well in all of the major browsers and supports most features that are necessary for a nice pop-up. It does not have all of the features of jQueryUI Dialog. This library includes everything needed in one javascript and css file.

Based on my research I found the two best libraries to be colorbox and shadowbox.

These libraries offer very similar features and they both work well with all of the major browsers. For us ColorBox was the right choice because it has is under an MIT license, the size of the library is smaller than ShadowBox and it supports auto-resize-to-fit-content which ShadowBox didn't have.

View 3 Comment(s)

RabbitMQ Message Queue

Mike Peters, April 16

RabbitMQ is an open source messaging server, allowing you to scale by queuing jobs to be processed offline.

If you're looking to scale your service, integrating message-queues into your application is a MUST. Without offline processing, you'll quickly hit a glass ceiling on maximum throughput.

In the world of message-queues, there are lots of different solutions. From the over-simplistic PHP Dropr, through RabbitMQ, ActiveMQ, Fuse, ZeroMQ and commercial enterprise queue systems.

There's a great post by the makers of SecondLife, sharing their evaluation of 9 popular message-queue systems here. Take the time to review available message-queues and test which one best suits your needs.

Here at SPI, our original implementation of distributed message-queues was MySQL based. Two master-master databases, sharing a queue table. While it did work for our needs, databases suck for message queues and we needed a better solution.

We had a short run with ActiveMQ -- another popular message-queue solution, but were not happy with the test results. RabbitMQ is a lot faster (x40 times), supports AMQP and is better suited for distributed clusters.

Let's cover the steps to install RabbitMQ on your server.

1. Getting started

Download the latest version of RabbitMQ from this page.

Use the Bin version. No need to recompile.

unsetenv JAVA_HOME
mkdir /usr/tmp
fetch "http://www.rabbitmq.com/releases/rabbitmq-server/v1.7.2/rabbitmq-server-generic-unix-1.7.2.tar.gz"
tar xvfz rabbitmq-server-generic-unix-1.7.2.tar.gz

2. Download JRE

Download and install the version of JRE matching your system (32bit or 64bit) from the Java SE Download page.

3. Install Erlang

If you're on FreeBSD, simply do:

cd /usr/ports/lang/erlang
make all
make install

Otherwise, Download Erlang.

4. Start RabbitMQ

Changedir to the path where you installed RabbitMQ and start it with:

cd bin
./rabbitmq-server -detached

Simple right? You now have the RabbitMQ up and running (in the background), ready to relay messages.

It's important to understand the terminology and building blocks (Queues, Exchanges, Bindings) of RabbitMQ.

Take the time to review the Getting Started Slides and FAQ.

5. Communicating with RabbitMQ

Now that we have RabbitMQ up and running we can start the fun part of writing and reading messages.

For the purpose of this guide, we'll use PHP to communicate with RabbitMQ.

There are several PHP-wrappers for RabbitMQ. Our favorite one is the clean and simple PHP amqp lib.

The code below uses php-amqplib to connect and send messages to a RabbitMQ server.

The publisher (writes messages to queue):

require_once('amqp.inc');

$HOST = 'localhost';
$PORT = 5672;
$USER = 'guest';
$PASS = 'guest';
$VHOST = '/';
$EXCHANGE = 'router';
$QUEUE = 'msgs';

$conn = new AMQPConnection($HOST, $PORT, $USER, $PASS);
$ch = $conn->channel();
$ch->access_request($VHOST, false, false, true, true);

$msg_body = implode(' ', array_slice($argv, 1));
$msg = new AMQPMessage($msg_body, array('content_type' => 'text/plain'));
$ch->basic_publish($msg, $EXCHANGE);

$ch->close();
$conn->close();

The consumer (reads messages from queue):

require_once('amqp.inc');

$HOST = 'localhost';
$PORT = 5672;
$USER = 'guest';
$PASS = 'guest';
$VHOST = '/';
$EXCHANGE = 'router';
$QUEUE = 'msgs';
$CONSUMER_TAG = 'consumer';

$conn = new AMQPConnection($HOST, $PORT, $USER, $PASS);
$ch = $conn->channel();
$ch->access_request($VHOST, false, false, true, true);

$ch->queue_declare($QUEUE);
$ch->exchange_declare($EXCHANGE, 'direct', false, false, false);
$ch->queue_bind($QUEUE, $EXCHANGE);

function process_message($msg) {
global $ch, $CONSUMER_TAG;

echo "\n--------\n";
echo $msg->body;
echo "\n--------\n";

$ch->basic_ack($msg->delivery_info['delivery_tag']);

// Cancel callback
if ($msg->body === 'quit') {
$ch->basic_cancel($CONSUMER_TAG);
}
}

$ch->basic_consume($QUEUE, $CONSUMER_TAG, false, false, false, false, 'process_message');

// Loop as long as the channel has callbacks registered
while(count($ch->callbacks)) {
$ch->wait();
}

$ch->close();
$conn->close();

--

Further reading:
* Why Databases suck for messaging
* Introduction to Message Queues and how RabbitMQ works
* SecondLife (the game) evaluation of 9 message-queue solutions
* RabbitMQ Administration
* RabbitMQ Clustering

Realtime PHP Syntax Checking

Mike Peters, April 14

If you love Agile Development and PHP as much as we do, you're probably going to have developers editing critical PHP scripts on live machines.

While many PHP editors include built-in syntax checking, it doesn't hurt to have a fail-safe in place. We wanted to come up with a way to apply php -l SCRIPT syntax checking on any PHP script as soon as it is updated.

At first the idea was to write a plugin for Mercurial - our version control system, so that whenever a changeset is pushed, the syntax will be checked.

Unfortunately this doesn't cover changes done via SSH or files uploaded via FTP by clients.

We came up with a shell script, executed by cron every minute, that iterates through all files changed within the last 3 days, applying "php -l" on those files and sending an email to the appropriate developer with the information about the error.

Regardless of how files are updated, we now have an automated way to syntax-check every single change to a PHP script.

The emails generated by the script follow this format:

Oops! Found a FATAL syntax error in a production PHP script...

DIRECTORY: /magnetic/

---------------------------------------------
FILE: ./affiliate/offline_credit_commission.php
---------------------------------------------
Parse error: syntax error, unexpected '<' in /offline_credit_commission.php on line 8
changeset: 2673:fe54a1e4dbe0
user: Doug
date: Wed Apr 14 00:29:26 2010 +0300
summary: Change offline credit affiliate commission

CheckSyntax cronjob:

#!/bin/sh

# Change to source directory directory
cd $(dirname $0)

# Generate report
./CheckSyntax.sh >log.$$
a=$?

# Email report
# TODO: Update these with your settings
if [ $a = 1 ]
then
./sendEmail -f CheckSyntax@mycompany.com \
-t developers@mycompany.com \
-u "Check Syntax" \
-s mail.mymailserver.com \
-m < log.$$
fi

# Remove temporary file
rm -rf log.$$

# (eof)

CheckSyntax.sh:

#!/usr/local/bin/bash

##################################
# Config
##################################
#

# TODO: Update these with your settings
php="/usr/local/bin/php"
hg="/usr/local/bin/hg"
def="/magnetic"

#
##################################

# Directory passed in parameters?
if [ "$1" != "" ]
then
# Get the directory from parameters
dir=$1
else
# No parameter, use default
dir=$def
fi

# Change to the directory we need to check
cd $dir

# Prepare tmp file
tmp=/tmp/CheckSyntax.$$

# Header
echo "Oops! Found a FATAL syntax error in a production PHP script..."
echo ""
echo "DIRECTORY: $dir"

# Find PHP files
# Note: '-newerct' only workds under FreeBSD
find . -newerct '3 days ago' ! -path "*/temp/*" | grep "\.php$" >$tmp

# Loop files
cnt=0
err=0
tot=0
while read i
do
# If file isn't a file, skip
[ ! -f "$i" ] && continue;

((tot++))

# Syntax check
a=$($php -l $i | grep "^No syntax errors detected in")

# Is it OK?
if [ "$a" = "" ]
then
# Not OK, display details
echo ""
echo "----------------------------------------------------------------------------------------------"
echo "FILE: $i"
echo "----------------------------------------------------------------------------------------------"
a=$($php -d display_errors=1 -l $i | grep "Parse error")
echo $a | sed "s/<[a-zA-Z\/][^>]*>//g"
$hg log $i | head -n 5 | grep -v "^$"
((cnt++))
err=1
fi
done < $tmp

echo ""
echo "--"
echo ""
echo "$cnt BAD FILES"
echo "$tot TOTAL FILES"
echo ""

# Remove tmp file
rm -rf $tmp

# 0 - no error | 1 - error
exit $err

# (eof)

-

To install, save the two files (CheckSyntax and CheckSyntax.sh), chmod a+rx, update the email settings in CheckSyntax and add this line to your cronjob:

# CheckSyntax
# TODO: Update path with the location of CheckSyntax
*/1 * * * * /home/CheckSyntax

How to create a PHP Daemon

Mike Peters, April 14

A daemon is a Linux program that runs in the background.

Most daemons are written in C. While C is faster and more robust than PHP, looking at development time and cost, PHP generally scores a lot better than C.

Converting your C daemons to PHP where applicable, makes the code more accessible to a wider percentage of your development force and allows better re-use and connecting of existing code.

To Facebook, keeping all code accessible to all developers was so important, that they've decided to come up with a way to compile PHP to C in real time. So you get PHP speed of development with C speed. The HipHop toolkit is available on an open source license.

The simple PHP4 functions included here, will let you easily create PHP daemons.

Features:
* Daemon will not start if another instance is already running
* Daemon PID saved to /var/run/
* Daemon log messages saved to /var/log/
* Run in foreground mode with the "-f" flag

Sample daemon:

require_once("common_daemon.php");

// Get parameters from command line
foreach ($argv as $parameter)
{
$run_in_foreground = Strcasecmp($parameter,"-f")==0;
}

// Start our daemon
if (!PHPDaemonStart("mydaemon", $run_in_foreground))
{
// Already running
return;
}

// Do something useful
while (1)
{
PHPDaemonLog("Hello, the time is ".date("h:i:s", time()));
sleep(1);
}

// Never here

The common_daemon.php functions:

// Give us eternity to execute the script.
ini_set("max_execution_time", "0");
ini_set("max_input_time", "0");
set_time_limit(0);

function PHPDaemonStart($name, $run_in_foreground=0)
{
// This must be global so that the handle is kept alive
// after this function exits
global $handle_lockfile;

// Remember the daemon name
global $phpdaemon_name;
$phpdaemon_name = $name;

// Remember if we're running in foreground mode
global $phpdaemon_run_in_foreground;
$phpdaemon_run_in_foreground = $run_in_foreground;

// Open PID file
$tmpfilename = "/var/run/$name.pid";
if (!($handle_lockfile = @fopen($tmpfilename,"a+")))
{
// Script already running - abort
return 0;
}

// Obtain an exlcusive lock on file
// (If script is running this will fail)
if (!@flock( $handle_lockfile, LOCK_EX | LOCK_NB, &$wouldblock) || $wouldblock)
{
// Script already running - abort
@fclose($handle_lockfile);
return 0;
}

// Write our PID
@ftruncate($handle_lockfile,0);
@fseek($handle_lockfile, 0, 0);
@fwrite($handle_lockfile, getmypid());
@fflush($handle_lockfile);

// If we need to run in foreground, we're done
if ($run_in_foreground)
{
return 1;
}

// If we're up to here we need to fork our process
$pid = pcntl_fork();
if ($pid==-1)
{
// Can't fork
return 0;
}

// If we are the parent
if ($pid)
{
// Sleep for 2 seconds, letting the forked process wake up
// and attempt to grab our lock
sleep(2);

// Release the lock
@flock($handle_lockfile, LOCK_UN);

// Close file
fclose($handle_lockfile);

// Kill parent process
// By now the child has the lock
die;
}
// (Else - we are the child)
else
{
// Open file
while (!($handle_lockfile = @fopen($tmpfilename, "a+")))
{
// Try again
usleep(100);
}

// Re-establish lock on file
while (!@flock($handle_lockfile, LOCK_EX, &$wouldblock))
{
// Try again
usleep(100);
}

// Write pid into file
@ftruncate($handle_lockfile,0);
@fseek($handle_lockfile, 0, 0);
@fwrite($handle_lockfile, getmypid());
@fflush($handle_lockfile);
}

// We have a lock - all good
return 1;
}

function PHPDaemonLog($str)
{
// This is the daemon name
global $phpdaemon_name;

// This indicates whether or not we are running in foreground mode
global $phpdaemon_run_in_foreground ;

// Open log file
$handle = @fopen("/var/log/$phpdaemon_name.log", "a");

// Obtain an exclusive lock (so that this is thread safe)
flock($handle, LOCK_EX, &$wouldblock);

// Write
$output = date("Y-m-d h:i:s", time())." $str\r\n";
@fwrite($handle, $output);

// If we are running in foreground mode, output to screen as well
if ($phpdaemon_run_in_foreground) echo $output;

// Release lock
flock($handle, LOCK_UN);

// Close file
fclose($handle);
}

function IsPHPDaemonRunning($name)
{
// Open PID file
$tmpfilename = "/var/run/$name.pid";
if (!($tmpfile = @fopen($tmpfilename,"r")))
{
// Script not running
return 0;
}

// Obtain an exlcusive lock on file
// (If script is running this will fail)
$not_running = flock( $tmpfile, LOCK_EX | LOCK_NB, &$wouldblock);

// Release lock if successful
if ($not_running) flock($tmpfile, LOCK_UN);

// Close file
@fclose($tmpfile);

// Return result
return !$not_running || $wouldblock;
}

View 1 Comment(s)

Cassandra PHP Wrapper

Mike Peters, April 7

Update: Check out the Cassandra PHP Wrapper 0.7

In a previous post, I explained how the Cassandra decentralized database can allow you to scale well beyond what's possible with MySQL.

The Cassandra Data Model and API are very different than traditional RDBMS way-of-thinking. That's why a lot of developers switching from MySQL to Cassandra are finding it difficult to grasp at first.

To simplify the migration from MySQL to Cassandra, we created a high-level PHP wrapper for Cassandra, using function prototypes and variable names that are close in meaning to RDBMS.

While there are several other high level PHP wrappers for Cassandra, none of the existing ones answered our requirements -

* Simple
* Don't throw exceptions, return error codes
* As close as possible to the low level Thrift
* Make it easy for RDBMS developers to adopt

This is how the CassandraDB class was born.

Adding records to the database is as simple as:

// Initialize Cassandra
$cassandra = new CassandraDB("SPI");

// Debug on
$cassandra->SetDisplayErrors(true);

// Insert record ("Columns" in Cassandra)
$record = array();
$record["name"] = "Mike Peters";
$record["email"] = "mike at softwareprojects.com";
if ($cassandra->InsertRecord('mytable', "Mike Peters", $record))
{
echo "Record (Columns) inserted successfully.\r\n";
}

// Print record
$record = $cassandra->GetRecordByKey('mytable', "Mike Peters");
print_r($record);

Output is

Array
(
[email] => mike at softwareprojects.com
[name] => Mike Peters
)

Adding record arrays:

// Initialize Cassandra
$cassandra = new CassandraDB("SPI");

// Debug on
$cassandra->SetDisplayErrors(true);

// Insert record array ("SuperColumns" in Cassandra)
$record = array();
$record["Mike Peters"] = array("name" => "Mike Peters", "email" => "Mike at Peters");
$record["Jonathan Ellis"] = array("name" => "Jonathan Ellis", "email" => "Jonathan at Ellis");
if ($cassandra->InsertRecordArray('my_super_table', "People", $record))
{
echo "RecordArray (SuperColumns) inserted successfully.\r\n";
}

// Print record array
$record = $cassandra->GetRecordByKey('my_super_table', 'People');
print_r($record);

Output is

Array
(
[Jonathan Ellis] => Array
(
[email] => Jonathan at Ellis
[name] => Jonathan Ellis
)

[Mike Peters] => Array
(
[email] => Mike at Peters
[name] => Mike Peters
)

)

--

Here's the complete Cassandra PHP Wrapper class. Enjoy!

// CassandraDB version 0.1
// Software Projects Inc
// http://www.softwareprojects.com
//

// Includes
$GLOBALS['THRIFT_ROOT'] = '/services/thrift/';
require_once $GLOBALS['THRIFT_ROOT'].'/packages/cassandra/Cassandra.php';
require_once $GLOBALS['THRIFT_ROOT'].'/packages/cassandra/cassandra_types.php';
require_once $GLOBALS['THRIFT_ROOT'].'/transport/TSocket.php';
require_once $GLOBALS['THRIFT_ROOT'].'/protocol/TBinaryProtocol.php';
require_once $GLOBALS['THRIFT_ROOT'].'/transport/TFramedTransport.php';
require_once $GLOBALS['THRIFT_ROOT'].'/transport/TBufferedTransport.php';

class CassandraDB
{
// Internal variables
protected $socket;
protected $client;
protected $keyspace;
protected $transport;
protected $protocol;
protected $err_str = "";
protected $display_errors = 0;
protected $consistency = 1;
protected $parse_columns = 1;

// Functions

// Constructor - Connect to Cassandra via Thrift
function CassandraDB ($keyspace, $host = "127.0.0.1", $port = 9160)
{
// Initialize
$this->err_str = '';

try
{
// Store passed 'keyspace' in object
$this->keyspace = $keyspace;

// Make a connection to the Thrift interface to Cassandra
$this->socket = new TSocket($host, $port);
$this->transport = new TFramedTransport($this->socket, 1024, 1024);
$this->protocol = new TBinaryProtocolAccelerated($this->transport);
$this->client = new CassandraClient($this->protocol);
$this->transport->open();
}
catch (TException $tx)
{
// Error occured
$this->err_str = $tx->why;
$this->Debug($tx->why." ".$tx->getMessage());
}
}

// Insert Column into ColumnFamily
// (Equivalent to RDBMS Insert record to a table)
function InsertRecord ($table /* ColumnFamily */, $key /* ColumnFamily Key */, $record /* Columns */)
{
// Initialize
$this->err_str = '';

try
{
// Timestamp for update
$timestamp = time();

// Build batch mutation
$cfmap = array();
$cfmap[$table] = $this->array_to_supercolumns_or_columns($record, $timestamp);

// Insert
$this->client->batch_insert($this->keyspace, $key, $cfmap, $this->consistency);

// If we're up to here, all is well
$result = 1;
}
catch (TException $tx)
{
// Error occured
$result = 0;
$this->err_str = $tx->why;
$this->Debug($tx->why." ".$tx->getMessage());
}

// Return result
return $result;
}

// Insert SuperColumn into SuperColumnFamily
// (Equivalent to RDMBS Insert record to a "nested table")
function InsertRecordArray ($table /* SuperColumnFamily */, $key_parent /* Super CF */,
$record /* Columns */)
{
// Initialize
$err_str = '';

try
{
// Timestamp for update
$timestamp = time();

// Build batch mutation
$cfmap = array();
$cfmap[$table] = $this->array_to_supercolumns_or_columns($record, $timestamp);

// Insert
$this->client->batch_insert($this->keyspace, $key_parent, $cfmap, $this->consistency);

// If we're up to here, all is well
$result = 1;
}
catch (TException $tx)
{
// Error occured
$result = 0;
$this->err_str = $tx->why;
$this->Debug($tx->why." ".$tx->getMessage());
}

// Return result
return $result;
}

// Get record by key
function GetRecordByKey ($table /* ColumnFamily or SuperColumnFamily */, $key, $start_from="", $end_at="")
{
// Initialize
$err_str = '';

try
{
return $this->get($table, $key, NULL, $start_from, $end_at);
}
catch (TException $tx)
{
// Error occured
$this->err_str = $tx->why;
$this->Debug($tx->why." ".$tx->getMessage());
return array();
}
}

// Print debug message
function Debug ($str)
{
// If verbose is off, we're done
if (!$this->display_errors) return;

// Print
echo date("Y-m-d h:i:s")." CassandraDB ERROR: $str\r\n";
}

// Turn verbose debug on/off (Default is off)
function SetDisplayErrors($flag)
{
$this->display_errors = $flag;
}

// Set Consistency level (Default is 1)
function SetConsistency ($consistency)
{
$this->consistency = $consistency;
}

// Build cf array
function array_to_supercolumns_or_columns($array, $timestamp=null)
{
if(empty($timestamp)) $timestamp = time();

$ret = null;
foreach($array as $name => $value) {
$c_or_sc = new cassandra_ColumnOrSuperColumn();
if(is_array($value)) {
$c_or_sc->super_column = new cassandra_SuperColumn();
$c_or_sc->super_column->name = $this->unparse_column_name($name, true);
$c_or_sc->super_column->columns = $this->array_to_columns($value, $timestamp);
$c_or_sc->super_column->timestamp = $timestamp;
}
else
{
$c_or_sc = new cassandra_ColumnOrSuperColumn();
$c_or_sc->column = new cassandra_Column();
$c_or_sc->column->name = $this->unparse_column_name($name, true);
$c_or_sc->column->value = $value;
$c_or_sc->column->timestamp = $timestamp;
}
$ret[] = $c_or_sc;
}

return $ret;
}

// Parse column names for Cassandra
function parse_column_name($column_name, $is_column=true)
{
if(!$column_name) return NULL;

return $column_name;
}

// Unparse column names for Cassandra
function unparse_column_name($column_name, $is_column=true)
{
if(!$column_name) return NULL;

return $column_name;
}

// Convert supercolumns or columns into an array
function supercolumns_or_columns_to_array($array)
{
$ret = null;
for ($i=0; $i<count($array); $i++)
foreach ($array[$i] as $object)
{
if ($object)
{
// If supercolumn
if (isset($object->columns))
{
$record = array();
for ($j=0; $j<count($object->columns); $j++)
{
$column = $object->columns[$j];
$record[$column->name] = $column->value;
}
$ret[$object->name] = $record;
}
// (Otherwise - not supercolumn)
else
{
$ret[$object->name] = $object->value;
}
}
}

return $ret;
}

// Get record from Cassandra
function get($table, $key, $super_column=NULL, $slice_start="", $slice_finish="")
{
try
{
$column_parent = new cassandra_ColumnParent();
$column_parent->column_family = $table;
$column_parent->super_column = $this->unparse_column_name($super_column, false);

$slice_range = new cassandra_SliceRange();
$slice_range->start = $slice_start;
$slice_range->finish = $slice_finish;
$predicate = new cassandra_SlicePredicate();
$predicate->slice_range = $slice_range;

$resp = $this->client->get_slice($this->keyspace, $key, $column_parent, $predicate, $this->consistency);

return $this->supercolumns_or_columns_to_array($resp);
}
catch (TException $tx)
{
$this->Debug($tx->why." ".$tx->getMessage());
return array();
}
}

// Convert array to columns
function array_to_columns($array, $timestamp=null) {
if(empty($timestamp)) $timestamp = time();

$ret = null;
foreach($array as $name => $value) {
$column = new cassandra_Column();
$column->name = $this->unparse_column_name($name, false);
$column->value = $value;
$column->timestamp = $timestamp;

$ret[] = $column;
}
return $ret;
}

// Get error string
function ErrorStr()
{
return $this->err_str;
}
}

CassandraDB Todo:

* Ability to add multiple nodes and try to connect to next-in-line automatically if connection fails
* Implement the remaining get_ functions
* Implement delete
* Implement update vs insert-fail-on-duplicate-key?
* Implement Order By desc

-

Recommended Further Reading:

* Cassandra in Production at Digg
* WTF is a SuperColumn?
* Cassandra Reads - How do they work
* Cassandra Writes - How do they work
* Cassandra Write Properties (Video)
* Engineering notes by Facebook
* RandomPartitioner vs OrderPreservingPartitioner

View 10 Comment(s)

« Previous Posts

» Next Posts

About Us | Contact us | Privacy Policy | Terms & Conditions