I recently worked on a project that involved integrating with MSN Messenger, creating my very own "IM Robot".

What can an IM Robot do?

Keep track of contacts online/offline status and when they were last seen.

Broadcast a message to all contacts.

Automatically answer common questions.

Notify contacts about new events. (See http://notify.me)

Keep track of code snippets

Which IM Library to Use

Setting up a IM robot can be a bit of work especially if starting from scratch. There are a lot of libraries out there that can be used to help simplify the process. The trouble is a lot of libraries are not kept up to date and fail to work as IM protocols change.

We did some digging and found a library that would provide a good foundation to build a IM Robot that can do just about anything. We say implementations in PHP, C, Java, Perl and Python. After some testing we concluded the Java MSN Library would be a very good fit.

How To Use It

Using this library with java is pretty straight forward. First the library must be added to the classpath. The step to take to complete this will depend on how your developing your java code. The most basic method to add a library to your classpath is to do this at run time with a command such as:

java -classpath MyLibrary.jar MyPackage.MyClass

A better approach would be to setup the classpath in a manifest file. The manifest file is then placed inside the jar file and tells the executable jar where to look for the libraries. This manifest file should look something like this (note the class-path on line 5):

Manifest-Version: 1.0
Ant-Version: Apache Ant 1.7.1
Created-By: 14.3-b01 (Sun Microsystems Inc.)
Main-Class: imstatus.Main
Class-Path: lib/jml-1.0b4-full.jar lib/httpcore-4.0.1.jar lib/mysql-co
nnector-java-5.1.6-bin.jar
X-COMMENT: Main-Class will be added automatically by build

This is setup so that the 3 required libraries are in the lib folder. These 3 libraries are needed for setting up an IM robot and can be downloaded from the following locations:

jml-1.0b4-full.jar

httpcore-4.0.1.jar

mysql-connector-java-5.1.6-bin.jar

Now that the libraries are setup we can begin to use them.

Developing the IM Robot Code

There are a few examples of using the Java MSN Library on the main page. However, they are a tad confusing as it creates a new BasicMessenger class. This is confusing as the library already has a BasicMessenger class which is abstract. The library also has a SimpleMessenger class which is a subclass of BasicMessenger. This class appears to be the correct implementation that we would want to use to create a new IM Robot.

However, the original authors made the constructor protected so that we cannot instantiate the class outside of the original package. Since we want a simple way to create an IM Robot I have modified the original source code to have a public constructor for the SimpleMessenger class.

With this new package we can very easily create a new IM Robot with the following two lines of code (make sure to replace yourLogin and yourPassword):

SimpleMessenger messenger = new SimpleMessenger(Email.parseStr("yourLogin@msn.com"), "yourPassword");
messenger.login();

With that code in our main funtion we can run it and test that the Robot automatically logs into Windows Live Messenger.

Of course that code just logs the Robot into Windows Live Messenger. The next step is to setup the robot to do something interesting. This is one feature that is very nice about the Java Msn Library as it as listeners for many different events. For example we can detect when the robot has finished logging in with the following:

messenger.addListener(new MsnAdapter() {
// Setup the login completed event

@Override
public void loginCompleted(MsnMessenger messenger) {
MsnOwner owner = messenger.getOwner();
owner.setInitStatus(MsnUserStatus.ONLINE);
owner.setStatus(MsnUserStatus.ONLINE);

// Setup the contact list event
messenger.addContactListListener(new ContactListAdapter());
}
});

Then we can take this a step further and detect when a status changes for one of the robots contacts with something like the following:

messenger.addListener(new MsnAdapter() {
// Setup the login completed event

@Override
public void loginCompleted(MsnMessenger messenger) {
MsnOwner owner = messenger.getOwner();
owner.setInitStatus(MsnUserStatus.ONLINE);
owner.setStatus(MsnUserStatus.ONLINE);

// Setup the contact list event
messenger.addContactListListener(new ContactListAdapter());
}
});

The above code will detect when the robot has finished logging in and then setup a new listener to detect when a contacts status has changed. The new listener invokes the ContactListAdapter class when a status has changed. This contactListAdapter class is setup as followes:

class ContactListAdapter extends MsnContactListAdapter {
@Override
public void contactStatusChanged(MsnMessenger messenger, MsnContact contact) {
System.out.println(contact.getEmail()+" is currently "+contact.getStatus());
// Can add code here to store the status in a database
}
}

We can still take this a step further and setup the robot to handle automatically adding contacts when a contact requests it. This logic can be added to the ContactListAdapter class with something like the following:

class ContactListAdapter extends MsnContactListAdapter {
@Override
public void contactListSyncCompleted(MsnMessenger messenger) {
MsnContact[] contacts = messenger.getContactList().getContactsInList(MsnList.AL);
for (int i = 0; i < contacts.length; i++) {
contactStatusChanged(messenger,contacts[i]);
}
}

@Override
public void contactAddedMe(MsnMessenger messenger, MsnContact contact) {
messenger.addFriend(contact.getEmail(), contact.getDisplayName());
}

@Override
public void contactAddedMe(MsnMessenger messenger, MsnContactPending[] pending){
for(int i=0; i messenger.addFriend(pending[i].getEmail(), pending[i].getDisplayName());
}
}

@Override
public void contactStatusChanged(MsnMessenger messenger, MsnContact contact) {
System.out.println(contact.getEmail()+" is currently "+contact.getStatus());
// Can add code here to store the status in a database
}
}

There you have it! Put all of the above code together and you will have a robot that knows how to automatically add contacts and keep track of when a contact's status changes.

There's a new kid on the block.

A new system designed to "simplify the deployment of Marketing Campaigns, Sales Funnels, and Product Launches from their current technical complexity down to Drag-and-Drop Ease" (Source: Kajabi.com)

Kajabi was created by Andy Jenkins (VideoBoss, formerly with Stompernet), Kenny Rueter and Travis Rosser. With this roster, you can bet your Inbox is about to be flooded with emails about how Kajabi is the best thing since sliced bread.

With this post, I wanted to address a question by one of our clients - how does Kajabi compare with SoftwareProjects' own Shopping Cart solution.

Disclaimer: Kajabi is still under development. The information I gathered for this post was collected by analyzing two recent product launches powered by Kajabi, along with a series of emails from the Kajabi team.

First things first: Kajabi is NOT a Shopping Cart

Kajabi is not designed to be an end-to-end shopping cart system.

They will integrate with all popular shopping cart systems, but it's just not designed to be a CRM / Affiliate & Lead Manager / Fulfillment / Backoffice, like SoftwareProjects or Infusionsoft.

You can expect limited built-in support for selling your content, but you're not going to get an executive dashboard that lets you manage the full 360 of your Internet Business.



Membership System

Kajabi offers a built-in membership system, where paying members login to access the content (videos / tutorials) they paid for.

You can customize the membership system skin to some degree and manage members from the admin panel. Think aMember plus a CMS (Content Management System).

At this point my understanding is there's going to be a single backoffice (unlike SPI's Customer's Backoffice and Affiliate's Backoffice), which will support a single product per Kajabai deployment.

Content Delivery

This is one area that keeps being emphasized in all communications I hear about Kajabi.

Still not clear how this is different from hosting all videos and MultiMedia on Amazon S3 or EdgeCast, like everyone are doing.

I'm guessing they use the same underlying technology and just make it easier to use.

Autoresponder

Kajabi will not have a built-in Autoresponder.

It is said to integrate with popular Autoresponder services such as AWeber, SoftwareProjects, iContact etc.

Help desk

Kajabi will not have a built-in help desk system.

Hosting

Kajabi is said to offer "Cloud Computing Technology, (so that) you can publish anything you create with Kajabi instantly to the web - with theoretically infinitely scalable virtual hardware resources".

They seem to be using Rackspace Cloud hosting, to create virtual instances of your site as needed, to accommodate traffic demands.

This sounds very similar to SoftwareProjects' Multi-Homed hosting, where we do what no other company has ever done before and guarantee 100% uptime, no matter what.

Battle Tested

Kajabi successfully powered the product launches of Andy Jenkins' Video Boss and Frank Kern's List Control.

While still under development, the technology has proven its ability to handle launch-day traffic without too many glitches.

Here at SoftwareProjects, we have been powering the majority of the big product launches you hear about, including Magnetic Sponsoring WWN, the ShoeMoney System, ListBuilding, PPC Classroom, Affiliate Classroom, OnlineMLMSecrets and others.

Then again... we've been doing this since 1998.

In Summary

We really look forward to working with the Kajabi team on the integration side.

Where Kajabi shines is simplifying the process of getting an info-marketing product up in no time.

I would describe it as an InfoMarketing CMS with a membership backoffice. It may not have all the bells and whistles, but as long as the integration is straightforward, you get the best of both worlds.

-

More on this topic:
* Kajabi's home page
* Kajabi Beta Testing portal
* Follow Kajabi on Twitter
* Andy Jenkins blog

Keynotes from a great presentation titled Cassandra Troubleshooting: out of the shadows, presented by Benjamin Black at the Cassandra Summit in San Francisco two weeks ago.

The slides are here

-

Is your Ring unbalanced?

That's because when you add one node at a time using RandomPartitioner, the new nodes takes over half of the most balanced node:

Note that as long as you're doubling-up the size of your cluster, everything will be balanced. But when you're growing one node at
a time, the cluster will be unbalanced.

To fix: Manually assign tokens.

How do you know which tokens to assign? Use this Python script:

Writes are slow

Make sure your commitlog is on a separate drive.

Writes are fast. Reads keep getting slower

Step 1:

Look at iostat -x to see if you're maxing out utilization

If you are, get more nodes

Step 2:

Look at nodetool tpstats

Focus on the middle column (pending) and specifically:
* Row-Read-STage
* Message-Deserializer-pool

If these two are high (4096 is the max), it means your client is sending too many reads to this node.

Update your client or get more nodes to distribute reads.

Step 3:

Adjust memtable settings

When does a memtable get flushed to disk?

Size: When it gets to a certain size
Time: If it hasn't been flushed in x seconds
Operations: When certain operations occur

If you're flushing memtables too often, you're triggering follow-up effects (compactions, sstable merges) that is consuming a lot of bandwidth.

You want less frequent memtable flush, which leads to less frequent compaction and less disk bandwidth demand.

If memtable is not compatible with your data needs, you begin consuming huge amounts of your bandwidth on compactions.

once a minute = bad

Step 4:

Use SSDs for the disk drives. Makes no difference on the commit log drive.

I inserted a bunch of data, now my nodes are flapping

Flapping = nodes are marked down/up

Step 1:

Monitor swap (vmstat on linux, swapinfo on freebsd)

mmap takes 2gb per segment.

Swapping can delay gossip long enough to cause a node
to be marked down.

Swapping is bad.

To fix: Change DiskAccessMode in the Cassandra config file, to mmap_index_only

We avoid risking driving ourselves into swap by the JVM
allocating large chunks of mmap blocks.

Step 2:

Tell the O/S you want to avoid swapping if possible.
On FreeBSD: add this line to /etc/sysctl.conf

On Linux, echo 0 into /proc/sys/vm/swappiness

In our ongoing journey to deliver faster, more reliable shopping carts, email marketing and hosting solutions, we're always on the lookout for new technologies and techniques to deal with a lot of data.

I'd like to share a few presentations we recently came across, that talk about scaling in the real world:

* MySpace: Massive scaling a .Net website with the Microsoft Robotic Studio
* Twitter: Big Data in Real-Time at Twitter
* Scale at Facebook
* Yahoo: Selectively Materializing User's Event Feeds
* Four Kitchens: Scalable queuing with flexible service levels
* Cassandra at Mahalo
* Cassandra at CloudKick

If you're using Cassandra over PHP, you are probably aware of several bugs in the PHP Thrift interface that get in the way.

Courtesy of Jonathan Ellis, here's a list of known patches you need to apply to your PHP Thrift library:

* https://issues.apache.org/jira/browse/THRIFT-347 (TSocket Timeout)
* https://issues.apache.org/jira/browse/THRIFT-638 (Block until recv timeout)
* https://issues.apache.org/jira/browse/THRIFT-780 (Prevent aborts)
* https://issues.apache.org/jira/browse/THRIFT-788 (Multiget more than 17 keys)

To make your life easier, you can download a patched version of Thrift 0.2 for PHP, with all patches applied, here.

-

A great presentation by Cliff Moon: Dealing with the Thrift API

View 2 Comment(s)

A new version of the SoftwareProjects Cassandra PHP Wrapper (db_cassandra.php) is now available, for Cassandra 0.7.

This version of the PHP Wrapper adds support for:
* Adding Keyspaces and Column families on the fly
* Better error logging
* Disconnect on destructor to prevent java fd leaks
* Reconnect on timeout, up to 10 times
* Deletes
* Updated all API calls to 0.7

You can download the Cassandra 0.7 PHP wrapper, modify and distribute it freely.

Please let us know if you find any issues.

Usage Examples

Adding a new Keyspace with one column family


Adding records and reading back values


Setup tips

The PHP Wrapper was tested with the 0.7 trunk version, June 30, 2010 build.

Make sure you patch your Thrift TSocket as described here (comments section).

-

Further Reading

If you're just getting started with Cassandra, you'll enjoy the selection of hand picked Cassandra tutorials listed below. These are the best of the best:

= Tutorials
* Start here: WTF is a SuperColumn
* Cassandra Basics - Indexing
* Introduction to Cassandra
* History of Cassandra (Video)
* The shift to NoSQL

= Case Studies
* Scaling Twitter with Cassandra
* Cassandra at Digg
* Cassandra at RackSpace

= Internals
* How Reads work in Cassandra
* How Writes work in Cassandra
* Cassandra Replication and Consistency
* Random Partitioner vs Order Preserving Partitioner (OPP)
* Cassandra 101
* Do you really need SQL to do it all in Cassandra
* Let's play with Cassandra

View 6 Comment(s)

In a previous post, we wrote about PHP real time syntax checking. A method to send email alerts when any critical scripts on your live servers fail to pass PHP syntax validation.

The problem with this approach is that it alerts you of a problem after-the-fact.

Someone uploads a bad PHP script which breaks a critical module, you are alerted, but by that time users are already affected.

Today we'll take it this concept one step further by writing a hook for the Mercurial version control system, preventing developers from checking-in scripts that don't pass syntax-checking.

Installing PHP Syntax Check Mercurial hook

Step 1: The shell script

Save this shell script under your Mercurial folder, calling it: php_syntax.sh


Step 2: Add hook to Mercurial config file

Update your Mercurial hgweb.config, adding the new hook under the 'hooks' section, like this:

[hooks]
pretxnchangegroup.syntax_check = /usr/home/mercurial/php_syntax.sh

--

Replace '/usr/home/mercurial' with the path where you saved php_syntax.sh

Attempting to push changes to the repository that don't pass php-syntax check, are now blocked.

Users will be presented with an error message and the transaction rolled back, until the user fixes the problem and pushes changes again.