Call us Toll-Free:
1-800-218-1525
Live ChatEmail us
Mercurial is great distributed version control system that we all use and love here at SPI.

If you're not familiar with distributed version control systems, the key thing to understand is that, unlike a central repository, Mercurial pulls all revisions to your local machine, so that you have the entire project history from day one.

This allows you to check-in / check-out / revert / branch all locally, without having to interface with a remote server repository until you're ready to push changes to the server.

Working on a Windows machine, with the server running on Linux/FreeBSD, introduces an interesting problem of "Case Folding Collision".

What is Case Folding Collision?

Linux/FreeBSD are case sensitive file systems. This means that hello.gif and HELLO.gif are two different files.

Windows on the other hand, is a case insensitive file system. You can't have both hello.gif and HELLO.gif in the same directory on a Windows machine.

Using Mercurial to pull/push between a Linux/FreeBSD and your local Windows machine, becomes problematic when you have multiple files with the same name different capitalization in the same folder.

Pulling changes from Mercurial aborts with an aggressive error, like

Quote:
abort: case-folding collision between backoffice/assets/images/med-wwn-platinum.gif and backoffice/assets/images/med-wwn-Platinum.gif

How to fix Mercurial Case Folding Collision

On the Linux/FreeBSD machine, we have to enter Mercurial "debug mode", go back to the bad version that introduced the files with same name different capitalization, delete them in that old revision and then go back to the current revision.

Here's how it's done:

Step 1: Enter Mercurial debug mode

hg debugsetparents REVISION

The revision should be the one you are attempting to update to.

hg debugrebuildstate

Step 2: Remove rogue files

Now we want to remove all the files in error.

You can use 'hg manifest tip' to check for the files in error.

hg rm -A -f FILENAME
hg forget FILENAME

Step 3: Commit changes

hg ci -m "Fixed case problems" -u root
hg merge -f

Step 4: Commit again (incase we had multiple heads)

hg ci -m "Merged head" -u root

Step 5: Go back to tip

hg co -C tip

Note: Often when dealing with case folding collision, you'll have more than one rogue file. Make sure you delete ALL of them on step 2, so that you don't have to repeat the process multiple times.

View 2 Comment(s)
Want to find out if your server is under DoS attack?

Or maybe you want to optimize your architecture, understanding the network footprint of each of your servers?

Using a combination of handy shell commands, it's easy to construct a list of all IP addresses connected to your server, sorted by count:

On FreeBSD:
netstat -nat | sed -n -e '/ESTABLISHED/p' | awk '{print $5}' | sed 's/\./ /g' | awk '{print $1"."$2"."$3"."$4}' | sort | uniq -c | sort -n

On Linux:
netstat -ntu | sed -e 's/::ffff://g' | awk '{print $5}' | cut -d : -f1 | sort | uniq -c | sort -n

Sample output:

1 114.56.110.115
1 114.56.110.90
1 67.128.224.129
2 114.56.110.85
4 114.56.110.99
5 208.53.167.125
7 67.128.224.128
14 114.56.110.116
15 114.56.110.125
18 114.56.110.84

Blocking offenders is as easy as:

On Linux:
/sbin/iptables -I INPUT -s 114.56.110.84 -j DROP

On FreeBSD:
echo "ALL : 114.56.110.84 : deny" >> /etc/hosts.allow


View 4 Comment(s)
If you're using Mercurial HG and keep running into the "premature EOF reading chunk" error, there's hope. As part of this post, I will walk you through the process of repairing a damaged repository where HG .i files are corrupt and no one can successfully "Pull Changes".

Step 1: Connect to your HG server, cd to the repository folder and run
hg verify

If you get something like

Quote:
error_file_name.extension@553: unpacking 646ac1c08e97: ./../Objects/stringobject.c:4124: bad argument to internal function
10662 files, 567 changesets, 11475 total revisions
1 integrity errors encountered!

Voila, you're dealing with HG index corruption.

Step 2: Backup the file path_to_project/.hg/store/data/error_file_name.extension.i backup/error_file_name.extension.i

Step 3: In the path_to_project/ folder create a new empty file
echo "" >> new_file.extension

Step 4:
hg add new_file.extension

Step 5:
hg commit

You might need to do hg commit -u hg_user_name

Step 6: Go to path_to_project/.hg/store/data/ and
cp new_file.extension.i error_file_name.extension.i

This will replace the current file

Step 7: Pull the files from your local HG client and you should be all set!

-

Tip: To avoid unnecessary merges, always "Pull Changes" - "Commit" and "Push" as close to each other as possible.

iContact 2.0 API Integration PHP Example

Mike Peters, December 25, 2009
One of the cool features of SPI's Autoresponder service, is its ability to seamlessly integrate with third party autoresponders.

Once you designate an autoresponder as a third-party one, SPI will continue applying your predefined rules for adding/removing members from that autoresponder, while notifying the third-party service (GetResponse, iContact, AWeber) automatically.

Over the last few days, we have battling with iContact's new API. iContact announced their 2.0 API back on February 2009, but it's still labeled as "beta" and there's little to no documentation available online.

The SPI-iContact integration was using version 1.0 and our intention was to continue using that version until 2.0 is officially released.

Unfortunately, with no prior warning, version 1.0 stopped working and is no longer honoring requests to add contacts / subscriptions records.

iContact's API external login page is down. The help section is down:



And the developer forums are ghost town USA.



Ok, rant over.

We really like the folks at iContact. They offer an easy to use and inexpensive entry-level autoresponder service. I just wish they would update the API documentation.

The purpose of this post is to help others who are going through integrating with the iContact 2.0 API by providing simple PHP examples for adding a contract, subscripting a contact to a list and unsubscribing.

We haven't been able to find anything like this online, so I'm pretty sure this will help others going through the integration process.

The code below is self-contained. No need to use any other libraries.


function IContactLogin($account_id, $key, $user, $pass, &$client_folder_id)
{
 
// Build iContact authentication
 
$headers = array(
 
'Accept: text/xml',
 
'Content-Type: text/xml',
 
'Api-Version: 2.0',
 
'Api-AppId: ' . $key,
 
'Api-Username: ' . $user,
 
'Api-Password: ' . $pass
 
);

 
// Connect to iContact to retrieve the client folder id
 
$ch=curl_init("https://app.icontact.com/icp/a/$account_id/c/");
 
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
 
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
 
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
 
$buf = curl_exec($ch);
 
curl_close($ch);

 
// Extract client folder id from response
 
$client_folder_id = "";
  if ((
$pos=strpos($buf,"<clientFolderId>"))!==false)
  {
   
$client_folder_id = substr($buf, strlen("<clientFolderId>")+$pos);
    if ((
$pos=strpos($client_folder_id,"<"))!==false)
    {
     
$client_folder_id = substr($client_folder_id, 0, $pos);
    }
  }

 
// If we have a non empty client_folder_id,
  // then everything worked well
 
$result = ($client_folder_id+0 > 0);

 
// Return result
 
return $result;
}


function
IContactSubscribe($account_id, $key, $user, $pass, $email, $list_id, &$result_str)
{
 
// Get client folder id
 
if (!IContactLogin($account_id, $key, $user, $pass, &$client_folder_id))
  {
   
$result_str = "Failed retrieving client_folder_id for '$user'";
    return
0;
  }

 
// Build iContact authentication
 
$headers = array(
 
'Accept: text/xml',
 
'Content-Type: text/xml',
 
'Api-Version: 2.0',
 
'Api-AppId: ' . $key,
 
'Api-Username: ' . $user,
 
'Api-Password: ' . $pass
 
);

 
// Find contact_id for the given 'email'
 
$ch=curl_init("https://app.icontact.com/icp/a/$account_id/c/$client_folder_id/contacts/?email=".URLEncode($email));
 
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
 
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
 
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
 
$buf = curl_exec($ch);
 
curl_close($ch);

 
// Extract contactId from response
 
$contact_id = "";
  if ((
$pos=strpos($buf,"<contactId>"))!==false)
  {
   
$contact_id = substr($buf, $pos+strlen("<contactId>"));
    if ((
$pos=strpos($contact_id,"<"))!==false)
    {
     
$contact_id = substr($contact_id,0,$pos);
    }
  }

 
// If we don't have a contactId, can't add subscription
 
if (empty($contact_id))
  { 
   
$result_str = "Failed finding a contact with the email address of '$email'";
    return
0;
  }

 
// Build subscription record
 
$data = '<?xml version="1.0" encoding="UTF-8"?>'."\r\n<subscriptions>\r\n";
 
$data.= "<subscription>\r\n";
 
$data.= "<contactId>$contact_id</contactId>\r\n";
 
$data.= "<listId>$list_id</listId>\r\n";
 
$data.= "<status>normal</status>\r\n";
 
$data.= "</subscription>\r\n</subscriptions>";

 
// Add subscription
 
$ch=curl_init("https://app.icontact.com/icp/a/$account_id/c/$client_folder_id/subscriptions/");
 
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
 
curl_setopt($ch, CURLOPT_POSTFIELDS, $data);
 
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
 
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
 
$buf = curl_exec($ch);
 
curl_close($ch);

 
// Extract subscriptionID from response
 
$subscription_id = "";
  if ((
$pos=strpos($buf,"<subscriptionId>"))!==false)
  {
   
$subscription_id = substr($buf, $pos+strlen("<subscriptionId>"));
    if ((
$pos=strpos($subscription_id,"<"))!==false)
    {
     
$subscription_id = substr($subscription_id,0,$pos);
    }
  }

 
// If we have a subscription id OR this subscription already existed, we're good
 
$result = !empty($subscription_id) || strpos($buf,"could not be updated")!==false;

 
// Set result string
 
$result_str = ($result ? "Updated subscription $subscription_id" : $buf);

 
// Return result
 
return $result;
}

function
IContactAddContact($account_id, $key, $user, $pass, $email, $firstname, $lastname,
&
$result_str)
{
 
// Get client folder id
 
if (!IContactLogin($account_id, $key, $user, $pass, &$client_folder_id))
  {
   
$result_str = "Failed retrieving client_folder_id for '$user'";
    return
0;
  }
 
 
// Build iContact authentication
 
$headers = array(
 
'Accept: text/xml',
 
'Content-Type: text/xml',
 
'Api-Version: 2.0',
 
'Api-AppId: ' . $key,
 
'Api-Username: ' . $user,
 
'Api-Password: ' . $pass
 
);
         
 
// Build contact record
 
$data = '<?xml version="1.0" encoding="UTF-8"?>'."\r\n<contacts>\r\n";
 
$data.= "<contact>\r\n";
 
$data.= "<email>$email</email>\r\n";
 
$data.= "<firstName>$firstname</firstName>\r\n";
 
$data.= "<lastName>$lastname</lastName>\r\n";
 
$data.= "<status>normal</status>\r\n";
 
$data.= "</contact>\r\n</contacts>";

 
// Add contact
 
$ch=curl_init("https://app.icontact.com/icp/a/$account_id/c/$client_folder_id/contacts/");
 
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
 
curl_setopt($ch, CURLOPT_POSTFIELDS, $data);
 
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
 
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
 
$buf = curl_exec($ch);
 
curl_close($ch);

 
// Extract contactId from response
 
$contact_id = "";
  if ((
$pos=strpos($buf,"<contactId>"))!==false)
  {
   
$contact_id = substr($buf, $pos+strlen("<contactId>"));
    if ((
$pos=strpos($contact_id,"<"))!==false)
    {
     
$contact_id = substr($contact_id,0,$pos);
    }
  }

 
// If we have a contact id, we're good
 
$result = !empty($contact_id);

 
// Set result string
 
$result_str = ($result ? "Added new contact $contact_id" : $buf);

 
// Return result
 
return $result;
}

function
IContactUnsubscribe($account_id, $key, $user, $pass, $email, $list_id, &$result_str)
{
 
// Get client folder id
 
if (!IContactLogin($account_id, $key, $user, $pass, &$client_folder_id))
  {
   
$result_str = "Failed retrieving client_folder_id for '$user'";
    return
0;
  }

 
// Build iContact authentication
 
$headers = array(
 
'Accept: text/xml',
 
'Content-Type: text/xml',
 
'Api-Version: 2.0',
 
'Api-AppId: ' . $key,
 
'Api-Username: ' . $user,
 
'Api-Password: ' . $pass
 
);

 
// Find contact_id for the given 'email'
 
$ch=curl_init("https://app.icontact.com/icp/a/$account_id/c/$client_folder_id/contacts/?email=".URLEncode($email));
 
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
 
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
 
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
 
$buf = curl_exec($ch);
 
curl_close($ch);

 
// Extract contactId from response
 
$contact_id = "";
  if ((
$pos=strpos($buf,"<contactId>"))!==false)
  {
   
$contact_id = substr($buf, $pos+strlen("<contactId>"));
    if ((
$pos=strpos($contact_id,"<"))!==false)
    {
     
$contact_id = substr($contact_id,0,$pos);
    }
  }

 
// If we don't have a contactId, can't add subscription
 
if (empty($contact_id))
  {
   
$result_str = "Failed finding a contact with the email address of '$email'";
    return
0;
  }

 
// Build subscription record
 
$data = '<?xml version="1.0" encoding="UTF-8"?>'."\r\n<subscriptions>\r\n";
 
$data.= "<subscription>\r\n";
 
$data.= "<contactId>$contact_id</contactId>\r\n";
 
$data.= "<listId>$list_id</listId>\r\n";
 
$data.= "<status>unsubscribed</status>\r\n";
 
$data.= "</subscription>\r\n</subscriptions>";

 
// Add subscription
 
$ch=curl_init("https://app.icontact.com/icp/a/$account_id/c/$client_folder_id/subscriptions/");
 
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
 
curl_setopt($ch, CURLOPT_POSTFIELDS, $data);
 
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
 
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
 
$buf = curl_exec($ch);
 
curl_close($ch);

 
// Extract subscriptionID from response
 
$subscription_id = "";
  if ((
$pos=strpos($buf,"<subscriptionId>"))!==false)
  {
   
$subscription_id = substr($buf, $pos+strlen("<subscriptionId>"));
    if ((
$pos=strpos($subscription_id,"<"))!==false)
    {
     
$subscription_id = substr($subscription_id,0,$pos);
    }
  }

 
// If we have a subscription id OR this subscription already unsubscribed, we're good
 
$result = !empty($subscription_id) || strpos($buf,"could not be updated")!==false;

 
// Set result string
 
$result_str = ($result ? "Updated subscription $subscription_id" : $buf);

 
// Return result
 
return $result;
}


View 3 Comment(s)

How to run SSH on a different port

Mike Peters, December 21, 2009
By default SSH runs on port 22. One of the easiest ways to make it more difficult for hackers to run dictionary attacks on your server, attempting to brute-force ssh-login, is to run ssh on a different port.

Open up /etc/ssh/sshd_config and add this line at the top:

Port 1234

Replace 1234 with the port number you'd like to run ssh on. If there is already a Port line in your sshd_config file, comment it with a #

Now, Restart sshd by issuing:

/etc/rc.d/init.d/sshd restart

Test ssh works properly by connecting locally (replace 1234 with the new port number):

ssh -l USERNAME localhost -p 1234


View 1 Comment(s)

MapReduce 101: Parallel Programming

Mike Peters, December 21, 2009
Two of the best video tutorials explaining the basics of MapReduce.

If you're used to MySQL, you can think of "Map" as several queries running "SELECT" where data is extracted from multiple tables. And you can think of "Reduce" as the "GROUP BY" operator where data is joined together to form a short result.





For more information about MapReduce, see Google's Introduction to Parallel Programming and MapReduce
« Previous Posts » Next Posts



About Us  |  Contact us  |  Privacy Policy  |  Terms & Conditions