There are a few commands under FreeBSD that depend on procfs (process file system).

FreeBSD doesn't mount it by default.

This tutorial describes how to mount /proc on FreeBSD and how to get FreeBSD to do it automatically when rebooting.

1. Mounting /proc

To mount /proc, run the following command:


Applications and commands depending on procfs will now work correctly.

2. Mount /proc automatically when rebooting

To get /proc to be mounted automatically when rebooting, simply add this line in /etc/fstab:


There you go, /proc will now be mounted automatically at boot time.

DDos (Denial Of Service) Attacks are distributed hits to your server, coming from multiple sources at the same time.

Unlike an attack from a single location, where the source IP address can be blocked on the firewall level, denial of service attacks are very difficult to stop.

DDoS attacks recently silenced the MPAA, Aiplex and took down Malaysian Government critics. DDoS attacks are back and they're bigger than ever.

New technology makes it too easy to launch low orbit ion cannon attacks and bring sites down to their knees.

Here are a few simple things you can do to protect your servers against a DDoS attack:

Have a contingency plan

Much like recovering from a failed harddrive, you need to plan ahead.

Avoid single points of failure and make sure you have at least two separate machines, running your web-servers and databases. Multi-homed hosting can really help here.

When the attack comes in, you'll be able to switch the ip address until the storm calms down.

Disable Ping-flood attacks

Add this to your /etc/sysctl.conf:

And run this on the shell to apply the changes immediately:

Use obscure ports for anything other than HTTP

Change your MySQL (/etc/my.cnf port=1234), FastCGI and all other daemons to run on unique port numbers.

Install all the latest security patches

Duh!

Use private ip addresses for inter-server communications

If you have more than one machine on the same LAN, use the LAN private ip addresses to communicate between the machines.

This is particularly helpful when your data-center decides to null-route the public-facing ip address of your database server (why is it open in the first place?) and you want to allow the web server to continue communicating with the database uninterrupted.

Using private LAN ip addresses is more efficient and ensures no interruptions in case your public-facing ip address gets null-routed.

Use a Firewall

Hardware firewalls rock, but the good ones can get very expensive to acquire and manage.

These two software firewalls are great for brute force detection and advanced policies that can detect anomalies common to DDoS attacks: APF and BFD. Both are from R-FX Networks

Software Projects, a full-service Internet Marketing & Web Development agency, providing services to more than 3,000 businesses in 14 countries, is looking for PHP superstars!

If you're passionate about technology, love challenges and looking to work with the best in the industry, send us your resume.

== Job description

We're looking for the absolute best PHP developers, to join our team of professionals, working on a platform of 81 Internet Marketing services (http://www.softwareprojects.com/reso...form-1192.html)

Full time, reliable and passionate PHP developers, capable of working 8 hours a day independently without constant supervision.

The successful candidates will be responsible to spearhead development of new Internet Marketing services, collaborating with other Software Projects engineers and working directly with our clients.

== Why work for us?

Software Projects has been recognized for its continued growth, entrepreneurship, product excellence, passionate culture and superb working environment:

* Named "Largest Full service Internet Marketing and Web Development agency" by Inc. Magazine
* Ranked in Inc. Magazine's list of 5000 fastest growing private companies in America
* One of only 60 companies to earn Google Qualified company status
* Active contributer to the MySQL and Cassandra communities

We love our people and compensate them generously for their contributions to our success.

== Requirements:

* Highly motivated, ability to work alone
* Excellent communication skills
* Fast Learner
* Passionate and Persistent
* Multi-tasker, Team player

== Skills:

* Excellent problem solving skills
* Minimum 4 years PHP experience
* Minimum 3 years MySQL experience
* Fluent with FreeBSD/Linux
* CVS, HG, SVN or similar version-control experience

== Nice to have:

* Familiarity with Cassandra
* Java experience
* Understanding of Agile development principles
* Experience with Internet marketing, Shopping carts, PPC, Email Marketing, SEO

The Mercurial version control system offers a powerful mechanism to let you perform automated actions in response to events that occur in a repository. In some cases, you can even control Mercurial's response to those events. No comments

With this post, I'll walk you through how I developed a Mercurial hook that automatically pushes files from one central location, to a remote server.

Why do we need this auto-push-remotely hook?

Here at SoftwareProjects, as part of our multi-homed hosting solution, we host servers in multiple data-centers, where multiple servers run the same websites. Think geo-load balancing.

Replicating a website across several locations has several benefits:

Throughput - It will be able to handle more traffic, since the load is distributed across several machines.

Better reliability - if one machine is down, the others can still serve user requests.

Easier maintenance - Upgrades can be implemented one machine at a time, with zero impact to the end users.

More moving parts

Of course, running multiple servers does add some complexity as the website data needs to be synchronized on all the servers.

Often Rsync is used for this purpose and setup to run every X minutes via cron.

Rysnc has some disadvantages. It has to recursively scan all the directories and search for files that have changed before sending them to another server. This can be a bit time consuming and also requires more CPU. It means that the content may be out of sync by roughly 5-10 minutes. Having the content live on some servers and not on others can be an issue as different users will have a different experience.

Another solution instead of using Rsync is to automatically synchronize the file or files to the other servers as soon as it has changed. This can easily be done when using version control to send the content to a central server. Then the central server can send the content to all of the other servers.

The Mercurial Hook

Automatically synchronizing files to other servers can be setup with Mercurial. This just requires setting up a new hook on the changegroup event.

This hook can be setup in mercurial's config file with the following two lines of code:

[hooks]
changegroup.hg_sync = /usr/home/code/hg_sync.sh

We also need to create the hg_sync.sh script that is executed during the changroup event. Below is an example script that will automatically push the changes to all other servers.

#!/usr/local/bin/bash

# Determine the repository that recieved the push
p=`/bin/pwd`

# See if it is a repository that we want to synchronize across servers
if [ "$p" == "/usr/home/admin/htdocs" ]; then
# Get each server to syncrhonize with
while read current_line
do
# Push the content to the other server
/usr/local/bin/hg push -f $current_line
done < /usr/home/code/hg_sync_servers.txt
fi

exit 0;

This script will first determine if the changegroup event is occurring on the correct repository, based on the path. Then it will push to each server that is listed in the file /usr/home/code/hg_sync_servers.txt. This text file can look something like the following:

ssh://user@code1.domain.com//usr/home/admin/htdocs
ssh://user@code2.domain.com//usr/home/admin/htdocs
ssh://user@code3.domain.com//usr/home/admin/htdocs

With the above script setup on the changegroup event the output from an "hg push" will contain the following line:

running hook changegroup.hg_sync: /usr/home/code/hg_sync.sh

How to install

The last step is to add a hook to all of the servers listed in the hg_sync_servers.txt file. These servers need to be setup to automatically run "hg update" after they are pushed to. This is simply a matter of adding the following line to the mercurial config file on these servers:

changegroup = /usr/local/bin/hg update >&2

This setup will allow all files to be automatically synchronized to all servers just after the central repository receives the files. This allows for files to be seen on all servers faster than what can be accomplished with a tool such as rsync.

View 3 Comment(s)

I recently worked on a project that involved integrating with MSN Messenger, creating my very own "IM Robot".

What can an IM Robot do?

Keep track of contacts online/offline status and when they were last seen.

Broadcast a message to all contacts.

Automatically answer common questions.

Notify contacts about new events. (See http://notify.me)

Keep track of code snippets

Which IM Library to Use

Setting up a IM robot can be a bit of work especially if starting from scratch. There are a lot of libraries out there that can be used to help simplify the process. The trouble is a lot of libraries are not kept up to date and fail to work as IM protocols change.

We did some digging and found a library that would provide a good foundation to build a IM Robot that can do just about anything. We say implementations in PHP, C, Java, Perl and Python. After some testing we concluded the Java MSN Library would be a very good fit.

How To Use It

Using this library with java is pretty straight forward. First the library must be added to the classpath. The step to take to complete this will depend on how your developing your java code. The most basic method to add a library to your classpath is to do this at run time with a command such as:

java -classpath MyLibrary.jar MyPackage.MyClass

A better approach would be to setup the classpath in a manifest file. The manifest file is then placed inside the jar file and tells the executable jar where to look for the libraries. This manifest file should look something like this (note the class-path on line 5):

Manifest-Version: 1.0
Ant-Version: Apache Ant 1.7.1
Created-By: 14.3-b01 (Sun Microsystems Inc.)
Main-Class: imstatus.Main
Class-Path: lib/jml-1.0b4-full.jar lib/httpcore-4.0.1.jar lib/mysql-co
nnector-java-5.1.6-bin.jar
X-COMMENT: Main-Class will be added automatically by build

This is setup so that the 3 required libraries are in the lib folder. These 3 libraries are needed for setting up an IM robot and can be downloaded from the following locations:

jml-1.0b4-full.jar

httpcore-4.0.1.jar

mysql-connector-java-5.1.6-bin.jar

Now that the libraries are setup we can begin to use them.

Developing the IM Robot Code

There are a few examples of using the Java MSN Library on the main page. However, they are a tad confusing as it creates a new BasicMessenger class. This is confusing as the library already has a BasicMessenger class which is abstract. The library also has a SimpleMessenger class which is a subclass of BasicMessenger. This class appears to be the correct implementation that we would want to use to create a new IM Robot.

However, the original authors made the constructor protected so that we cannot instantiate the class outside of the original package. Since we want a simple way to create an IM Robot I have modified the original source code to have a public constructor for the SimpleMessenger class.

With this new package we can very easily create a new IM Robot with the following two lines of code (make sure to replace yourLogin and yourPassword):

SimpleMessenger messenger = new SimpleMessenger(Email.parseStr("yourLogin@msn.com"), "yourPassword");
messenger.login();

With that code in our main funtion we can run it and test that the Robot automatically logs into Windows Live Messenger.

Of course that code just logs the Robot into Windows Live Messenger. The next step is to setup the robot to do something interesting. This is one feature that is very nice about the Java Msn Library as it as listeners for many different events. For example we can detect when the robot has finished logging in with the following:

messenger.addListener(new MsnAdapter() {
// Setup the login completed event

@Override
public void loginCompleted(MsnMessenger messenger) {
MsnOwner owner = messenger.getOwner();
owner.setInitStatus(MsnUserStatus.ONLINE);
owner.setStatus(MsnUserStatus.ONLINE);

// Setup the contact list event
messenger.addContactListListener(new ContactListAdapter());
}
});

Then we can take this a step further and detect when a status changes for one of the robots contacts with something like the following:

messenger.addListener(new MsnAdapter() {
// Setup the login completed event

@Override
public void loginCompleted(MsnMessenger messenger) {
MsnOwner owner = messenger.getOwner();
owner.setInitStatus(MsnUserStatus.ONLINE);
owner.setStatus(MsnUserStatus.ONLINE);

// Setup the contact list event
messenger.addContactListListener(new ContactListAdapter());
}
});

The above code will detect when the robot has finished logging in and then setup a new listener to detect when a contacts status has changed. The new listener invokes the ContactListAdapter class when a status has changed. This contactListAdapter class is setup as followes:

class ContactListAdapter extends MsnContactListAdapter {
@Override
public void contactStatusChanged(MsnMessenger messenger, MsnContact contact) {
System.out.println(contact.getEmail()+" is currently "+contact.getStatus());
// Can add code here to store the status in a database
}
}

We can still take this a step further and setup the robot to handle automatically adding contacts when a contact requests it. This logic can be added to the ContactListAdapter class with something like the following:

class ContactListAdapter extends MsnContactListAdapter {
@Override
public void contactListSyncCompleted(MsnMessenger messenger) {
MsnContact[] contacts = messenger.getContactList().getContactsInList(MsnList.AL);
for (int i = 0; i < contacts.length; i++) {
contactStatusChanged(messenger,contacts[i]);
}
}

@Override
public void contactAddedMe(MsnMessenger messenger, MsnContact contact) {
messenger.addFriend(contact.getEmail(), contact.getDisplayName());
}

@Override
public void contactAddedMe(MsnMessenger messenger, MsnContactPending[] pending){
for(int i=0; i messenger.addFriend(pending[i].getEmail(), pending[i].getDisplayName());
}
}

@Override
public void contactStatusChanged(MsnMessenger messenger, MsnContact contact) {
System.out.println(contact.getEmail()+" is currently "+contact.getStatus());
// Can add code here to store the status in a database
}
}

There you have it! Put all of the above code together and you will have a robot that knows how to automatically add contacts and keep track of when a contact's status changes.

View 6 Comment(s)

Keynotes from a great presentation titled Cassandra Troubleshooting: out of the shadows, presented by Benjamin Black at the Cassandra Summit in San Francisco two weeks ago.

The slides are here

-

Is your Ring unbalanced?

That's because when you add one node at a time using RandomPartitioner, the new nodes takes over half of the most balanced node:

Note that as long as you're doubling-up the size of your cluster, everything will be balanced. But when you're growing one node at
a time, the cluster will be unbalanced.

To fix: Manually assign tokens.

How do you know which tokens to assign? Use this Python script:

Writes are slow

Make sure your commitlog is on a separate drive.

Writes are fast. Reads keep getting slower

Step 1:

Look at iostat -x to see if you're maxing out utilization

If you are, get more nodes

Step 2:

Look at nodetool tpstats

Focus on the middle column (pending) and specifically:
* Row-Read-STage
* Message-Deserializer-pool

If these two are high (4096 is the max), it means your client is sending too many reads to this node.

Update your client or get more nodes to distribute reads.

Step 3:

Adjust memtable settings

When does a memtable get flushed to disk?

Size: When it gets to a certain size
Time: If it hasn't been flushed in x seconds
Operations: When certain operations occur

If you're flushing memtables too often, you're triggering follow-up effects (compactions, sstable merges) that is consuming a lot of bandwidth.

You want less frequent memtable flush, which leads to less frequent compaction and less disk bandwidth demand.

If memtable is not compatible with your data needs, you begin consuming huge amounts of your bandwidth on compactions.

once a minute = bad

Step 4:

Use SSDs for the disk drives. Makes no difference on the commit log drive.

I inserted a bunch of data, now my nodes are flapping

Flapping = nodes are marked down/up

Step 1:

Monitor swap (vmstat on linux, swapinfo on freebsd)

mmap takes 2gb per segment.

Swapping can delay gossip long enough to cause a node
to be marked down.

Swapping is bad.

To fix: Change DiskAccessMode in the Cassandra config file, to mmap_index_only

We avoid risking driving ourselves into swap by the JVM
allocating large chunks of mmap blocks.

Step 2:

Tell the O/S you want to avoid swapping if possible.
On FreeBSD: add this line to /etc/sysctl.conf

On Linux, echo 0 into /proc/sys/vm/swappiness

View 1 Comment(s)