How to dump memcache key/value pairs fast
Posted by dctrwatson in Memcached on December 10, 2010
Recently we migrated our memcached cluster to a new larger one. This needed to be done mostly for reliability and speed, but it’s also nice being able to have access to new stats like ‘reclaimed’ in 1.4.5.
We decided against migrating data from cluster to cluster because there are long-expire keys we no longer care about. Instead, we just ran some scripts to fill the new cluster with new data. After the scripts finished and the new cluster ran in production for a couple days, it only contained ~70% the data the old cluster possessed. We were curious what that old data was, so we decided to dump it and analyze.
Read the rest of this entry »
Apache Cassandra FreeBSD rc.d script
Posted by dctrwatson in Cassandra, FreeBSD on November 19, 2010
We are a FreeBSD shop, so naturally we run our Apache Cassandra cluster on it. Unfortunately, there isn’t an rc.d script for it. Using cassandra/bin/cassandra -p /var/run/cassandra.pid has been working fine without any grief. However, it would be nice to utilize the rc system built into FreeBSD. I took the Tomcat 6.0 rc script that it is in the port system (www/tomcat6) and adapted it for cassandra.
Nginx modules for a memcached page cache cluster
Posted by dctrwatson in Memcached, Nginx on October 29, 2010
Nginx already has a neat module included with it to proxy requests to a memcached server (memcached_pass). Combine that with the upstream round robin load balancer, you have the beginnings of a memcached page cache cluster for your nginx. However, calling each server sequentially can become quite expensive if the key resides in one of the last servers.
Introducing myvault
Posted by dctrwatson in Shell Script on October 13, 2010
I wanted someway from a Unix shell to edit an encrypted file that would store some semi-sensitive information. Google failed to bring me any acceptable candidates. So I decided to dive into the world of shell scripting and came up with something I’m going to call myvault.
It uses OpenSSL to generate a random 32 character password, which it then uses to symmetrically encrypt the file. The password is then asymmetrically encrypted using an RSA key. A new random 32 character password is generated every time it is run.
By default, it uses your SSH key to open a file called “.myvault” in your $HOME directory using Vim. You can pass it arguments to use a different encrypted file and/or private/public key(s). You can even encrypt/decrypt any file which it then spits to STDOUT for your use.
Now I’m no shell scripting genius nor a complete beginner. So I may have done things inefficiently or down right wrong. Please leave feedback for improvements or changes.
Check it out here: http://github.com/dctrwatson/myvault
Reset file’s modification time to their birth time
Posted by dctrwatson in FreeBSD on October 11, 2010
NOTE:This only works for FreeBSD.
This will change the modification time (ls -l) of the current directory and all files and directories under it back to their “birth time” (creation time). If you want to limit this use man find to figure out the proper arguments.
find . -exec sh -c 'touch -t $(date -r $(stat -f %B {}) "+%Y%m%d%H%M.%S") {}' \;
Just finished reading a few wh…
Posted by dctrwatson in Uncategorized on October 7, 2010
Just finished reading a few whitepapers on Java Memory Ergonomics and Garbage Collectors… so many jvm switches make much more sense now =D
Was working on a FreeBSD box t…
Posted by dctrwatson in Uncategorized on October 4, 2010
Was working on a FreeBSD box this morning and discovered an amazing command that is better than using find|grep : “whereis” is much prettier
I’m starting to get annoyed wi…
Posted by dctrwatson in Django on September 30, 2010
I’m starting to get annoyed with finding bugs in Django that are caused by following PostgreSQL idioms that MySQL is not compatible with! =(
Installing PostgreSQL 9.0 on Ubuntu 10.04
Posted by dctrwatson in Postgres, Ubuntu on September 29, 2010
PostgreSQL 9.0 final was released on 2010/09/20 which I’ve been waiting for to begin playing with it and evaluating it. The mainline Ubuntu Lucid apt repositories only have 8.4. According to the package maintainer, Martin Pitt, the postgresql-9.0 package won’t be added until Ubuntu Natty. Thankfully, he setup a backports repo for Ubuntu that we can pull from.
Read the rest of this entry »
Even though I know it is reall…
Posted by dctrwatson in Uncategorized on September 14, 2010
Even though I know it is really “mongo-d” I still read it as “mon-god” whenever I read articles about MongoDB like these: http://j.mp/94zWNq