Cassandra Metrics Graphite Reporter Agent

With the release of Cassandra 1.2, many new metrics were instrumented with Metrics with CASSANDRA-4009. However, getting those metrics into something like Graphite was still a polling process. Metrics does have Reporters that let Java Agents push metrics stored in the registry to various datastores (Graphite, Ganglia, etc.) Currently, this requires writing the agent code, compiling it and loading it into Cassandra. Soon there will be a way to just configure these reporters using metrics-reporters-config with CASSANDRA-4430. For now though, this simple agent will push metrics into Graphite while filtering out some noise.

Datastax has a blog post with a brief outline of how to enable the GraphiteReporter but it doesn’t go into much detail or release any code. This post augments it with the missing pieces.

As I’m not a Java engineer, I tried to do the bare minimum to get this working. Please submit pull requests to improve it!

1. Clone: git://github.com/disqus/graphite-reporter-agent.git

2. Modify com/disqus/metrics/reporter/GraphiteReporterAgent.java with the host and port of your carbon cache or relay (this should probably be properties or javaagent options but felt lazy due to CASSANDRA-4430).

These were obsoleted by Nikola Petrov by adding Maven support in Pull Request #1

3. Get metrics-graphite 2.0.3: wget http://search.maven.org/remotecontent?filepath=com/yammer/metrics/metrics-graphite/2.0.3/metrics-graphite-2.0.3.jar -O /usr/share/java/metrics-graphite-2.0.3.jar

4. Compile the agent: javac -cp /usr/share/cassandra/lib/metrics-core-2.0.3.jar:/usr/share/java/metrics-graphite-2.0.3.jar com/disqus/metrics/reporter/GraphiteReporterAgent.java

5. Make the jar: jar -cfM graphite-reporter.jar .

3. Run: mvn clean package dependency:copy-dependencies

4. Move the jars somewhere Cassandra can load it:
mv target/graphite-reporter-agent-1.0-SNAPSHOT.jar /usr/share/java/
mv target/dependency/metrics-graphite-2.0.3.jar /usr/share/java/

5. Tell Cassandra to start the agent and add metrics-graphite to the classpath.
In Ubuntu, edit /etc/default/cassandra:

JVM_EXTRA_OPTS="$JVM_EXTRA_OPTS -javaagent:/usr/share/java/graphite-reporter-agent-1.0-SNAPSHOT.jar"
EXTRA_CLASSPATH="/usr/share/java/metrics-graphite-2.0.3.jar"

6. Restart cassandra

7. ???

8. PROFIT!!!!

You should now start seeing stats under cassaandra.$(hostname -s) in your Graphite instance. The current configuration filters out metrics from Connection type, per-client Streaming metrics and the system, system_auth and system_traces column families.