None
Mind the gap.
Metrics provides a powerful toolkit of ways to measure the behavior of critical components in your production environment.
With modules for common libraries like Jetty, Logback, Log4j, Apache HttpClient, Ehcache, JDBI, Jersey and reporting backends like Ganglia and Graphite, Metrics provides you with full-stack visibility.
Getting Started will guide you through the process of adding Metrics to an existing application. We’ll go through the various measuring instruments that Metrics provides, how to use them, and when they’ll come in handy.
You need the metrics-core library as a dependency:
<dependencies>
<dependency>
<groupId>io.dropwizard.metrics</groupId>
<artifactId>metrics-core</artifactId>
<version>${metrics.version}</version>
</dependency>
</dependencies>
Note
Make sure you have a metrics.version property declared in your POM with the current version,
which is 3.1.0.
Now it’s time to add some metrics to your application!
A meter measures the rate of events over time (e.g., “requests per second”). In addition to the mean rate, meters also track 1-, 5-, and 15-minute moving averages.
private final Meter requests = metrics.meter("requests");
public void handleRequest(Request request, Response response) {
requests.mark();
// etc
}
This meter will measure the rate of requests in requests per second.
A Console Reporter is exactly what it sounds like - report to the console. This reporter will print every second.
ConsoleReporter reporter = ConsoleReporter.forRegistry(metrics)
.convertRatesTo(TimeUnit.SECONDS)
.convertDurationsTo(TimeUnit.MILLISECONDS)
.build();
reporter.start(1, TimeUnit.SECONDS);
So the complete Getting Started is
package sample;
import com.codahale.metrics.*;
import java.util.concurrent.TimeUnit;
public class GetStarted {
static final MetricRegistry metrics = new MetricRegistry();
public static void main(String args[]) {
startReport();
Meter requests = metrics.meter("requests");
requests.mark();
wait5Seconds();
}
static void startReport() {
ConsoleReporter reporter = ConsoleReporter.forRegistry(metrics)
.convertRatesTo(TimeUnit.SECONDS)
.convertDurationsTo(TimeUnit.MILLISECONDS)
.build();
reporter.start(1, TimeUnit.SECONDS);
}
static void wait5Seconds() {
try {
Thread.sleep(5*1000);
}
catch(InterruptedException e) {}
}
}
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>somegroup</groupId>
<artifactId>sample</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>Example project for Metrics</name>
<dependencies>
<dependency>
<groupId>io.dropwizard.metrics</groupId>
<artifactId>metrics-core</artifactId>
<version>${metrics.version}</version>
</dependency>
</dependencies>
</project>
Note
Make sure you have a metrics.version property declared in your POM with the current version,
which is 3.1.0.
To run
mvn package exec:java -Dexec.mainClass=sample.First
The centerpiece of Metrics is the MetricRegistry class, which is the container for all your
application’s metrics. Go ahead and create a new one:
final MetricRegistry metrics = new MetricRegistry();
You’ll probably want to integrate this into your application’s lifecycle (maybe using your
dependency injection framework), but static field is fine.
A gauge is an instantaneous measurement of a value. For example, we may want to measure the number of pending jobs in a queue:
public class QueueManager {
private final Queue queue;
public QueueManager(MetricRegistry metrics, String name) {
this.queue = new Queue();
metrics.register(MetricRegistry.name(QueueManager.class, name, "size"),
new Gauge<Integer>() {
@Override
public Integer getValue() {
return queue.size();
}
});
}
}
When this gauge is measured, it will return the number of jobs in the queue.
Every metric in a registry has a unique name, which is just a dotted-name string like
"things.count" or "com.example.Thing.latency". MetricRegistry has a static helper method
for constructing these names:
MetricRegistry.name(QueueManager.class, "jobs", "size")
This will return a string with something like "com.example.QueueManager.jobs.size".
For most queue and queue-like structures, you won’t want to simply return queue.size(). Most of
java.util and java.util.concurrent have implementations of #size() which are O(n),
which means your gauge will be slow (potentially while holding a lock).
A counter is just a gauge for an AtomicLong instance. You can increment or decrement its value.
For example, we may want a more efficient way of measuring the pending job in a queue:
private final Counter pendingJobs = metrics.counter(name(QueueManager.class, "pending-jobs"));
public void addJob(Job job) {
pendingJobs.inc();
queue.offer(job);
}
public Job takeJob() {
pendingJobs.dec();
return queue.take();
}
Every time this counter is measured, it will return the number of jobs in the queue.
As you can see, the API for counters is slightly different: #counter(String) instead of
#register(String, Metric). While you can use register and create your own Counter
instance, #counter(String) does all the work for you, and allows you to reuse metrics with the
same name.
Also, we’ve statically imported MetricRegistry’s name method in this scope to reduce
clutter.
A histogram measures the statistical distribution of values in a stream of data. In addition to minimum, maximum, mean, etc., it also measures median, 75th, 90th, 95th, 98th, 99th, and 99.9th percentiles.
private final Histogram responseSizes = metrics.histogram(name(RequestHandler.class, "response-sizes"));
public void handleRequest(Request request, Response response) {
// etc
responseSizes.update(response.getContent().length);
}
This histogram will measure the size of responses in bytes.
A timer measures both the rate that a particular piece of code is called and the distribution of its duration.
private final Timer responses = metrics.timer(name(RequestHandler.class, "responses"));
public String handleRequest(Request request, Response response) {
final Timer.Context context = responses.time();
try {
// etc;
return "OK";
} finally {
context.stop();
}
}
This timer will measure the amount of time it takes to process each request in nanoseconds and provide a rate of requests in requests per second.
Metrics also has the ability to centralize your service’s health checks with the
metrics-healthchecks module.
First, create a new HealthCheckRegistry instance:
final HealthCheckRegistry healthChecks = new HealthCheckRegistry();
Second, implement a HealthCheck subclass:
public class DatabaseHealthCheck extends HealthCheck {
private final Database database;
public DatabaseHealthCheck(Database database) {
this.database = database;
}
@Override
public HealthCheck.Result check() throws Exception {
if (database.isConnected()) {
return HealthCheck.Result.healthy();
} else {
return HealthCheck.Result.unhealthy("Cannot connect to " + database.getUrl());
}
}
}
Then register an instance of it with Metrics:
healthChecks.register("postgres", new DatabaseHealthCheck(database));
To run all of the registered health checks:
final Map<String, HealthCheck.Result> results = healthChecks.runHealthChecks();
for (Entry<String, HealthCheck.Result> entry : results.entrySet()) {
if (entry.getValue().isHealthy()) {
System.out.println(entry.getKey() + " is healthy");
} else {
System.err.println(entry.getKey() + " is UNHEALTHY: " + entry.getValue().getMessage());
final Throwable e = entry.getValue().getError();
if (e != null) {
e.printStackTrace();
}
}
}
Metrics comes with a pre-built health check: ThreadDeadlockHealthCheck, which uses Java’s
built-in thread deadlock detection to determine if any threads are deadlocked.
To report metrics via JMX:
final JmxReporter reporter = JmxReporter.forRegistry(registry).build();
reporter.start();
Once the reporter is started, all of the metrics in the registry will become visible via JConsole or VisualVM (if you install the MBeans plugin):
Tip
If you double-click any of the metric properties, VisualVM will start graphing the data for that property. Sweet, eh?
Metrics also ships with a servlet (AdminServlet) which will serve a JSON representation of all
registered metrics. It will also run health checks, print out a thread dump, and provide a simple
“ping” response for load-balancers. (It also has single servlets–MetricsServlet,
HealthCheckServlet, ThreadDumpServlet, and PingServlet–which do these individual
tasks.)
To use this servlet, include the metrics-servlets module as a dependency:
<dependency>
<groupId>io.dropwizard.metrics</groupId>
<artifactId>metrics-servlets</artifactId>
<version>${metrics.version}</version>
</dependency>
Note
Make sure you have a metrics.version property declared in your POM with the current version,
which is 3.1.0.
From there on, you can map the servlet to whatever path you see fit.
In addition to JMX and HTTP, Metrics also has reporters for the following outputs:
STDOUT, using ConsoleReporter from metrics-coreCSV files, using CsvReporter from metrics-coremetrics-coremetrics-gangliametrics-graphiteThis goal of this document is to provide you with all the information required to effectively use the Metrics library in your application. If you’re new to Metrics, you should read the Getting Started guide first.
The central library for Metrics is metrics-core, which provides some basic functionality:
The starting point for Metrics is the MetricRegistry class, which is a collection of all the
metrics for your application (or a subset of your application). If your application is running
alongside other applications in a single JVM instance (e.g., multiple WARs deployed to an
application server), you should use per-application MetricRegistry instances with different
names.
Each metric has a unique name, which is a simple dotted name, like com.example.Queue.size.
This flexibility allows you to encode a wide variety of context directly into a metric’s name. If
you have two instances of com.example.Queue, you can give them more specific:
com.example.Queue.requests.size vs. com.example.Queue.responses.size, for example.
MetricRegistry has a set of static helper methods for easily creating names:
MetricRegistry.name(Queue.class, "requests", "size")
MetricRegistry.name(Queue.class, "responses", "size")
These methods will also elide any null values, allowing for easy optional scopes.
A gauge is the simplest metric type. It just returns a value. If, for example, your application
has a value which is maintained by a third-party library, you can easily expose it by registering a
Gauge instance which returns that value:
registry.register(name(SessionStore.class, "cache-evictions"), new Gauge<Integer>() {
@Override
public Integer getValue() {
return cache.getEvictionsCount();
}
});
This will create a new gauge named com.example.proj.auth.SessionStore.cache-evictions which will
return the number of evictions from the cache.
Given that many third-party library often expose metrics only via JMX, Metrics provides the
JmxAttributeGauge class, which takes the object name of a JMX MBean and the name of an attribute
and produces a gauge implementation which returns the value of that attribute:
registry.register(name(SessionStore.class, "cache-evictions"),
new JmxAttributeGauge("net.sf.ehcache:type=Cache,scope=sessions,name=eviction-count", "Value"));
A ratio gauge is a simple way to create a gauge which is the ratio between two numbers:
public class CacheHitRatio extends RatioGauge {
private final Meter hits;
private final Timer calls;
public CacheHitRatio(Meter hits, Timer calls) {
this.hits = hits;
this.calls = calls;
}
@Override
public Ratio getRatio() {
return Ratio.of(hits.getOneMinuteRate(),
calls.getOneMinuteRate());
}
}
This gauge returns the ratio of cache hits to misses using a meter and a timer.
A cached gauge allows for a more efficient reporting of values which are expensive to calculate:
registry.register(name(Cache.class, cache.getName(), "size"),
new CachedGauge<Long>(10, TimeUnit.MINUTES) {
@Override
protected Long loadValue() {
// assume this does something which takes a long time
return cache.getSize();
}
});
A derivative gauge allows you to derive values from other gauges’ values:
public class CacheSizeGauge extends DerivativeGauge<CacheStats, Long> {
public CacheSizeGauge(Gauge<CacheStats> statsGauge) {
super(statsGauge);
}
@Override
protected Long transform(CacheStats stats) {
return stats.getSize();
}
}
A counter is a simple incrementing and decrementing 64-bit integer:
final Counter evictions = registry.counter(name(SessionStore.class, "cache-evictions"));
evictions.inc();
evictions.inc(3);
evictions.dec();
evictions.dec(2);
All Counter metrics start out at 0.
A Histogram measures the distribution of values in a stream of data: e.g., the number of results
returned by a search:
final Histogram resultCounts = registry.histogram(name(ProductDAO.class, "result-counts");
resultCounts.update(results.size());
Histogram metrics allow you to measure not just easy things like the min, mean, max, and
standard deviation of values, but also quantiles like the median or 95th percentile.
Traditionally, the way the median (or any other quantile) is calculated is to take the entire data set, sort it, and take the value in the middle (or 1% from the end, for the 99th percentile). This works for small data sets, or batch processing systems, but not for high-throughput, low-latency services.
The solution for this is to sample the data as it goes through. By maintaining a small, manageable reservoir which is statistically representative of the data stream as a whole, we can quickly and easily calculate quantiles which are valid approximations of the actual quantiles. This technique is called reservoir sampling.
Metrics provides a number of different Reservoir implementations, each of which is useful.
A histogram with a uniform reservoir produces quantiles which are valid for the entirely of the histogram’s lifetime. It will return a median value, for example, which is the median of all the values the histogram has ever been updated with. It does this by using an algorithm called Vitter’s R), which randomly selects values for the reservoir with linearly-decreasing probability.
Use a uniform histogram when you’re interested in long-term measurements. Don’t use one where you’d want to know if the distribution of the underlying data stream has changed recently.
A histogram with an exponentially decaying reservoir produces quantiles which are representative of (roughly) the last five minutes of data. It does so by using a forward-decaying priority reservoir with an exponential weighting towards newer data. Unlike the uniform reservoir, an exponentially decaying reservoir represents recent data, allowing you to know very quickly if the distribution of the data has changed. Timers use histograms with exponentially decaying reservoirs by default.
A histogram with a sliding window reservoir produces quantiles which are representative of the past
N measurements.
A histogram with a sliding time window reservoir produces quantiles which are strictly
representative of the past N seconds (or other time period).
Warning
While SlidingTimeWindowReservoir is easier to understand than
ExponentiallyDecayingReservoir, it is not bounded in size, so using it to sample a
high-frequency process can require a significant amount of memory. Because it records every
measurement, it’s also the slowest reservoir type.
A meter measures the rate at which a set of events occur:
final Meter getRequests = registry.meter(name(WebProxy.class, "get-requests", "requests"));
getRequests.mark();
getRequests.mark(requests.size());
Meters measure the rate of the events in a few different ways. The mean rate is the average rate of events. It’s generally useful for trivia, but as it represents the total rate for your application’s entire lifetime (e.g., the total number of requests handled, divided by the number of seconds the process has been running), it doesn’t offer a sense of recency. Luckily, meters also record three different exponentially-weighted moving average rates: the 1-, 5-, and 15-minute moving averages.
Hint
Just like the Unix load averages visible in uptime or top.
A timer is basically a histogram of the duration of a type of event and a meter of the rate of its occurrence.
final Timer timer = registry.timer(name(WebProxy.class, "get-requests"));
final Timer.Context context = timer.time();
try {
// handle request
} finally {
context.stop();
}
Note
Elapsed times for it events are measured internally in nanoseconds, using Java’s high-precision
System.nanoTime() method. Its precision and accuracy vary depending on operating system and
hardware.
Metrics can also be grouped together into reusable metric sets using the MetricSet interface.
This allows library authors to provide a single entry point for the instrumentation of a wide
variety of functionality.
Reporters are the way that your application exports all the measurements being made by its metrics.
metrics-core comes with four ways of exporting your metrics:
JMX, console,
SLF4J, and CSV.
With JmxReporter, you can expose your metrics as JMX MBeans. To explore this you can use
VisualVM (which ships with most JDKs as jvisualvm) with the VisualVM-MBeans plugins installed
or JConsole (which ships with most JDKs as jconsole):
Tip
If you double-click any of the metric properties, VisualVM will start graphing the data for that property. Sweet, eh?
Warning
We don’t recommend that you try to gather metrics from your production environment. JMX’s RPC API is fragile and bonkers. For development purposes and browsing, though, it can be very useful.
To report metrics via JMX:
final JmxReporter reporter = JmxReporter.forRegistry(registry).build();
reporter.start();
For simple benchmarks, Metrics comes with ConsoleReporter, which periodically reports all
registered metrics to the console:
final ConsoleReporter reporter = ConsoleReporter.forRegistry(registry)
.convertRatesTo(TimeUnit.SECONDS)
.convertDurationsTo(TimeUnit.MILLISECONDS)
.build();
reporter.start(1, TimeUnit.MINUTES);
For more complex benchmarks, Metrics comes with CsvReporter, which periodically appends to a set
of .csv files in a given directory:
final CsvReporter reporter = CsvReporter.forRegistry(registry)
.formatFor(Locale.US)
.convertRatesTo(TimeUnit.SECONDS)
.convertDurationsTo(TimeUnit.MILLISECONDS)
.build(new File("~/projects/data/"));
reporter.start(1, TimeUnit.SECONDS);
For each metric registered, a .csv file will be created, and every second its state will be
written to it as a new row.
It’s also possible to log metrics to an SLF4J logger:
final Slf4jReporter reporter = Slf4jReporter.forRegistry(registry)
.outputTo(LoggerFactory.getLogger("com.example.metrics"))
.convertRatesTo(TimeUnit.SECONDS)
.convertDurationsTo(TimeUnit.MILLISECONDS)
.build();
reporter.start(1, TimeUnit.MINUTES);
Metrics has other reporter implementations, too:
Metrics also provides you with a consistent, unified way of performing application health checks. A health check is basically a small self-test which your application performs to verify that a specific component or responsibility is performing correctly.
To create a health check, extend the HealthCheck class:
public class DatabaseHealthCheck extends HealthCheck {
private final Database database;
public DatabaseHealthCheck(Database database) {
this.database = database;
}
@Override
protected Result check() throws Exception {
if (database.ping()) {
return Result.healthy();
}
return Result.unhealthy("Can't ping database");
}
}
In this example, we’ve created a health check for a Database class on which our application
depends. Our fictitious Database class has a #ping() method, which executes a safe test
query (e.g., SELECT 1). #ping() returns true if the query returns the expected result,
returns false if it returns something else, and throws an exception if things have gone
seriously wrong.
Our DatabaseHealthCheck, then, takes a Database instance and in its #check() method,
attempts to ping the database. If it can, it returns a healthy result. If it can’t, it returns
an unhealthy result.
Note
Exceptions thrown inside a health check’s #check() method are automatically caught and
turned into unhealthy results with the full stack trace.
To register a health check, either use a HealthCheckRegistry instance:
registry.register("database", new DatabaseHealthCheck(database));
You can also run the set of registered health checks:
for (Entry<String, Result> entry : registry.runHealthChecks().entrySet()) {
if (entry.getValue().isHealthy()) {
System.out.println(entry.getKey() + ": OK");
} else {
System.out.println(entry.getKey() + ": FAIL");
}
}
The metrics-ehcache module provides InstrumentedEhcache, a decorator for
Ehcache caches:
final Cache c = new Cache(new CacheConfiguration("test", 100));
MANAGER.addCache(c);
this.cache = InstrumentedEhcache.instrument(registry, c);
Instrumenting an Ehcache instance creates gauges for all of the Ehcache-provided statistics:
hits |
The number of times a requested item was found in the cache. |
in-memory-hits |
Number of times a requested item was found in the memory store. |
off-heap-hits |
Number of times a requested item was found in the off-heap store. |
on-disk-hits |
Number of times a requested item was found in the disk store. |
misses |
Number of times a requested item was not found in the cache. |
in-memory-misses |
Number of times a requested item was not found in the memory store. |
off-heap-misses |
Number of times a requested item was not found in the off-heap store. |
on-disk-misses |
Number of times a requested item was not found in the disk store. |
objects |
Number of elements stored in the cache. |
in-memory-objects |
Number of objects in the memory store. |
off-heap-objects |
Number of objects in the off-heap store. |
on-disk-objects |
Number of objects in the disk store. |
mean-get-time |
The average get time. Because ehcache supports JDK1.4.2, each get
time uses System.currentTimeMillis(), rather than nanoseconds.
The accuracy is thus limited. |
mean-search-time |
The average execution time (in milliseconds) within the last sample period. |
eviction-count |
The number of cache evictions, since the cache was created, or statistics were cleared. |
searches-per-second |
The number of search executions that have completed in the last second. |
accuracy |
A human readable description of the accuracy setting. One of “None”, “Best Effort” or “Guaranteed”. |
It also adds full timers for the cache’s get and put methods.
The metrics are all scoped to the cache’s class and name, so a Cache instance named users
would have metric names like net.sf.ehcache.Cache.users.get, etc.
The metrics-ganglia module provides GangliaReporter, which allows your application to
constantly stream metric values to a Ganglia server:
final GMetric ganglia = new GMetric("ganglia.example.com", 8649, UDPAddressingMode.MULTICAST, 1);
final GangliaReporter reporter = GangliaReporter.forRegistry(registry)
.convertRatesTo(TimeUnit.SECONDS)
.convertDurationsTo(TimeUnit.MILLISECONDS)
.build(ganglia);
reporter.start(1, TimeUnit.MINUTES);
The metrics-graphite module provides GraphiteReporter, which allows your application to
constantly stream metric values to a Graphite server:
final Graphite graphite = new Graphite(new InetSocketAddress("graphite.example.com", 2003));
final GraphiteReporter reporter = GraphiteReporter.forRegistry(registry)
.prefixedWith("web1.example.com")
.convertRatesTo(TimeUnit.SECONDS)
.convertDurationsTo(TimeUnit.MILLISECONDS)
.filter(MetricFilter.ALL)
.build(graphite);
reporter.start(1, TimeUnit.MINUTES);
If you prefer to write metrics in batches using pickle, you can use the PickledGraphite:
final Graphite pickledGraphite = new PickledGraphite(new InetSocketAddress("graphite.example.com", 2004));
final GraphiteReporter reporter = GraphiteReporter.forRegistry(registry)
.prefixedWith("web1.example.com")
.convertRatesTo(TimeUnit.SECONDS)
.convertDurationsTo(TimeUnit.MILLISECONDS)
.filter(MetricFilter.ALL)
.build(pickledGraphite);
reporter.start(1, TimeUnit.MINUTES);
The metrics-httpclient module provides InstrumentedHttpClientConnManager and
InstrumentedHttpClients, two instrumented versions of Apache HttpClient 4.x classes.
InstrumentedHttpClientConnManager is a thread-safe HttpClientConnectionManager implementation which
measures the number of open connections in the pool and the rate at which new connections are
opened.
InstrumentedHttpClients follows the HttpClients builder pattern and adds per-HTTP method timers for
HTTP requests.
The default per-method metric naming and scoping strategy can be overridden by passing an
implementation of HttpClientMetricNameStrategy to the InstrumentedHttpClients.createDefault method.
A number of pre-rolled strategies are available, e.g.:
HttpClient client = InstrumentedHttpClients.createDefault(registry, HttpClientMetricNameStrategies.HOST_AND_METHOD);
The metrics-jdbi module provides a TimingCollector implementation for JDBI, an SQL
convenience library.
To use it, just add a InstrumentedTimingCollector instance to your DBI:
final DBI dbi = new DBI(dataSource);
dbi.setTimingCollector(new InstrumentedTimingCollector(registry));
InstrumentedTimingCollector keeps per-SQL-object timing data, as well as general raw SQL timing
data. The metric names for each query are constructed by an StatementNameStrategy instance, of
which there are many implementations. By default, StatementNameStrategy uses
SmartNameStrategy, which attempts to effectively handle both queries from bound objects and raw
SQL.
The metrics-jersey module provides InstrumentedResourceMethodDispatchAdapter, which allows
you to instrument methods on your Jersey 1.x resource classes:
An instance of InstrumentedResourceMethodDispatchAdapter must be registered with your Jersey
application’s ResourceConfig as a singleton provider for this to work.
public class ExampleApplication {
private final DefaultResourceConfig config = new DefaultResourceConfig();
public void init() {
config.getSingletons().add(new InstrumentedResourceMethodDispatchAdapter(registry));
config.getClasses().add(ExampleResource.class);
}
}
@Path("/example")
@Produces(MediaType.TEXT_PLAIN)
public class ExampleResource {
@GET
@Timed
public String show() {
return "yay";
}
}
The show method in the above example will have a timer attached to it, measuring the time spent
in that method.
Use of the @Metered and @ExceptionMetered annotations is also supported.
Jersey 2.x changed the API for how resource method monitoring works, so a new
module metrics-jersey2 provides InstrumentedResourceMethodApplicationListener,
which allows you to instrument methods on your Jersey 2.x resource classes:
The metrics-jersey2 module provides InstrumentedResourceMethodApplicationListener, which allows
you to instrument methods on your Jersey 2.x resource classes:
An instance of InstrumentedResourceMethodApplicationListener must be registered with your Jersey
application’s ResourceConfig as a singleton provider for this to work.
public class ExampleApplication extends ResourceConfig {
.
.
.
register(new InstrumentedResourceMethodApplicationListener (new MetricRegistry()));
config = config.register(ExampleResource.class);
.
.
.
}
@Path("/example")
@Produces(MediaType.TEXT_PLAIN)
public class ExampleResource {
@GET
@Timed
public String show() {
return "yay";
}
}
The show method in the above example will have a timer attached to it, measuring the time spent
in that method.
Use of the @Metered and @ExceptionMetered annotations is also supported.
The metrics-jetty8 (Jetty 8.0), metrics-jetty9-legacy (Jetty 9.0), and metrics-jetty9
(Jetty 9.1 and higher) modules provides a set of instrumented equivalents of Jetty classes:
InstrumentedBlockingChannelConnector, InstrumentedHandler, InstrumentedQueuedThreadPool,
InstrumentedSelectChannelConnector, and InstrumentedSocketConnector.
The Connector implementations are simple, instrumented subclasses of the Jetty connector types
which measure connection duration, the rate of accepted connections, connections, disconnections,
and the total number of active connections.
InstrumentedQueuedThreadPool is a QueuedThreadPool subclass which measures the ratio of idle
threads to working threads as well as the absolute number of threads (idle and otherwise).
InstrumentedHandler is a Handler decorator which measures a wide range of HTTP behavior:
dispatch times, requests, resumes, suspends, expires, the number of active, suspected, and
dispatched requests, as well as meters of responses with 1xx, 2xx, 3xx, 4xx, and
5xx status codes. It even has gauges for the ratios of 4xx and 5xx response rates to
overall response rates. Finally, it includes meters for requests by the HTTP method: GET,
POST, etc.
The metrics-log4j and metrics-log4j2 modules provide InstrumentedAppender, a Log4j Appender implementation
(for log4j 1.x and log4j 2.x correspondingly) which records the rate of logged events by their logging level.
You can add it to the root logger programmatically.
For log4j 1.x:
InstrumentedAppender appender = new InstrumentedAppender(registry);
appender.activateOptions();
LogManager.getRootLogger().addAppender(appender);
For log4j 2.x:
Filter filter = null; // That's fine if we don't use filters; https://logging.apache.org/log4j/2.x/manual/filters.html
PatternLayout layout = null; // The layout isn't used in InstrumentedAppender
InstrumentedAppender appender = new InstrumentedAppender(metrics, filter, layout, false);
appender.start();
LoggerContext context = (LoggerContext) LogManager.getContext(false);
Configuration config = context.getConfiguration();
config.getLoggerConfig(LogManager.ROOT_LOGGER_NAME).addAppender(appender, level, filter);
context.updateLoggers(config);
The metrics-logback module provides InstrumentedAppender, a Logback Appender
implementation which records the rate of logged events by their logging level.
You add it to the root logger programmatically:
final LoggerContext factory = (LoggerContext) LoggerFactory.getILoggerFactory();
final Logger root = factory.getLogger(Logger.ROOT_LOGGER_NAME);
final InstrumentedAppender metrics = new InstrumentedAppender(registry);
metrics.setContext(root.getLoggerContext());
metrics.start();
root.addAppender(metrics);
The metrics-jvm module contains a number of reusable gauges and
metric sets which allow you to easily instrument JVM internals.
Supported metrics include:
Metrics comes with metrics-json, which features two reusable modules for Jackson.
This allows for the serialization of all metric types and health checks to a standard, easily-parsable JSON format.
The metrics-servlets module provides a handful of useful servlets:
HealthCheckServlet responds to GET requests by running all the [health checks](#health-checks)
and returning 501 Not Implemented if no health checks are registered, 200 OK if all pass, or
500 Internal Service Error if one or more fail. The results are returned as a human-readable
text/plain entity.
HealthCheckServlet requires that the servlet context has a HealthCheckRegistry named
com.codahale.metrics.servlets.HealthCheckServlet.registry. You can subclass
MetricsServletContextListener, which will add a specific HealthCheckRegistry to the servlet
context.
ThreadDumpServlet responds to GET requests with a text/plain representation of all the live
threads in the JVM, their states, their stack traces, and the state of any locks they may be
waiting for.
MetricsServlet exposes the state of the metrics in a particular registry as a JSON object.
MetricsServlet requires that the servlet context has a MetricRegistry named
com.codahale.metrics.servlets.MetricsServlet.registry. You can subclass
MetricsServletContextListener, which will add a specific MetricRegistry to the servlet
context.
MetricsServlet also takes an initialization parameter, show-jvm-metrics, which if "false" will
disable the outputting of JVM-level information in the JSON object.
PingServlet responds to GET requests with a text/plain/200 OK response of pong. This is
useful for determining liveness for load balancers, etc.
AdminServlet aggregates HealthCheckServlet, ThreadDumpServlet, MetricsServlet, and
PingServlet into a single, easy-to-use servlet which provides a set of URIs:
/: an HTML admin menu with links to the following:/healthcheck: HealthCheckServlet/metrics: MetricsServlet/ping: PingServlet/threads: ThreadDumpServletYou will need to add your MetricRegistry and HealthCheckRegistry instances to the servlet
context as attributes named com.codahale.metrics.servlets.MetricsServlet.registry and
com.codahale.metrics.servlets.HealthCheckServlet.registry, respectively. You can do this using
the Servlet API by extending MetricsServlet.ContextListener for MetricRegistry:
public class MyMetricsServletContextListener extends MetricsServlet.ContextListener {
public static final MetricRegistry METRIC_REGISTRY = new MetricRegistry();
@Override
protected MetricRegistry getMetricRegistry() {
return METRIC_REGISTRY;
}
}
And by extending HealthCheckServlet.ContextListener for HealthCheckRegistry:
public class MyHealthCheckServletContextListener extends HealthCheckServlet.ContextListener {
public static final HealthCheckRegistry HEALTH_CHECK_REGISTRY = new HealthCheckRegistry();
@Override
protected HealthCheckRegistry getHealthCheckRegistry() {
return HEALTH_CHECK_REGISTRY;
}
}
Then you will need to register servlet context listeners either in you web.xml or annotating the class with @WebListener if you are in servlet 3.0 environment. In web.xml:
<listener>
<listener-class>com.example.MyMetricsServletContextListener</listener-class>
</listener>
<listener>
<listener-class>com.example.MyHealthCheckServletContextListener</listener-class>
</listener>
You will also need to register AdminServlet in web.xml:
<servlet>
<servlet-name>metrics</servlet-name>
<servlet-class>com.codahale.metrics.servlets.AdminServlet</servlet-class>
</servlet>
<servlet-mapping>
<servlet-name>metrics</servlet-name>
<url-pattern>/metrics/*</url-pattern>
</servlet-mapping>
The metrics-servlet module provides a Servlet filter which has meters for status codes, a
counter for the number of active requests, and a timer for request duration. By default the filter
will use com.codahale.metrics.servlet.InstrumentedFilter as the base name of the metrics.
You can use the filter in your web.xml like this:
<filter>
<filter-name>instrumentedFilter</filter-name>
<filter-class>com.codahale.metrics.servlet.InstrumentedFilter</filter-class>
</filter>
<filter-mapping>
<filter-name>instrumentedFilter</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>
An optional filter init-param name-prefix can be specified to override the base name
of the metrics associated with the filter mapping. This can be helpful if you need to instrument
multiple url patterns and give each a unique name.
<filter>
<filter-name>instrumentedFilter</filter-name>
<filter-class>com.codahale.metrics.servlet.InstrumentedFilter</filter-class>
<init-param>
<param-name>name-prefix</param-name>
<param-value>authentication</param-value>
</init-param>
</filter>
<filter-mapping>
<filter-name>instrumentedFilter</filter-name>
<url-pattern>/auth/*</url-pattern>
</filter-mapping>
You will need to add your MetricRegistry to the servlet context as an attribute named
com.codahale.metrics.servlet.InstrumentedFilter.registry. You can do this using the Servlet API
by extending InstrumentedFilterContextListener:
public class MyInstrumentedFilterContextListener extends InstrumentedFilterContextListener {
public static final MetricRegistry REGISTRY = new MetricRegistry();
@Override
protected MetricRegistry getMetricRegistry() {
return REGISTRY;
}
}
If you’re looking to integrate with something not provided by the main Metrics libraries, check out the many third-party libraries which extend Metrics:
Many, many thanks to:
MetricRegistry#name.ScheduledReporter and JmxReporter now implement Closeable.metrics-jetty9.Access-Control-Allow-Origin to MetricsServlet.Meter EWMA rates.AdminServletContextListener in favor of MetricsServlet.ContextListener and
HealthCheckServlet.ContextListener.HealthCheckServlet and MetricsServlet.DefaultWebappMetricsFilter to InstrumentedFilter.MetricsContextListener to InstrumentedFilterContextListener and made it fully
abstract to avoid confusion.MetricsServletContextListener to AdminServletContextListener and made it fully
abstract to avoid confusion.SharedMetricRegistries, a singleton for sharing named metric registries.metrics-ehcache.metrics-jersey.metrics-log4j.metrics-logback.metrics-jetty9’s InstrumentedHandler.MetricsContextListener to metrics-servlet.MetricsServletContextListener to metrics-servlets.Counting interface.SlidingWindowReservoir to a synchronized implementation.Slf4jReporter’s logging of 99th percentiles.GraphiteReporter.JmxReporter.ScheduledReporter#report() for manual reporting.HealthCheck and
InstrumentedResourceMethodDispatchProvider.SlidingWindowReservoir.metrics-jetty9, removing InstrumentedConnector and improving
the API.sun.misc.HttpClient metrics.com.codahale.metrics package, with the corresponding changes in Maven
artifact groups. This should allow for an easier upgrade path without classpath conflicts.MetricRegistry no longer has a name.metrics-jetty9 for Jetty 9.JmxReporter takes an optional domain property to disambiguate multiple reporters.MetricRegistryListener.Base.Counter, Meter, and EWMA to use JSR133’s LongAdder instead of
AtomicLong, improving contended concurrency.MetricRegistry#buildMap(), allowing for custom map implementations in
MetricRegistry.MetricRegistry#removeMatching(MetricFilter).metrics-json to optionally depend on metrics-healthcheck.metrics-jetty8.com.example.Thing, allowing for very flexible
scopes, etc.MetricSet for sets of metrics.metrics-jvm.metrics-json.metrics-guice, metrics-scala, and metrics-spring.metrics-servlet to metrics-servlets.metrics-web to metrics-servlet.metrics-jetty to metrics-jetty8.InstrumentedSslSelectChannelConnector and InstrumentedSslSocketConnector.Unsafe in InstrumentedResourceMethodDispatchProvider with type erasure
trickery.InstrumentedClientConnManager to extend PoolingClientConnectionManager instead of
the deprecated ThreadSafeClientConnManager.ExponentiallyDecayingSample with long periods of inactivity.DnsResolver instances to InstrumentedClientConnManager.metrics-guice.InstrumentedHttpClient.VirtualMachineMetrics and
metrics-servlet.metrics-ehcache.metrics-spring now support @Gauge-annotated fields.GraphiteReporter up for extension.group and type to metrics-annotations, metrics-guice, metrics-jersey,
and metrics-spring.GangliaReporter.NullPointerException errors in metrics-spring.metrics-spring, including allowing custom Clock instances.InstanceNotFoundException exceptions thrown while unregistering a metric
in JmxReporter to TRACE. It being WARN resulted in huge log dumps preventing process
shutdowns when applications had ~1K+ metrics.metrics-spring.GangliaReporter.InstrumentationModule in metrics-guice now uses the default MetricsRegistry and
HealthCheckRegistry.JmxReporter.GraphiteReporter.ThreadLocalRandom for UniformSample and
ExponentiallyDecayingSample to reduce lock contention on random number generation.Ordered from TimedAnnotationBeanPostProcessor in metrics-spring.#timerContext() to Scala Timer.Error instances thrown during health checks.enable static methods to CsvReporter and changed
CsvReporter(File, MetricsRegistry) to CsvReporter(MetricsRegistry, File).InstrumentedEhcache.GangliaReporter.metrics-guice.metrics-httpclient to consistently associate metrics with the org.apache class
being extended.metrics-httpclient.InstrumentedAppender in metrics-log4j. It no longer forwards events to an
appender. Instead, you can just attach it to your root logger to instrument logging.InstrumentedAppender in metrics-logback. No major API changes.@ExceptionMetered-annotated resource methods in metrics-jersey.Snapshot instances from concurrently modified collections.MetricsServlet’s thread dumps where one thread could be missed.RatioGauge and PercentGauge.InstrumentedQueuedThreadPool’s percent-idle gauge to be a ratio.MetricsServlet into a set of focused servlets: HealthCheckServlet,
MetricsServlet, PingServlet, and ThreadDumpServlet. The top-level servlet which
provides the HTML menu page is now AdminServlet.metrics-spring.MetricsServlet.@Timed etc. to metrics-annotations.metrics-jersey, which provides a class allowing you to automatically instrument all
@Timed, @Metered, and @ExceptionMetered-annotated resource methods.metrics-scala from com.yammer.metrics to
com.yammer.metrics.scala.CounterMetric to Counter.GaugeMetric to Gauge.HistogramMetric to Histogram.MeterMetric to Meter.TimerMetric to Timer.ToggleGauge, which returns 1 the first time it’s called and 0 every time after
that.VirtualMachineMetrics to a non-singleton class.Utils.Meter and Timer.LoggerMemoryLeakFix.DeathRattleExceptionHandler now logs to SLF4J, not syserr.MetricsRegistry#groupedMetrics().Metrics#allMetrics().Metrics#remove(MetricName).MetricsRegistry#threadPools() and #newMeterTickThreadPool() and added
#newScheduledThreadPool.MetricsRegistry#shutdown().ThreadPools#shutdownThreadPools() to #shutdown().HealthCheck’s abstract name method with a required constructor parameter.HealthCheck#check() is now protected.DeadlockHealthCheck from com.yammer.metrics.core to com.yammer.metrics.utils.HealthCheckRegistry#unregister(HealthCheck).VirtualMachineMetrics and MetricsServlet: commited to committed.MetricsRegistry#createName to protected.MetricsRegistry now.Metrics.newJmxGauge and MetricsRegistry.newJmxGauge are deprecated.VirtualMachineMetrics.Snapshot, which calculates quantiles.Percentiled to Sampling and dropped percentile and percentiles in favor of
producing Snapshot instances. This affects both Histogram and Timer.Summarized to Summarizable.CsvReporter’s construction parameters.VirtualMachineMetrics.GarbageCollector to
VirtualMachineMetrics.GarbageCollectorStats.metrics-servlet to metrics-guice.metrics-aop.newJmxGauge from both Metrics and MetricsRegistry. Just use JmxGauge.JmxGauge to com.yammer.metrics.util.MetricPredicate to com.yammer.metrics.core.NameThreadFactory into ThreadPools and made ThreadPools package-visible.Timer#values(), Histogram#values(), and Sample#values(). Use getSnapshot()
instead.Timer#dump(File) and Histogram#dump(File), and Sample#dump(File). Use
Snapshot#dump(File) instead.DeathRattleExceptionHandler.VirtualMachineMetrics.metrics-jetty.TimerMetric#time() and TimerContext.GangliaReporter.UniformSample.metrics-httpclient for instrumenting Apache HttpClient 4.1.public methods in metrics-guice.@ExceptionMetered to metrics-guice.GangliaReporter.CvsReporter, which outputs metric values to .csv files.GangliaReporter.Metrics.shutdown() and improved metrics lifecycle behavior.metrics-web.metrics-servlet now responds with 501 Not Implememented when no health checks have been
registered.metrics-servlet.ExponentiallyDecayingSample.ConsoleReporter.metrics-aop for Guiceless support of method annotations.metrics-jdbi which adds instrumentation to JDBI.GraphiteReporter.GangliaReporter.InstrumentedHandler in metrics-jetty.#dump(File) to HistogramMetric and TimerMetric.Metrics.removeMetric().metrics-jetty.vm output of MetricsServlet.com.sun.mangement-based GC instrumentation in favor of a
java.lang.management-based one. getLastGcInfo has a nasty native memory leak in it, plus
it often returned incorrect data.GraphiteReporter.Clock interface for timers for non-wall-clock timing.MetricsRegistry and HealthCheckRegistry.MetricsServlet for disabling the jvm section.MetricsServlet.metrics-scala module
which is now the only cross-built module. All other modules dropped the Scala version suffix in
their artifactId.GraphiteReporter.GraphiteReporter when dealing with unavailable servers.MetricsServlet when a gauge throws an exception.MetricsServlet menu page.JmxReporter.metrics-ehcache, for the instrumentation of Ehcache instances.metrics-jetty’s InstrumentedHandler.GraphiteReporter.GraphiteReporter.MetricsServlet’s links when the servlet has a non-root context path.pretty query parameter for MetricsServlet to format the JSON object for human
consumption.no-cache headers to the MetricsServlet responses.4xx or 5xx status
codes.provided dependency. Thanks to Mårten Gustafson (@chids) for
the patch.metrics-core: A dependency-less project with all the core metrics.metrics-graphite: A reporter for the [Graphite](http://graphite.wikidot.com)
aggregation system.metrics-guice: Guice AOP support.metrics-jetty: An instrumented Jetty handler.metrics-log4j: An instrumented Log4J appender.metrics-logback: An instrumented Logback appender.metrics-servlet: The Metrics servlet with context listener.VirtualMachineMetrics’ initialization.@Gauge annotation.ExponentiallyDecayingSample. Thanks to Martin Traverso (@martint) for
the patch.java.util.logging.@Timed and @Metered.HealthCheck#name().Metrics.newJmxGauge().HealthChecks.JmxReporter lag.ExponentiallyDecayingSample.UniformSample.ExponentiallyDecayingSample.jackon-mapper.JettyHandler.Servlet dependency optional.JmxReporter initialization.Counter#++ and Counter#--.Timer#update.MeterMetric.median to Timer.p95 to Timer (95th percentile).p98 to Timer (98th percentile).p99 to Timer (99th percentile).TimedToggle, which may or may not be useful at all.Timer instances (i.e., those which have recorded no timings yet) no longer explode when
asked for metrics for that which does not yet exist.$ characters messing up
JMX’s good looks.Timer, giving it 99.9% confidence level with a %5 margin of error
(for a normally distributed variable, which it almost certainly isn’t.)Sample#iterator returns only the recorded data, not a bunch of zeros.Timer, Meter, and LoadMeter to their own attributes, which allows for
easy export of Metrics data via JMX to things like Ganglia or whatever.Timer now uses Welford’s algorithm for calculating running variance, which means no more
hilariously wrong standard deviations (e.g., NaN).Timer now supports +=(Long) for pre-recorded, nanosecond-precision timings.Sample to use an AtomicReferenceArray