Version: 9.4.20.v20190813 |

Chapter 14. Java Management Extensions (JMX)
Previous | Part III. Jetty Administration Guide Home | Next

##### Contact the core Jetty developers at www.webtide.com

private support for your internal/customer projects ... custom extensions and distributions ... versioned snapshots for indefinite support ... scalability guidance for your apps and Ajax/Comet projects ... development services for sponsored feature development

## Chapter 14. Java Management Extensions (JMX)

The Java Management Extensions (JMX) API is a standard API for managing and monitoring resources such as applications, devices, services, and the Java virtual machine.

Typical uses of the JMX technology include:

- Consulting and changing application configuration
- Accumulating and making available statistics about application behavior
- Notifying of state changes and erroneous conditions

The JMX API includes remote access, so a remote management program can interact with a running application for these purposes.

## Using JMX with Jetty

Jetty’s architecture is based on POJO components (see Jetty Architecture). These components are organized in a tree and each component may have a lifecycle that spans the `Server` lifetime, or a web application lifetime, or even shorter lifetimes such as that of a TCP connection.

Every time a component is added or removed from the component tree, an event is emitted, and `Container.Listener` implementations can listen to those events and perform additional actions.

One such `Container.Listener` is `MBeanContainer` that uses `ObjectMBean` to create an MBean from an arbitrary POJO, and register/unregister the MBean to/from the platform `MBeanServer`.

Jetty components are annotated with Jetty JMX Annotations and provide specific JMX details so that `ObjectMBean` can build a more precise representation of the JMX metadata associated with the component POJO.

Therefore, when a component is added to the component tree, `MBeanContainer` is notified, it creates the MBean from the component POJO and registers it to the `MBeanServer`. Similarly, when a component is removed from the tree, `MBeanContainer` is notified, and unregisters the MBean from the `MBeanServer`.

The Jetty MBeans can be accessed via any JMX console such as Java Mission Control (JMC), VisualVM, JConsole or others.

### Configuring JMX

This guide describes the various ways to initialize and configure the Jetty JMX integration. Configuring the Jetty JMX integration only registers the Jetty MBeans into the platform `MBeanServer`, and therefore the MBeans can only be accessed locally (from the same machine), not from remote machines.

This means that this configuration is enough for development, where you have easy access (with graphical user interface) to the machine where Jetty runs, but it is typically not enough when the machine Jetty where runs is remote, or only accessible via SSH or otherwise without graphical user interface support. In these cases, you have to enable JMX Remote Access.

#### Standalone Jetty Server

JMX is not enabled by default in the Jetty distribution. To enable JMX in the Jetty distribution run the following, where `{$jetty.home}` is the directory where you have the Jetty distribution installed, and `${jetty.base}` is the directory where you have your Jetty configuration (see the documentation for Jetty base vs. home examples):

    $ cd ${jetty.base}
    $ java -jar {$jetty.home}/start.jar --add-to-start=jmx

Running the above command will append the available configurable elements of the `jmx` module to the `{$jetty.base}/start.ini` file, or create the `${jetty.base}/start.d/jmx.ini` file.

#### Embedded Jetty Server

When running Jetty embedded into an application, create and configure an `MBeanContainer` instance as follows:

    Server server = new Server();

    // Setup JMX.
    MBeanContainer mbeanContainer = new MBeanContainer(ManagementFactory.getPlatformMBeanServer());
    server.addBean(mbeanContainer);

    // Export the loggers as MBeans.
    server.addBean(Log.getLog());

Because logging is initialized prior to the `MBeanContainer` (even before the `Server` itself), it is necessary to register the logger manually via `server.addBean()` so that the loggers may show up in the JMX tree as MBeans.

#### Using the Jetty Maven Plugin with JMX

If you are using the Jetty Maven plugin you should copy the `${jetty.home}/etc/jetty-jmx.xml` file into your webapp project somewhere, such as `src/main/config/etc/`, then add a `<jettyXml>` element to the `<configuration>` element of the Jetty Maven Plugin:

    <plugin>
      <groupid>org.eclipse.jetty</groupid>
      <artifactid>jetty-maven-plugin</artifactid>
      <version>9.4.20.v20190813</version>
      <configuration>
        <scanintervalseconds>10</scanintervalseconds>
        <jettyXml>src/main/config/etc/jetty-jmx.xml</jettyXml>
      </configuration>
    </plugin>

### Using JConsole or Java Mission Control to Access Jetty MBeans

The simplest way to access the MBeans that Jetty publishes is to use Java Mission Control (JMC) or JConsole.

Both these tools can connect to local or remote JVMs to display the MBeans.

For local access, you just need to start JConsole or JMC and then choose from their user interface the local JVM you want to connect to.

For remote access, you need first to enable JMX Remote Access in Jetty.

### Enabling JMX Remote Access

There are two ways of enabling remote connectivity so that JConsole or JMC can connect to the remote JVM to visualize MBeans.

- Use the `com.sun.management.jmxremote` system property on the command line. Unfortunately, this solution does not work well with firewalls and is not flexible.
- Use Jetty’s `jmx-remote` module or - equivalently - the `ConnectorServer` class.

`ConnectorServer` will use by default RMI to allow connection from remote clients, and it is a wrapper around the standard JDK class `JMXConnectorServer`, which is the class that provides remote access to JMX clients.

Connecting to the remote JVM is a two step process:

- First, the client will connect to the RMI registry to download the RMI stub for the `JMXConnectorServer`; this RMI stub contains the IP address and port to connect to the RMI server, i.e. the remote `JMXConnectorServer`.
- Second, the client uses the RMI stub to connect to the RMI server (i.e. the remote `JMXConnectorServer`) typically on an address and port that may be different from the RMI registry address and port.

The configuration for the RMI registry and the RMI server is specified by a `JMXServiceURL`. The string format of an RMI `JMXServiceURL` is:

    service:jmx:rmi://<rmi_server_host>:<rmi_server_port>/jndi/rmi://<rmi_registry_host>:<rmi_registry_port>/jmxrmi

Default values are:

    rmi_server_host = localhost
    rmi_server_port = 1099
    rmi_registry_host = localhost
    rmi_registry_port = 1099

With the default configuration, only clients that are local to the server machine can connect to the RMI registry and RMI server - this is done for security reasons. With this configuration it would still be possible to access the MBeans from remote using a SSH tunnel.

By specifying an appropriate `JMXServiceURL`, you can fine tune the network interfaces the RMI registry and the RMI server bind to, and the ports that the RMI registry and the RMI server listen to. The RMI server and RMI registry hosts and ports can be the same (as in the default configuration) because RMI is able to multiplex traffic arriving to a port to multiple RMI objects.

If you need to allow JMX remote access through a firewall, you must open both the RMI registry and the RMI server ports.

Examples:

    service:jmx:rmi:///jndi/rmi:///jmxrmi
      rmi_server_host = local host address
      rmi_server_port = randomly chosen
      rmi_registry_host = local host address
      rmi_registry_port = 1099

    service:jmx:rmi://0.0.0.0:1099/jndi/rmi://0.0.0.0:1099/jmxrmi
      rmi_server_host = any address
      rmi_server_port = 1099
      rmi_registry_host = any address
      rmi_registry_port = 1099

    service:jmx:rmi://localhost:1100/jndi/rmi://localhost:1099/jmxrmi
      rmi_server_host = loopback address
      rmi_server_port = 1100
      rmi_registry_host = loopback address
      rmi_registry_port = 1099

### Note

When `ConnectorServer` is started, its RMI stub is exported to the RMI registry. The RMI stub contains the IP address and port to connect to the RMI object, but the IP address is typically the machine host name, not the host specified in the `JMXServiceURL`.

To control the IP address stored in the RMI stub you need to set the system property `java.rmi.server.hostname` with the desired value. This is especially important when binding the RMI server host to the loopback address for security reasons. See also JMX Remote Access via SSH Tunnel.

#### Enabling JMX Remote Access in Standalone Jetty Server

Similarly to enabling JMX in a standalone Jetty server, you enable the `jmx-remote` module:

    $ cd ${jetty.base}
    $ java -jar {$jetty.home}/start.jar --add-to-start=jmx-remote

#### Enabling JMX Remote Access in Embedded Jetty

When running Jetty embedded into an application, create and configure a `ConnectorServer`:

    Server server = new Server();

    // Setup JMX
    MBeanContainer mbeanContainer = new MBeanContainer(ManagementFactory.getPlatformMBeanServer());
    server.addBean(mbeanContainer);

    // Setup ConnectorServer
    JMXServiceURL jmxURL = new JMXServiceURL("rmi", null, 1999, "/jndi/rmi:///jmxrmi");
    ConnectorServer jmxServer = new ConnectorServer(jmxURL, "org.eclipse.jetty.jmx:name=rmiconnectorserver");
    server.addBean(jmxServer);

The `JMXServiceURL` above specifies that the RMI server binds to the wildcard address on port 1999, while the RMI registry binds to the wildcard address on port 1099 (the default RMI registry port).

#### JMX Remote Access Authorization

The standard `JMXConnectorServer` provides several options to authorize access. For a complete guide to controlling authentication and authorization in JMX, see Authentication and Authorization in JMX RMI connectors.

To authorize access to the `JMXConnectorServer` you can use this configuration, where the `jmx.password` and `jmx.access` files have the format specified in the blog entry above:

    <New id="ConnectorServer" class="org.eclipse.jetty.jmx.ConnectorServer">
      <Arg>
        <New class="javax.management.remote.JMXServiceURL">
          <Arg type="java.lang.String">rmi</Arg>
          <Arg type="java.lang.String" />
          <Arg type="java.lang.Integer">1099</Arg>
          <Arg type="java.lang.String">/jndi/rmi:///jmxrmi</Arg>
        </New>
      </Arg>
      <Arg>
        <Map>
          <Entry>
            <Item>jmx.remote.x.access.file</Item>
            <Item>
              <New class="java.lang.String"><Arg><Property name="jetty.base" default="." />/resources/jmx.access</Arg></New>
            </Item>
          </Entry>
          <Entry>
            <Item>jmx.remote.x.password.file</Item>
            <Item>
              <New class="java.lang.String"><Arg><Property name="jetty.base" default="." />/resources/jmx.password</Arg></New>
            </Item>
          </Entry>
        </Map>
      </Arg>
      <Arg>org.eclipse.jetty.jmx:name=rmiconnectorserver</Arg>
      <Call name="start" />
    </New>

Similarly, in code:

    JMXServiceURL jmxURL = new JMXServiceURL("rmi", null, 1099, "/jndi/rmi:///jmxrmi");
    Map<String, Object> env = new HashMap<>();
    env.put("jmx.remote.x.access.file", "resources/jmx.access");
    env.put("jmx.remote.x.password.file", "resources/jmx.password");
    ConnectorServer jmxServer = new ConnectorServer(jmxURL, env, "org.eclipse.jetty.jmx:name=rmiconnectorserver");
    jmxServer.start();

Calling `ConnectorServer.start()` may be explicit as in the examples above, or can be skipped when adding the `ConnectorServer` as a bean to the `Server`, so that starting the `Server` will also start the `ConnectorServer`.

#### Securing JMX Remote Access with TLS

The JMX communication via RMI happens by default in clear-text.

It is possible to configure the `ConnectorServer` with a `SslContextFactory` so that the JMX communication via RMI is encrypted:

    <New id="ConnectorServer" class="org.eclipse.jetty.jmx.ConnectorServer">
      <Arg>
        <New class="javax.management.remote.JMXServiceURL">
          <Arg type="java.lang.String">rmi</Arg>
          <Arg type="java.lang.String" />
          <Arg type="java.lang.Integer">1099</Arg>
          <Arg type="java.lang.String">/jndi/rmi:///jmxrmi</Arg>
        </New>
      </Arg>
      <Arg />
      <Arg>org.eclipse.jetty.jmx:name=rmiconnectorserver</Arg>
      <Arg><Ref refid="sslContextFactory" /></Arg>
    </New>

Similarly, in code:

    SslContextFactory.Server sslContextFactory = new SslContextFactory.Server();
    sslContextFactory.setKeyStorePath("/path/to/keystore");
    sslContextFactory.setKeyStorePassword("secret");

    JMXServiceURL jmxURL = new JMXServiceURL("rmi", null, 1099, "/jndi/rmi:///jmxrmi");
    ConnectorServer jmxServer = new ConnectorServer(jmxURL, null, "org.eclipse.jetty.jmx:name=rmiconnectorserver", sslContextFactory);

It is possible to use the same `SslContextFactory` used to configure the Jetty `ServerConnector` that supports TLS for the HTTP protocol. This is used in the XML example above: the `SslContextFactory` configured for the TLS `ServerConnector` is registered with an id of `sslContextFactory` which is referenced in the XML via the `Ref` element.

The keystore must contain a valid certificate signed by a Certification Authority.

The RMI mechanic is the usual one: the RMI client (typically a monitoring console) will connect first to the RMI registry (using TLS), download the RMI server stub that contains the address and port of the RMI server to connect to, then connect to the RMI server (using TLS).

This also mean that if the RMI registry and the RMI server are on different hosts, the RMI client must have available the cryptographic material to validate both hosts.

Having certificates signed by a Certification Authority simplifies by a lot the configuration needed to get the JMX communication over TLS working properly.

If that is not the case (for example the certificate is self-signed), then you need to specify the required system properties that allow RMI (especially when acting as an RMI client) to retrieve the cryptographic material necessary to establish the TLS connection.

For example, trying to connect using the JDK standard `JMXConnector` with both the RMI server and the RMI registry to `domain.com`:

    // System properties necessary for an RMI client to trust a self-signed certificate.
    System.setProperty("javax.net.ssl.trustStore", "/path/to/trustStore");
    System.setProperty("javax.net.ssl.trustStorePassword", "secret");

    JMXServiceURL jmxURL = new JMXServiceURL("service:jmx:rmi:///jndi/rmi://domain.com:1100/jmxrmi")
    Map<String, Object> clientEnv = new HashMap<>();
    // Required to connect to the RMI registry via TLS.
    clientEnv.put(ConnectorServer.RMI_REGISTRY_CLIENT_SOCKET_FACTORY_ATTRIBUTE, new SslRMIClientSocketFactory());
    try (JMXConnector client = JMXConnectorFactory.connect(jmxURL, clientEnv))
    {
        Set<ObjectName> names = client.getMBeanServerConnection().queryNames(null, null);
    }

Similarly, to launch JMC:

    $ jmc -vmargs -Djavax.net.ssl.trustStore=/path/to/trustStore -Djavax.net.ssl.trustStorePassword=secret

Note that these system properties are required when launching the `ConnectorServer` too, on the server, because it acts as an RMI client with respect to the RMI registry.

#### JMX Remote Access with Port Forwarding via SSH Tunnel

You can access JMX MBeans on a remote machine when the RMI ports are not open, for example because of firewall policies, but you have SSH access to the machine using local port forwarding via a SSH tunnel.

In this case you want to configure the `ConnectorServer` with a `JMXServiceURL` that binds the RMI server and the RMI registry to the loopback interface only: `service:jmx:rmi://localhost:1099/jndi/rmi://localhost:1099/jmxrmi`.

Then you setup the local port forwarding with the SSH tunnel:

    $ ssh -L 1099:localhost:1099 <user>@<machine_host>

Now you can use JConsole or JMC to connect to `localhost:1099` on your local computer. The traffic will be forwarded to `machine_host` and when there, SSH will forward the traffic to `localhost:1099`, which is exactly where the `ConnectorServer` listens.

When you configure `ConnectorServer` in this way, you must set the system property `-Djava.rmi.server.hostname=localhost`, on the server.

This is required because when the RMI server is exported, its address and port are stored in the RMI stub. You want the address in the RMI stub to be `localhost` so that when the RMI stub is downloaded to the remote client, the RMI communication will go through the SSH tunnel.

Previous | Top | Next
Using Annotations with Jetty Embedded | Home | Jetty JMX Annotations

See an error or something missing? Contribute to this documentation at Github!(Generated: 2019-08-13)