Roller Telemetry Handoff Specification: ECS JSON Logging & Prometheus Metrics

Normative Handoff Specification for Coding Agents. This document defines the exact implementation plan for bringing Apache Roller (~/source/roller, service="roller", service_instance="roller-prod") onto the Simple Agility Telemetry Contract. It fulfills items R1 through R5 in SimpleAgilityTelemetryAdoption and satisfies architectural decisions SA-5 (instance labeling) and SA-6 (Elastic Common Schema).

flowchart TD
    subgraph Client_Requests["Inbound Traffic"]
        REQ["HTTP Request<br/>(Optional: traceparent or X-Correlation-Id)"]
    end

    subgraph Roller_Container["Roller Spring Boot Application (roller-app-1)"]
        TCF["TraceCorrelationFilter<br/>(Extracts / Mints trace.id & transaction.id)"]
        MDC["Log4j2 ThreadContext<br/>(trace.id, transaction.id, http.request.method, url.path)"]
        APP["Application Core & Controllers<br/>(Business Logic & MCP Server)"]
        LOG4J["Log4j2 JsonTemplateLayout<br/>(classpath:EcsLayout.json, service.name='roller')"]
        ACT["Spring Boot Actuator<br/>(Port 8090, /actuator/prometheus)"]
        
        REQ --> TCF
        TCF -->|Enrich| MDC
        TCF --> APP
        APP -->|Log Event| LOG4J
    end

    subgraph Observability_Plane["Observability Ingestion (Host docker2)"]
        STDOUT["Docker Stdout Stream<br/>(ECS JSON One-Line Objects)"]
        ALLOY_LOGS["Grafana Alloy (Logs Scrape)<br/>(Relabels service='roller', service_instance='roller-prod')"]
        ALLOY_PROM["Grafana Alloy (Metrics Scrape)<br/>(Scrapes 127.0.0.1:8090/actuator/prometheus)"]
        LOKI[("Loki (Central Ingestion)")]
        PROM[("Prometheus (Central Ingestion)")]
        
        LOG4J --> STDOUT
        STDOUT --> ALLOY_LOGS
        ALLOY_LOGS --> LOKI
        ACT --> ALLOY_PROM
        ALLOY_PROM --> PROM
    end

1. Maven Dependency Bill of Materials (app/pom.xml)

Roller uses Spring Boot with Log4j2 and already imports org.apache.logging.log4j:log4j-bom (${log4j2.security.version} = 2.26.1). Two dependencies must be added to app/pom.xml:

1.1 Log4j2 JSON Template Layout (For ECS Layout)

Add under <dependencies> in app/pom.xml:

<!-- Supplies EcsLayout.json template and JsonTemplateLayout -->
<dependency>
    <groupId>org.apache.logging.log4j</groupId>
    <artifactId>log4j-layout-template-json</artifactId>
</dependency>

(Version is managed by the existing log4j-bom import; do not specify a hardcoded version).

1.2 Micrometer Prometheus Registry

Roller already includes spring-boot-starter-actuator. To enable Prometheus format export, add:

<!-- Prometheus metrics registry for Spring Boot Actuator -->
<dependency>
    <groupId>io.micrometer</groupId>
    <artifactId>micrometer-registry-prometheus</artifactId>
</dependency>

2. Log4j2 ECS Configuration (app/src/main/resources/log4j2.xml)

2.1 The Console Appender

Replace the existing Console appender in app/src/main/resources/log4j2.xml with the ECS JSON template layout:

<Console name="console" target="SYSTEM_OUT">
    <JsonTemplateLayout eventTemplateUri="classpath:EcsLayout.json">
        <EventTemplateAdditionalField key="service.name" value="roller"/>
    </JsonTemplateLayout>
</Console>

2.2 Critical Logger Appender Trap (Additivity Notice)

In Roller's default log4j2.xml, logger categories (e.g. org.apache.roller, roller.sso, roller.comments, roller.admin) specify:

<Logger name="org.apache.roller" level="info" additivity="false">
    <AppenderRef ref="asyncRoller"/>
</Logger>
IMPORTANT

Because additivity="false" is set and asyncRoller writes only to file (${logging.dir}/roller.log), messages sent to org.apache.roller will not reach console (stdout) unless explicitly referenced.

In containerized deployments (Docker), the observability plane (jakemon) captures logs from stdout/stderr, not local disk files. To ensure application logs reach Docker stdout in ECS format, add <AppenderRef ref="console"/> to each primary logger or define an asynchronous console appender:

<Async name="asyncConsole">
    <AppenderRef ref="console"/>
</Async>

<Loggers>
    <!-- Ensure both file and stdout appenders are referenced -->
    <Logger name="org.apache.roller" level="info" additivity="false">
        <AppenderRef ref="asyncRoller"/>
        <AppenderRef ref="console"/>
    </Logger>
    <Logger name="roller.sso" level="info" additivity="false">
        <AppenderRef ref="asyncRoller"/>
        <AppenderRef ref="console"/>
    </Logger>
    <Logger name="roller.comments" level="info" additivity="false">
        <AppenderRef ref="asyncRoller"/>
        <AppenderRef ref="console"/>
    </Logger>
    <Logger name="roller.admin" level="info" additivity="false">
        <AppenderRef ref="asyncRoller"/>
        <AppenderRef ref="console"/>
    </Logger>
    
    <Root level="info">
        <AppenderRef ref="console"/>
    </Root>
</Loggers>

3. Request Correlation Filter (TraceCorrelationFilter.java)

To support end-to-end tracing across the Simple Agility ecosystem, create TraceCorrelationFilter.java in app/src/main/java/org/apache/roller/weblogger/ui/core/filters/:

3.1 Trace Extraction & Precedence

  1. W3C traceparent Header: Check for standard W3C header traceparent (format: 00-<32-hex-trace-id>-<16-hex-parent-id>-<flags>). Extract the 32-hex character trace-id.
  2. X-Correlation-Id Header: If traceparent is absent, check for incoming X-Correlation-Id.
  3. Generated ID: If neither is present, generate a clean 32-hex-character string (UUID.randomUUID().toString().replace("-", "")).

3.2 ThreadContext Injection & Cleaning

Inject the following fields into Log4j's ThreadContext:

3.3 Reference Implementation

package org.apache.roller.weblogger.ui.core.filters;

import jakarta.servlet.Filter;
import jakarta.servlet.FilterChain;
import jakarta.servlet.ServletException;
import jakarta.servlet.ServletRequest;
import jakarta.servlet.ServletResponse;
import jakarta.servlet.http.HttpServletRequest;
import jakarta.servlet.http.HttpServletResponse;
import org.apache.logging.log4j.ThreadContext;
import org.springframework.core.Ordered;
import org.springframework.core.annotation.Order;
import org.springframework.stereotype.Component;

import java.io.IOException;
import java.util.UUID;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

@Component
@Order(Ordered.HIGHEST_PRECEDENCE)
public class TraceCorrelationFilter implements Filter {

    private static final Pattern TRACEPARENT_PATTERN = 
        Pattern.compile("^[0-9a-f]{2}-([0-9a-f]{32})-[0-9a-f]{16}-[0-9a-f]{2}$");

    @Override
    public void doFilter(ServletRequest req, ServletResponse res, FilterChain chain)
            throws IOException, ServletException {
        if (!(req instanceof HttpServletRequest request) || !(res instanceof HttpServletResponse response)) {
            chain.doFilter(req, res);
            return;
        }

        String traceId = resolveTraceId(request);
        String transactionId = UUID.randomUUID().toString();

        ThreadContext.put("trace.id", traceId);
        ThreadContext.put("transaction.id", transactionId);
        ThreadContext.put("http.request.method", request.getMethod());
        ThreadContext.put("url.path", request.getRequestURI());
        
        String ip = request.getHeader("X-Forwarded-For");
        ThreadContext.put("client.ip", ip != null ? ip.split(",")[0].trim() : request.getRemoteAddr());
        
        String ua = request.getHeader("User-Agent");
        if (ua != null) {
            ThreadContext.put("user_agent.original", ua);
        }

        // Propagate in response headers for client tracing
        response.setHeader("X-Correlation-Id", traceId);
        response.setHeader("traceparent", "00-" + traceId + "-" + transactionId.replace("-", "").substring(0, 16) + "-01");

        try {
            chain.doFilter(request, response);
        } finally {
            ThreadContext.clearMap();
        }
    }

    private String resolveTraceId(HttpServletRequest request) {
        String tp = request.getHeader("traceparent");
        if (tp != null) {
            Matcher m = TRACEPARENT_PATTERN.matcher(tp.trim().toLowerCase());
            if (m.matches()) {
                return m.group(1);
            }
        }
        String corr = request.getHeader("X-Correlation-Id");
        if (corr != null && !corr.isBlank()) {
            return corr.trim();
        }
        return UUID.randomUUID().toString().replace("-", "");
    }
}

4. Actuator & Prometheus Configuration (application.properties)

In app/src/main/resources/application.properties:

Roller already isolates its management server to port 8090 (management.server.port=8090). Configure Actuator exposure as follows:

# Management & Observability configuration
management.server.port=8090
management.endpoints.web.exposure.include=health,info,prometheus
management.endpoint.health.show-details=when_authorized
management.endpoint.prometheus.enabled=true

# Metric tags
management.metrics.tags.application=roller
management.metrics.tags.service=roller
NOTE

Actuator metrics will be served on internal port 8090 at path /actuator/prometheus. Port 8090 is strictly accessible on localhost / container network and is not exposed to public Cloudflare ingress.


5. Observability Plane Scrape Configuration (jakemon)

Once Roller exposes /actuator/prometheus on port 8090, jakemon's Alloy agent on host docker2 (hosts/docker2/apps.alloy) configures the scrape target:

// Roller production Prometheus scrape (Port 8090)
prometheus.scrape "roller_prod" {
  targets = [{
    __address__ = "127.0.0.1:8090",
  }]()
  metrics_path    = "/actuator/prometheus"
  scrape_interval = "60s"
  forward_to      = [prometheus.remote_write.central.receiver]

  extra_metrics = {
    service          = "roller",
    service_instance = "roller-prod",
  }
}

6. Verification and Acceptance Criteria

When a coding agent implements these items in ~/source/roller, the implementation must pass the following checks:

Check 1: ECS Console Log Verification

Run Roller locally or inspect Docker stdout:

# Verify log lines are valid JSON and contain ECS required keys
docker logs roller-app-1 2>&1 | tail -n 20 | jq .

Must produce lines containing:

Check 2: Traceparent Propagation

Send a request with a known traceparent:

curl -v -H "traceparent: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01" http://localhost:8080/

Verify:

  1. HTTP response carries X-Correlation-Id: 4bf92f3577b34da6a3ce929d0e0e4736.
  2. Loki query {service_instance="roller-prod"} | json | trace_id="4bf92f3577b34da6a3ce929d0e0e4736" matches the exact log line.

Check 3: Prometheus Metrics Scraping

Query port 8090:

curl -s http://localhost:8090/actuator/prometheus | grep "jvm_memory_used_bytes"

Must return standard Prometheus metrics.

Check 4: Conformance Script Gate

Run the conformance script from the Wikantik repo:

~/source/jspwiki/bin/simple-agility-conformance.sh

All Roller log and metric checks must report PASS.


See Also