Overview

Axis2 2.0.1 includes streaming message formatters for JSON responses. These formatters wrap the transport OutputStream with a FlushingOutputStream that pushes data to the HTTP layer every 64 KB (configurable), converting a single buffered response into a stream of HTTP/2 DATA frames or HTTP/1.1 chunked transfer encoding segments.

Both GSON and Moshi variants are provided as drop-in replacements for their respective base formatters. No service code changes are required.

Problem - Large HTTP Responses and Reverse Proxies

When an Axis2 service returns a large JSON response (hundreds of MB), the default formatter serializes the entire response into memory before writing it to the wire. A reverse proxy (nginx, AWS ALB, or similar) may reject the response due to body-size limits or buffering timeouts, returning 502 Bad Gateway to the client even though the Axis2 service completed successfully.

The streaming formatter eliminates this by flushing incrementally during GSON/Moshi serialization. The proxy never sees the full response body as a single buffer; it forwards chunks as they arrive.

Request path (client to server): The streaming formatter operates on the response path only. For large HTTP POST request bodies, note that neither HTTP/1.1 (RFC 9110) nor HTTP/2 (RFC 9113) define a limit on request body size. HTTP/2 is actually better suited for large requests because the body is sent as DATA frames with flow control — the sender and receiver negotiate how much data to send at a time, preventing buffer overflow. In practice, size rejections come from infrastructure layers, not the HTTP spec:

• Reverse proxies (CloudFlare, nginx) — client_max_body_size or equivalent; increase or remove the limit
• Load balancers (AWS ALB/NLB) — ALB imposes no body size limit when routing to EC2/ECS targets. The 1MB limit applies only to the ALB→Lambda integration path (an AWS hard limit that cannot be increased). NLB operates at TCP level and has no body size limit
• Web servers (Tomcat, WildFly) — maxPostSize / max-post-size; set to -1 for unlimited

When a large POST is rejected, debug the specific error and the infrastructure layer that imposed the limit — the fix is usually a configuration change, not a code refactor. Breaking requests into smaller payloads is a last resort when the infrastructure limit cannot be changed.

Available Variants

Both variants share the same FlushingOutputStream implementation in the org.apache.axis2.json.streaming package.

Configuration

<!-- Recommended: streaming + field filtering (wraps MoshiStreamingMessageFormatter) -->
<messageFormatter contentType="application/json"
    class="org.apache.axis2.json.streaming.FieldFilteringMessageFormatter"/>

<!-- OR: streaming only (Moshi, no field filtering) -->
<messageFormatter contentType="application/json"
    class="org.apache.axis2.json.streaming.MoshiStreamingMessageFormatter"/>

<!-- OR: streaming only (GSON variant) -->
<messageFormatter contentType="application/json"
    class="org.apache.axis2.json.streaming.JSONStreamingMessageFormatter"/>

GET /services/MyService?fields=status,result
# Response: {"response":{"status":"SUCCESS","result":0.0245}}
# (other top-level fields omitted)

// The response POJO — what the Axis2 service method returns
public class ServiceResponse {
    private String status;
    private long responseTimeMs;
    private Map<String, Object> data;   // parsed from backend JSON
}

// At runtime, "data" contains:
//   "records" -> List<Map<String, Object>>   (100+ fields per element)
//   "metadata" -> Map<String, Object>
//   "diagnostics" -> Map<String, Object>

response.getData()                 // Map<String, Object>  = "data"
    .get("records")                // List<Map>            = "data.records"
    .get(0).get("id")              // Object               = "data.records.id"

?fields=status,data.records.id,data.records.name
   |       |         |          |
   |       |         |          +-- 3rd level: key inside each List element
   |       |         +------------- 2nd level: key inside the data Map
   |       +----------------------- 1st level: field on the response POJO
   +------------------------------- top-level POJO field (no dots)

{"response": {
    "status": "SUCCESS",
    "data": {
        "records": [
            {"id":"item-1", "name":"Widget A", ... 125 more fields ...},
            {"id":"item-2", "name":"Widget B", ... 125 more fields ...}
        ],
        "metadata": {...},
        "diagnostics": {...}
    }
}}

{"response": {
    "status": "SUCCESS",
    "data": {
        "records": [
            {"id":"item-1"},
            {"id":"item-2"}
        ]
    }
}}

<parameter name="streamingFlushIntervalBytes">131072</parameter>

How It Works

The streaming formatter is structurally identical to the standard GSON/Moshi formatter with one difference: before creating the JsonWriter, it wraps the transport OutputStream with a FlushingOutputStream:

// Inside writeTo():
OutputStream flushingStream = new FlushingOutputStream(outputStream, flushInterval);
// ... GSON/Moshi serialization proceeds normally against the flushing stream

During serialization, every time the accumulated bytes exceed the flush interval, the FlushingOutputStream calls flush() on the underlying transport stream. This triggers the servlet container (Tomcat, WildFly/Undertow) to send the buffered bytes as an HTTP/2 DATA frame or HTTP/1.1 chunk. The reverse proxy receives and forwards each chunk independently.

The three serialization paths (fault response, element response, and object response) all benefit from flushing without any path-specific changes.

Services That Benefit

The streaming formatter applies to all Axis2 JSON-RPC services in the deployment. Any service that returns a large JSON response benefits transparently:

• BigDataH2Service — enterprise big data processing with large record sets. The streaming formatter prevents proxy rejections as response sizes grow into the hundreds of MB range.
• FinancialBenchmarkService — portfolio variance, Monte Carlo VaR, and scenario analysis. Responses are typically small (1-10 KB) but the formatter operates transparently with no overhead on small payloads.
• Any custom service — services deployed as .aar archives benefit without code changes once the formatter is configured in axis2.xml.

Testing

The streaming formatter has been tested on:

• WildFly 32 (local) — all services produce valid JSON
• WildFly 32 behind a reverse proxy (HTTP/2 ALPN on port 8443) — all services produce bit-identical results compared to the non-streaming formatter

To verify the formatter is active, enable DEBUG logging for org.apache.axis2.json.streaming and look for:

MoshiStreamingMessageFormatter: using FlushingOutputStream with 65536 byte flush interval

Axis2/C Equivalent

Axis2/C achieves the same streaming behavior natively through Apache httpd's mod_h2. During JSON response generation, the C service calls ap_rflush(r) periodically to flush the response bucket brigade. This causes mod_h2 to emit HTTP/2 DATA frames incrementally — the same 64KB chunked pattern as the Java formatter, with identical proxy behavior. Both implementations produce matching HTTP/2 frame sequences for the same payload.