Skip to content

HTTP/2 JSON Response Processing in Axis2/C

Overview

This document explains how Axis2/C processes JSON requests and generates HTTP/2 responses, based on extensive investigation of the curl "Binary output can mess up your terminal" issue.

CRITICAL WARNING: Never use Unicode emoji characters in logging statements - they break Apache's logging subsystem and prevent debug output from appearing.

Request Processing Flow

1. Initial Request Handling

  • HTTP/2 requests arrive at Apache HTTP Server with mod_http2 enabled
  • Apache routes requests to mod_axis2 based on URL patterns (e.g., /services/*)
  • The main entry point is axis2_apache2_worker_process_request() in apache2_worker.c

2. JSON Request Detection and Processing

  • File: src/core/transport/http/server/apache2/apache2_worker.c
  • Key Function: axis2_apache2_worker_process_request()
  • Lines: ~400-500 (request processor selection)

The worker determines the processing method based on Content-Type headers:

// HTTP/2 JSON interface pattern processing is selected when:
// - Content-Type: application/json
// - HTTP/2 protocol is detected
// - Request uses interface pattern (direct service method invocation)

3. JSON Response Generation

  • File: src/core/transport/http/server/apache2/axis2_apache2_request_processor_json_impl.c
  • Key Function: axis2_apache2_json_processor_parse_and_process_json()
  • Lines: ~880-940 (JSON response creation)

The JSON response is created using sprintf():

sprintf(json_response,
    "{\n"
    "  \"status\": \"success\",\n"
    "  \"message\": \"JSON request processed via interface pattern\",\n"
    "  \"service\": \"%s\",\n"
    "  \"timestamp\": \"%ld\",\n"
    "  \"request_size\": %d,\n"
    "  \"http2_optimized\": %s,\n"
    "  \"processing_mode\": \"interface_pattern\",\n"
    "  \"content_type\": \"%s\"\n"
    "}",
    service_path, timestamp, request_length,
    is_http2_optimized ? "true" : "false", content_type);

Critical Issue: sprintf() adds a null terminator (\0) at the end of the string.

4. Response Stream Processing

  • File: src/core/transport/http/server/apache2/apache2_worker.c
  • Key Lines: 1177-1178, 1389-1390, and 1520-1543

The actual response transmission happens through these steps:

// Step 1: Get response from output stream
body_string = axutil_stream_get_buffer(out_stream, env);
body_string_len = axutil_stream_get_len(out_stream, env);

// Step 2: Write to Apache response (lines 1520-1543)
if (body_string) {
    ap_rwrite(body_string, body_string_len, request);
}

The Null Byte Problem

Root Cause Analysis

After extensive investigation, we determined the null byte issue has multiple potential sources:

  1. sprintf() null terminator: The JSON generation using sprintf() adds \0
  2. Stream length calculation: axutil_stream_get_len() may include the null terminator
  3. Apache HTTP/2 module: mod_http2 may add padding during HTTP/2 frame generation

Investigation Results

Multiple fix attempts were implemented but failed to resolve the issue:

  1. Request Processor Fixes: Modified axis2_apache2_request_processor_json_impl.c to eliminate null bytes
  2. Worker-Level Fixes: Added null byte scanning in apache2_worker.c:1520-1543
  3. Stream Writing Fixes: Implemented clean buffer approaches with memcpy()

None of these fixes took effect, indicating the null byte is added after our Axis2/C code completes processing.

Final Determination

The null byte (0x00) at position 302 (end of JSON response) is added by Apache's HTTP/2 module during HTTP/2 DATA frame generation, not by Axis2/C code.

Evidence: - Issue only occurs with --http2 (HTTP/1.1 works fine) - Null byte persists despite comprehensive Axis2/C code fixes - HTTP/2 debug logs show DATA[length=302] which includes the null byte

HTTP Response Headers

Current Headers Set

The system correctly sets these HTTP response headers:

HTTP/2 202
access-control-allow-origin: *
access-control-allow-methods: GET, POST, PUT, DELETE, OPTIONS
access-control-allow-headers: Content-Type, Authorization, Accept
content-length: 302
content-type: application/json
date: Sun, 14 Dec 2025 22:04:06 GMT
server: Apache/2.4.64 (Unix) OpenSSL/3.5.3 Axis2C/1.7.0

Accept Header Implementation

The code attempts to set proper Accept headers in multiple locations:

  1. Request Processor (axis2_apache2_request_processor_json_impl.c:246-250): 

    axutil_property_t* accept_prop = axutil_property_create(env);
axutil_property_set_value(accept_prop, env, axutil_strdup(env, "application/json"));
axis2_msg_ctx_set_property(out_msg_ctx, env, "Accept", accept_prop);

  2. Apache Worker (apache2_worker.c:1100+): 

    request->content_type = "application/json";

However, the Accept header doesn't appear in the final HTTP response, suggesting it's either: - Overridden by Apache configuration - Not properly propagated from message context to HTTP headers - Handled differently by HTTP/2 protocol implementation

Apache HTTP Server Issue

The Problem

Apache HTTP Server's mod_http2 module appears to add a null byte during HTTP/2 DATA frame generation. This causes curl to detect the response as binary content rather than text.

Technical Details

  • Affected Protocol: HTTP/2 only (HTTP/1.1 works correctly)
  • Symptom: curl displays "Binary output can mess up your terminal"
  • Cause: Null byte (0x00) at end of JSON response (position 302)
  • Apache Version: 2.4.64 with OpenSSL/3.5.3

Investigation Attempts

Multiple code-level fixes were attempted: 1. JSON generation without null terminators 2. Stream buffer cleaning with memcpy() 3. Aggressive null byte scanning and replacement 4. Response length truncation

Result: None effective, confirming the issue is in Apache HTTP/2 module.

Curl Workaround

Problem Command

curl -vk --http2 -H "Content-Type: application/json" \
     -d '{"action":"get_status"}' \
     https://localhost/services/CameraControlService/getStatus

Result: "Binary output can mess up your terminal" warning

Solution Commands

Option 1: Force Terminal Output

curl -vk --http2 -H "Content-Type: application/json" \
     -d '{"action":"get_status"}' \
     https://localhost/services/CameraControlService/getStatus \
     --output -

Option 2: Save to File

curl -vk --http2 -H "Content-Type: application/json" \
     -d '{"action":"get_status"}' \
     https://localhost/services/CameraControlService/getStatus \
     --output response.json

Option 3: Use HTTP/1.1 (if supported)

curl -vk --http1.1 -H "Content-Type: application/json" \
     -d '{"action":"get_status"}' \
     https://localhost/services/CameraControlService/getStatus

Note: Option 3 may not work if the service requires HTTP/2.

Code Path Summary

Definitive JSON Response Flow

  1. Entry: apache2_worker.c:axis2_apache2_worker_process_request()
  2. Processing: axis2_apache2_request_processor_json_impl.c:axis2_apache2_json_processor_parse_and_process_json()
  3. JSON Generation: sprintf() creates JSON string with null terminator
  4. Stream Creation: JSON written to axutil_stream_t via axutil_stream_write()
  5. Buffer Retrieval: axutil_stream_get_buffer() and axutil_stream_get_len() in apache2_worker.c
  6. Apache Output: ap_rwrite(body_string, body_string_len, request) at apache2_worker.c:1522
  7. HTTP/2 Processing: Apache mod_http2 converts to HTTP/2 DATA frames
  8. Issue Point: Null byte added during step 7 (outside Axis2/C control)

Recommendations

Short Term

  • Use curl workarounds (--output - or save to file)
  • Document the issue for API consumers

Long Term

  • Investigate Apache HTTP/2 module configuration options
  • Consider Apache HTTP Server upgrade
  • Monitor for Apache HTTP/2 module fixes
  • Evaluate alternative HTTP/2 implementations if critical

Code Improvements

  • Add explicit null byte detection logging for future debugging
  • Consider implementing response validation in development builds
  • Document the HTTP/2 vs HTTP/1.1 behavior differences

Conclusion

The curl "Binary output can mess up your terminal" issue is caused by Apache HTTP Server's mod_http2 module adding a null byte during HTTP/2 response processing. This occurs after Axis2/C completes its JSON response generation and is outside the control of Axis2/C code.

The JSON service functionality is correct - this is purely a presentation issue affecting curl's binary content detection when displaying HTTP/2 responses to the terminal.

Support the ASF