FreeSWITCH mod_xml_rpc WebSocket Frame Length Truncation Heap Out-of-Bounds Write (CVE-2026-116785)

FreeSWITCH mod_xml_rpc WebSocket Frame Length Truncation Heap Out-of-Bounds Write (CVE-2026-116785)

On October 28, 2026, the FreeSWITCH repository posted GHSA-jj89-q47c-gh94, corresponding to CVE-2026-116785, CVSS 3.1 9.1 (AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:H/A:H), CWE-122 / CWE-681 / CWE-787, affected < 1.11.2, fixed >= 1.11.2. The reporter is Tristan Madani (Talence Security), and the advisory was published by andywolk. The GHSA page lists neither the hash of the fixing commit nor a Patch section; NVD's references mark d557ca6d29e6da4f10824dc158100906a592b91a directly as Patch (source is [email protected]), and it does indeed match the file entity of v1.11.2.

The advisory's conclusion can be compressed into three sentences: ws_read_frame() decodes the 64-bit extended length with a 32-bit byte swap, and the result lands in a signed 32-bit variable; when the highest bit of the surviving half-word is 1, it becomes negative, bypassing the signed comparison "length > buffer size", and is then treated by the buffer-filling loop as a huge unsigned count; the write goes past the 64 KiB receive buffer, and remote code execution has not been demonstrated.

All three sentences are correct, but the reader cannot get "which line, why, and exactly how many places were changed". This article fills in that gap: laying out the dozen or so lines of ws_read_frame() from fetching the frame header to memcpy one by one, computing "why a negative number can fool two checks at once" down to the specific byte values, and then going through the fix side hunk by hunk — after which you find the fix is not one commit: the same function was changed three times in the same afternoon, while the release notes give only one semantic description. There are two more details the advisory does not mention: the conf/minimal configuration set actually loads mod_xml_rpc by default; and the 1.10 branch still has no corresponding fixed release.

Verification boundaries: source taken from signalwire/freeswitch tag v1.11.1 (c2c59645f6, 2026-05-26); the fix side taken from the API diffs of three commits and the file entity of tag v1.11.2 (3f13ad1b1d, 2026-08-09); the numerical path recomputed with a script following C expression semantics. This article did not compile FreeSWITCH, nor did it send a single packet to a running service — any sentence involving runtime behavior is marked as derived from source.

1. Exposure surface: two switches, one exception

mod_xml_rpc is the module that connects FreeSWITCH's command bus to XML-RPC/HTTP; /RPC2 exposes the freeswitch.api and freeswitch.management methods (mod_xml_rpc.c:1228-1231), and /socket exposes a WebSocket upgrade endpoint. This module is not loaded by default — conf/vanilla/autoload_configs/modules.conf.xml:14 is a commented line:

        <!-- <load module="mod_xml_rpc"/> -->
      

The same line (each at :14) is also commented in the insideout and sbc configuration sets. But conf/minimal is not. conf/minimal/autoload_configs/modules.conf.xml:9 is an uncommented <load module="mod_xml_rpc"/>, neatly placed under the <!-- XML Interfaces --> section:

        <!-- XML Interfaces -->
    <load module="mod_xml_rpc"/>
      

This is not an isolated typo-like case; it looks like a design trade-off — "the minimal configuration set should keep a door open that can run commands". It means the statement "the module is off by default" holds only for the vanilla/insideout/sbc configuration sets; on an instance booted with minimal, the first switch is already on.

The second switch is the WebSocket endpoint itself. enable-websocket does not appear even once in the xml_rpc.conf.xml of the five conf trees (grep file by file, count is 0 everywhere); only the module's own sample config src/mod/xml_int/mod_xml_rpc/conf/autoload_configs/xml_rpc.conf.xml:9 leaves a commented line:

        <!-- <param name="enable-websocket" value="true"/> -->
      

The parsing code is at mod_xml_rpc.c:134-135, the value goes through switch_true(); because globals is memset to zero when the module loads (:172), the default is off. So the full condition for "can be reached" is: the module is loaded and enable-websocket is explicitly set to true. Once both are satisfied, the service listens on the module's http-port (:128-129, default 8080, :143-145), and the bind address comes from abyss's socket_unix.c:887:

        name.sin_addr.s_addr = INADDR_ANY;
      

i.e. 0.0.0.0. The bar for both steps is low — edit two files and restart once. So the real risk of this hole is not "every FreeSWITCH can be hit", but "any instance that has ever opened this module to build a monitoring panel, integrate with a CTI backend, or run automated dial testing is on the exposure surface, and panel-style usage usually needs websocket to work".

There is one more point that is easy to read backwards: the stock xml_rpc.conf.xml actually already has the auth parameters filled in (conf/vanilla/autoload_configs/xml_rpc.conf.xml:6-8):

        <param name="auth-realm" value="freeswitch"/>
    <param name="auth-user" value="freeswitch"/>
    <param name="auth-pass" value="works"/>
      

That is, an instance started with the default configuration and only those two lines uncommented looks like it "has authentication". The next section explains why these three parameters are completely ineffective for /socket.

2. Auth order: /socket is dispatched before the password check

abyss's handler list is arranged as three items in this module: xmlrpc_server_abyss_set_handler(..., "/RPC2", ...) (mod_xml_rpc.c:1245, via setHandler → ServerAddHandler3, libs/xmlrpc-c/src/xmlrpc_server_abyss.c:747), handler_hook (:819, registered at :1258), auth_hook (:709, registered at :1259). ListAdd appends at the tail (libs/xmlrpc-c/lib/abyss/src/data.c:105-129), but runUserHandler iterates from the end of the list backwards (server.c:565-567), so the execution order is auth_hook→handler_hook→/RPC2: the last-registered auth_hook runs first.

/socket is dispatched in auth_hook's first branch, and handler_hook is never asked before that dispatch. websocket_hook itself never returns TRUE (:603, :604, :706 are all FALSE), so after the WS session ends and it returns FALSE, the loop does continue down to handler_hook — only by then the connection is already closed, and handler_hook hits the return FALSE at :861 for /socket. This function only recognizes the four substrings /api/, /webapi/, /txtapi/, /xmlapi/ (strstr matching, not prefix comparison, :848-859). The first block of auth_hook is not authentication, it is dispatch:

        if (globals.enable_websocket && !strncmp(r->requestInfo.uri, "/socket", 7)) {
		// Chrome has no Authorization support yet
		// https://code.google.com/p/chromium/issues/detail?id=123862
		return websocket_hook(r);
	}
      

(mod_xml_rpc.c:719-723)

Three things happen at once in these five lines. First, the URI test is a 7-byte prefix comparison: /socket, /socket/, and /socketanything all match. Second, on a match it unconditionally returns websocket_hook(r), and the function does not continue further. Third, the comment states the motivation plainly: browsers cannot send an Authorization header during the WebSocket handshake (it cites chromium issue 123862 from 2011), so this layer simply does not validate — not an omission, a deliberate skip.

Validation comes later. Requests with the /domains/ prefix call http_directory_auth at :744, and the remaining non-/pub URIs call the same function at :780-782:

Article title: FreeSWITCH mod_xml_rpc WebSocket frame length truncation heap out-of-bounds write (CVE-2026-116785)

Body:

        } else {
			if (globals.realm && strncmp(r->requestInfo.uri, "/pub", 4)) {
				ret = !http_directory_auth(r, NULL);
			}
		}
      

(mod_xml_rpc.c:779-783)

So /socket and authentication are two sequential branches in the same function; the former comes first and does not loop back. The concrete scheme at this layer is HTTP Basic, not Digest: http_directory_auth takes the scheme name from the authorization header and compares it with "basic" (:410, :416), first uses switch_b64_decode to decode the credentials into user:pass (:418-421), then reassembles user:pass from auth-user/auth-pass or the user directory and hands it to xmlrpc_base64Encode for encoding, and finally does a strcmp against the original token (encoding at :458, :482; comparison at :460, :484). The challenge returned on failure is

        switch_snprintf(z, sizeof(z), "Basic realm=\"%s\"", domain_name ? domain_name : globals.realm);
	ResponseAddField(r, "WWW-Authenticate", z);
	ResponseStatus(r, 401);
      

(mod_xml_rpc.c:546-548)

There is no digest branch in the code. The GHSA text describes this layer as "HTTP digest checks", which does not match the source: what is implemented here is Basic, the scheme name comparison is strcasecmp(x, "basic") (:416), and the failure response is Basic realm= (:546). This difference does not affect the conclusion—regardless of whether this layer is called Basic or Digest, it comes after return websocket_hook(r)—but it does affect the packet-capture criterion during troubleshooting: what you see in traffic is 401 + Basic realm=, not 401 + nonce.

Conclusion: any client that can establish a TCP connection to 8080 and send a request carrying Upgrade: websocket / Sec-WebSocket-Protocol: websocket / Sec-WebSocket-Key / Sec-WebSocket-Version can get a 101 and then enter ws_read_frame(). The gate is at :603: none of the four headers may be missing (if (!key || !version || !proto || !upgrade) return FALSE;), and :604 further requires the values of upgrade and sec-websocket-protocol to contain websocket (strncasecmp, case-insensitive); a key/version/proto with an empty-string value is blocked at the !key || !version || !*proto check in ws_handshake_kvp (ws.c:231), and an empty upgrade cannot pass :604. Nothing else is validated: it does not check whether the key is valid base64, does not check whether the version is 13, and the Sec-WebSocket-Accept in the 101 response is computed on the fly from the key—whatever the client gives, it signs. Whether auth-realm/auth-user/auth-pass are filled in, and with what, is irrelevant to this path.

III. Frame parsing chain: from 14 bytes to memcpy

After websocket_hook gets the 101, it enters the frame-reading loop (mod_xml_rpc.c:625-626), calling ws_read_frame(wsh, &opcode, &data) each time. The connection handle wsh_t is allocated once via malloc(sizeof(*wsh)) in ws_init (ws.c:336); the struct is defined in ws.h:60-75:

        typedef struct wsh_s {
	ws_tsession_t *tsession;
	char buffer[65536];
	char wbuffer[65536];
	size_t buflen;
	issize_t datalen;
	issize_t wdatalen;
	char *payload;
	issize_t plen;
	issize_t rplen;
	SSL *ssl;
	int handshake;
	uint8_t down;
	int secure;
	uint8_t close_sock;
} wsh_t;
      

issize_t is an alias for int (ws.h:29). The receive buffer buffer and the send buffer wbuffer are both 65536-byte inline arrays—they are not pointers but two contiguous regions within the same heap allocation, and this point determines where the out-of-bounds write lands in Section V.

Entering the function. datalen records the number of bytes read into buffer this time, and need records "how many bytes in total are required according to the current header fields". At the start it grabs 14 bytes:

        if ((wsh->datalen = ws_raw_read(wsh, wsh->buffer, 14)) < need) {
		while (!wsh->down && (wsh->datalen += ws_raw_read(wsh, wsh->buffer + wsh->datalen, 14 - wsh->datalen)) < need) ;
      

(ws.c:406-407)

14 is the maximum fixed-length portion of "2-byte base header + 8-byte extended length + 4-byte mask key"; this number will later determine the size of the minimum attack frame. The fill-in loop only guarantees reaching need (which is 2 at this point) bytes, and once reached it proceeds.

opcode takes the low 4 bits (:417) and is dispatched to the WSOC_TEXT/WSOC_BINARY/WSOC_PING/WSOC_PONG branch (:427-432). Then, in order:

        int mask = (wsh->buffer[1] >> 7) & 1;

if (mask) {
				need += 4;
				...
			}

wsh->plen = wsh->buffer[1] & 0x7f;
			wsh->payload = &wsh->buffer[2];

if (wsh->plen == 127) {
				uint64_t *u64;

need += 8;
				...
				u64 = (uint64_t *) wsh->payload;
				wsh->payload += 8;

wsh->plen = ntohl((u_long)*u64);

} else if (wsh->plen == 126) {
      

(ws.c:434-465, omitting two "header not long enough" checks of need > wsh->datalen)

• The highest bit of buffer[1] is the MASK bit; RFC 6455 requires that frames sent by clients have it set to 1. If set to 1, need += 4, reserving 4 bytes for the mask key.

• buffer[1] & 0x7f is the 7-bit length code, with values 0–127. 126 means "followed by a 2-byte length", and 127 means "followed by an 8-byte length".

• payload first points to buffer[2], i.e., the first byte after the header; in the 127 branch it is further +8 to skip the 8-byte extended length; when masked it is further +4 to skip the mask key (:481-484). Note that payload always points to "the data region within the frame", not "a copy of the data within the frame"—it is a pointer inside wsh->buffer, which is the precondition for the later memcpy being able to write to other fields of the struct.

• The line at :463 is the seed of the entire vulnerability: the formal parameter of ntohl() is uint32_t, so the 64-bit value (u_long)*u64 passed in is truncated to the low 32 bits, then a 32-bit byte-order swap is performed, and the result is assigned to the int-typed plen.

Continuing downward, the remaining-length calculation and the first gate:

        need = (wsh->plen - (wsh->datalen - need));

if ((need + wsh->datalen) > (issize_t)wsh->buflen) {
				/* too big - Ain't nobody got time fo