fix: send Malformed Packet CONNACK for CONNECT reserved bit=1 in MQTT 5.0

[liooooo29]

Problem

When a MQTT 5.0 CONNECT packet has the reserved bit (bit 0 of Connect Flags) set to 1, Mosquitto silently closes the connection without sending a CONNACK packet.

Per the MQTT 5.0 OASIS specification §3.1.2.3, bit 0 of the Connect Flags byte is reserved and must be set to 0.

Current Behavior

// read_and_verify_connect_flags() in handle_connect.c
if((*connect_flags & 0x01) != 0x00){
    log__printf(NULL, MOSQ_LOG_INFO, "Protocol error ...");
    return MOSQ_ERR_PROTOCOL;  // Connection closed without CONNACK
}

Expected Behavior

Per [MQTT-3.1.2-3] and §4.13, the server should send a CONNACK with Reason Code 0x81 (Malformed Packet) before closing the connection.

Fix

if((*connect_flags & 0x01) != 0x00){
    log__printf(NULL, MOSQ_LOG_INFO, "Protocol error ...");
    if(context->protocol == mosq_p_mqtt5){
        send__connack(context, 0, MQTT_RC_MALFORMED_PACKET, NULL);
    }
    return MOSQ_ERR_MALFORMED_PACKET;
}

Test Case

Verified with an automated MQTT 5.0 conformance test (T-70) that sends CONNECT with reserved bit=1 and expects a CONNACK with Reason Code 0x81.

References

• MQTT 5.0 OASIS Standard (2019-03-07), §3.1.2.3: Connect Flags
• [MQTT-3.1.2-3]: The Server MUST validate that the reserved flag is set to 0