Is your feature request related to a problem? Please describe.
As a developer of embedded products using NetX Duo, my target platform is always constrained when it comes to RAM and I am not able to handle a maximally sized MQTT message (256 MB + max 5 bytes of fixed header). Typically, the client-server protocol will be based on messages maximally a few kB in size.
But if there is some bug in the broker or if the device needs to connect to brokers that the device vendor does not have full control over - e.g. a third party broker for telemetry aggregation or home automation hub - there is a risk that a large message anyway comes over the wire.
Worst case this will lead to NetX Duo packet pool exhaustion when all packets in the pool are allocated while the MQTT message is not yet complete (these are the packets queued in NXD_MQTT_CLIENT .nxd_mqtt_client_processing_packet).
In the slightly less bad case, nxd_mqtt_client_message_get() will return NXD_MQTT_INSUFFICIENT_BUFFER_SPACE. But there is no information on how much more is needed and in practise it anyway means that the size is above what the application layer reasonably expects. In this case, the message stays in the receive queue and will be "returned" over and over again until the application decides to drop the connection.
Even though it can be argued that the broker or the original publisher is violating the application level protocol, which reasonably sets some maximum expected size, the device with NetX Duo will be blamed for becoming unresponsive.
If TLS is not used, injecting such a message into the TCP stream can also be used as the base of a DOS-attack.
Describe the solution you'd like
I would like a parameter to nxd_mqtt_client_connect() to set the maximum size I am prepared to accept.
If an MQTT message arrives where remaining length exceeds this, nxd_mqtt_client.c will just read the packets from the TCP layer and throw them away until remaining length is exhausted and the next message begins.
An extra feature would be a call-back when this happens with maybe message type and size so that the application can log that some traffic will not be handled or other diagnostics.
Describe alternatives you've considered
There is a call-back nxd_mqtt_packet_receive_notify that can signal that a packet is consumed but to me it seems it would require the application to duplicate much of the MQTT parsing logic, handling of MQTT headers that split a packet boundary etc.
Another approach would be have a message received watchdog (perhaps based on MQTT ping) and just reboot the device if there hasn't been a successful message received in whatever time is reasonable for the business logic. This is a bit brutal and will leave the broker with dead sockets for a while until they are cleared out.
Open issues
If the message is discarded it will not be ACKed. So if the subscription is with QoS > 0, the broker will resend it on next connection (unless the client sets the Clean Session flag) and this could make the system end up in a loop. Perhaps it should be supported to store the message ID so that it can be acked even if this is lying to the broker about the message status?
Is your feature request related to a problem? Please describe.
As a developer of embedded products using NetX Duo, my target platform is always constrained when it comes to RAM and I am not able to handle a maximally sized MQTT message (256 MB + max 5 bytes of fixed header). Typically, the client-server protocol will be based on messages maximally a few kB in size.
But if there is some bug in the broker or if the device needs to connect to brokers that the device vendor does not have full control over - e.g. a third party broker for telemetry aggregation or home automation hub - there is a risk that a large message anyway comes over the wire.
Worst case this will lead to NetX Duo packet pool exhaustion when all packets in the pool are allocated while the MQTT message is not yet complete (these are the packets queued in
NXD_MQTT_CLIENT .nxd_mqtt_client_processing_packet).In the slightly less bad case,
nxd_mqtt_client_message_get()will returnNXD_MQTT_INSUFFICIENT_BUFFER_SPACE. But there is no information on how much more is needed and in practise it anyway means that the size is above what the application layer reasonably expects. In this case, the message stays in the receive queue and will be "returned" over and over again until the application decides to drop the connection.Even though it can be argued that the broker or the original publisher is violating the application level protocol, which reasonably sets some maximum expected size, the device with NetX Duo will be blamed for becoming unresponsive.
If TLS is not used, injecting such a message into the TCP stream can also be used as the base of a DOS-attack.
Describe the solution you'd like
I would like a parameter to
nxd_mqtt_client_connect()to set the maximum size I am prepared to accept.If an MQTT message arrives where remaining length exceeds this,
nxd_mqtt_client.cwill just read the packets from the TCP layer and throw them away until remaining length is exhausted and the next message begins.An extra feature would be a call-back when this happens with maybe message type and size so that the application can log that some traffic will not be handled or other diagnostics.
Describe alternatives you've considered
There is a call-back
nxd_mqtt_packet_receive_notifythat can signal that a packet is consumed but to me it seems it would require the application to duplicate much of the MQTT parsing logic, handling of MQTT headers that split a packet boundary etc.Another approach would be have a message received watchdog (perhaps based on MQTT ping) and just reboot the device if there hasn't been a successful message received in whatever time is reasonable for the business logic. This is a bit brutal and will leave the broker with dead sockets for a while until they are cleared out.
Open issues
If the message is discarded it will not be ACKed. So if the subscription is with QoS > 0, the broker will resend it on next connection (unless the client sets the Clean Session flag) and this could make the system end up in a loop. Perhaps it should be supported to store the message ID so that it can be acked even if this is lying to the broker about the message status?