Replace bundled snmplib with shared libnetsnmp#2454
Conversation
The bundled library is a copy of the long deceased UCD-SNMP implementation which has since been merged into the current NET-SNMP implementation widely available as libnetsnmp. Switch to using NET-SNMP as a shared library and remove this outdated library code from Squid.
|
Draft for now as it needs more testing, particularly of SNMP packet behaviours with multi-process Squid. Squid is expected to build with --enable-snmp fine, and work as before with no visible changes. |
|
|
||
| reset(); | ||
|
|
||
| // XXX: fix this copying for net-snmp struct members: |
There was a problem hiding this comment.
Not sure how many of these fields are relevant to Squids SNMP v1 / v2c implementation.
rousskov
left a comment
There was a problem hiding this comment.
PR description: The bundled library is a copy of the long deceased UCD-SNMP implementation which has since been merged into the current Net-SNMP implementation widely available as libnetsnmp.
Current Squid SNMP code is no longer a copy. We have fixed many problems in previously copied code. This difference is important when deciding how to handle SNMP problems in Squid. Please replace "is" with "was" (at least).
PR description: Switch to using Net-SNMP as a shared library ... Net-SNMP is itself quite outdated in a lot of ways but, is still more up to date than this old Squid code, and there does not appear to be any better SNMP library available.
I support migration to Net-SNMP, even though we would not be able to fix that library when new SNMP vulnerability reports against Squid start to come in. Judging by Net-SNMP activity on GitHub, there will be such reports. Those reports would be against code external to Squid. While that fact does not really help anybody who uses SNMP in Squid, cares about CVEs, but cannot invest in fixing Net-SNMP library, Squid Project does not have the resources to secure its SNMP code. We can only hope that we are going to waste less time on working around Net-SNMP bugs than on maintaining currently embedded SNMP code.
| Must(community != nullptr); | ||
| xfree(community); | ||
| } | ||
| xfree(peername); |
There was a problem hiding this comment.
This PR contains a lot of unnecessary out-of-scope code changes. It is too difficult for me to untangle them from changes actually required to support the new/replacement library. Please remove all unnecessary changes from this PR, including changes that move otherwise-unchanged code around.
There was a problem hiding this comment.
The code has gone through two levels of review before submission. These changes are the result of the "new code matches Squid coding style requirements" review.
There was a problem hiding this comment.
The code has gone through two levels of review before submission. These changes are the result of the "new code matches Squid coding style requirements" review.
Since the end result is problematic, the reviewers made a mistake and/or their suggestions were misinterpreted/misapplied. Can you point me to the reviews you are using to reject my change request?
| { | ||
| switch (type) { | ||
| case SMI_INTEGER: | ||
| case ASN_INTEGER: |
There was a problem hiding this comment.
Can Squid define an SMI_INTEGER and similar constants instead of changing this and similar Squid code using those constants? If the answer is "yes", then we should do that, at least for the library migration PR.
There was a problem hiding this comment.
That would be the bad practice of adding unnecessary technical debt.
The goal of this PR is to remove the old library, not to preserve its dead API forever.
There was a problem hiding this comment.
That would be the bad practice of adding unnecessary technical debt.
We need to balance many evils here. Creating temporary debt may reduce the number of errors and speed up this PR acceptance. If those goals are valuable, then creating that debt can be the lesser evil.
The goal of this PR is to remove the old library, not to preserve its dead API forever.
A "forever" preservation has not been requested. The actual duration of that debt existence depends on many factors. We even have the power to limit it by this PR lifetime (although that may require higher cooperation levels than we were able to achieve in the last few years).
| peername = xstrdup(session.peername); | ||
| } | ||
| if (session.localname) { | ||
| localname = xstrdup(session.localname); |
There was a problem hiding this comment.
A different way is needed to correctly support copying of net-snmp objects. A migration PR should not implement that and should not add copying of members that Squid is not already using (or, where absolutely necessary, their direct equivalents).
For example, why do we need to copy localname in this PR?
There was a problem hiding this comment.
The members of the session struct have changed between the SNMPv1 based library and the SNMPv3 based library. All the protocol-agnostic members need to be copied to preserve session consistency when the SNMPv1/2 is acting.
Only the SNMPv3 and AgentX protocol fields are possibly optional - since those are supposed to be rejected, or at least undefined behaviour from Squid perspective.
There was a problem hiding this comment.
All the protocol-agnostic members need to be copied to preserve session consistency when the SNMPv1/2 is acting
How do you define "session consistency" in this context? Why is singling out localname among many other new fields necessary and sufficient?
I hope that this PR can leave localname and numerous other new snmp_session fields untouched/unmentioned (including packing/unpacking code) while other PR(s) can eventually focus on correctly identifying information that needs to be relayed and correctly relaying that information across Squid kid processes. If some library code is not compatible with that plan, let's explicitly disclose that incompatibility and discuss how to address it.
| { | ||
| switch (type) { | ||
| case SMI_INTEGER: | ||
| case ASN_INTEGER: |
There was a problem hiding this comment.
That would be the bad practice of adding unnecessary technical debt.
We need to balance many evils here. Creating temporary debt may reduce the number of errors and speed up this PR acceptance. If those goals are valuable, then creating that debt can be the lesser evil.
The goal of this PR is to remove the old library, not to preserve its dead API forever.
A "forever" preservation has not been requested. The actual duration of that debt existence depends on many factors. We even have the power to limit it by this PR lifetime (although that may require higher cooperation levels than we were able to achieve in the last few years).
| Must(community != nullptr); | ||
| xfree(community); | ||
| } | ||
| xfree(peername); |
There was a problem hiding this comment.
The code has gone through two levels of review before submission. These changes are the result of the "new code matches Squid coding style requirements" review.
Since the end result is problematic, the reviewers made a mistake and/or their suggestions were misinterpreted/misapplied. Can you point me to the reviews you are using to reject my change request?
| peername = xstrdup(session.peername); | ||
| } | ||
| if (session.localname) { | ||
| localname = xstrdup(session.localname); |
There was a problem hiding this comment.
All the protocol-agnostic members need to be copied to preserve session consistency when the SNMPv1/2 is acting
How do you define "session consistency" in this context? Why is singling out localname among many other new fields necessary and sufficient?
I hope that this PR can leave localname and numerous other new snmp_session fields untouched/unmentioned (including packing/unpacking code) while other PR(s) can eventually focus on correctly identifying information that needs to be relayed and correctly relaying that information across Squid kid processes. If some library code is not compatible with that plan, let's explicitly disclose that incompatibility and discuss how to address it.
| #define INETADDRESSTYPE_ENUMS | ||
|
|
||
| #define INETADDRESSTYPE_UNKNOWN 0 | ||
| #define INETADDRESSTYPE_IPV4 1 |
There was a problem hiding this comment.
This code is located inside #if HAVE_NET_SNMP_TYPES_H, implying that it is needed for compatibility with that external library header. AFAICT, modern net-snmp/types.h does not define these INETADDRESSTYPE_ constants. If the external library does not actually require us to define these constants, then please place these definitions into snmp_core.h (and drop the ones Squid is not using).
| #include "ip/Address.h" | ||
| #include "log/access_log.h" | ||
| #include "mgr/Registration.h" | ||
| #include "snmp_core.h" |
There was a problem hiding this comment.
AFAICT, snmp_core.h has no #if SQUID_SNMP protection. Why do we want to start including it unconditionally?
|
I adjusted PR title to emphasize that we are not just removing a bundled library, but are replacing it with a shared library. |
The bundled library is a copy of the long deceased UCD-SNMP
implementation which has since been merged into the current
Net-SNMP implementation widely available as libnetsnmp.
Switch to using Net-SNMP as a shared library and remove this
outdated library code from Squid.
Net-SNMP is itself quite outdated in a lot of ways but, is
still more up to date than this old Squid code, and there
does not appear to be any better SNMP library available.