After rolling out SNMPv3 with USM (user based security model, security level authNoPriv) on a HP ProCurve based network an external party added Nagios to the mix. A few weeks later, while planing the implementation of a central log server, I found the local log buffer on all switches to be filled with SNMP security violations. All originating from the Nagios server IP address.
I had a look and shortly after found out that the Nagios installation utilizes a Perl script to parse the requested values from the HP switches. This script in turn utitlized the well known Perl module Net::SNMP from David M. Town while my monitoring setup used the Net-SNMP implementation.
Breaking the Perl script down to the bare minimum and comparing the packets on the wire to the Net-SNMP utils traffic with wireshark I soon found that the Perl module sents one additional packet.
First of all kudos to David for explaining the issue and crafting a patch to work around the problem.
In the screenshot above you can see what the traffic looks like, using e.g. snmpget. Comparing this to RFC 4314 section 4 you'll recognize one packet being sent for the discovery and getting answered followed by a second packet with the actual request and the answer to that one too.
Now here we've the packets sent when using this small sample utilizing Perl and Net::SNMP.
You'll soon notice that we now have two packets sent within the discovery process.
Going back to RFC 4314 section 4
you should read on and now find out that for authenticated communication there should
be another message being sent to establish time syncronisation. Here we have the explanation
for our strange second packet being sent and the cause for the SNMP security access violations.
IMHO it would be helpful if you could ignore those invalid packages on the management networks but AFAIK that's currently not possible.
David replied, after some back and forth, with a patch
for Net::SNMP USM.pm (diffed against version 6.0) which adopts the Net-SNMP behaviour.
The core Net::SNMP distribution will still follow strictly the RFC recommendations.
With this patch applied Net::SNMP will synchronize on only one probing packet picking up the the values from snmpEngineBoots and snmpEngineTime as the value of the msgAuthoritativeEngineBoots and msgAuthoritativeEngineTime aswell instead of sending a second packet for those two values.