I run into the follow scenario 1-2 times a week. While there is a fairly easy work-around, I feel it could be better handled or logged. There are other ways to get a server into this state, but here is the way I can guarantee to reproduce it.

• Provision VM with 1 NIC
• Add new NIC to host in VCenter
• Run Puppet - New NIC shows in Foreman
• Edit New NIC and set it to managed, primary, provisioned
• Set New NIC to same DNS name (hostname) (at this point it will have a different mac, same/different ip doesnt seem to matter)
• DO NOT SAVE YET
• Delete Old Nic
• Submit

(this can all be done on a physical server as well, but you'd need to physically add a new NIC and rerun puppet, just easier to reproduce on VM)

It will fail at this point because "Unable to set DHCP entry BadRequest" - due to:


E, [2016-11-08T08:10:15.465701 #1545] ERROR -- : Failed to add DHCP reservation for 2016test.deploy.lab.beer.town (172.23.41.164 / 00:50:56:a4:49:6b): Entry already exists
D, [2016-11-08T08:10:15.465758 #1545] DEBUG -- : Failed to add DHCP reservation for 2016test.deploy.lab.beer.town (172.23.41.164 / 00:50:56:a4:49:6b): Entry already exists (Proxy::DHCP::Error)


This appears to be due to no dependancy checks on the NICs. I feel it should first attempt an omshell delete of the old NIC, and omshell dhcp create of the new NIC. However, the disconnect appears to be that all checks are related to hostnames, not mac or IP.

If you follow the same set of instructions above, but change the hostname to anything aside from the original hostname, it all works. Of course you now have a new hostname and DNS name on the NIC that are not truly the correct hostname.

I am sure there are other workarounds, but my current work around is to:

• Edit Old-Nic and copy/paste/set the new Mac address (and ip if that has changed) of the New-Nic, into the Old-Nic Mac.
• Delete New-Nic
• Save

relevant logs:
Proxy - https://gist.github.com/tek0011/1fa1086b1f06cb708286b09e27522390#file-foreman-proxy
Production - https://gist.github.com/tek0011/044be7ef1ffdb0eb8ebcaaf8daa70fd9

Let me know if you need anything more.