from trilobite@lemmy.ml to selfhosted@lemmy.world on 30 May 06:01
https://lemmy.ml/post/48054883
Hi, so I have a little Proxmox box with two VMs: VM1 and VM2 which is a clone of VM1. I change the mac of VM2 to avoid conflict and I reset the machine ID of VM1. I then have a seperate pfSense machine machine that that acts as router, firewall and DHCP server. Proxmox is on the 192.168.20.1/24 domain. In the DHCP server, Proxmox get IP 192.168.20.8 explicitly assigned. All good to this point. I’ve set VMs on pfSense to get the 192.168.20.9X addresses assigned. VM1 gets 192.168.20.91 assigned, while VM2 should be getting 192.168.20.92.
But this is what actually happens:
-
VM1 gets 192.168.20.106 assigned, despite telling pfSense to assign it 192.168.20.91. This happens even with VM2 shutdown. The DHCP Lease table is showing 91 up and running and does not list 106. Yet, the ARP table shows 106 assigned and no 91 assigned. This is even with me deleting the 106 entry from the ARP table several times and rebooting both the VM and the Proxmox server.
-
The VM is definately getting 106 assigned as I can log into it with 106 IP but 91 doesn’t respond (no route to host).
Is this something to do with the bridge configuration on Proxmox? Iv’e added a screenshot of what I see. It doesn’t seem to be that complicated to setup a bridge?
I can’t get my head around this so tips are welcome.
EDIT: I’ve just run ‘sudo ip’ on the VM and i see the ens18 interface with the MAC I assigned to it and the 106 IP assigned to this interface. There are then seven of ‘vethXXX’ interfaces. Not sure what these are. There are also four ‘brXXXX’ interfaces, one ‘loXXXX’ interface and one ‘docker0’ interface, the latter probably used by the docker subsystem running on the VM. I imagine the ‘brXXXX’ interfaces are the docker containers themselves (I think I have four running). But what are the ‘vethXXXX’ interfaces? Sounds like its something to do with “virtual interface”. Why so many and what is creating these?
threaded - newest
It sounds like you’ve got another DHCP server running on that segment.
See message above. I doubt it. I would have had this problem a lot earlier with other machines.
Do you have proxy arp setup on PVE? If not, try flushing the arp table in your pfSense.
don’t I have not configure proxy arp on PVE. the only arp is what I see cached by pfSense. tried flushing the table in pfSense with not luck … the VM is still hooking up to 106 and this is confirmed in the pfSense ARP table even after flushing, despite the DHCP lease table saying that the VM is hooked to 91.
Having additional virtual network interfaces on VMs is completely normal, ens18 does indeed sound like the right one you should be looking at.
Seconding the other commenter who mentioned the possibility of a second DHCP server.
Is your Proxmox host wired via ethernet to the pfsense? Or are there WiFi APs in the mix anywhere
But if there was another DHCP server hiding somewhere, i would have had this problem years ago with all the other machines that are using the same router/DHCP server?
Is there a reason you sre using dhcp instead of assigning ips manually?
Dhcp is great if you don’t care about stuff, in my experiece as soon as you start caring you should do it manually
Wrong. It’s 2026. You should be setting static dhcp entries and using dhcp to ensure static IPs, not avoiding dhcp. Using manually assigned static IPs just means you’ve built a fragile unique snowflake.
I don’t know what I’m doing, please explain.
DHCP can be set to specifically assign the same IP to specific devices, reserving them and ensuring that no other systems will use the same IP accidentally. So your servers will consistently get that same IP address assigned to them every time, no worry about the ip address changing unexpectedly.
Can I just check with @tiptoes@sh.itjust.works and @Eggymatrix@sh.itjust.works that what I’m doing is what you are saying.
I’m not setting static IPs at client level. Would be a nightmare. What I do is assign IPs on the DHCP server i.e MAC address on VM1 is XX:XX:XX:XX so I set this MAC address in DHCP to correspond to IP 192.168.20:XX so that the machine gets a unique IP all the time. Is this what you meant? I thought everyone would be doing this nowadays as it so easy to manage, except when something goes wrong like in this case.
No what I meant is I don’t use dhcp at all, but apparently I am an old gremlin and all these youngsters do it some variation of your way. If you want ro do it like that I leave you with them.
No worries, I just don’t want to get you confused
Oh ok, it’s what I did. I set the client’s IP as fixed on my router.
Dhcp and ensuring it works is someting I don’t care about. I tell the kernel wich packets it should be interested in and how to sign itself and that will be it thank you very much.
I configure static routes and have all my machines and network segments neatly arranged in my database. I setup a new machine I know exactly what address it should have,it goes up and until it has a problem it will keep the address it was installed with.
Its 2026 I work like that, you can have your opinions I have mine. The difference is that I depend on one less thing that I don’t care about, so I have more profit margin.
Very old school; yes you can certainly do all of that and track all of that yourself. We all used to do it that way……But it’s 2026….just as you’d use a real editor rather than edlin, or password managers rather than text files, the new ways ARE better, easier and more consistent. Making sure dhcp works is one of the modern (honestly not that modern) basics that make sure your network is set up properly and isn’t hiding some misconfiguration gremlins that only work because of some static ip and route workaround you implemented years ago and worked “until now”.
You can have some weird workaround creep up yesrs later also with dhcp and dns ,😉
Heh true enough. And you can run into issues with package updates breaking something. But hopefully that breaks things for others too so there’s some shared misery there rather than just having to curse your past self for decisions you no longer remember? 🤣
Maybe I’m misunderstanding but what I mean is that I assign static IP via DHCP based on the MAC. I’m not setting static IPs at client level. i.e MAC address on VM1 is XX:XX:XX:XX so I set this MAC address in DHCP to correspond to IP 192.168.20:XX so that the machine gets a unique IP all the time. Is this what you meant?
.
Well, because its all managed in one place rather than having to go and configure loads of machines
Doing it that way is a sure way of loosing your mind at a certain scale.
Don’t do that.
Ensure your DHCP and DNS work as expected and save your headache for the future when you want to expand the homelab to something like https certification or IPv6.
Static IPs should be used sparingly. Like for servers.
I agree with you. I am not at a scale were it matters so I don’t bother. I only have a couple dozen servers to harness for work. Point is that OP does not sound to be at that scale either
My scale is ±5 servers. And of them only the ones that are critical like my AD/DHCP/DNS or my backup servers are actually static.
I’d peobably get an aneurysm seeing your network ;), /j
But if it’s tidy and documented, more power to you :)
Well my entire network doc is a 30ish lines .txt file that looks like this example:
Then some other general stuff on some other 10.0.1.x network etc
Once you take care assigning static blocks and it makes sense you don’t have any surprises. Db1 crashed? Lookup its config data, reinstall machine, recover from backup and nothing else needs to be configured.
You very much want dhcp
Inside your DHCP server you should set a static IP assignment.
That is just static ip with extra steps
It avoids all the issues with static IPs though
Manually setting static IPs leads to a incredibly confusing network setup and you are liable to end up with duplicate IPs
I had this exact fucking problem. Check if you’re using cloud-init and if there’s a holdover entry that may be overlooked.
For me, it was under the
/var/run/folder’s netplan, not the usual/etc/netplan.No
netplanfolder under either/etc/or/var/run/Nice. I’m struggling hard to keep cloud-init out of my pve setup.
Acronyms, initialisms, abbreviations, contractions, and other phrases which expand to something larger, that I’ve seen in this thread:
7 acronyms in this thread; the most compressed thread commented on today has 16 acronyms.
[Thread #322 for this comm, first seen 30th May 2026, 20:30] [FAQ] [Full list] [Contact] [Source code]
Use dhclient -v to See the client side, check logs in the DHCP Server. Sniff maybe.
What a surprise … if I were to believe this I would file for madness state support. <img alt="" src="https://lemmy.ml/pictrs/image/7f1c50db-dc7c-447e-a2b8-d213946e6f56.png"> Look like the VM is having a nice chat with the DHCP sever and both agree that the IP should be 192.168.2’.91. Then one of the two cheat, and actually work on 106. Logs in DHCP server a showing nothing. I even told pfSense to ignore the machine ID and it had no effect whatsoever. <img alt="" src="https://lemmy.ml/pictrs/image/fe1cc880-fcf6-4a45-b452-ca83768adb1e.png"> If there was another DHCP server hiding somewhere, dhclient would have picked that up presumably.
I think you have a PFsense problem not a proxmox problem.
I have encountered something similar to this in the past with PF sense. What fixed it for me is shut down the machine in question, let the DHCP lease show offline in PF sense, then use that very line on the status - DHCP leases page to assign the static IP address to it. Then when I booted it back up it worked.
Also copy and paste the MAC address right out of the DHCP leases table if you are adding it manually. I believe it may be case sensitive.
Just to be clear, this is what is in the ARP table on pfSense: <img alt="" src="https://lemmy.ml/pictrs/image/a8caa0d5-c268-4163-b607-e81b0577a34c.png"> and this is what is in the DHCP lease table in pfSense. <img alt="" src="https://lemmy.ml/pictrs/image/69721083-8f68-46e8-895e-63ecae17008b.png"> What I’ve concluded is that the DHCP sees the VM is online, it probably offered the 91 IP, and just shows it online. The ARP table is showing what the actual assigned IP is (106) and SSH login attempts confirm this. There is no 106 entry in the DHCP table. I would ignore the VM2 element of the equation i described above for now. I added just to describe the conflict that arose when I switched it on. I woudl also say that VM1 was backed up on another Proxmox server I had runnina and then restore it on this new Proxmox server I have with a bigger NVMe.
Yeah this still sounds very much like what I had happen. pfSense tries really hard to hang on to that old random dhcp lease sometimes.
Don’t worry about ARP- that just shows what currently exists.
You might try turn off the vm, delete the static mapping, then delete the DHCP lease in status - dhcp leases, then add the static mapping again and turn the vm back on.
Also on pfSense check /var/dhcpd/var/db/dhcpd.leases . Chances are your VM is in there. Turn off VM, stop DHCP service on pfSense, delete lease from that file, restart DHCP service, check static mapping, turn on VM.
Let me know if that works…
a) turned off VM b) deleted the static mapping and recreated c) change the MAC on the VM and then the same MAC on pfSense d) checked /var/dhcpd/var/db/dhcpd.leases and there is no sign of any 192.168.20.XX lower than 20.110 (from 110 I leave the space availble for occasional wifi access of devices not in my home) e) Rebooted pfSense
absolutely the same problem again :-(
Hmm Are you wedded to that particular Mac address? If not, shut down the VM, delete the virtual Network card, then make a new virtual Network card. Copy paste the Mac of that new card into pfsense with the static mapping, and fire up the VM. See what happens.
If that doesn’t work, I remember something it was possible for proxmox to do some kind of routed Network system. To investigate that, delete all static mappings, fire up the VM, and just look at what Mac address it shows getting the DHCP lease. Is it the one that shows as being assigned to the VM?
Clearly a problem on the VM. Run
dhclient -v ens18; for i in $(seq 60); do ip a s dev ens18; sleep 1; done
Just to see if its broken immediately or if another process probably fks it up later
.
ran the above and the following pops up. the MAC ending is c3 is the new one I assigned to the 20.91 address on DHCP pfsense server about an hr ago.
EDIT: wondering whether this may be a network manager problem on the VM client? See here
EDIT2: Even tried running
ip addr flush dev <your_adapter_id>as suggested here but no effect at allEDIT3: This is now solved. It was a client problem. Somewhere buried in the system, a static IP had been set up on this machine in the past I image.
<img alt="" src="https://lemmy.ml/pictrs/image/07f74647-de6f-4b98-b787-be98e9311d37.png">
When running
ntmui, the 106 address was configured as static address. Deleted it and now only sees the 91 address. Didn’t realise you coudl set two IPs against same interface. This is the page that helped following advice from @nibbler@discuss.tchncs.de of runnigndhclient -v ens18; for i in $(seq 60); do ip a s dev ens18; sleep 1; done