I solved this problem before I posted, but thought it might help someone else. My problem was:
I am trying to remove a device via the web interface. The device is offline, a deleted VM. I select the Edit button for the device and in the pop up window Remove the device. I am prompted to restart syncthing, and do so. The device always reappears. This happens on both Linux and MacOSX machines.
It turns out that all these machines were connected to one “introducer” node. As long as the old device was still listed in the introducer, it would automagically reappear in all the other nodes. I first removed it from the introducer node, and it then was removed properly in all other nodes, when I manually removed via the web interface.