Before deleting the VPN, make sure you have deleted the OpenVPN and IP Security configurations. Here are the steps to remove a VPN configuration already performed:
On the Network menu of the VPN tab select the VPN to remove by clicking the More button and clicking Delete.
Display the Delete VPN window and re-type the name of the VPN configuration to be deleted. Click the Confirm button to continue with the VPN configuration delete process.
Wait for the removal to complete. and a "VPN (VPN name) has been successfully deleted" notice that the VPN was deleted successfully.