Same as destroying a building, you can either select the flag and then hit the "destroy" button in the lower right hand corner or simply press "d" which is the keyboard shortcut.
thx for the tip I found a bug this way Since I play the game in 'window' mode, i didn't see the destroy Button, because the window header bar costs about 12 to 16 pixels in height. They are missing at the bottom -> no destroy button