Well, if you have created a wrong tag or you need to recreate it for any reasons, GitHub Desktop will not allow you to do that.
So you need to use console git tool to delete local and remote tags.
# Define tag name variable
TAG=v0.0.2
# Delete local tag
git tag -d $TAG
# Delete remote tag
git push --delete origin $TAG
This way local and remote tags will be deleted.
You can also delete tag on GitHub site. But you need to delete an associated release first.