The Delete Task Button appears in the Toolbar when editing a task that has already been saved.

When you click the button, you are prompted to make sure you really want to delete the task. Click OK to delete the task or Cancel to not delete the task.