To delete a category, delete all the line items. A category will not delete when you drag out all the line items out to a different category. The work around is to add a line item to that category and then delete the item. The category will then go away as expected.
For more articles related to this topic, click here.