You can only delete a horse if there are no transaction or activity against the horse record. For example, if the horse was accidentally added. To do this:
Go to Horses
Find the relevant horse record in the table
Hover over the record and click on the gear at the end of the row
Click delete
Confirm deletion
The best option is to make the horse "Non-current". This will drop the horse from appearing on your horse lists but you will still have access to the record. To make a horse Non-current
Go to Horses
Open the relevant horse record
Click on Horse details
Untick "Current Horse"
Click save