If the expense is "pending", you can delete it in the finance section: https://tools.2houses.com/finances/expenses
However, you can't delete an accepted/refused expense.