How can I get rid of an abusive caregiver that I can no longer afford?

If they live with you maybe hard to get rid of them. If not and still show up to work, call the cops. If living with you call Adult protection services .

How can the caregiver tell you No??? Do they live with You? Is this person a caregiver that turned into a relationship?
Easiest was is to have locks changed. When caregiver shows up, have them bodily removed. They can't show the cops they own the property so they have to leave.
There must be more to this story we are not getting?

Give him/her a formal written notice and stop paying. Better yet, don't open the door.