Twitching with dementia?

Twitching in our sleep is really quite normal.
I would not worry further about this unless you see it occurring frequently or when awake.
This was likely deep REM sleep and dreams.

My Husband had what the Hospice Nurse called a Myoclonic seizure she observed it once. She said it really was not anything to worry about.
That said if it was or it wasn't I would have done the same thing about it....nothing.
My Husband had dementia. I think Vascular dementia along with the diagnosed Alzheimer's. There is no way on Gods good earth that I would have put him through testing even if he was not on Hospice.
Putting him through any testing would have been traumatic for him (and me, getting him to testing, dealing with sedation after and on and on for no good reason as I would not have authorized any treatment.)

Your mom has dementia...
so I guess my question to you is this...
What treatment would you authorize for her.
No treatment will cure, reverse, stop the dementia.
IF she is in no pain I would not do anything.
IF there is pain that can be helped with medication to some degree.
But the biggest decision would be her QUALITY of life.
If there is something going on that would not impact QUALITY of life then do nothing.

this is one of the decisions where "Benefit VS Burden" comes into play

What to do about body jerking with Alzheimer's/dementia

Sudden twitching or jerking, known as myoclonus, is another condition that sometimes happens with Alzheimer's. The person's arms, legs, or whole body may jerk. This can look like a seizure, but the person doesn't pass out. Tell the doctor right away if you see these signs. The doctor may prescribe one or more medicines to help reduce symptoms.

https://www.nia.nih.gov/health/alzheimers-caregiving/caring-person-late-stage-alzheimers-disease

The above website provides a wealth if information about AD and late stage/end of life symptoms.

Best of luck to you.