I'm afraid my father who suffers from Parkinson's Disease may try to hurt one of my children. What can I do?

My Fil who had a stroke two years ago and also suffers from PD, got a little confused the other day. My one year old was playing in the corner and for some reason he thought stairs was in the same area. Well, because of lack of proper balance we usually assist him in walking. My five year old comes running up the stairs to tell me that grandpa is "trying to get J*****"". I explained to my fil that he cannot attempt to go after the baby, well that lead to him getting upset. I know I was wrong for walking out the room for a second and leaving my son but now I am afraid that he will unconsciously hurt one of the smaller children.