What do you do about a relative that has moved into my 88-year-old father's home, but does nothing for him?

My Father was asked by his grand daughter if she & newly wed Husband could move in with him shortly after my Mom passed and my Dad was alone. He agree and gave them a deal telling them that they should save their money to get their own home. They said they were only staying two years. It's been 10 years and they have 3 children now and 4th on the way, The house is too small for this even though I converted my Fathers office into a bedroom for the two oldest children and boy and girl. My father's health is failing two heart attacks and bad hip and kidney. They just left on a multi-state 11 day vacation in a rent RV with a master bedroom that has a king size bed. They have already had 3 vacations that I know of this year. They do nothing to take care of the yard, or the house. The 3 times I've been down to check on my Dad, he stays in his room ( they got the master bedroom). The grand daughter mentioned earlier this year ( before she was pregnant) that the house was too small and they might be moving...but now there is no discussion...no timeline of moving out. I am concerned that he will just pass while they are out and about or with the children. They still have him babysit for hours at a time sometime a day or night and he can barely walk with a walker. What is something happened to Grandpa in front of the Children, they could help, an they'd be traumatized. My brother lives nearby and takes my dad to the hospital and doctor, and lunch once in a while, but stays out of the affairs generally and says nothing. I live out of state, but my Dad has asked me numerous times to live with him. What do I do, I don't want to be the bad guy.