How to respond to parent that is asking to talk to his mother?

I recommend educating yourself a bit to what is "normal" in this sort of decline. Watch some Teepa Snow videos, particularly those about communications with someone who is, as Oliver Sacks (neurologist and writer) used to say "a very real world, but not YOUR world". There is little pain there for your father, so don't let it cause you so much suffering. Just accept that for your dad, his own mom is very much alive. What you say may have been absolutely just fine. Nothing has to "make sense" anymore. It is just a matter of "comfort care" of the mind. A sort of trying whatever works: deflection, distraction, subject changing, understanding.

It is so sad, this loss of reality. Just try to understand that at some point, when the fears of EARLY dementia are over, and the person has passed into another realm completely, the things you see are sometimes more painful for YOU than for the one you love.
How kind you are to want to be such a comfort. I wish you the very best.

Perfect advice has been given by the others: yes to therapeutic fibs and redirection. They are perfectly ethical and moral for the greater purpose of keeping your LO calm and peaceful -- something they are now no longer able to do for themselves due to their dementia.

My friend told his mom, "we are going to see her this weekend" for years. He learned to do so after a couple of times reminding mom that her mom had passed, resulting in her collapsing in the bed in tears as if hearing the news for the first time. He learned it was more humane to tell the kinder lie.

My mom asks about my Dad a lot, and since he travelled constantly I tell her, ‘You know how he is, I’m sure he’ll get in touch with us shortly.’ If there are any habits or activities your grandma did, maybe fold those into the explanation?

As my father’s passing was really horrific, I also often up in the car yelling or crying or both after those conversations. But incorporating his habits into explaining he’s away for the moment does seem to work.

Wishing you the best!

Try any number of these:
"Dad, grandma went to the store and she can't be reached, we will try later."
"We can call after lunch, would you like a sandwich?"
"Grandma has a doctor's appointment today so we will call her tomorrow. Let's have some tea (or whatever he likes as a snack)."

When you are redirecting someone acknowledge what they want then offer the redirection.

You can also talk about his mom, your grandma. Ask him about her. You might learn things you did not know. It sounds like you are hurting either from the loss of your grandma or just the fact that your dad thinks she is still alive. In either case talking about her might help you get more comfortable.