Mom lost 40 pounds in 6 months. Is this normal for 98 yrs old with dementia?

I used to help my mom with one meal every day so I got to see how staff handled feeding, some were very good at getting people to eat ("come on and try it, I was cooking all day and I'm really hoping you'll like it!") while others were very quick to mark someone as refusing to eat and moving on. There are a lot of very, very calorie dense nutritional supplements that can be given to those with unintended weight loss and your facility should be well aware of what they are, she also needs be monitored by the dietitian to come up with a meal plan tailored to her needs and preferences.
Some weight loss may be unavoidable as appetite decreases toward the end of life and this is something that needs to be discussed in a care plan meeting.

She's in a nursing home, and they did a swallow test. That was fine. When I help her eat, I see she's very picky about texture as well as taste. If she doesn't like something, she spits it out.

It's never "normal" to lose that much weight that fast.

It's also not acceptable to fail to support a resident with meals if that resident is unable to feed herself. What has the facility got to say for itself?

Residents should be monitored for weight loss and appropriate measures taken to address it. This is a huge red flag, that level of weight loss could hardly be excused as unnoticed.

Wait, your comment about aides had me thinking she is in a facility but your profile says she is living at home, which is it pamelac?

What kind of trouble does she have with eating?

If it's utensils, please have her doc order an OT assessment; there are adaptive utensils that really helped my mom.

Swallowing difficulties need to be assessed by a speech language pathologist, as noted by Mac.