What costs are involved in a once or twice a week caregiver, and where should I start looking?

When and if you do need “hired” help, be sure to weigh the pros and cons of using an agency vs. using someone independently. You can find cheaper rates by getting someone who works independently but those through an agency are bonded and insured. I’ve heard stories of the family forced to pay for medical bills of the hired independent caregiver that got injured in their home. Also consider if that caregiver gets sick or doesn’t show for some reason, would you be strapped? Checking with churches is a good idea as some of the others have commented. If your Mom qualifies, there are county aging offices that provide a bounty of free or sliding scale services. Best of luck to you. There are lots of supportive people on this site.

You can get a "Companion" for a lot less than a CNA or a Nurse. Neither of which you need at this point.
Have you looked into Adult Day Care? The one my Husband went to they had a bus/van come to the house and pick him up, he spent the day there had lunch and if he chose to participate there were activities and then the ride home. After I was asked to find other arrangements I found another Day Care that I drove him to. This was a a local IL, AL and Memory Care facility, he would spend the day in the Memory Care wing. I got a few days a week for myself and he got lunch and a break from me!.

If she is not a candidate for Day Care if you are a member of a local Church or other place of worship you can ask there if there is a program where someone will sit with her for a while. Or they may know someone looking for a part time job. Many Senior Centers have volunteers that will do that as well.
I contacted the local Community College and I hired 2 people that had just taken the CNA certification class, they were going on to Nursing school and I said I could accommodate their schedules. I hired them for $15.00 and hour, far less than what I would have paid if I went through an agency. But you do not at this point need a trained CNA.
There is also a site Care.com and they have sitters that post their info and you can hire someone in your area. There is a fee to sign up. I do not know how or if they are background checked.
Right now a program like Meals on Wheels or I think the Salvation Army has Golden Diners that would get the meals in as well as having someone check on her. But that will not help with the social aspect. For that a Senior Center or Adult Day care might be a great option.

The cost for a certified nursing assistant in my area in PA is $23.00/hour. That adds up pretty fast.

Well, it depends on the level of care you want for her. If she’s on Medicare or Medicaid, you might investigate a Home Health aide. If all you’re interested in is someone to visit, check with your church or Senior Center. Many have volunteers who make friendly visits. There’s also Meals on Wheels if she qualifies. Their people will often stay and visit for a while.