Can we remove my brother as POA due to past fraud against my mother?

In 2013 our family lawyer of 20 years retired and his firm was bought out by a large law firm. My brother had convinced my mother to move her living will from the law firm to another law firm and had got her to sign a release and he picked up the file and original will on March 08, 2014. He visited my mother and told my sister he was heading over to the new lawyer. We thought nothing more of this incident until my sister (who had limited power of attorney) tried to order new cheques for my mother January 2015. They had lost the copy of the power of attorney so she phoned the new law firm to get a copy from the file. We were both shocked when there was no sign of the will or file, despite an extensive search. We then contacted the first firm and they produced a receipt signed by my brother showing he had picked both the will and the file.
The file contained a legal loan agreement executed in July 28, 2009 when my brother borrowed my mother’s last $6,000.00 to prevent his house going into foreclosure. From 2007 to 2009, he had borrowed $26,000.00 from my mother who ended up paying his mortgage, hydro, cable, food etc. He left her with .27 cents in her savings account. He also needed money due to a divorce he was going through at the time. He paid her back in small increments over the two years and then continued to borrow. At the time the loan agreement was signed, he owed my mother $10,400.00. He also moved into my mother’s house for four months but ended up staying two and half years.
When the house sold short and he was unable to pay off the loan, they set up a payment plan of $200.00 month starting in 2010, after he started receiving his city pension. When he was remiss in paying this amount each month my mother went back to the family lawyer in 2011 and added a codicil that the outstanding amount of loan was to be deducted from his 1/3 of the estate upon her death and that if he was living in her house at the time of her death, he had two months to vacate.
When my brother was confronted by my sister about the missing will, he gave several excuses as to why he did not take the will to the new lawyer and finally blamed first lawyer, stating they had lost the will. My sister and I were frantic, as we had been advised by estate lawyers at both firms that if my mother were to die, she die intestate. It not only would have benefited him financially but more importantly, this was a Living Will and stated that she wanted no life saving measures taken if she were to end up in hospital and had details re her funeral wishes and small legacies to several organizations. I finally got fed up with the lies; this was theft and I reported it to the police, which I did. When he was informed I had reported him, the will magically was found in my mother’s dresser draw the next day, a drawer that had been searched 100 times during our search for the copy of her will. However the loan agreement and codicil had been destroyed, leaving an outstanding balance of $7,800.00.
We had to get a new will drawn up by and it cost my mother. She was very distressed and angry at the situation. Her behaviour was so erratic, my sister phoned Community Care Access Centre on June 1, 2015 and arranged for a mental assessment. Due to the high volume, the assessment could not be booked until November, 2015.
Then we found a letter from CRA indicating my mother’s Guaranteed Income Supplement of $560.00 per month was being cancelled as of August 2015. My brother, who supposedly was filing her income tax and is an economics major, failed to file and ignored the two warning letters. In addition, he was putting his bills in with my mother’s bills in the hope she will mistakenly pay them for him. He had failed to make the house tax payments in time and file for her sewer and water rebates. My mother was growing worse during 2014-2015 and my sister and I tried every means possible to get her assessed and finally got her to the doctor in December 2015. Although she was diagnosed with advance dementia, my brother kept trying to get her to write a handwritten will to leave the house to him. Her handwriting was so bad it was eligible and she could not spell her name nor his name right.
Then in January 2016, she fell and broke her hip and was in first the hospital and then a Rehab. She was finally transferred to Long Term Care April 6, 2016.
At this point, the house needs to be sold to finance any items or care that my mother needs. AND IT IS NOT OUR MONEY My brother already asked that he get a Gift Deed of $20,000 when the house sells. We told him the money had to be put aside for our mother's care and no one could touch anything until she dies, but he was insisting this is completely legal. After consulting with my mother’s lawyer he was told it was illegal. Now he is badgering us to use ‘HIS” 1/3 share of the house to purchase the house and we wait for out share when mom dies. Can we remove him as POA?