You have no income tax liability, but depending upon the size of the remaining balance you may face federal gift tax issues. Your son has no tax liability; your generosity in paying off his mortgage is a gift to him.
On a sale of a principal residence, you may avoid tax under the rules.