If you have a $1 million mortgage with 5% interest you are paying $50k interest per year. Meaning you are paying most of your interest at the beginning of the loan. If you are down to $750k after 10 years and refinance to 2% you have already paid around $350k in interest . The bank has already gotten the biggest chunk of the profits but takes a "loss" on the remaining portion.