Royal Bank of Scotland could have to pay higher than expected settlements to cover the cost of involvement in the mis-selling of U.S. mortgage-backed securities, The Times reported on Friday, pushing RBS shares lower.

The part-nationalized bank, which had set aside 1.9 billion pounds ($3 billion) to deal with the claims, could be asked by the Federal Housing Finance Agency (FHFA) to pay settlements of more than 5 billion pounds over the sale of about $32 billion of mortgage-backed debt in the United States, the report said.

Spokeswomen for RBS and for the FHFA declined to comment.

The bank agreed to pay $99.5 million in June to settle claims it misrepresented more than $2 billion of mortgage-backed bonds during the U.S. housing bubble between 2005 and 2007.

The federal court in Connecticut is handling a second case which addresses RBS' sale of $30.4 billion in mortgage securities to Freddie Mac and Fannie Mae.

Shares in RBS fell 1.4 percent to 388.5 pence by 1538 GMT (1038 ET).

"We believe that a 5 billion settlement would be towards the top end of market expectations for this particular issue, and slightly above the levels where other banks have settled in relation to the assets involved, but it's not inconceivable," said RBC Capital Markets banks analyst Leigh Goodwin.

UBS in 2013 agreed to pay $885 million to settle a lawsuit that alleged the bank misledFannie Mae and Freddie Mac into buying $4.5 billion of problem mortgage bonds.

The Times report, citing unnamed sources familiar with the situation, said RBS's 1.9 billion pounds provision was unlikely to be enough to cover the expected claims.

"The FHFA settlement is likely to be the largest of the outstanding litigation and conduct hits that RBS still faces, but there are many others, and they are likely to drag on for many years," Goodwin added.

RBS Chief Executive Ross McEwan said in October the bank would not pay a dividend until it had strengthened its capital position and had more clarity over future misconduct charges.

In September the FHFA had reached a total of $17.9 billion of settlements, stemming from lawsuits it filed in 2011 to recoup losses on roughly $200 billion of mortgage-backed securities purchased by Freddie Mac and Fannie Mae.