Top Tory signs Free Speech Charter - as colleagues snub call to put residents’ interests first