There are serious reasons to support free speech clause, say Peer and MP