More than 3,200 lawyers have written to the prime minister urging him to "rethink" plans to restrict jury trials for all but the most serious cases.