Conservatives say Channel 4 broke impartiality rules while Labour claims the PM is "hiding from scrutiny".