You can't seriously defend the proposition that Dred Scott was a non-factor. It was certainly treated as such by the leading figures of the day. It contributed to the schism at the 1860 D Convention, fueled the growth of the R Party, & the R reaction fed Ssouthern paranoia.
You're assuming there was a bipartisan consensus (in a country without stable parties) for the ultimate outcome, which was radical in a way that discredited judicial resolution.