That’s the proposal in this new paper from law professors Sidney Shapiro and Richard Murphy:
In administrative law, as elsewhere, empirical studies confirm that ideology affects judicial review of agency action to an alarming degree. After summarizing and analyzing empirical studies of judicial review in administrative law, this article proposes four models of judicial behavior that can explain the politicization of judging. While all four models are based on the literature, the article uniquely compares and contrasts them. Based on the models, the article then explores three improbable possibilities for ameliorating politicized judging. The first and least promising contemplates making administrative law clearer and more determinate, thereby depriving judges of the discretion necessary for political preferences to affect their decision-making. The second requires judges to discuss openly their political and policy preferences when explaining their administrative law decisions. The third possibility is the most plausible, least ambitious, and simplest: Require five-judge panels for a well-defined set of significant agency actions – e.g., legislative rules with significant economic effects promulgated through notice-and-comment.
The paper can be found here on SSRN. It’s an interesting proposal. As more studies show ideological effects for judging, I suspect we’ll see similar proposals go forward at the legislative level.