Philip Hamburger has a new book out on the history of the administrative state. From an "originalist" perspective the administrative state is arguably unconstitutional, perhaps unquestionably so. But good luck in getting rid of it. I mean that only half facetiously; as a libertarian I really would like to see administrative agencies disappear. But I don't see it happening anytime soon.
Hamburger is famous for arguing against the concept of "Separation of Church and State." One common mantra of the anti-Separation crowd is that those words are not found in the Constitution. The counter is but the concept is. Likewise with "Separation of Powers." Those words are not found in the Constitution; but the concept is.
And administrative law violates the "Separation of Powers." The legislative, executive and judicial branches of government each has specific functions that the others may not perform. The problem with administrative law is that these "agencies" all perform quasi-legislative, executive and judicial acts as though they were some fourth branch of government (not mentioned in the US Constitution!).
Anyway you can hear Hamburger talk on the matter below.