Also see these two posts (one and two) by Ilya Somin on whether blacks were part of "We The People." It's often said that, if you looked at what America's Founders did, "rights" belonged to white propertied Protestant males. Not exactly (it's complicated). Viewed through the lens of "states' rights," the states did different things. Blacks, women, non-Protestants had "rights" in some states, not others.
Justice Thomas' point is interesting. The Constitution unmoored from the Declaration gives us a pro-slavery Founding. Read together, we get an anti-slavery Founding.