"Would you rather we spin our own cloth, etc.?"

There's multiple ways to answer this:

a) There's a distinction between an end-user's view of the world and a developer/manufacturer's. From that perspective the analogy with spinning cloth might be misleading.

b) You're absolutely right that there always must be some set of abstractions we rely on, and that it can't be turtles all the way down. I'm not claiming everyone needs to fork every library they use. I'm claiming that even if they do so it shouldn't be cause for worry to anyone (besides themselves).

c) There is in fact lots of different ways to spin cloth. I don't think people in the cloth-spinning industry worry that somebody else in the world might use a different way to spin cloth. Think about all the competing kinds of wrenches, on metric vs imperial scales, etc. I think I might have found this disquieting and kinda ugly if I thought about it a few years ago. Now I have no problem with it. The tower of babel seems like a relatively minor issue to me now. If you're manufacturing a car or a crane you just decide what kind of wrench you're going to rely on and keep on cranking. Just avoid mixing multiple wrenches in the same assembly line.