Well, I have a bachelor's in physics, which is not really enough to answer this question properly. But I'll fire off my knowledge.
I think the first issue with the question is the assumption of "solid",
What do you mean by solid? We have a sense for what that is on a macro scale, but on the scale of particles, it really doesn't have the same meaning.
When you think solid, you probably think "Cannot push hand through", or possibly "Truck interacted with 'solid', need new truck"
That's very different for small things, and the best information we have is that they just don't behave like this.
Instead, they have an indeterminate position governed by a wave function. (If they have a position at all, plenty of evidence to suggest they're waves. Schrodinger didn't like that, see Cats).
The other problem is, when you want to figure out what these little things look like, you have to smash them with a photon. And smashing small things with high energy photons leads to disastrous complications for the sample. Unfortunately, no matter how we try shoot the thing, we just can't nail down all of its information.
We can only see so much detail. Though that hasn't stopped theorists from suggesting smaller constituent waves, (Strings), but since they have no good way to prove their theory, that line of discussion just remains a beautiful simplification.
Now, is it "Turtles all the way down"?
That's a philosophical discussion, rather than a physical one, we have no evidence one way or another. And honestly, trying to apply your real-world physical intuition to the quantum level has mostly just gotten people into trouble. (Einstein: Not a fan of Quantum)
In the words of my prof, "The Math works out"