Clay comes from the yellow Goop piles in the sewers, and you need a pickaxe to mine them. There's no clay lying around before that, so if you've been searching every room for it, get the pickaxe first.
Where do you find clay in Deep Dish Dungeon?
In the sewers. Look for the yellowish blobs labeled Goop pile. They look a bit like sand, which is why a lot of players walk right past them.

Get the pickaxe
You can't mine goop without it. See where to find the pickaxe.
Go back down to the sewers
Take the stairs down into the sewers and look for the yellow piles.
Mine the Goop pile
Hit the pile with the pickaxe and the clay goes straight into your inventory. One pile gave five clay, another gave three, so the amount varies.

How do you remove goo in Deep Dish Dungeon?
Mine it with the pickaxe. The same yellow goo blocks some passages, and clearing it is how you collect clay, so you get both at once.
Swinging the pickaxe at the floor gets you nothing. Aim at the Goop pile itself.
What is clay used for?
The smelter. Bring clay and scrap metal back to camp to build one, and from then on you can smelt scrap into iron rods.