Not sure exactly what size you are looking for but you can buy smaller quantities here. They have a 4.25" square listed. I purchased a 4" x 5 1/3" square but it looks like it is only available by case.
If you are innovative, you can use plastic cups and drill holes in them. I found some nice 33 oz ones at the dollar store, but they fell over easy-and they were not that narrow at the bottom(like your standard convenience store cup).
I've had success finding small, tall pots by searching tree farm suppliers. Sapling pots seem to be pretty much universally tall and narrow, and come in all sorts of sizes. Not sure of the prices, since I never had adequate funding and motivation to place an order, but once I started looking for sapling pots I was able to find lots of vendors.