Hi,
it is not an easy advice, since I do not really know your budget, but I usually recommend a new kaya goban (or better) whenever possible, as you were looking for.
Glass go stones are a good option, as are yunzi (slightly heavier and less "white" than glass).
Since this will (likely) not be your final go set, if you intend to carry it around (e.g. to your local Go club) I would recommend a foldable goban, that is much more easier to carry than a standard one. This said, with the exception of bamboo boards (that are very heavy and not so beautiful) I don't know where you can find one.
I am based in Europe, so I don't have direct experience, but Yellow Mountain Imports have several go sets
https://www.ymimports.com/collections/go-game-setsPlease enjoy your first set
quest and don't forget to share a picture of the result here
Galation