Buying physical bullion
The smallest gold and silver products cost under $100. Just note that the smallest items carry higher premiums (a larger markup over the metal price), so they suit gifts or getting started more than long-term value. See What size bullion should I buy? and What makes up the price of bullion?
Starting small and regularly
If you'd like to build up a storage holding gradually with small amounts, Ainslie Saver lets you start from $50 with much lower premiums than small physical items. It's designed for regular, modest deposits.