How can I disable collection page swatches?

You can disable the app for the collection's page or any other page in the app settings. Navigate to App settings and then under Page settings, disable the app for the collections page.