You can fix this in 3 steps.
In the Sources box in the lower left corner, click "+" and select a type. If you're on a PC, you'll select Game Capture. If you're on a Mac, you'll select Window Capture. Click OK to confirm.
As you select from the Window list, select your Minecraft window.
Back to the Sources box, make sure you move the Game Capture or Window Capture to the top of the list so that it's the default for all future recordings.