You're visible to others only when your visibility is switched on.
If you turn visibility off, two things happen: you stop appearing in other people's feeds, and your own feed goes quiet too, you won't see anyone else's profiles while you're invisible.
Think of it as a two-way switch. Visible means you can see and be seen. Invisible means neither.
Related:
How do I turn off discoverability?
What can others see about me?
Can someone track my location on Zeeën?