This classification is available for physical games as well. We currently don’t display it in the sidebar during browsing since there are some other player count relevant tags that are commonly used by physical game creators. As more developers start classifying with the updated system then we may change what filters we display.
You can trigger the filter to manually display by adding a player count to the URL, eg. https://itch.io/physical-games/two-player