Whereas I see toys and games separately: you play a game, following its rules to complete objectives; but you play with a toy, maybe making up your own self-enforced rules (though something like Mouse Trap blurs the lines a bit—arguably more fun to play with the Rube Goldberg contraption than to play the game). I would classify something like Cosmic Osmo as a highly interactive toy, as it has no definable objectives, no win state, no loss state.
Not judging—many visual novels are absolutely not games, but I’m planning to buy the rest of the SciADV VNs as soon as they’re available. Because game-ness doesn’t matter as long as you enjoy it.