In that form, it might be unbalanced, but that's an excellent idea, to use two axes of evaluation. Four suits and four ranks, each of which has a particular meaning, can be used for a lot of things in a game.
For instance, you could give everyone a card at the beginning of the scene. The "lead player" in that scene reveals their card and opens the scene. Having a card which matches the lead's card in suit or rank means something significant, and you can play your character accordingly. Maybe you reveal right away, and then both of you become "lead players" of a sort, as a connection arises between you two. Then anyone who matches either of you can do the same, etc.
Or you could base the core mechanic on Crazy Eights, come to think of it, since that game doesn't rely on numbers. The value of the card is only relevant insofar as it is distinct from the other values. If you wanted to do a condensed version, you could only have face cards and Aces, deal everyone a couple cards, and then let Aces be wild, or something like that. Ooh, that's cool. You could go a lot of directions with that.