I think how much of an issue physical theft is depends on how you intend to use your key, and your expectations.
The way I use my key is as a 2nd factor. Defending against physical theft is out of scope for me because: Physical theft would not be catastrophic, it is only 1 of 4 things they would need to compromise my account. In addition to the key they would need to know (1) where I used it (2) what username or email address was used (3) my password.
The point of two factor is to have two separate factors that complement each other (the key is resistant to phishing, hacking, malware, etc, but weak against physical theft, your login credentials are resistant to physical theft but more vulnerable to phishing/hacking/malware/etc)
But if you are intending to use your key as your one and only factor, then yes, I can see how you would be concerned with physical theft (but I don’t think multiple keys would sufficiently mitigate this).
edit: I would also add that if your primary concern is physical theft, I dont think a security key is the best option for you, unless you can protect it with a PIN as well (and I believe you can with some).