Sure. Why keep people around if they're non-deployable? *Everyone* in the military has to be able to deploy, even if they are non-combat types. This includes having all vaccines up to date, being in good dental/physical health, having an up to date will and relevant powers of attorneys, qualified on your weapon, etc. There are often big 'PDP's' (Pre-Deployment Processing) where they go unit by unit to get everyone ready to deploy. If you are physically injured, or pregnant, you are non-deployable and subject to discharge. Sometimes, getting pregnant and non-deployable is a relatively easy way to get out of military service - I've seen it happen. It's typically an Honorable Discharge (unless there are other circumstances). But, claiming some kind of religious/political/philosophical exemption for a vaccine isn't going to cut it. When you enlist, you assert you are willing to 'go to war' if necessary, and that means learning to shoot/kill, deploy when ready, know rules of engagement, etc. You also have to be vaccinated as required - it's not a choice (thus, the discharge). Since these folks are willingly disobeying an order to get vaccinated, I'd move past an Honorable Discharge and go with General Under Honorable. Won't hurt much, but federal employment would not happen.
And, as you've noted, it's not a ton of people (well, it is for the Marines given how small they are). Boot 'em.