Edit: obligatory explanation (thanks mods for squaring me away)…
What you see via the UI isn’t “all that exists”. Unlike Reddit, where everything is a black box, there are a lot more eyeballs who can see “under the hood”. Any instance admin, proper or rogue, gets a ton of information that users won’t normally see. The attached example demonstrates that while users will only see upvote/downvote tallies, admins can see who actually performed those actions.
Edit: To clarify, not just YOUR instance admin gets this info. This is ANY instance admin across the Fediverse.
I do think the current system is bad, it should be clear that everyone has access to the voter names, but you would be able to do the same with posts, voter names are just another data source.
Most people would just join a community that share their world view, you don’t need to “shadow-ban” everyone with a different bias.
I don’t know if I care about voting being public, often I see good posts being down voted because they express a different opinion. If voting isn’t anonymous, maybe people would put more thought into their votes.