I would be a proponent of only upvotes that are public, just like Kbin does.

I might in fact switch to Kbin just for that, but Voyager doesn’t work with Kbin.

Only upvotes / favorites is what Mastodon uses as well. It’s the downvotes of those who don’t react that can make a platform toxic. Not because the downvoters intend to, but because the author of the downvoted comment doesn’t know why it’s downvoted.