Santa is a robot moderator. Santa will decide if you’re naughty or nice. Santa has no chill.
Hi everyone!
The slrpnk admins were nice enough to let me try a little moderation experiment. I made a moderation bot called Santa, which tries to ease the amount of busywork for moderators, and reduce the level of unpleasantness in conversations.
If someone’s interactions are attracting a lot of downvotes compared to their upvotes, they are probably not contributing to the community, even if they are not technically breaking any rules. That’s the simple core of it. Then, on top of that, the bot gives more weight to users that other people upvote frequently, so it is much more accurate than simply adding up the up and down vote totals. In testing, it seemed to do a pretty good job figuring out who was productive and not.
Most people upvote more than they downvote. To accumulate a largely negative opinion from the community, your content has to be very dislikable. The current configuration bans less than 3% of the users that it evaluates, but there are some vocal posters in that 3%, which is the whole point.
It is currently live and moderating !pleasantpolitics@slrpnk.net. It is experimental. Please don’t test it by posting bad content there. If you have a generally good posting history, it will probably let you get away with being obnoxious, and it won’t be a good test. Test it by posting good things that you think will attract real-life jerks, and let it test its banhammer against them instead of you.
FAQ
Q: What if I am banned?
A: You may be a jerk. Sorry you had to find out this way.
It’s not hard to accumulate more weighted upvotes than downvotes. In the current configuration, 97% of the users on Lemmy manage to do it. If you are one of the 3%, it is because the community consensus is that your content is more negative than positive.
It’s also possible that the pattern of voting arrived at some outcome for you that really isn’t fair. I studied the bot’s moderation decisions a lot, trying to get the algorithm right, but it’s impossible for any moderation system to be perfect. If you feel strongly that you were moderated unfairly, comment below and I’ll look into it and tell you some examples of things you posted that drew negative rank, and what I think of the bot’s decision.
Q: How long do bans last?
A: Bans are transient and based on user sentiment going back one month from the present day. If you have not posted much in the last month, even a single downvoted comment could result in a ban. That’s an unfortunate by-product of making it hard for throwaway accounts to cause problems. If that happened to you, it should be easy to reverse the ban in a few days by engaging and posting outside of the moderated community, showing good faith and engagement, and bringing your average back up.
If you are at all a frequent poster on Lemmy and received a ban, you might have some negative rank in your average, and your ban may be indefinite until your habitual type of postings and interactions changes, and your previous interactions age past the one month limit.
Q: How can I avoid getting banned?
A: Engage positively with the community, respect others’ opinions, and contribute constructively. Santabot’s algorithm values the sentiment of trusted community members, so positive interactions are key.
If you want to hear examples of positive and negative content from your history, let me know and I can help. Pure voting totals are not always a good guideline to what the bot is reacting to.
Q: How does it work?
A: The code is in a Codeberg repository. There’s a more detailed description of the algorithm there, or you can look at the code.
Q: Won’t this create an echo chamber?
A: It might. I looked at its moderation decisions a lot and it’s surprisingly tolerant of unpopular opinions as long as they’re accompanied by substantial posting outside of the unpopular opinion. More accurately, the Lemmy community is surprisingly tolerant of a wide range of opinions, and that consensus is reflected when the bot parses the global voting record.
If you’re only posting your unpopular opinion, or you tend to get in arguments about it, then that’s going to be an issue, much more so than someone who expresses an unusual opinion but still in a productive fashion.
Q: Won’t people learn to fake upvotes for themselves and trick the bot?
A: They might. The algorithm is resistant to it but not perfectly. I am worried about that, to be honest, much more than about the bot’s decisions about aboveboard users being wrong all that often.
What do you think?
It may sound like I’ve got it all figured out, but I don’t think I do. Please let me know what you think. The bot is live on !pleasantpolitics@slrpnk.net so come along and give it a try. Post controversial topics and see if the jerks arrive and overwhelm the bot. Or, just let me know in the comments. I’m curious what the community thinks.
Thank you!
Edit: I have retuned the bot and adjusted some numbers above to reflect its current configuration. It was based on pure parity of downvotes against upvotes, but it now weights downvotes more, so the number of users banned has gone up from about 0.5% to about 3%.
Loving this. Very walkaway (the book, not the community) vibes.
As I posted in the other thread, I’m very interested to see how this works out. I am definitely curious to see what the bot thinks of some of my posting habits if you are able to share that.
Sure. You have a pretty large amount of comments from the last month, pretty heavily voted on, with a ratio of about 2.7:1 positive rankings. This morning it needed to be 1:1 or more to post, and now I’ve changed it to be 2:1, but 2.7:1 is still well over the line.
Interactions it looks at highly positively are things like this:
- https://slrpnk.net/comment/9258523
- https://slrpnk.net/comment/9140065
- https://slrpnk.net/comment/9237926
Interactions it looks at highly negatively are things like this:
Your user is a great example of a hard situation for the bot to judge. To me, all five comments are perfectly reasonable. But you’re getting downvotes from some highly trusted users on the last two, so it counts them as negative things that are outweighed by the weight of other interactions you’ve had.
If someone was only posting things like the last two comments, would that be ban-worthy? To the bot it would be. I would probably agree with that in most cases, even though the comments are fine, since it’s indicative of a single-issue account, always getting in disagreements, which usually isn’t indicative of good things for the contribution level of that user. But it’s something to watch closely since ranking the last two comments negatively starts to smell like creating a single-viewpoint echo chamber.
I see this as it reaching a right judgement given pretty difficult data to interpret.
How will this be audited to ensure fascists don’t game the down votes to quell pro-solarpunk, pro-liberation messaging?
Gaming the system is, I think, more unlikely than it might seem. In my auditing leading up to making it live, the problem was the opposite of that. The average fascist account, if it’s not banned outright, might have a “weight” of plus or minus single digits, whereas slrpnk admins might have a weight of several hundred. Some people were getting banned just because of a single downvote from one of the admins, applied to a reasonable comment, outweighed the whole community’s consensus.
I am watching the results, to some extent, and depending on good people who do receive moderation saying something if it seems unreasonable. I think it is possible to create a network of artificial votes to game the system, but you have to do a lot. It’s resistant to simply massively inserting fake votes from some random account to throw off the tally. You have to engineer artificial trust for yourself, and outweigh a community consensus of millions of votes. I think that, if it even takes off to the point that defeating it becomes a focal point, the level of voting that’s required to game the system will be large enough to be obvious during an audit.
Good to know about that issue with the weight. I guess I need to stick better to the “down-vote etiquette” as by our CoC.
I wouldn’t stress about it too much. I played with the tuning and did more detailed spot checking of its judgements, like the examples I sent you in DM, until I agreed with its judgements almost all the time that I checked. That’s why
SMOOTHING_FACTORis so much higher now than it used to be, to reduce the influence of single high-profile accounts. I just meant that in my testing before I had a chance to tweak it extensively, the problem was more often an overly “pro-solarpunk” judgement than the other way.
Fantastic. Glad to know you thought about this
