Sunday, May 29, 2011

How to test blog moderation?

I discovered two days ago how to change the settings of this blog to allow anonymous comments, which I did. The next day, an anonymous comment appeared (about the move to living in virtual reality): "you still have to go number 2 physically though ... " Thus I am immediately faced with a deep, fascinating question: to delete or not to delete, that is the question.

I occasionally participate in blogs, and my comments often get deleted. For example, once I tried to put a comment on Archbishop Dolan's blog, correcting some information (about the sexual abuse scandal, if memory serves) by giving a few facts, with links to supporting documents. But my comment never appeared. I was surprised because it expressed no opinion and only stated facts. How can one object to that? That blog sometimes contains criticisms in the comments, so I had not thought that comment moderation was unfair. I spent a few days observing the blog regularly after that, and now have a conjecture: the moderator lets through a balance of positive and begative comments, but biases them so that the negative comments are emotional rants rather than rational arguments. It gives the illusion of being open-minded while subtly veering readers in a certain direction. If my theory is true, that's a pretty sneaky manipulation!

Another time, on a similar site, I tried to post a comment. I knew the moderator was ruthless in deleting whatever he didn't like, so I tried to game the system by posting a quote from his own writing, carefully chosen so that in the context of his post it would be a subtle rebuttal of his rant. I figured that he surely would feel guilty about deleting that post, and was hoping to pass his filter successfully because of that. I naively thought I was being clever. What happened next was impressive: my comment got marked as "waiting to be moderated" before appearing... but half a day later. So, yes, it did appear. But by that time, many readers had moved on to some other topic. Additionally, it appeared at the place in the list of comments where I had originally posted it, so it was buried deep underneath all the comments that appeared in the intervening twelve hours, and so I expect that no one read it. Clever! He outwitted me.

So, the main purpose of moderation is propaganda: then moderation is most effective if it does not consist in stupidly deleting everything that does not fit the moderator's agenda, but in letting some carefully filtered, well-chosen criticism go through.

I have lost a lot of my excitement about blogs as measuring the pulse of public opinion, when I realized that comments do not express the vox populi but the moderator's portrayal of the vox populi. It's a sham.

One burning question: how to test such theories? I would love to see some formal study of blog moderation, preferably extended to information control in general. How can the user figure out whether a blog is biased, and how badly? In both of the above examples, I realized it by mere chance, and I did not repeat the experiment. What about blogs that claim to follow certain rules for moderation: how can we verify that they follow the stated rules, when we do not get to see the deleted comments?

7 comments:

  1. Zero-knowledge proofs.

    ReplyDelete
  2. Um, thanks anonymous, but can you expand on your hint? I know how to use zero-knowledge proofs for graph isomorphism, but I don't see what this has to do with it ... :)

    ReplyDelete
  3. Why not have (a) a clearly defined set of rules, and (b) two comment feeds, one moderated with the article, and one unmoderated available for someone to check and verify that the rules are followed.

    ReplyDelete
  4. Kunal, is there a way to do that which is user-friendly enough that reader lambda can understand it?

    I doubt that owners of big blogs would be willing to do that, though. They probably value their power too much to be ready to yield it. If we can't get them to participate in a specific protocol, can we still test for bias?

    ReplyDelete
  5. Here's another idea. Design the blogging software system so that it displays a "moderation score" prominently near the top of the blog and also a "rate the moderator" link. Following the link leads you to a page where you see a random sample of comment submissions from a few weeks ago which you can rate as either spam or not spam. (Maybe, comments automatically judged to be clearly spam, such as viagra ads, are never included in the sample.) If the classification made by the moderator conflicts with the majority opinion of the readers, then this negatively affects the moderation score.

    Of course, for this to work, you'll have to trust that the blog owner is not tampering with the setup described. But the blogging software itself can be open-source and developed by a separate entity. Also, the user base needs to be active enough to voluntarily test the moderation. But I guess if the users are suspicious about hidden agendas, then they will naturally be interested in doing it.

    ReplyDelete
  6. Arnab: that sounds promising. By the way, I just found your comment buried in my automatic spam folder!

    ReplyDelete
  7. Unfortunately, some sites or brands see comment moderation more as a positioning and public relations tool rather than one for genuine engagement and discussion.

    ReplyDelete