Google and Jigsaw create Perspective technology to identify abusive comments

Any website that allows contributions from visitors -- whether it is a platform such as Twitter or a site that has a comments section -- has to contend with the problem of abuse, swearing and harassment. Manually keeping on top of this sort of content can be a hellish task, but Google and fellow Alphabet subsidiary Jigsaw have launched a new technology called Perspective to help automate things.

The technology uses machine learning to identify what are being called "toxic comments" (defined as "a rude, disrespectful, or unreasonable comment that is likely to make you leave a discussion") so they can be dealt with appropriately.

The Perspective API is in the relatively early stages of development, but it has already been tested on the New York Times website. Other website developers will be able to get their hands on the API so publishers can take a more hands-off approach to dealing with comment abuse. "The API uses machine learning models to score the perceived impact a comment might have on a conversation. Developers and publishers can use this score to give realtime feedback to commenters or help moderators do their job, or allow readers to more easily find relevant information."

Google explains a little about how the technology works:

Perspective reviews comments and scores them based on how similar they are to comments people said were "toxic" or likely to make someone leave a conversation. To learn how to spot potentially toxic language, Perspective examined hundreds of thousands of comments that had been labeled by human reviewers. Each time Perspective finds new examples of potentially toxic comments, or is provided with corrections from users, it can get better at scoring future comments.

With Perspective running, publishers are free to choose what it does. It could be used to automatically block certain content, or it could simply flag it up for moderation. Alternatively, the poster could be alerted to the "toxicity" of their words and be invited to reconsider what they are saying.

Google says that the technology will expand over the coming year:

Perspective joins the TensorFlow library and the Cloud Machine Learning Platform as one of many new machine learning resources Google has made available to developers. This technology is still developing. But that’s what’s so great about machine learning—even though the models are complex, they’ll improve over time. When Perspective is in the hands of publishers, it will be exposed to more comments and develop a better understanding of what makes certain comments toxic.

While we improve the technology, we’re also working to expand it. Our first model is designed to spot toxic language, but over the next year we’re keen to partner and deliver new models that work in languages other than English as well as models that can identify other perspectives, such as when comments are unsubstantial or off-topic.

In the long run, Perspective is about more than just improving comments. We hope we can help improve conversations online.

Image credit: Happy Stock Photo / Shutterstock