Table of Contents

  1. What Is Automated Content Moderation and Why Do We Need It?
  2. Defining the Role of AI in Content Moderation
  3. Content Moderation Tools
    1. — Text Moderation
    2. — Image Moderation
    3. — Video Moderation
  4. AI Content Moderation in Practice: Popular Use Cases by Market Leaders
    1. Case 1: Amazon
    2. Case 2: Facebook
    3. Case 3: YouTube
    4. Case 4: Twitter
  5. Final Thoughts: Should We Automate Content Moderation with AI?

In the era of social media, there are certain rules that both users and content creators must obey. But who is setting these rules and why?

Due to the constant access to the Internet and the abundance of content created and posted on various platforms, online users are at great risk of being exposed to inappropriate content. This usually includes flagged content and is categorized as violence (e.g., threats), sexually explicit material, and potentially illegal content. Yet even though adults can easily distinguish such indecent information, children are naturally unable to do so. Also, as more and more harmful content appears online, it can seriously affect the mental health of moderators. The consequences appear dire.

In fact, large teams are behind the content moderation process. But today, content moderation heavily relies on artificial intelligence. Brands, customers, and individual users produce an immense amount of image, video, and text data online. Considering how much data is being produced on a continuous basis, it’s necessary to monitor this information to protect communities, children, and brands.

Harmful, unlawful, and offensive posts can easily harm a community member, especially children, or damage the reputation of a brand. Mental health disorders are, by far, the most common and dangerous outcomes of improper online content management. Thus, smart content moderation matters to everyone.

Let’s learn more about how artificial intelligence has transformed content moderation and see if it’s a better way to process digital content compared to humans!

What Is Automated Content Moderation and Why Do We Need It?

Social media is a big part of our lives

The Internet should be a safe place for all its users, but the question of safety and responsibility is up to us. Or is it not really?

According to Gartner, the C-suite will prioritize user-generated content moderation services in 30% of large businesses by 2024. What does this imply for content moderators? Companies would have to expand their moderation capabilities and policies, as well as invest in content moderation tools to automate the process and scale up. As a result, online users will be able to contribute to enforcing moderation and reporting content violations with the help of AI.

AI-powered content moderation is a crucial achievement in the history of social media management since it helps effectively develop and maintain a product that is subsequently sold to consumers. This way, artificial intelligence becomes a part of the business strategies, which shape brand identity and strengthen user engagement. For instance, two major platforms, Facebook and YouTube, used AI to block graphic violence and sexually explicit/pornographic content. As a result, they managed to improve their reputation and expand their audience.

Therefore, originally human labor, content moderation is a fundamental and structural aspect of social networks. It’s a defining attribute of online platforms, which grows in importance as the volume of content increases. However, as the problematic content grows both in volume and severity, international organizations and states are concerned about the impact of such content on users. They develop appropriate measures to regulate these platforms because traditional content moderation practices have raised critical issues throughout its development, including:

  • The lack of standardization;
  • Subjective decisions;
  • The working conditions and practices of human moderators;
  • The psychological effects of continuous exposure to harmful content.

Most importantly, the inability to manage and regulate content online leads to serious mental health issues. The detrimental effects of digital hate speech worsen the injustices and prejudice experienced by some communities, especially racism. All of this served as the reason for using artificial intelligence in order to make social media a safe and responsible platform for humans.

Let’s see how AI became our hero of the hour in delivering automated, fast, and smart content moderation that makes an online space safe for everyone!

Defining the Role of AI in Content Moderation

How do online users feel about AI in taking over content moderation

AI content moderation is about creating machine learning algorithms that can detect inappropriate content and take over the tedious human work of scrolling through hundreds and thousands of posts every day. However, machines can still miss some important nuances, like misinformation, bias, or hate speech. So achieving one hundred percent clear, safe, and user-friendly content on the Internet seems almost impossible.

Giving a single definition of AI in content moderation is tricky. On the one hand, AI content moderation has little to do with artificial intelligence per se. However, in the context of legislation and policy discussions, the term “AI in content moderation” might apply to a number of automated procedures used throughout various stages of content moderation. These procedures may include a simple process, like keyword filters, or more complex ones that rely on ML algorithms and tools.

Organizations often follow an established procedure for online content moderation workflow, using one of the main content moderation methods:

  1. Pre-moderation. Content moderation is performed before it is published on social media.
  2. Post-moderation. Content is screened and reviewed after being published.
  3. Reactive moderation. This method relies on users in detecting inappropriate content.
  4. Distributed moderation. The decision to remove content is distributed among online community members.
  5. Automated moderation. This method relies on artificial intelligence.

Most of the modern platforms use artificial intelligence to accomplish automated content moderation. An essential feature that satisfies the requirements for transparency and efficacy of content moderation is the ability of AI systems to offer specific analytics on content that has been “actioned.” Simply said, artificial intelligence offers a much more desirable solution to many issues that emerged as a result of poor content moderation and inefficient human labor.

Some of the most common practical applications of algorithmic content moderation include copyright, terrorism, toxic speech, and political issues (transparency, justice, depoliticization). Here, AI can cover a much broader spectrum of abusive and toxic content, remove it fast, and protect the psychological health of both users and human moderators.

Content Moderation Tools

The most popular algorithmic systems for content moderation

When machine learning algorithms are in play, AI systems require large-scale processing of user data to create new tools. However, the implementation of content moderation tools by companies and platforms must be transparent to their users in terms of speech, privacy, and access to information.

These machine learning tools can do so by training on labeled datasets, including web pages, social media postings, examples of speech in different languages and from different communities, etc. If the dataset is properly labeled, according to the ML model task (recommendation, classification, or prediction), the final tools will be able to decipher the communication of various groups and detect abusive content. However, like any other technology, AI tools used for moderation must be designed and used in accordance with international human rights law.

Do you need help with data annotation for your AI project in content moderation? Make sure to contact our team of true data experts at Label Your Data, who can handle your sensitive data in the most secure and effective way!

Now, let’s examine AI content moderation technologies in greater depth!

— Text Moderation

If you think logically, online content is usually associated with text and human language, which we can easily define whether it’s acceptable for the public or not. Plus, the volume of text information exceeds that of images or videos. But how do machines tackle this task?

Natural Language Processing (NLP) is used for moderating textual content in a way similar to humans. NLP tools are trained to predict the emotional tone of the text message, aka sentiment analysis, or classify it (e.g., a hate speech classifier). Such tools are trained on the text devoid of features like usernames or URLs, but it was not until recently that emojis have been included in sentiment analysis. An excellent example of using NLP tools for AI content moderation is Google/Jigsaw’s Perspective API.

Other ML models are used for the text generation, like OpenAI's predictive tool known as GPT-2 language model. The dataset it was trained on consisted of 8 million web pages!

— Image Moderation

What about the visual aspect of working with questionable and harmful content? AI-enabled automation of image detection and identification ranges from simple to more complex systems. In general, images require a bit more sophisticated data handling and ML algorithms.

There are so-called “hash values,” distinct numerical values that are produced as a result of running an ML algorithm in a file. The hash function creates a unique digital fingerprint for a file by computing the numerical value depending on its attributes. The same hash function may be used to check if freshly uploaded content matches the hash value of previously recognized content. You can check Microsoft’s PhotoDNA tool to get the full picture.

Among other ML tools for image moderation, computer vision (CV) methods are used to identify certain objects or characteristics in an image, such as nudity, weaponry, or logos. Besides, OCR (optical character recognition) tools can be useful for detecting and analyzing text in the images and make it machine-readable for further NLP tasks (i.e., deciphering the meaning of the text).

— Video Moderation

Image generation methods are also gaining traction in AI content moderation. For instance, Generative Adversarial Networks (GAN, based on generative algorithms), train an ML model to identify a manipulated image or video. Most importantly, GANs help to detect deepfakes, videos that represent fictional characters, actions, and claims. Have you ever seen a deepfake video with President Obama? As you can see, deepfake technology is quite a controversial issue prevalent in the online space and television, too. It might provoke disinformation and threaten privacy and dignity rights. So, having AI handle the problem is an important step towards responsible social media and automated content moderation.

AI-powered content moderation with AWS

Case 1: Amazon

The international online community is hungry for good content on different platforms. But what defines good content? It’s the one that is comprehensive and safe for the audience, including images, videos, text, and audio data. It’s both the private content that individual users share and the content produced by the brands for marketing purposes. As such, conventional, human-led content moderation becomes insufficient in handling the increasing scope of content and protecting online users.

Thus, companies require novel, robust strategies to tackle content moderation, otherwise, they risk damaging their brand reputation, harming the online community, and ultimately disengaging the users. For this reason, AI-powered content moderation with AWS (Amazon Web Services) provides a scalable solution to current moderation issues, using both machine learning and deep learning techniques (e.g., NLP). They are designed to maintain users’ safety and engagement, cut operational costs, and increase accuracy.

The AWS tool for content moderation, Amazon Rekognition, is based on ML and DL. It can identify inappropriate or offensive content (e.g., explicit nudity, suggestiveness, violence, drugs, etc.) at an 80% accuracy rate, as well as remove it from the platform. It’s a great way to automate (or semi-automate) the moderation of millions of images and videos and speed up the process using content moderation APIs, with no need for ML expertise.

Case 2: Facebook

Facebook is arguably one of the most popular social media networks known today. However, its content moderation issues are on everyone’s lips, like the Christchurch Attacks or a major Cambridge Analytica lawsuit. In the latter case, there was a violation of the privacy of millions of Facebook users.

The first case, however, affected hundreds and thousands of Facebook users, allowing a terrorist to livestream the massacre. The video was later re-uploaded on other platforms, like YouTube and Twitter. Facebook handled the incident both manually and computationally, and now each video would be hashed, compared to the database, and finally blocked if a match was found. Nevertheless, the video was still shown to thousands of viewers for at least 17 minutes. This is what the consequences of poor content moderation look like.

But the lesson was learned, and soon Facebook started using artificial intelligence for proactive moderation. Mark Zuckerberg, the CEO of Facebook, claims that its AI system discovers 90% of flagged content and that the remaining 10% is uncovered by human moderators. The ML model was used to predict hate speech and then another system defined the next action: delete, demote, or send to human moderators.

More specifically, AI content moderation at Facebook detects and flags potentially problematic content (i.e., text, images), using their in-house systems, such as Deep Text and FastText. They also had a multilingual system called XLM-R (RoBERTa). However, these systems proved inefficient, so the platform had to develop more accurate classifiers for hate speech detection and automated content review on both Facebook and Instagram. One such example is their newest system, RIO, which was trained based on the performance of prediction and deployment success.

However, just recently, Facebook enlisted the support of one of the biggest content moderation partners, Accenture. From now on, Facebook relies on Accenture in moderating its content by building a scalable infrastructure to prevent harmful content from appearing on its site.

Case 3: YouTube

While Facebook professes incredible performance, other platforms show varying degrees of success. Let’s have a look at YouTube. As reported by Financial Times, human moderators actively eliminated 11 million videos on YouTube in the second quarter of 2020, which is double the average rate. Since nearly 50% of removal appeals were upheld when AI was in charge of moderation, as opposed to fewer than 25% of them when judgments were handled by humans, the accuracy of the removals was also lower.

Additionally, due to the recent pandemic restrictions and trends toward remote work, the number of human moderators has significantly dropped. As a result, YouTube has started to depend more on AI systems lately. Almost 98% of the videos on YouTube that were removed for violent extremism were flagged by ML algorithms.

One of its noteworthy algorithmic moderating systems is known as Content ID, which is used in the copyright domain (hash-matching audio and video content). The platform has also managed to regulate toxic speech in the uploaded content by training ML classifiers. They are trained to predict hate, harassment, as well as swearing, and inappropriate language in a video. In this case, AI benefits both the users and the advertisers on the platform.

Case 4: Twitter

Twitter has been long criticized for not being able to respond efficiently to harassment online. Therefore, this platform has been working on its internal AI-powered tool for content moderation, and eventually came up with Quality Filter. This filter can predict low-quality, spammy, or automated content on the platform using NLP, labeled training data, and established parameters for predictive ML models.

However, Quality Filter was designed to not remove potentially inappropriate content, but to make it less visible to users, given the First Amendment stance on freedom of expression. So, the paradigm shift towards automation in content moderation practices made us rely more on artificial intelligence in deciding what content we should trust or expose ourselves to.

Final Thoughts: Should We Automate Content Moderation with AI?

Can we trust AI in moderating our content?

Despite AI demonstrating such fantastic performance in moderating online content, it remains unclear whether companies and platforms should opt for full-scale automation. We must acknowledge that even though AI is effective and successful in moderating large volumes of problematic content, it is still an auxiliary and educational asset.

Thanks to many ML tools, automated content moderation turned out to be a comprehensive solution to modern issues regarding harmful, toxic, and unsafe content. However, it’s better to let humans make the final decision whether a user should be banned or content should be deleted. It’s all about a mindful approach to combining AI and human work and achieving the most optimal results. This is how we (together with AI) can build in trust and responsibility in social networks. With responsible AI practices and clear guidelines for content moderation, we can ensure that the online space is a safe space for all its users, including international, multicultural, and diverse communities.

But never forget about the data and trust the professionals only, like we are at Label Your Data. Send your request and find out the solutions that we can offer for your specific project in machine learning!

Subscibe for Email Notifications Get Notified ⤵

Receive weekly email each time we publish something new:

Please read our Privacy notice

Subscribe me for updates
✔︎ Congrats! You are on the list.

Data Annotatiion Quote Get Instant Data Annotation Quote

What type of data do you need to annotate?

Get My Quote ▶︎