Text Moderation (Google)
This evaluator leverages Google's advanced text analysis capabilities to assess content across various safety attributes such as toxicity, profanity, and threats. It's designed to identify potentially harmful or sensitive content in text documents.
Configuration
Option | Description | Type | Default | Required | Constraints |
---|---|---|---|---|---|
Attributes | Specific attributes to check within the text. An empty list implies all are checked. |
|
|
|
|
Threshold | The confidence threshold to flag an attribute as concerning. |
|
|
|
|
Metrics
This evaluator reports the following metrics to provide detailed insights into the moderation process:
Name | Key | Description | Min | Max |
---|---|---|---|---|
Toxicity | toxicity | The probability that the text is toxic. | 0 | 1 |
Derogatory | derogatory | The probability that the text is derogatory. | 0 | 1 |
Violence | violence | The probability that the text contains violent content. | 0 | 1 |
Sex | sex | The probability that the text contains sexual content. | 0 | 1 |
Insult | insult | The probability that the text contains insulting content. | 0 | 1 |
Profanity | profanity | The probability that the text contains profanity. | 0 | 1 |
Threat | threat | The probability that the text contains threats of violence. | 0 | 1 |
Weapons | weapons | The probability that the text mentions firearms and weapons. | 0 | 1 |
Public Safety | public_safety | The probability that the text mentions public safety issues. | 0 | 1 |
Health | health | The probability that the text mentions health issues. | 0 | 1 |
Religion | religion | The probability that the text mentions religion and belief. | 0 | 1 |
Illicit Drugs | illicit_drugs | The probability that the text mentions illicit drugs. | 0 | 1 |
War | war | The probability that the text mentions war and conflict. | 0 | 1 |
Financial | financial | The probability that the text mentions finance-related topics. | 0 | 1 |
Political | political | The probability that the text mentions political content. | 0 | 1 |
Legal | legal | The probability that the text discusses legal issues. | 0 | 1 |
Additional Information
You can learn more about Google Cloud's text moderation at Google Cloud's text moderation documentation.
Last updated