Language Detector (Modelmetry)

This evaluator detects the language of the input and compares it to the expected language or to the language of the output, depending on the configuration.

Configuration

Option	Description	Type	Default	Required	Constraints
CheckType	What should be checked by this evaluator	string	input_matches_output	false	Enum: input_matches_output, input_matches_language, output_matches_fulfillment
ExpectedLanguage	The expected language of the input and/or output	string	None	false	Length: 2 Pattern: ^[a-z]{2}$
WordCountThreshold	Minimum number of words to check, as the language detection can be unreliable for very short texts	int	2	false	Min: 0
ConfidenceThreshold	Minimum confidence threshold for the language detection. If the confidence is lower than this, the evaluation will be skipped.	float64	0.4	false	Min: 0 Max: 1

Metrics

This evaluator reports the following metrics based on the analysis results:

Name	Key	Description	Min	Max
Input Matches Output Language	input_matches_output_language	Whether the input language matches the output language	0	1
Input Language As Expected	input_language_as_expected	Whether the input language matches the expected language	0	1
Output Language As Expected	output_language_as_expected	Whether the output language matches the expected language	0	1