Modelmetry Documentation
Back to site
  • Documentation
  • Getting Started
    • Concepts
  • Platform
    • Evaluators
    • Automations
    • Analytics
    • API Keys
    • Secrets
    • Team
      • Memberships
      • Invitations
      • Roles & Permissions
  • Evaluators
    • List of Evaluators
    • Prompt Shields (Azure)
    • DLP PII Detector (Google)
    • Text Moderation (Google)
    • Boolean LLM-as-Judge (Modelmetry)
    • Competitor Blocklist (Modelmetry)
    • Emotion Analysis (Modelmetry)
    • HTTP Request (Modelmetry)
    • Language Detector (Modelmetry)
    • Score LLM-as-Judge (Modelmetry)
    • Word Counter (Modelmetry)
  • Developers
    • Modelmetry JS SDK
    • Modelmetry Python SDK
Powered by GitBook
On this page
  • Create/Edit an automation
  • Delete an automation
  1. Platform

Automations

An automation is a specific flow that, given a specific rule will, if valid, execute a set of pre-defined actions. At this moment, automations get assessed for each new ingested trace and check.

You can view all your existing automations by clicking on Automations in the sidebar.

Automation Attribute
Description

Name

A name for the automation to easily identify it later.

Enabled

Whether the automation is currently enabled. If disabled, the automation will not be evaluated and will never trigger. This is useful for drafts.

Sampling Rate

How often the automation should be evaluated, between 0 and 1. A value of 1 means every time, 0.5 half the time, and 0 never.

Rule

Each automation has a set of rules that determine when it should trigger. The rules are evaluated against the incoming data.

Actions

The actions to perform when the automation triggers. You must ensure your actions work with the resource type detected from the rule (trace). The currently available actions are: - Send Webhook - Run Evaluation

An automation can only target one resource type (i.e., trace, check). Therefore, an automation rule such as trace.name == "some-name" && check.outcome == "pass" will always be skipped. If you need to run the same set of actions for both resources, create two distinct automations.

Create/Edit an automation

To create an automation:

  1. Go to Automations

  2. Click on + New

To edit an automation:

  1. Go to Automations

  2. Click Edit for the automation you want to change

Delete an automation

  1. Go to Automations

  2. Click Edit for the automation you want to change

  3. Click the Delete automation button

PreviousEvaluatorsNextAnalytics

Last updated 5 months ago