Signals catch recurring failures in your agent's production traffic. Until now, Latitude discovered them for you automatically. Now you can define your own, with exactly the detection logic you want, over the API, the SDKs, and MCP.

A custom Signal is a name, a description, and a detector. The detector is either a judge (natural-language criteria that compiles to an LLM check) or a rule (deterministic conditions like tool failures, empty output, cost thresholds, or regex matches over the conversation). Scope any Signal to a subset of traffic with filters, and set a triage priority so the important ones surface first.

Create one with the SDK

await client.signals.create("banking-assistant", {
  name: "Dispute resolution failures with churn risk",
  description:
    "User raised a transaction dispute the assistant failed to resolve, then signaled intent to leave.",
  priority: "urgent",
  evaluation: {
    settings: {
      kind: "judge",
      criteria:
        "The user mentioned a disputed charge or unauthorized transaction, the assistant did not confirm a resolution or refund, AND the user expressed intent to leave with phrases like 'close my account', 'switch banks', or 'cancel everything'.",
    },
  },
  filters: {
    tools: [{ op: "eq", value: "dispute_lookup" }],
    cost: [{ op: "gt", value: 4_000_000 }], // microcents, so > $0.04
  },
})

await client.signals.create("banking-assistant", {
  name: "Dispute resolution failures with churn risk",
  description:
    "User raised a transaction dispute the assistant failed to resolve, then signaled intent to leave.",
  priority: "urgent",
  evaluation: {
    settings: {
      kind: "judge",
      criteria:
        "The user mentioned a disputed charge or unauthorized transaction, the assistant did not confirm a resolution or refund, AND the user expressed intent to leave with phrases like 'close my account', 'switch banks', or 'cancel everything'.",
    },
  },
  filters: {
    tools: [{ op: "eq", value: "dispute_lookup" }],
    cost: [{ op: "gt", value: 4_000_000 }], // microcents, so > $0.04
  },
})

await client.signals.create("banking-assistant", {
  name: "Dispute resolution failures with churn risk",
  description:
    "User raised a transaction dispute the assistant failed to resolve, then signaled intent to leave.",
  priority: "urgent",
  evaluation: {
    settings: {
      kind: "judge",
      criteria:
        "The user mentioned a disputed charge or unauthorized transaction, the assistant did not confirm a resolution or refund, AND the user expressed intent to leave with phrases like 'close my account', 'switch banks', or 'cancel everything'.",
    },
  },
  filters: {
    tools: [{ op: "eq", value: "dispute_lookup" }],
    cost: [{ op: "gt", value: 4_000_000 }], // microcents, so > $0.04
  },
})

await client.signals.create("banking-assistant", {
  name: "Dispute resolution failures with churn risk",
  description:
    "User raised a transaction dispute the assistant failed to resolve, then signaled intent to leave.",
  priority: "urgent",
  evaluation: {
    settings: {
      kind: "judge",
      criteria:
        "The user mentioned a disputed charge or unauthorized transaction, the assistant did not confirm a resolution or refund, AND the user expressed intent to leave with phrases like 'close my account', 'switch banks', or 'cancel everything'.",
    },
  },
  filters: {
    tools: [{ op: "eq", value: "dispute_lookup" }],
    cost: [{ op: "gt", value: 4_000_000 }], // microcents, so > $0.04
  },
})

await client.signals.create("banking-assistant", {
  name: "Dispute resolution failures with churn risk",
  description:
    "User raised a transaction dispute the assistant failed to resolve, then signaled intent to leave.",
  priority: "urgent",
  evaluation: {
    settings: {
      kind: "judge",
      criteria:
        "The user mentioned a disputed charge or unauthorized transaction, the assistant did not confirm a resolution or refund, AND the user expressed intent to leave with phrases like 'close my account', 'switch banks', or 'cancel everything'.",
    },
  },
  filters: {
    tools: [{ op: "eq", value: "dispute_lookup" }],
    cost: [{ op: "gt", value: 4_000_000 }], // microcents, so > $0.04
  },
})

await client.signals.create("banking-assistant", {
  name: "Dispute resolution failures with churn risk",
  description:
    "User raised a transaction dispute the assistant failed to resolve, then signaled intent to leave.",
  priority: "urgent",
  evaluation: {
    settings: {
      kind: "judge",
      criteria:
        "The user mentioned a disputed charge or unauthorized transaction, the assistant did not confirm a resolution or refund, AND the user expressed intent to leave with phrases like 'close my account', 'switch banks', or 'cancel everything'.",
    },
  },
  filters: {
    tools: [{ op: "eq", value: "dispute_lookup" }],
    cost: [{ op: "gt", value: 4_000_000 }], // microcents, so > $0.04
  },
})

await client.signals.create("banking-assistant", {
  name: "Dispute resolution failures with churn risk",
  description:
    "User raised a transaction dispute the assistant failed to resolve, then signaled intent to leave.",
  priority: "urgent",
  evaluation: {
    settings: {
      kind: "judge",
      criteria:
        "The user mentioned a disputed charge or unauthorized transaction, the assistant did not confirm a resolution or refund, AND the user expressed intent to leave with phrases like 'close my account', 'switch banks', or 'cancel everything'.",
    },
  },
  filters: {
    tools: [{ op: "eq", value: "dispute_lookup" }],
    cost: [{ op: "gt", value: 4_000_000 }], // microcents, so > $0.04
  },
})

await client.signals.create("banking-assistant", {
  name: "Dispute resolution failures with churn risk",
  description:
    "User raised a transaction dispute the assistant failed to resolve, then signaled intent to leave.",
  priority: "urgent",
  evaluation: {
    settings: {
      kind: "judge",
      criteria:
        "The user mentioned a disputed charge or unauthorized transaction, the assistant did not confirm a resolution or refund, AND the user expressed intent to leave with phrases like 'close my account', 'switch banks', or 'cancel everything'.",
    },
  },
  filters: {
    tools: [{ op: "eq", value: "dispute_lookup" }],
    cost: [{ op: "gt", value: 4_000_000 }], // microcents, so > $0.04
  },
})

await client.signals.create("banking-assistant", {
  name: "Dispute resolution failures with churn risk",
  description:
    "User raised a transaction dispute the assistant failed to resolve, then signaled intent to leave.",
  priority: "urgent",
  evaluation: {
    settings: {
      kind: "judge",
      criteria:
        "The user mentioned a disputed charge or unauthorized transaction, the assistant did not confirm a resolution or refund, AND the user expressed intent to leave with phrases like 'close my account', 'switch banks', or 'cancel everything'.",
    },
  },
  filters: {
    tools: [{ op: "eq", value: "dispute_lookup" }],
    cost: [{ op: "gt", value: 4_000_000 }], // microcents, so > $0.04
  },
})

await client.signals.create("banking-assistant", {
  name: "Dispute resolution failures with churn risk",
  description:
    "User raised a transaction dispute the assistant failed to resolve, then signaled intent to leave.",
  priority: "urgent",
  evaluation: {
    settings: {
      kind: "judge",
      criteria:
        "The user mentioned a disputed charge or unauthorized transaction, the assistant did not confirm a resolution or refund, AND the user expressed intent to leave with phrases like 'close my account', 'switch banks', or 'cancel everything'.",
    },
  },
  filters: {
    tools: [{ op: "eq", value: "dispute_lookup" }],
    cost: [{ op: "gt", value: 4_000_000 }], // microcents, so > $0.04
  },
})

await client.signals.create("banking-assistant", {
  name: "Dispute resolution failures with churn risk",
  description:
    "User raised a transaction dispute the assistant failed to resolve, then signaled intent to leave.",
  priority: "urgent",
  evaluation: {
    settings: {
      kind: "judge",
      criteria:
        "The user mentioned a disputed charge or unauthorized transaction, the assistant did not confirm a resolution or refund, AND the user expressed intent to leave with phrases like 'close my account', 'switch banks', or 'cancel everything'.",
    },
  },
  filters: {
    tools: [{ op: "eq", value: "dispute_lookup" }],
    cost: [{ op: "gt", value: 4_000_000 }], // microcents, so > $0.04
  },
})

The judge detects the pattern going forward from the moment you create it. Deterministic rule-based Signals are backfilled over your recent history, so you see matches immediately.

Or create one from your AI agent over MCP

Point any MCP client at Latitude and describe the Signal in plain language:

Create an urgent Signal in banking-assistant called "Dispute resolution failures with churn risk." Flag sessions where the user raised a transaction dispute the assistant failed to resolve and then said something like "close my account" or "switch banks." Only run it on sessions that used the dispute_lookup tool and cost more than four cents.

Latitude turns that into the same Signal, judge criteria and filters included.

Available today in the TypeScript SDK, the Python SDK, and over MCP.