Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add empty Alerts Page + mock API behind feature flag #754

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

flxwu
Copy link
Contributor

@flxwu flxwu commented Dec 23, 2023

What does this PR do?

In the spirit of Trunk-Based Development, I am adding a boilerplate Alerts page + a mock API for it behind a feature flag.

#714

image

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • Chore (refactoring code, technical debt, workflow improvements)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Refactor (does not change functionality, e.g. code style improvements, linting)
  • This change requires a documentation update

Mandatory Tasks

  • Make sure you have self-reviewed the code. A decent size PR without self-review might be rejected.

Copy link

vercel bot commented Dec 23, 2023

@flxwu is attempting to deploy a commit to the langfuse Team on Vercel.

A member of the Team first needs to authorize it.

@flxwu
Copy link
Contributor Author

flxwu commented Dec 23, 2023

State as of latest PR (flxwu#4):

Most important attributes of an Alert:

// Currently only COST_PER_USER is supported, we can add COST_PER_TRACE, TOKENS_PER_USER etc in the future.
alertMetric       AlertMetric
// When this threshold is crossed, the alert will be fired
alertThreshold    Float
// The webhook to call when the alert is triggered
triggerWebhookUrl String?     @map("trigger_webhook_url")

We are now able to automatically trigger Alerts when their threshold is crossed:

  1. See all alerts from the database on the UI
  2. On every new generation, check whether we should trigger any of the alerts in the database
  3. If yes, we trigger the Alerts by calling their Webhook URL.

Showcase

https://www.loom.com/share/cb9c1c31ad67414ea9f43c2599086b76?sid=9cc52abe-8084-48ea-b931-671e6ac2a794

We can see that the total cost that triggered the alert was 1.57$: (on top is the webhook logging, bottom is the user consumption dashboard with date range all-time)
image

@flxwu
Copy link
Contributor Author

flxwu commented Dec 23, 2023

State as of flxwu#5

We are now able to add alerts through the UI:

https://www.loom.com/share/4bfc522ae511490cb256be3ca57df28d?sid=70da53b1-0b86-48bf-b66c-1ff37c693674

@flxwu flxwu force-pushed the 12-23-feat_add_empty_Alerts_Page_mock_API_behind_feature_flag branch from 688afa5 to 22a1765 Compare December 29, 2023 08:36
@flxwu flxwu force-pushed the 12-23-feat_add_empty_Alerts_Page_mock_API_behind_feature_flag branch from 22a1765 to dbab01d Compare December 29, 2023 12:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant