Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Odd/inconsistent bullet styles when using Claude 3 Sonnet and the conversational API #38390

Closed
stephendonner opened this issue May 16, 2024 · 3 comments

Comments

@stephendonner
Copy link

stephendonner commented May 16, 2024

Description

Odd/inconsistent bullet styles when using Claude 3 Sonnet and the conversational API

Found while testing #37575

Steps to Reproduce

  1. install 1.67.85
  2. launch Brave using --enable-features=AIChat:conversation_api\/true
  3. switch to Claude 3 Sonnet (you'll have to go Premium first)
  4. load https://www.polygon.com/24158477/shogun-season-2-renewal-what-about-how
  5. click on Summarize this page
  6. examine the bulleted-list formatting

Actual result:

cascading formatting note the blue 🔷
Screen Shot 2024-05-16 at 4 01 28 PM Screen Shot 2024-05-16 at 3 50 40 PM

Expected result:

Screen Shot 2024-05-16 at 4 09 12 PM

Reproduces how often:

30-60% of the time? 🤷

Brave version (brave://version info)

Brave | 1.67.85 Chromium: 125.0.6422.60 (Official Build) beta (x86_64)
-- | --
Revision | d01060f5ec155e6eb64a335203663a29c0e5d81f
OS | macOS Version 11.7.10 (Build 20G1427)

Version/Channel Information:

  • Can you reproduce this issue with the current release? No
  • Can you reproduce this issue with the beta channel? Yes
  • Can you reproduce this issue with the nightly channel? Yes

cc @mattmcalister @bbondy @evq @LorenzoMinto @petemill @nullhook

@stephendonner
Copy link
Author

cc @brave/qa-team

@stephendonner
Copy link
Author

stephendonner commented May 16, 2024

FWIW, I don't seem to see this (as often?) using Claude 3 Haiku - is it on a different config/path?

Screen Shot 2024-05-16 at 4 27 29 PM

@stephendonner
Copy link
Author

Another example, where it becomes Mixtral-like in its predilection for emojis

Screen Shot 2024-05-16 at 4 58 06 PM

@stephendonner stephendonner changed the title Odd/inconsistent bullet styles when using Claude 3 Sonnet and the conversational API Odd/inconsistent bullet styles when using Claude 3 Sonnet and the conversational API May 17, 2024
@LorenzoMinto LorenzoMinto self-assigned this May 31, 2024
@evq evq closed this as completed May 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Done
Development

No branches or pull requests

2 participants