insufficient_quota

[silvacarl2]

Describe the bug
keep getting insufficient_quota when testing with google colab.

To Reproduce
run the colab example

Expected behavior
it is probably rate limiting because it is trying to call openai too fast.

Screenshots
run the colab example

Additional context
I think if there is a parameter to adjust the rate it is accessing openai API as a parameter to this call smart_scraper_graph.run(), it should work fine.

anotehr approach would be to support some open hosted LLMs, like together.ai llama-3-70b, and it may not have this issue.

[PeriniM]

Thanks for the feedback! We will add it, also it is possible to integrate a retry mechanism using tenacity

[silvacarl2]

ok cool, i will check that out.

also, i need to see ow to get this to work with open LLMs.

[VinciGit00]

Let us know man

[VinciGit00]

Do u have any news?

[silvacarl2]

have not had a chance to focus on this yet but will do.

[VinciGit00]

please try the new version

[silvacarl2]

still get this error from OpenAI: RateLimitError

[VinciGit00]

This is because the website you want to scrape is too big, you have to increase your biking plan on OpenAI