Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v1.33.1 now requires --v8-flags=--max-old-space-size=8192 to prevent OOME #18935

Closed
nleck opened this issue May 1, 2023 · 16 comments · Fixed by #20135 or #23838
Closed

v1.33.1 now requires --v8-flags=--max-old-space-size=8192 to prevent OOME #18935

nleck opened this issue May 1, 2023 · 16 comments · Fixed by #20135 or #23838
Assignees
Labels
upstream Changes in upstream are required to solve these issues

Comments

@nleck
Copy link

nleck commented May 1, 2023

Several different programs were working up until the latest release, but now I get the following error.

<--- Last few GCs --->
78[28692:0x118008000] 9509 ms: Mark-Compact (reduce) 1399.5 (1424.9) -> 1399.0 (1424.9) MB, 29.54 / 0.00 ms (+ 7.8 ms in 7 steps since start of marking, biggest step 1.6 ms, walltime since start of marking 43 ms) (average mu = 0.430, current mu = 0.408[28692:0x118008000] 9588 ms: Mark-Compact (reduce) 1399.9 (1425.3) -> 1399.4 (1425.4) MB, 57.17 / 0.00 ms (+ 0.0 ms in 0 steps since start of marking, biggest step 0.0 ms, walltime since start of marking 59 ms) (average mu = 0.351, current mu = 0.277

<--- JS stacktrace --->

Fatal JavaScript out of memory: Reached heap limit

@bartlomieju
Copy link
Member

This is caused by #18844 in which we upgraded to newer rusty_v8 release. We had to disable "pointer compression" in our V8 build, because it was causing memory leaks (#18369).

The root issue is a V8 bug (https://bugs.chromium.org/p/v8/issues/detail?id=13640&q=garbage%20collection&can=2) which it appears now got some traction and might be fixed in the near(-ish) future. Once that happens we will turn back on pointer compression. For now the only remedy on your side is to increase the limit for old space.

@nleck
Copy link
Author

nleck commented May 2, 2023

@bartlomieju should we close this bug ( assuming there's another one tracking it ) ?

@bartlomieju
Copy link
Member

Let's keep it open for tracking purposes :)

@bartlomieju bartlomieju self-assigned this May 2, 2023
@bartlomieju bartlomieju added the upstream Changes in upstream are required to solve these issues label May 2, 2023
@callionica
Copy link

I get the message "Fatal JavaScript out of memory: Ineffective mark-compacts near heap limit" in a previously working and untouched app. I assume that's the same bug?

@bartlomieju
Copy link
Member

That's correct, currently Deno requires more baseline heap size to function properly (because of the disabled pointer compression) so unfortunately you will have to increase the limits on your side as well.

@masx200
Copy link

masx200 commented May 5, 2023

I'm also experiencing this problem, and running deno test is the same error.

deno 1.33.1 (release, x86_64-pc-windows-msvc)
v8 11.4.183.1
typescript 5.0.3

@callionica
Copy link

Does 1.33.2 do anything to fix/workaround this? Is my app unusual for hitting this? I'd kind of expect a lot of noise and activity around a regression like this, but this issue doesn't have many comments.

@bartlomieju
Copy link
Member

Does 1.33.2 do anything to fix/workaround this? Is my app unusual for hitting this? I'd kind of expect a lot of noise and activity around a regression like this, but this issue doesn't have many comments.

No, there's no change in 1.33.2 to fix this. There's really nothing we can do until https://bugs.chromium.org/p/v8/issues/detail?id=13640&q=garbage%20collection&can=2 is fixed.

Is my app unusual for hitting this?

Maybe you have a very low heap size limit set? Most people use the default heap sizes so there's no problem of running out of heap size, besides overall slightly increased memory usage.

@denizdogan
Copy link

Is there any way to configure VSCode to use this mysterious flag for the language server? I couldn't find any obvious setting for it.

@dsherret
Copy link
Member

dsherret commented Jun 8, 2023

@denizdogan see #19088 (comment) -- It should be higher by default now though so you might not run into it as often.

@bajtos
Copy link
Contributor

bajtos commented Jun 26, 2023

When running our custom runtime built on Deno in a Docker container with 256MB memory, we run into this issue too.

Maybe you have a very low heap size limit set? Most people use the default heap sizes so there's no problem of running out of heap size, besides overall slightly increased memory usage.

What is the recommended configuration to avoid OOM? What is the recommended minimum memory size for running Deno-based apps in a container?

@bartlomieju
Copy link
Member

What is the recommended configuration to avoid OOM?

I can't really say, since it depends on the amount of internal JavaScript runtime code you have, you'll have to measure it experimentally.

What is the recommended minimum memory size for running Deno-based apps in a container?

The default setting should work fine in most cases, if you are storing huge objects in memory then you will need to increase the default heap size. It really depends on the given workload and there's no single one-size-fits-all in this case.

@bartlomieju
Copy link
Member

Reopening because of #20253. We discovered that after enabling pointer compression again we're still leaking memory. From https://bugs.chromium.org/p/v8/issues/detail?id=13640&q=garbage%20collection&can=2 it appears that the bug is still not fixed on V8 side.

@bartlomieju bartlomieju reopened this Aug 23, 2023
cscheid added a commit to quarto-dev/quarto-cli that referenced this issue Jan 25, 2024
@eduardobbrito
Copy link

Reopening because of #20253. We discovered that after enabling pointer compression again we're still leaking memory. From https://bugs.chromium.org/p/v8/issues/detail?id=13640&q=garbage%20collection&can=2 it appears that the bug is still not fixed on V8 side.

@bartlomieju it seems the bug was marked as fixed earlier this month.

I'm still having this issue on deno 1.43.3

@bartlomieju
Copy link
Member

It only made its way to v12.6-lkgr branch a few days ago. Follow #23838 for updates when it lands in Deno.

@mmastrac
Copy link
Contributor

@eduardobbrito If you're on Linux, would you mind giving the PR artifact from 23838 a try with pointer compression enabled? Grab it from the artifacts dropdown at the top of https://github.com/denoland/deno/pull/23838/checks.

mmastrac added a commit that referenced this issue May 23, 2024
v8 12.6 w/pointer compression enabled.

Closes #18935
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
upstream Changes in upstream are required to solve these issues
Projects
None yet
9 participants