importing manim should not trigger `pygments.styles.get_all_styles` #3713

astro-angelfish · 2024-04-21T10:25:50Z

Upon running import manim, and in particular during the initialization of manim.mobject.text.code_mobject.Code, a styles_list attribute is populated with the output of pygments.styles.get_all_styles().

This causes a lot of overhead and can lead to serious breakage (as documented by the original report below).

Tasks:

remove the Code.styles_list attribute,
rewrite the documentation to say that a list of all styles can be geneated by calling pygments.styles.get_all_styles yourself,
the example in the docstring of Code should be rewritten to not use Code.styles_list[15], but instead the explicit name of a code style.

Original report:

Description of bug / unexpected behavior

I tried to make a video and wanted to see the demo by manim -pql scene.py MemoryMapping. But it fails with an exception.

I also tried to do sudo pacman -Rscn cython as others say that cython uses gdb but did nothing useful.

Expected behavior

The command works correctly and gives me a video

How to reproduce the issue

Code for reproducing the problem

from manim import *

class MemoryMapping(Scene):
    def construct(self):
        phy_mem_all = Reactangle(width=2.0, height=4.0)
        self.play(Create(phy_mem_all))

Additional media files

Images/GIFs

I have no media because nothing succeed.

Logs

Terminal output

Traceback (most recent call last):
  File "/usr/bin/manim", line 5, in <module>
    from manim.__main__ import main
  File "/usr/lib/python3.11/site-packages/manim/__init__.py", line 68, in <module>
    from .mobject.text.code_mobject import *
  File "/usr/lib/python3.11/site-packages/manim/mobject/text/code_mobject.py", line 30, in <module>
    class Code(VGroup):
  File "/usr/lib/python3.11/site-packages/manim/mobject/text/code_mobject.py", line 158, in Code
    styles_list = list(get_all_styles())
                  ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/pygments/styles/__init__.py", line 60, in get_all_styles
    for name, _ in find_plugin_styles():
  File "/usr/lib/python3.11/site-packages/pygments/plugin.py", line 83, in find_plugin_styles
    yield entrypoint.name, entrypoint.load()
                           ^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/importlib/metadata/__init__.py", line 202, in load
    module = import_module(match.group('module'))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/voltron/__init__.py", line 95, in <module>
    setup_env()
  File "/usr/lib/python3.11/site-packages/voltron/__init__.py", line 24, in setup_env
    env = Environment(setup_logging=False,
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/scruffy/env.py", line 49, in __init__
    self.add(**kwargs)
  File "/usr/lib/python3.11/site-packages/scruffy/env.py", line 114, in add
    self._children[key].prepare()
  File "/usr/lib/python3.11/site-packages/scruffy/file.py", line 439, in prepare
    self.load()
  File "/usr/lib/python3.11/site-packages/scruffy/file.py", line 445, in load
    self._pm.load_plugins(self.path)
  File "/usr/lib/python3.11/site-packages/scruffy/plugin.py", line 65, in load_plugins
    self.load_plugins(filepath)
  File "/usr/lib/python3.11/site-packages/scruffy/plugin.py", line 61, in load_plugins
    mod = imp.load_module(modname, file, path, descr)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/imp.py", line 235, in load_module
    return load_source(name, filename, file)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/imp.py", line 172, in load_source
    module = _load(spec)
             ^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/voltron/plugins/debugger/dbg_gdb.py", line 663, in <module>
    class GDBCommand(DebuggerCommand, gdb.Command):
                                      ^^^^^^^^^^^
AttributeError: module 'gdb' has no attribute 'Command'

System specifications

System Details

OS (with version, e.g., Windows 10 v2004 or macOS 10.15 (Catalina)): Linux AL-1S 6.8.7-arch1-1 Update ReadMe to provide clarity... #1 SMP PREEMPT_DYNAMIC Wed, 17 Apr 2024 15:20:28 +0000 x86_64 GNU/Linux
RAM: 16G
Python version (python/py/python3 --version): Python 3.11.8
Installed modules (provide output from pip list):
https://pastebin.com/TzLmr6nV

LaTeX details

LaTeX distribution (e.g. TeX Live 2020): texlive-basic 2024.2-1
Installed LaTeX packages:

tlmgrlist.txt

FFMPEG

Output of ffmpeg -version:

ffmpeg version n6.1.1 Copyright (c) 2000-2023 the FFmpeg developers
built with gcc 13.2.1 (GCC) 20230801
configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-amf --enable-avisynth --enable-cuda-llvm --enable-lto --enable-fontconfig --enable-frei0r --enable-gmp --enable-gnutls --enable-gpl --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libdav1d --enable-libdrm --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libharfbuzz --enable-libiec61883 --enable-libjack --enable-libjxl --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libplacebo --enable-libpulse --enable-librav1e --enable-librsvg --enable-librubberband --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpl --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-libzimg --enable-nvdec --enable-nvenc --enable-opencl --enable-opengl --enable-shared --enable-vapoursynth --enable-version3 --enable-vulkan
libavutil      58. 29.100 / 58. 29.100
libavcodec     60. 31.102 / 60. 31.102
libavformat    60. 16.100 / 60. 16.100
libavdevice    60.  3.100 / 60.  3.100
libavfilter     9. 12.100 /  9. 12.100
libswscale      7.  5.100 /  7.  5.100
libswresample   4. 12.100 /  4. 12.100
libpostproc    57.  3.100 / 57.  3.100

Additional comments

I also installed blackarch on the top of archlinux. so there are a lot of packages. blackarch might have packages that causes the error but i didn't find anything useful. Manim was installed via paru -S manim

The text was updated successfully, but these errors were encountered:

behackl · 2024-04-21T11:00:05Z

There appears to be some weird interaction between pygments (which manim uses in order to highlight code visualized via the Code mobject) and voltron, a package that we do not depend on in any way (your pip list output is gigantic!). The immediate workaround on your end would be to create a separate virtual environment and use manim from there:

$ python3.11 -m venv py-manim
$ source py-manim/bin/activate
$ pip install manim
$ manim -pql ...

However, I do see an issue with manim itself here too: we should probably avoid running pygments.styles.get_all_styles() on import manim, as this forces pygments to run a bunch of code that could probably be delayed to the point where Code is actually instantiated.

astro-angelfish · 2024-04-22T11:16:04Z

There appears to be some weird interaction between pygments (which manim uses in order to highlight code visualized via the Code mobject) and voltron, a package that we do not depend on in any way (your pip list output is gigantic!). The immediate workaround on your end would be to create a separate virtual environment and use manim from there:
$ python3.11 -m venv py-manim
$ source py-manim/bin/activate
$ pip install manim
$ manim -pql ...
However, I do see an issue with manim itself here too: we should probably avoid running pygments.styles.get_all_styles() on import manim, as this forces pygments to run a bunch of code that could probably be delayed to the point where Code is actually instantiated.

Thanks. Creating a separate environment works for me.

I'm not going to close this issue because there's still an issue with manim itself. If it is required please feel free to help me close this issue.

behackl · 2024-04-23T21:45:59Z

No, this issue should be left open; I've just updated title and description with more explicit tasks -- this is actually also a good issue for a first contribution to the library.

goldenphoenix713 · 2024-06-07T17:47:09Z

Regarding the attribute removal, would it be a good idea, instead, to add a class method (say get_styles_list or something similar) that more or less accomplishes the same thing, but has the added benefit that it won't run during import. The reason being, it's a bit odd to specifically point someone to another library for a list of available code styles, especially if it's not as widely known as some. This class method would really just be a wrapper, but it might be a bit more cohesive.

behackl changed the title ~~manim fails with exception, making manim completely unavailable.~~ importing manim should not trigger pygments.styles.get_all_styles Apr 23, 2024

behackl added good first issue Good for newcomers maintenance refactoring, typos, removing clutter/dead code, and other code quality improvements labels Apr 23, 2024

behackl modified the milestone: v0.18.1 Apr 23, 2024

astro-angelfish mentioned this issue Apr 28, 2024

pwntools compatibility issue due to gdb related problems #3736

Closed

goldenphoenix713 mentioned this issue Jun 7, 2024

fix: importing manim should not trigger pygments.styles.get_all_styles #3797

Open

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

importing manim should not trigger `pygments.styles.get_all_styles` #3713

importing manim should not trigger `pygments.styles.get_all_styles` #3713

astro-angelfish commented Apr 21, 2024 •

edited by behackl

behackl commented Apr 21, 2024 •

edited

astro-angelfish commented Apr 22, 2024

behackl commented Apr 23, 2024

goldenphoenix713 commented Jun 7, 2024

importing manim should not trigger pygments.styles.get_all_styles #3713

importing manim should not trigger pygments.styles.get_all_styles #3713

Comments

astro-angelfish commented Apr 21, 2024 • edited by behackl

Description of bug / unexpected behavior

Expected behavior

How to reproduce the issue

Additional media files

Logs

System specifications

Additional comments

behackl commented Apr 21, 2024 • edited

astro-angelfish commented Apr 22, 2024

behackl commented Apr 23, 2024

goldenphoenix713 commented Jun 7, 2024

importing manim should not trigger `pygments.styles.get_all_styles` #3713

importing manim should not trigger `pygments.styles.get_all_styles` #3713

astro-angelfish commented Apr 21, 2024 •

edited by behackl

behackl commented Apr 21, 2024 •

edited