generate is a function but it can get loaded as a module in a "corner" case.

[kechan]

I ran into this experimenting in a rather disorganized manner in jupyter notebook.

from mlx_vlm import load, generate
from mlx_vlm.generate import get_model_and_processors

//do some coding, run inference
// and then later

from mlx_vlm import load, generate # do this again by mistake "redundantly"

then get error:

output = generate(model, processor, "http://images.cocodataset.org/val2017/000000039769.jpg", prompt, verbose=False)

TypeError: 'module' object is not callable

this is python 3.10.9 and bet it got something to do with how modules get loaded. generate as a artifact exists as both a method and a module so in this rather contrived way, it caused problem.

this is just a doc of this behaviour in case if this helps others.

[Blaizzy]

What OS are you running this notebook on?

[kechan]

Sonoma 14.2.1 (23C71)
VSCode Version: 1.89.1
Jupyter notebook
python 3.10.9

all mlx related are latest or bleeding edge.

I don't think this is a big problem for now. Just thought this may help debug should someone in future run into something strange.

[Blaizzy]

The generate you want to use is from utils.

Which you can import this way:

from mlx_vlm import load, generate

The generate.py is used for the CLI. Which you can use:

python -m mlx_vlm.generate ...

[Blaizzy]

Could you provide more context and reproducible example?

[kechan]

In a jupyterbook cell, run this:

from mlx_vlm import load, generate
from mlx_vlm.generate import get_model_and_processors
from mlx_vlm import load, generate <--- repeat this

and then use generate to run inference and you should get the error.

[Blaizzy]

I see,

I fixed it, here #24 :)

[Blaizzy]

Thanks for letting me know!