Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Depthwise 1D convolution needed for Mamba prefill #8571

Closed
esmalTT opened this issue May 16, 2024 · 0 comments
Closed

Depthwise 1D convolution needed for Mamba prefill #8571

esmalTT opened this issue May 16, 2024 · 0 comments
Labels
LLM_feature mamba P1_critical prefill LLM models have prefill mode and it's optimization is usually separated from decode mode.

Comments

@esmalTT
Copy link
Contributor

esmalTT commented May 16, 2024

We require a 1D depth-wise convolution to implement the Mamba prefill phase. There is currently no support for 1D convolution in tt-metal.

We require something that implements the same behaviour has the following PyTorch layer:

torch.nn.Conv1d(
    in_channels=5120,
    out_channels=5120,
    bias=True,
    kernel_size=4,
    groups=5120,
    padding=3,
)
@esmalTT esmalTT changed the title Support 1D depthwise convolution Depth-wise 1D convolution needed for Mamba prefill May 16, 2024
@esmalTT esmalTT added LLM_feature prefill LLM models have prefill mode and it's optimization is usually separated from decode mode. labels May 16, 2024
@esmalTT esmalTT changed the title Depth-wise 1D convolution needed for Mamba prefill Depthwise 1D convolution needed for Mamba prefill May 21, 2024
@esmalTT esmalTT closed this as completed May 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
LLM_feature mamba P1_critical prefill LLM models have prefill mode and it's optimization is usually separated from decode mode.
Projects
None yet
Development

No branches or pull requests

1 participant