[Question] Single forward pass through ChatModule

[caenopy]

❓ General Questions

An older example shows that you could previously run _forward_tokens() on a ChatModule to get logits. The API has since changed, so how would I do this now?

Thanks!

@CharlieFRuan

[CharlieFRuan]

Thanks for the issue, will take a look this weekend. A low-level forward/sample API may still be helpful; DebugChat may be a reasonable place to put it in.

[caenopy]

Thanks Charlie! Any update on the status of this?