Feature Request/Bug Report

[disturbedlegend]

Jan team:

Heavy user that loves your app. Please consider the following improvements:

• Automated, dynamic GPU memory allocation to avoid OOM errors - just like ComfyUI does.
• Setting to override Jan's minimum ('lower of') context provisioning algorithm, but instead provision entire context specified in the GGUF profile - to avoid mid-processing OOM failure/prompt to allocate more to context.

Thank you,
disturbedlegend

[disturbedlegend]

Jan team,

Your start-low context autosizing feature is a pain-in-the ... - can you at last consider a switch that honors and explicitly provisions the context memory set in model configuration, whether its been configured to draw that from the GGUF or manually configured (32768)?

At least for me, anytime I receive a "You've run out of context, do you want to truncate or expand?" prompt, it'll crash and lose whatever processing was completed thus far.