You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Ollama recently introduced support for reasoning models with a think=True parameter that enables chain-of-thought reasoning (see Ollama's thinking blog post). When enabled, the model provides both its reasoning process (thinking field) and the final answer (content field).
I'm trying to use this feature with instructor in JSON mode, but I'm unable to access the thinking output.
Current Setup
importinstructorfrompydanticimportBaseModelclassQueryPlan(BaseModel):
steps: list[str]
explanation: strclient=instructor.from_provider(
"ollama/qwen3:14b",
mode=instructor.Mode.JSON,
)
# Trying to pass think=True via extra_bodyresponse=client.chat.completions.create(
model="qwen3:14b",
messages=[{"role": "user", "content": "Create a plan for X"}],
response_model=QueryPlan,
temperature=0.2,
extra_body={"think": True} # <-- Attempting to enable thinking
)
Questions
Is the think parameter in extra_body actually passed to the Ollama API?
When I pass extra_body={"think": True}, does instructor forward this to Ollama's API?
Is thinking supported in JSON mode (instructor.Mode.JSON)?
Since the thinking field isn't part of my Pydantic schema, would it be stripped out even if Ollama returns it?
Is there a way to access the raw response that includes the thinking field?
Are there other modes that would preserve the thinking output?
Would instructor.Mode.TOOLS or another mode better support capturing both structured output and thinking?
How can I access both the thinking process and the validated response?
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
-
Ollama recently introduced support for reasoning models with a
think=Trueparameter that enables chain-of-thought reasoning (see Ollama's thinking blog post). When enabled, the model provides both its reasoning process (thinkingfield) and the final answer (contentfield).I'm trying to use this feature with instructor in JSON mode, but I'm unable to access the thinking output.
Current Setup
Questions
Is the
thinkparameter inextra_bodyactually passed to the Ollama API?extra_body={"think": True}, does instructor forward this to Ollama's API?Is thinking supported in JSON mode (
instructor.Mode.JSON)?thinkingfield isn't part of my Pydantic schema, would it be stripped out even if Ollama returns it?Are there other modes that would preserve the thinking output?
instructor.Mode.TOOLSor another mode better support capturing both structured output and thinking?How can I access both the thinking process and the validated response?
QueryPlan)Environment
Related
Thank you.
Beta Was this translation helpful? Give feedback.
All reactions