Add support for transcribe image and audio transcription for gemini, anthropic, mistral and ollama.

[rajeevrajeshuni]

Description

Add support for transcribe image and audio transcription for gemini, anthropic, mistral and ollama.

Type of Change

• New feature (non-breaking change that adds functionality)
• Code refactoring

Pre-submission Checklist

• My code follows the project's coding standards and style guidelines.
• My commits have clear and descriptive messages

DCO Affirmation

I affirm that all code in every commit of this pull request conforms to the terms of the Topoteretes Developer Certificate of Origin.

Implementation Questions for Review

I need guidance on the following architectural decisions:

1. Video Transcription: For video transcription, what is the intended approach?

   • Do we extract the audio and pass it to the transcription/audio model (e.g., Whisper, or an LLM with audio support)?
   • Or do we expect to send the video file/stream directly to a supported model (if any)?

2. Standardizing Return Types for Auto Transcription: The implementation currently involves model-specific methods (e.g., a distinct method for OpenAI and different approaches for Mistral). How should we establish a consistent, unified return type (e.g., a common TranscriptionResult object) across all models to ensure a standardized user interface?

Future Scope

• Groq Support: Should we also add an adapter for Groq?

Testing Notes

I have tested core components of the new features using a separate validation script.

• Test Failure Context: I am currently encountering a test setup error when running python cognee/tests/test_library.py, specifically: LLMAPIKeyNotSetError: LLM API key is not set. (Status code: 422). I need help on the correct environment variable or configuration file location for setting the LLM API key to resolve this and run the full test suite.

[pull-checklist]

Please make sure all the checkboxes are checked:

• I have tested these changes locally.
• I have reviewed the code changes.
• I have added end-to-end and unit tests (if applicable).
• I have updated the documentation and README.md file (if necessary).
• I have removed unnecessary code and debug statements.
• PR title is clear and follows the convention.
• I have tagged reviewers or team members for feedback.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

✨ Finishing touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment

Tip

📝 Customizable high-level summaries are now available in beta!

You can now customize how CodeRabbit generates the high-level summary in your pull requests — including its content, structure, tone, and formatting.

• Provide your own instructions using the high_level_summary_instructions setting.
• Format the summary however you like (bullet lists, tables, multi-section layouts, contributor stats, etc.).
• Use high_level_summary_in_walkthrough to move the summary from the description to the walkthrough section.

Example instruction:

"Divide the high-level summary into five sections:

1. 📝 Description — Summarize the main change in 50–60 words, explaining what was done.
2. 📓 References — List relevant issues, discussions, documentation, or related PRs.
3. 📦 Dependencies & Requirements — Mention any new/updated dependencies, environment variable changes, or configuration updates.
4. 📊 Contributor Summary — Include a Markdown table showing contributions:
   | Contributor | Lines Added | Lines Removed | Files Changed |
5. ✔️ Additional Notes — Add any extra reviewer context.
   Keep each section concise (under 200 words) and use bullet or numbered lists for clarity."

Note: This feature is currently in beta for Pro-tier users, and pricing will be announced later.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[github-actions]

Hello @rajeevrajeshuni, thank you for submitting a PR! We will respond as soon as possible.