This is a Bedrock (and other generative AI tools from AWS) API forwarding tool that can issue virtual keys, log chats, and manage costs.
It is compatible with any OPENAI client that can define Host and API Key.
Docker Image: DockerHub, Public ECR
Docker Image for Lambda: DockerHub, Public ECR
- Supports all current and future large language models from Bedrock (supported through bedrock-converse).
- Supports models deployed through Sagemaker LMI (partial models).
- Supports other forms of custom models, including Ollama, etc.
- More AI workflow applications, such as internet search, AWS command executors, etc.
- Create API keys. Keys can be created for regular users and administrators. Regular users can chat, while administrators can manage API keys and costs.
- Record the cost of each call and use it as a basis for cost control.
- Cost control. You can set monthly quotas and account balances for each API key. When the monthly quota or account balance is insufficient, it cannot be used.
- Calculate the overall cost.
Important
The cost calculation of this project cannot be used as the basis for AWS billing. Please refer to the AWS bill for actual expenses. Please refer to the official website for the Bedrock pricing.
Models and their parameters can be defined from the backend.
Once defined, models can be bound to groups or API Keys.
-
Bedrock App Profiles Support - Added support for Bedrock application profiles, allowing better configuration management for different deployment scenarios. #87
-
PostgreSQL Database Upgrade - Upgraded standalone PostgreSQL database from version 16.3 to 16.10 for improved performance and security. #89
-
Zero-Length Messages Fix - Fixed an issue where Bedrock would fail when processing zero-length messages. #90
-
Tool Use Enhancement - Improved tool use functionality with bug fixes and stability improvements. #91 #92
-
bedrock-converse Provider Fix - Fixed an issue with Claude new models: 'The model returned the following errors: temperature and top_p cannot both be specified for this model. Please use only one.'
-
Frontend Caching Enhancement - Added 1-month caching for frontend pages served by koa-static-server.
This connector is an open-source software aimed at providing proxy services for using Bedrock services. We make our best efforts to ensure the security and legality of the software, but we are not responsible for the users' behavior.
The connector is intended solely for personal learning and research purposes. Users shall not use it for any illegal activities, including but not limited to hacking, spreading illegal information, etc. Otherwise, users shall bear the corresponding legal responsibilities themselves. Users are responsible for complying with the laws and regulations in their respective jurisdictions and shall not use the connector for any illegal or non-compliant purposes. The developers and maintainers of this software shall not be liable for any disputes, losses, or legal liabilities arising from the use of this connector.
We reserve the right to modify or terminate the connector's code at any time without further notice. Users are expected to understand and comply with the relevant local laws and regulations.
If you have any questions regarding this disclaimer, please feel free to contact us through the open-source channels.
See CONTRIBUTING for more information.
This library is licensed under the MIT-0 License. See the LICENSE file.