Skip to content

Commit cb58d4d

Browse files
langchain-infralangchain-infra
authored
Merge pull request #3 from langchain-ai/infra/docs
2 parents 2580015 + 5cf7041 commit cb58d4d

File tree

5 files changed

+524
-1
lines changed

5 files changed

+524
-1
lines changed

‎charts/README.md‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
# Langchain Helm Charts
2+
3+
This repository contains Helm charts for deploying Langchain applications on Kubernetes.
4+
5+
## License
6+
7+
Copyright © 2023 Langchain Inc.
8+
9+
Licensed under the Apache License, Version 2.0 (the "License");
10+
you may not use this file except in compliance with the License.
11+
You may obtain a copy of the License at
12+
13+
<http://www.apache.org/licenses/LICENSE-2.0>
14+
15+
Unless required by applicable law or agreed to in writing, software
16+
distributed under the License is distributed on an "AS IS" BASIS,
17+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
18+
See the License for the specific language governing permissions and
19+
limitations under the License.

‎charts/langsmith/README.md‎

Lines changed: 343 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,343 @@
1+
# langsmith
2+
3+
![Version: 0.1.0](https://img.shields.io/badge/Version-0.1.0-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 0.1.0](https://img.shields.io/badge/AppVersion-0.1.0-informational?style=flat-square)
4+
5+
Helm chart to deploy the langsmith application and all services it depends on.
6+
7+
## Deploying Langsmith with Helm
8+
9+
### Prerequisites
10+
11+
Ensure you have the following tools/items ready.
12+
13+
1. A working Kubernetes cluster that you can access via `kubectl`
14+
1. Recommended: Atleast 4 vCPUs, 16GB Memory available
15+
1. You may need to tune resource requests/limits for all of our different services based off of organization size/usage
16+
2. Valid Dynamic PV provisioner or PVs available on your cluster. You can verify this by running:
17+
18+
```jsx
19+
kubectl get storageclass
20+
```
21+
2. `Helm`
22+
1. `brew install helm`
23+
3. Langsmith License Key
24+
1. You can get this from your Langchain representative. Contact us at support@langchain.dev for more information.
25+
3. SSL(optional)
26+
1. This should be attachable to a load balancer that
27+
4. OpenAI API Key(optional).
28+
1. Used for natural language search feature. Can specify OpenAI key in browser as well for the playground feature.
29+
5. Oauth Configuration(optional).
30+
1. You can configure oauth using the `values.yaml` file. You will need to provide a `client_id` and `client_issuer_url` for your oauth provider. We currently support anything that is OIDC compliant.
31+
6. External Postgres(optional).
32+
1. You can configure external postgres using the `values.yaml` file. You will need to provide connection parameters for your postgres instance.
33+
7. External Redis(optional).
34+
1. You can configure external redis using the `values.yaml` file. You will need to provide a connection url for your redis instance.
35+
36+
### Configure your Helm Charts:
37+
38+
1. Create a copy of `values.yaml`
39+
2. Override any values in the file. Refer to the `values.yaml` documentation below to see all configurable values. Some values we recommend tuning:
40+
1. Resources
41+
2. SSL(If on EKS or some other cloud provider)
42+
1. Add an annotation to the `frontend.service` object to tell your cloud provider to provision a load balancer with said certificate attached
43+
3. OpenAI Api Key
44+
4. Images
45+
5. Oauth
46+
47+
Bare minimum config file `langsmith_config.yaml`:
48+
49+
```yaml
50+
secrets:
51+
langsmithLicenseKey: ""
52+
53+
```
54+
55+
Example `EKS` config file with certificates setup using ACM:
56+
57+
```jsx
58+
frontend:
59+
service:
60+
annotations:
61+
service.beta.kubernetes.io/aws-load-balancer-type: "nlb"
62+
service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "https"
63+
service.beta.kubernetes.io/aws-load-balancer-ssl-cert: "<certificate arn>"
64+
```
65+
66+
### Deploying to Kubernetes:
67+
68+
1. Verify that you can connect to your Kubernetes cluster(note: We highly suggest installing into an empty namespace)
69+
1. Run `kubectl get pods`
70+
71+
Output should look something like:
72+
73+
```bash
74+
kubectl get pods ⎈ langsmith-eks-2vauP7wf 21:07:46
75+
No resources found in default namespace.
76+
```
77+
78+
2. Navigate to the directory containing the helm charts provided by Langchain.
79+
3. Run `helm install langsmith . --values langsmith_config.yaml`
80+
4. Run `kubectl get pods`
81+
1. Output should now look something like:
82+
83+
```bash
84+
langsmith-backend-6ff46c99c4-wz22d 1/1 Running 0 3h2m
85+
langsmith-frontend-6bbb94c5df-8xrlr 1/1 Running 0 3h2m
86+
langsmith-hub-backend-5cc68c888c-vppjj 1/1 Running 0 3h2m
87+
langsmith-playground-6d95fd8dc6-x2d9b 1/1 Running 0 3h2m
88+
langsmith-postgres-0 1/1 Running 0 9h
89+
langsmith-queue-5898b9d566-tv6q8 1/1 Running 0 3h2m
90+
langsmith-redis-0 1/1 Running 0 9h
91+
```
92+
93+
### Validate your deployment:
94+
95+
1. Run `kubectl get services`
96+
97+
Output should look something like:
98+
99+
```bash
100+
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
101+
langsmith-backend ClusterIP 172.20.140.77 <none> 1984/TCP 35h
102+
langsmith-frontend LoadBalancer 172.20.253.251 <external ip> 80:31591/TCP 35h
103+
langsmith-hub-backend ClusterIP 172.20.112.234 <none> 1985/TCP 35h
104+
langsmith-playground ClusterIP 172.20.153.194 <none> 3001/TCP 9h
105+
langsmith-postgres ClusterIP 172.20.244.82 <none> 5432/TCP 35h
106+
langsmith-redis ClusterIP 172.20.81.217 <none> 6379/TCP 35h
107+
```
108+
109+
2. Curl the external ip of the `langsmith-frontend` service:
110+
111+
```bash
112+
curl <external ip>/tenants
113+
[{"id":"00000000-0000-0000-0000-000000000000","has_waitlist_access":true,"created_at":"2023-09-13T18:25:10.488407","display_name":"Personal","config":{"is_personal":true,"max_identities":1},"tenant_handle":"default"}]%
114+
```
115+
116+
3. Visit the external ip for the `langsmith-frontend` service on your browser
117+
118+
The Langsmith UI should be visible/operational
119+
120+
![./langsmith_ui.png](langsmith_ui.png)
121+
122+
### Using your deployment:
123+
124+
We typically validate deployment using the following Jupyter notebook:
125+
126+
1. [https://github.com/langchain-ai/langchain/blob/master/docs/docs/guides/langsmith/walkthrough.ipynb](https://github.com/langchain-ai/langchain/blob/master/docs/docs/guides/langsmith/walkthrough.ipynb)
127+
2. For `"LANGCHAIN_ENDPOINT"` you will want to use `<external ip>/api`
128+
3. For `LANGCHAIN_HUB_API_URL` you will want to use `<external ip>/api-hub`
129+
4. For `"LANGCHAIN_API_KEY"` you will want to set an API key you generate. If not using oauth, you can set this to some random value `"foo"`
130+
5. Run through the notebook and validate that all cells complete successfully.
131+
132+
## FAQ:
133+
134+
1. How can we upgrade our application.
135+
- We plan to release new minor versions of the Langsmith application every 6 weeks. This will include release notes and all changes should be backwards compatible. To upgrade, you will need to follow the upgrade instructions in the Helm README and run a `helm upgrade langsmith --values <values file>`
136+
2. Backups
137+
- Currently, we rely on PVCs/PV to power storage for our application. We strongly encourage setting up `Persistent Volume` backups or moving to a managed service for `Postgres` to support disaster recovery
138+
3. Load Balancers
139+
- Currently, our application spins up one load balancer using a k8s service of type `LoadBalancer` for our frontend. If you do not want to setup a load balancer you can simply port-forward the frontend and use that as your external ip for the application.
140+
4. Authentication
141+
- Currently, our self-hosted solution supports either oauth or no auth.
142+
5. Using External `Postgres` or `Redis`
143+
- You can configure external postgres or redis using the external sections in the `values.yaml` file. You will need to provide the connection url for the database/redis instance.
144+
6. Networking
145+
- Our deployment only needs egress for a few things:
146+
- Fetching images (If mirroring your images, this may not be needed)
147+
- Talking to any LLMs
148+
- Your VPC can set up rules to limit any other access.
149+
150+
## Values
151+
152+
| Key | Type | Default | Description |
153+
|-----|------|---------|-------------|
154+
| backend.autoscaling.enabled | bool | `false` | |
155+
| backend.autoscaling.maxReplicas | int | `5` | |
156+
| backend.autoscaling.minReplicas | int | `1` | |
157+
| backend.autoscaling.targetCPUUtilizationPercentage | int | `80` | |
158+
| backend.containerPort | int | `1984` | |
159+
| backend.deployment.affinity | object | `{}` | |
160+
| backend.deployment.annotations | object | `{}` | |
161+
| backend.deployment.labels | object | `{}` | |
162+
| backend.deployment.nodeSelector | object | `{}` | |
163+
| backend.deployment.podSecurityContext | object | `{}` | |
164+
| backend.deployment.replicaCount | int | `1` | |
165+
| backend.deployment.resources | object | `{}` | |
166+
| backend.deployment.securityContext | object | `{}` | |
167+
| backend.deployment.tolerations | list | `[]` | |
168+
| backend.migrations.affinity | object | `{}` | |
169+
| backend.migrations.annotations | object | `{}` | |
170+
| backend.migrations.labels | object | `{}` | |
171+
| backend.migrations.nodeSelector | object | `{}` | |
172+
| backend.migrations.podSecurityContext | object | `{}` | |
173+
| backend.migrations.replicaCount | int | `1` | |
174+
| backend.migrations.resources | object | `{}` | |
175+
| backend.migrations.securityContext | object | `{}` | |
176+
| backend.migrations.tolerations | list | `[]` | |
177+
| backend.name | string | `"backend"` | |
178+
| backend.service.annotations | object | `{}` | |
179+
| backend.service.labels | object | `{}` | |
180+
| backend.service.port | int | `1984` | |
181+
| backend.service.type | string | `"ClusterIP"` | |
182+
| commonAnnotations | object | `{}` | |
183+
| commonLabels | object | `{}` | |
184+
| config.existingSecretName | string | `""` | |
185+
| config.langsmithLicenseKey | string | `""` | |
186+
| config.oauth.enabled | bool | `false` | |
187+
| config.oauth.oauthClientId | string | `""` | |
188+
| config.oauth.oauthIssuerUrl | string | `""` | |
189+
| config.openaiApiKey | string | `""` | |
190+
| frontend.autoscaling.enabled | bool | `false` | |
191+
| frontend.autoscaling.maxReplicas | int | `5` | |
192+
| frontend.autoscaling.minReplicas | int | `1` | |
193+
| frontend.autoscaling.targetCPUUtilizationPercentage | int | `80` | |
194+
| frontend.containerPort | int | `8080` | |
195+
| frontend.deployment.affinity | object | `{}` | |
196+
| frontend.deployment.annotations | object | `{}` | |
197+
| frontend.deployment.labels | object | `{}` | |
198+
| frontend.deployment.nodeSelector | object | `{}` | |
199+
| frontend.deployment.podSecurityContext | object | `{}` | |
200+
| frontend.deployment.replicaCount | int | `1` | |
201+
| frontend.deployment.resources | object | `{}` | |
202+
| frontend.deployment.securityContext | object | `{}` | |
203+
| frontend.deployment.tolerations | list | `[]` | |
204+
| frontend.name | string | `"frontend"` | |
205+
| frontend.service.annotations | object | `{}` | |
206+
| frontend.service.httpPort | int | `80` | |
207+
| frontend.service.httpsPort | int | `443` | |
208+
| frontend.service.labels | object | `{}` | |
209+
| frontend.service.type | string | `"LoadBalancer"` | |
210+
| fullnameOverride | string | `""` | |
211+
| hubBackend.autoscaling.enabled | bool | `false` | |
212+
| hubBackend.autoscaling.maxReplicas | int | `5` | |
213+
| hubBackend.autoscaling.minReplicas | int | `1` | |
214+
| hubBackend.autoscaling.targetCPUUtilizationPercentage | int | `80` | |
215+
| hubBackend.containerPort | int | `1985` | |
216+
| hubBackend.deployment.affinity | object | `{}` | |
217+
| hubBackend.deployment.annotations | object | `{}` | |
218+
| hubBackend.deployment.labels | object | `{}` | |
219+
| hubBackend.deployment.nodeSelector | object | `{}` | |
220+
| hubBackend.deployment.podSecurityContext | object | `{}` | |
221+
| hubBackend.deployment.replicaCount | int | `1` | |
222+
| hubBackend.deployment.resources | object | `{}` | |
223+
| hubBackend.deployment.securityContext | object | `{}` | |
224+
| hubBackend.deployment.tolerations | list | `[]` | |
225+
| hubBackend.name | string | `"hub-backend"` | |
226+
| hubBackend.service.annotations | object | `{}` | |
227+
| hubBackend.service.labels | object | `{}` | |
228+
| hubBackend.service.port | int | `1985` | |
229+
| hubBackend.service.type | string | `"ClusterIP"` | |
230+
| images.backendImage.pullPolicy | string | `"Always"` | |
231+
| images.backendImage.repository | string | `"docker.io/langchain/langchainplus-backend"` | |
232+
| images.backendImage.tag | string | `"latest"` | |
233+
| images.frontendImage.pullPolicy | string | `"Always"` | |
234+
| images.frontendImage.repository | string | `"docker.io/langchain/langchainplus-frontend-dynamic"` | |
235+
| images.frontendImage.tag | string | `"latest"` | |
236+
| images.hubBackendImage.pullPolicy | string | `"Always"` | |
237+
| images.hubBackendImage.repository | string | `"docker.io/langchain/langchainhub-backend"` | |
238+
| images.hubBackendImage.tag | string | `"latest"` | |
239+
| images.imagePullSecrets | list | `[]` | |
240+
| images.playgroundImage.pullPolicy | string | `"Always"` | |
241+
| images.playgroundImage.repository | string | `"docker.io/langchain/langchainplus-playground"` | |
242+
| images.playgroundImage.tag | string | `"latest"` | |
243+
| images.postgresImage.pullPolicy | string | `"Always"` | |
244+
| images.postgresImage.repository | string | `"docker.io/postgres"` | |
245+
| images.postgresImage.tag | string | `"14.7"` | |
246+
| images.redisImage.pullPolicy | string | `"Always"` | |
247+
| images.redisImage.repository | string | `"docker.io/redis"` | |
248+
| images.redisImage.tag | string | `"7"` | |
249+
| ingress.annotations | object | `{}` | |
250+
| ingress.enabled | bool | `false` | |
251+
| ingress.hostname | string | `""` | |
252+
| ingress.ingressClassName | string | `""` | |
253+
| ingress.labels | object | `{}` | |
254+
| ingress.tls | list | `[]` | |
255+
| nameOverride | string | `""` | |
256+
| playground.autoscaling.enabled | bool | `false` | |
257+
| playground.autoscaling.maxReplicas | int | `5` | |
258+
| playground.autoscaling.minReplicas | int | `1` | |
259+
| playground.autoscaling.targetCPUUtilizationPercentage | int | `80` | |
260+
| playground.containerPort | int | `3001` | |
261+
| playground.deployment.affinity | object | `{}` | |
262+
| playground.deployment.annotations | object | `{}` | |
263+
| playground.deployment.labels | object | `{}` | |
264+
| playground.deployment.nodeSelector | object | `{}` | |
265+
| playground.deployment.podSecurityContext | object | `{}` | |
266+
| playground.deployment.replicaCount | int | `1` | |
267+
| playground.deployment.resources | object | `{}` | |
268+
| playground.deployment.securityContext | object | `{}` | |
269+
| playground.deployment.tolerations | list | `[]` | |
270+
| playground.name | string | `"playground"` | |
271+
| playground.service.annotations | object | `{}` | |
272+
| playground.service.labels | object | `{}` | |
273+
| playground.service.port | int | `3001` | |
274+
| playground.service.type | string | `"ClusterIP"` | |
275+
| postgres.containerPort | int | `5432` | |
276+
| postgres.external.database | string | `""` | |
277+
| postgres.external.enabled | bool | `false` | |
278+
| postgres.external.existingSecretName | string | `""` | |
279+
| postgres.external.host | string | `""` | |
280+
| postgres.external.password | string | `""` | |
281+
| postgres.external.port | int | `5432` | |
282+
| postgres.external.user | string | `""` | |
283+
| postgres.name | string | `"postgres"` | |
284+
| postgres.service.annotations | object | `{}` | |
285+
| postgres.service.labels | object | `{}` | |
286+
| postgres.service.port | int | `5432` | |
287+
| postgres.service.type | string | `"ClusterIP"` | |
288+
| postgres.statefulSet.affinity | object | `{}` | |
289+
| postgres.statefulSet.annotations | object | `{}` | |
290+
| postgres.statefulSet.labels | object | `{}` | |
291+
| postgres.statefulSet.nodeSelector | object | `{}` | |
292+
| postgres.statefulSet.persistence.enabled | bool | `false` | |
293+
| postgres.statefulSet.persistence.size | string | `"8Gi"` | |
294+
| postgres.statefulSet.persistence.storageClassName | string | `""` | |
295+
| postgres.statefulSet.podSecurityContext | object | `{}` | |
296+
| postgres.statefulSet.resources | object | `{}` | |
297+
| postgres.statefulSet.securityContext | object | `{}` | |
298+
| postgres.statefulSet.tolerations | list | `[]` | |
299+
| queue.autoscaling.enabled | bool | `false` | |
300+
| queue.autoscaling.maxReplicas | int | `5` | |
301+
| queue.autoscaling.minReplicas | int | `1` | |
302+
| queue.autoscaling.targetCPUUtilizationPercentage | int | `80` | |
303+
| queue.deployment.affinity | object | `{}` | |
304+
| queue.deployment.annotations | object | `{}` | |
305+
| queue.deployment.labels | object | `{}` | |
306+
| queue.deployment.nodeSelector | object | `{}` | |
307+
| queue.deployment.podSecurityContext | object | `{}` | |
308+
| queue.deployment.replicaCount | int | `1` | |
309+
| queue.deployment.resources | object | `{}` | |
310+
| queue.deployment.securityContext | object | `{}` | |
311+
| queue.deployment.tolerations | list | `[]` | |
312+
| queue.name | string | `"queue"` | |
313+
| redis.containerPort | int | `6379` | |
314+
| redis.external.connectionUrl | string | `""` | |
315+
| redis.external.enabled | bool | `false` | |
316+
| redis.external.existingSecretName | string | `""` | |
317+
| redis.name | string | `"redis"` | |
318+
| redis.service.annotations | object | `{}` | |
319+
| redis.service.labels | object | `{}` | |
320+
| redis.service.port | int | `6379` | |
321+
| redis.service.type | string | `"ClusterIP"` | |
322+
| redis.statefulSet.affinity | object | `{}` | |
323+
| redis.statefulSet.annotations | object | `{}` | |
324+
| redis.statefulSet.labels | object | `{}` | |
325+
| redis.statefulSet.nodeSelector | object | `{}` | |
326+
| redis.statefulSet.persistence.enabled | bool | `false` | |
327+
| redis.statefulSet.persistence.size | string | `"8Gi"` | |
328+
| redis.statefulSet.persistence.storageClassName | string | `""` | |
329+
| redis.statefulSet.podSecurityContext | object | `{}` | |
330+
| redis.statefulSet.resources | object | `{}` | |
331+
| redis.statefulSet.securityContext | object | `{}` | |
332+
| redis.statefulSet.tolerations | list | `[]` | |
333+
334+
## Maintainers
335+
336+
| Name | Email | Url |
337+
| ---- | ------ | --- |
338+
| Ankush | <ankush@langchain.dev> | |
339+
340+
----------------------------------------------
341+
Autogenerated from chart metadata using [helm-docs v1.11.3](https://github.com/norwoodj/helm-docs/releases/v1.11.3)
342+
## Docs Generated by [helm-docs](https://github.com/norwoodj/helm-docs)
343+
`helm-docs -t .README.md.gotmpl`

0 commit comments

Comments
 (0)