feat: Add istio profile for E2E testing framework #728
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
✅ Deploy Preview for vllm-semantic-router ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
👥 vLLM Semantic Team NotificationThe following members have been identified for the changed files in this PR and have been automatically assigned: 📁
|
asaadbalum
force-pushed
the
istio_profile
branch
from
607b8e2 to
0ce7df0
Compare
Collaborator
|
@srampal PTAL, thanks |
Author
|
Hello Reviewers. |
asaadbalum
force-pushed
the
istio_profile
branch
2 times, most recently
from
c71d529 to
bccdccf
Compare
asaadbalum
force-pushed
the
istio_profile
branch
3 times, most recently
from
33927d7 to
0c3075f
Compare
Add comprehensive E2E testing profile for Istio service mesh integration, enabling validation of Semantic Router functionality within Istio's service mesh architecture. Implementation: - Add Istio profile with Helm-based installation of Istio components - Implement 4 Istio-specific test cases: * Sidecar health check and injection validation * Traffic routing through Istio Gateway and VirtualService * mTLS verification for secure service communication * Distributed tracing and observability integration - Integrate profile into CI/CD pipeline matrix - Update documentation with Istio profile usage and configuration Technical Details: - Uses Helm charts for Istio installation (base, istiod, gateway) - Automatic sidecar injection via namespace labeling - Configures Gateway, VirtualService, and DestinationRule - Validates mTLS, metrics collection, and tracing headers - Zero breaking changes to existing profiles Closes vllm-project#656 Signed-off-by: AI Assistant <[email protected]> Signed-off-by: Asaad Balum <[email protected]>
asaadbalum
force-pushed
the
istio_profile
branch
from
0c3075f to
9b579bb
Compare
Add Demo LLM (vLLM backend) deployment to Istio profile to fix common E2E test failures. Root cause: Istio profile was missing the vLLM backend that serves LLM requests. This caused 404 errors on /v1/chat/completions and 0% accuracy on all common tests (domain-classify, pii-detection, jailbreak-detection, etc.). Changes: - Add Step 4/6: Deploy Demo LLM backend (base-model.yaml) - Add deployDemoLLM() function to deploy vLLM backend - Add kubectl helper functions (kubectlApply, kubectlDelete, runKubectl) - Update cleanup functions to remove Demo LLM on teardown - Add timeoutDemoLLMDeploy constant for deployment timeout This fix brings the Istio profile in line with AIBrix and AI-Gateway profiles, which both deploy the vLLM backend for common tests. Expected result: All 17 tests (4 Istio-specific + 13 common) should now pass in CI. Signed-off-by: Asaad Balum <[email protected]>
asaadbalum
force-pushed
the
istio_profile
branch
from
cfa85ab to
41a0c63
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR implements the Istio profile for the E2E testing framework as requested in issue #656.
Overview
Adds comprehensive Istio service mesh integration testing for Semantic Router, enabling validation of sidecar injection, traffic routing, mTLS, and observability features.
What Changed
Istio Profile Implementation (
e2e/profiles/istio/)5-stage deployment pipeline:
Helm-based installation: Uses official Istio Helm charts from
istio-release.storage.googleapis.comConfigurable version: Override via
ISTIO_VERSIONenvironment variable (default: 1.28.0)Robust error handling: Panic recovery with defer-based cleanup for partial deployments
Health verification: 60-second stabilization period ensures mesh readiness before tests
Test Cases (4/4 Passing - 100%)
All tests required by issue #656 implemented and verified:
istio-sidecar-health-checkistio-traffic-routingistio-mtls-verificationistio-tracing-observabilityHelper Function Enhancement (
e2e/pkg/helpers/kubernetes.go)GetServiceByLabelInNamespace()for generic service lookup by namespaceGetEnvoyServiceName()as backward-compatible wrapper (hardcoded toenvoy-gateway-system)istio-systemnamespaceCI/CD Integration
istioto test matrix in.github/workflows/integration-test-k8s.ymlDocumentation (
e2e/README.md)Design Decisions
Helm vs istioctl
Decision: Use Helm charts for Istio installation
Rationale:
istioctlinstallation required)Test Scope
Focus: Istio-specific functionality (sidecar, gateway, mTLS, observability)
Excluded: Signal-decision engine tests (can be added in follow-up PR, following pattern from PR #695)
Kubernetes Version
Approach: Use Kind's default (v1.29.2 in CI)
Rationale: CI uses Kind v0.22.0 which defaults to K8s 1.29.2, meeting Istio 1.28+ requirements without explicit pinning
Testing Done
✅ All 4 Istio test cases pass (100% success rate)
✅ Istio installed via Helm (base, istiod, ingress gateway verified)
✅ Sidecar injection confirmed (istio-proxy container present and healthy)
✅ Traffic routing through Istio gateway successful (HTTP 200, Envoy headers detected)
✅ mTLS configuration verified (ISTIO_MUTUAL mode in DestinationRule)
✅ Observability features validated (Envoy metrics, Istio telemetry)
✅ Clean teardown confirmed (all Helm releases uninstalled, namespace deleted)
✅ Backward compatibility maintained (existing profiles unchanged)
Usage
Run all Istio tests
make e2e-test E2E_PROFILE=istio
Run with verbose output
make e2e-test E2E_PROFILE=istio E2E_VERBOSE=1
Test with specific Istio version
ISTIO_VERSION=1.28.0 make e2e-test E2E_PROFILE=istio
Setup only (for debugging)
make e2e-setup E2E_PROFILE=istio
Files Changed
e2e/profiles/istio/profile.go(607 lines) - Istio profile implementatione2e/testcases/istio_*.go(765 lines) - 4 test case implementationse2e/pkg/helpers/kubernetes.go- Generic service lookup helpere2e/cmd/e2e/main.go- Profile registratione2e/README.md- Istio profile documentation.github/workflows/integration-test-k8s.yml- CI integrationtools/make/e2e.mk- Make target help textTotal: 11 files changed, +1,522 lines, -10 lines
FIX #656