generated from pulumi/pulumi-provider-boilerplate
-
Notifications
You must be signed in to change notification settings - Fork 6
Description
Context
Known bugs should have dedicated regression tests to prevent them from recurring. Currently, only one regression test exists (Team partial creation failure, issue #73).
Part of Epic: #572
Known Bugs Requiring Tests
1. OIDC Issuer Policy Ordering (Issue #542)
- Bug: Policies array order causes false drift detection
- Test: Verify that policy order doesn't trigger unnecessary updates
- Location:
provider/pkg/resources/oidc_issuer_test.go
2. Team Partial Creation Failure (Issue #73) ✅
- Status: Test already exists in
examples/examples_yaml_test.go - Action: Ensure this test is preserved in pulumitest migration (Migrate YAML Integration Tests to pulumitest Framework #567)
3. Additional Bugs to Document
- Review closed issues with
kind/buglabel - Identify bugs without regression tests
- Add tests for each
Requirements
For each bug:
- Create unit test demonstrating the bug would occur without the fix
- Add integration test if unit test is insufficient
- Document the bug reference (issue number) in test comments
- Verify test fails if fix is removed
- Verify test passes with fix in place
Example: OIDC Policy Ordering Test
func TestOidcIssuer_PolicyOrderingDoesNotCauseDrift(t *testing.T) {
// Regression test for https://github.com/pulumi/pulumi-pulumiservice/issues/542
t.Run("Policy array order should not cause drift", func(t *testing.T) {
// Create mock with policies in order A, B, C
originalPolicies := []Policy{
{Name: "PolicyA", ...},
{Name: "PolicyB", ...},
{Name: "PolicyC", ...},
}
// API returns policies in order C, B, A (reordered)
apiPolicies := []Policy{
{Name: "PolicyC", ...},
{Name: "PolicyB", ...},
{Name: "PolicyA", ...},
}
// Call Diff()
// Assert: No changes detected (order shouldn't matter)
// Assert: replaces is empty
})
}Acceptance Criteria
- All known bugs have regression tests
- Tests are clearly labeled with issue numbers
- Tests fail when fix is removed (verified)
- Tests pass with fix in place
- Documentation updated with regression test patterns
Investigation Required
- Search GitHub issues for bugs:
is:issue label:kind/bug is:closed - Check PR history for bug fixes without tests
- Interview team about known production bugs
Reference
- Existing regression test:
examples/examples_yaml_test.go(Team partial creation) - Issue Ensure oidc auth policies order is consistent to prevent drifts #542: OIDC policy ordering
- Issue Specifying a non-existent member during Team resource creation causes the Team to be marked failed but doesn't delete the team #73: Team partial creation failure
Estimated Effort
1 week (includes investigation)