Skip to content

Add regression tests for known bugs #577

@rshade

Description

@rshade

Context

Known bugs should have dedicated regression tests to prevent them from recurring. Currently, only one regression test exists (Team partial creation failure, issue #73).

Part of Epic: #572

Known Bugs Requiring Tests

1. OIDC Issuer Policy Ordering (Issue #542)

  • Bug: Policies array order causes false drift detection
  • Test: Verify that policy order doesn't trigger unnecessary updates
  • Location: provider/pkg/resources/oidc_issuer_test.go

2. Team Partial Creation Failure (Issue #73) ✅

3. Additional Bugs to Document

  • Review closed issues with kind/bug label
  • Identify bugs without regression tests
  • Add tests for each

Requirements

For each bug:

  1. Create unit test demonstrating the bug would occur without the fix
  2. Add integration test if unit test is insufficient
  3. Document the bug reference (issue number) in test comments
  4. Verify test fails if fix is removed
  5. Verify test passes with fix in place

Example: OIDC Policy Ordering Test

func TestOidcIssuer_PolicyOrderingDoesNotCauseDrift(t *testing.T) {
    // Regression test for https://github.com/pulumi/pulumi-pulumiservice/issues/542
    t.Run("Policy array order should not cause drift", func(t *testing.T) {
        // Create mock with policies in order A, B, C
        originalPolicies := []Policy{
            {Name: "PolicyA", ...},
            {Name: "PolicyB", ...},
            {Name: "PolicyC", ...},
        }

        // API returns policies in order C, B, A (reordered)
        apiPolicies := []Policy{
            {Name: "PolicyC", ...},
            {Name: "PolicyB", ...},
            {Name: "PolicyA", ...},
        }

        // Call Diff()
        // Assert: No changes detected (order shouldn't matter)
        // Assert: replaces is empty
    })
}

Acceptance Criteria

  • All known bugs have regression tests
  • Tests are clearly labeled with issue numbers
  • Tests fail when fix is removed (verified)
  • Tests pass with fix in place
  • Documentation updated with regression test patterns

Investigation Required

  • Search GitHub issues for bugs: is:issue label:kind/bug is:closed
  • Check PR history for bug fixes without tests
  • Interview team about known production bugs

Reference

Estimated Effort

1 week (includes investigation)

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions