POC: EDP Prometheus metrics

[tbuchaillot]

Description

Summary:
Adds support for portal_app and portal_org labels in Prometheus metrics by extracting them from analytics record tags.

Changes:

• Added PortalAppTag() and PortalOrgTag() helper functions to extract portal identifiers from tags with portal_app_ and portal_org_ prefixes
• Extended label mapping in GetLabelsValues() to include portal_app and portal_org dimensions
• Returns portal_app_unknown / portal_org_unknown when tags are not present
• Added comprehensive test coverage (29 new test cases) for the new functionality

Use Case:
Enables tracking and filtering of Prometheus metrics by portal application and organization, allowing better observability and analytics segmentation for multi-tenant deployments.

Related Issue

Motivation and Context

How This Has Been Tested

Screenshots (if appropriate)

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)

Checklist

• Make sure you are requesting to pull a topic/feature/bugfix branch (right side). If pulling from your own
  fork, don't request your master!
• Make sure you are making a pull request against the master branch (left side). Also, you should start
  your branch off our latest master.
• My change requires a change to the documentation.
  • If you've changed APIs, describe what needs to be updated in the documentation.
• I have updated the documentation accordingly.
• Modules and vendor dependencies have been updated; run go mod tidy && go mod vendor
• I have added tests to cover my changes.
• All new and existing tests passed.
• Check your code additions will not fail linting checks:
  • go fmt -s
  • go vet

[probelabs]

🔍 Code Analysis Results

This PR introduces support for portal_app and portal_org labels in Prometheus metrics, allowing for better observability of API traffic originating from the Tyk Enterprise Developer Portal (EDP). The new labels are extracted from tags prefixed with portal-app- and portal-org- within analytics records. The changes are isolated to the Prometheus pump and are accompanied by thorough test coverage.

A significant portion of this PR consists of newly added documentation files related to MongoDB and a log file (logs/combined.log), which appear to be unrelated to the core change and may have been committed by mistake. These files add considerable noise to the review process.

Files Changed Analysis

• README.md: The documentation is updated to include portal_app and portal_org in the list of available labels for Prometheus metrics.
• pumps/prometheus.go: The core logic is implemented here. Two new helper functions, PortalAppTag and PortalOrgTag, are added to extract the relevant portal information from analytics record tags. The GetLabelsValues function is updated to include these new labels.
• pumps/prometheus_test.go: Adds extensive unit tests (256 new lines) to validate the new tag extraction logic and ensure the labels are correctly applied.
• logs/combined.log, mongo-aggregate-upsert-examples.md, tyk-pump-mongodb-analysis.md: These newly added files seem unrelated to the Prometheus metrics enhancement and should be reviewed for their relevance to this PR.

Architecture & Impact Assessment

What this PR accomplishes

This PR enhances the Prometheus pump to allow segmenting API metrics by the originating developer portal application and organization. This provides deeper visibility into API usage patterns for multi-tenant deployments using the Tyk Developer Portal.

Key technical changes introduced

• Tag Parsing: Introduces two functions, PortalAppTag and PortalOrgTag, to iterate through the Tags slice of an analytics.AnalyticsRecord and find the first tag matching the portal-app- or portal-org- prefix.
• Label Mapping: Extends the label mapping in GetLabelsValues to make portal_app and portal_org available for use in custom Prometheus metrics.
• Default Values: If no matching tags are found, it returns a default value (portal-app-unknown or portal-org-unknown) to ensure label consistency.

Affected system components

The changes are confined to the Prometheus pump within Tyk Pump. This is an additive change and will not affect existing Prometheus configurations unless the new labels are explicitly used. The functionality is dependent on the Tyk Gateway sending analytics records that contain the appropriate tags.

Component Interaction Diagram

graph TD
    A[Tyk Gateway] --sends--> B(AnalyticsRecord w/ Tags);
    B --processed by--> C[Tyk Pump];
    C --forwards to--> D{Prometheus Pump};
    subgraph D
        E[GetLabelsValues function]
    end
    B --'portal-app-*', 'portal-org-*' tags--> E;
    E --calls--> F[PortalAppTag / PortalOrgTag helpers];
    F --extract tag value--> E;
    E --adds to label set--> G[Prometheus Metric];
    G --exported to--> H[Prometheus Server];

Loading

Scope Discovery & Context Expansion

The change is localized to the Prometheus pump's data processing logic. The broader context relies on an end-to-end data flow where the Tyk Gateway (the data producer) correctly tags analytics records. The analytics.AnalyticsRecord struct, specifically its Tags []string field, serves as the contract. The implementation is self-contained and does not impact other pumps or parts of the Tyk Pump application.

Metadata

• Review Effort: 2 / 5
• Primary Label: enhancement

---

Powered by Visor from Probelabs

Last updated: 2025-11-11T08:18:36.500Z | Triggered by: synchronize | Commit: df94fd1

💡 TIP: You can chat with Visor using /visor ask <your question>

[probelabs]

🔍 Code Analysis Results

✅ Security Check Passed

No security issues found – changes LGTM.

✅ Architecture Check Passed

No architecture issues found – changes LGTM.

✅ Performance Check Passed

No performance issues found – changes LGTM.

✅ Quality Check Passed

No quality issues found – changes LGTM.

---

Powered by Visor from Probelabs

Last updated: 2025-11-14T09:39:21.452Z | Triggered by: synchronize | Commit: 95b3e23

💡 TIP: You can chat with Visor using /visor ask <your question>

[github-actions]

🚨 Jira Linter Failed

Commit: 95b3e23
Failed at: 2025-11-14 09:39:42 UTC

The Jira linter failed to validate your PR. Please check the error details below:

🔍 Click to view error details

failed to validate branch and PR title rules: branch name 'poc/edp-dimension' must contain a valid Jira ticket ID (e.g., ABC-123)

Next Steps

• Ensure your branch name contains a valid Jira ticket ID (e.g., ABC-123)
• Verify your PR title matches the branch's Jira ticket ID
• Check that the Jira ticket exists and is accessible

---

This comment will be automatically deleted once the linter passes.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
100.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud