Skip to content

Conversation

@wwyc
Copy link
Contributor

@wwyc wwyc commented Nov 24, 2025

Description

The current set up from #8362 uses an opt-in list to deprecate geo (city and subdivision) fields.

This PR changes the set up to use an opt-out approach to exempt specific apps and tables. In this case, any apps not listed in the config file (including future apps added) will have geo fields deprecated by default.

Related Tickets & Documents

Reviewer, please follow this checklist

@wwyc wwyc requested a review from a team as a code owner November 24, 2025 21:25
@dataops-ci-bot

This comment has been minimized.

@dataops-ci-bot

This comment has been minimized.


table = client.get_table(live_table)

# only glean tables have this label
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this mean we're explicitly only doing this for glean apps? I don't remember if this was discussed already

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes we are only deprecating the geo fields for glean apps for now. After the glean migration is completed the legacy tables would be deprecated.

@wwyc wwyc marked this pull request as draft November 24, 2025 23:59
@dataops-ci-bot

This comment has been minimized.

@dataops-ci-bot
Copy link

Integration report for "Updated ref to mapping method"

sql.diff

Click to expand!
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/monitoring_derived/shredder_targets_v1/query.py /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/monitoring_derived/shredder_targets_v1/query.py
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/monitoring_derived/shredder_targets_v1/query.py	2025-11-25 21:30:03.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/monitoring_derived/shredder_targets_v1/query.py	2025-11-25 21:25:34.000000000 +0000
@@ -14,6 +14,7 @@
 from google.cloud.bigquery import TableReference
 from google.cloud.exceptions import NotFound
 
+from bigquery_etl.cli.utils import get_glean_app_id_to_app_name_mapping
 from bigquery_etl.schema import Schema
 from bigquery_etl.shredder.config import (
     CLIENT_ID,
@@ -22,7 +23,6 @@
     SHARED_PROD,
     DeleteSource,
     find_glean_targets,
-    get_glean_channel_to_app_name_mapping,
 )
 
 FIND_TABLES_QUERY_TEMPLATE = """
@@ -169,7 +169,7 @@
                 dataset_name.replace("_derived", "_stable")
             ] = f"{dataset_name}.additional_deletion_requests_v1"
 
-    glean_channel_names = get_glean_channel_to_app_name_mapping()
+    glean_channel_names = get_glean_app_id_to_app_name_mapping()
 
     for table_name, stable_tables in upstream_stable_tables.items():
         deletion_tables: Set[DeleteSource] = set()
@@ -246,7 +246,7 @@
         bigquery_client = bigquery.Client()
         glean_delete_targets = find_glean_targets(pool, client=bigquery_client)
 
-    glean_channel_names = get_glean_channel_to_app_name_mapping()
+    glean_channel_names = get_glean_app_id_to_app_name_mapping()
     glean_app_name_to_channels = defaultdict(list)
     for channel, app_name in glean_channel_names.items():
         glean_app_name_to_channels[app_name].append(channel)

Link to full diff

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants