Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: baseline_clients_daily to only grab distribution_id for fenix app #5614

Merged
merged 1 commit into from
May 21, 2024
Merged

fix: baseline_clients_daily to only grab distribution_id for fenix app #5614

merged 1 commit into from
May 21, 2024

Conversation

kik-kik
Copy link
Contributor

@kik-kik kik-kik commented May 18, 2024 •
edited by data-sync-user

fix: baseline_clients_daily to only grab distribution_id for fenix app

This is to address an incorrect assumption that every baseline ping would have the same structure. It turns out the field containing distribution_id only exists for Fenix.

Checklist for reviewer:

  • Commits should reference a bug or github issue, if relevant (if a bug is referenced, the pull request should include the bug number in the title).
  • If the PR comes from a fork, trigger integration CI tests by running the Push to upstream workflow and provide the <username>:<branch> of the fork as parameter. The parameter will also show up
    in the logs of the manual-trigger-required-for-fork CI task together with more detailed instructions.
  • If adding a new field to a query, ensure that the schema and dependent downstream schemas have been updated.
  • When adding a new derived dataset, ensure that data is not available already (fully or partially) and recommend extending an existing dataset in favor of creating new ones. Data can be available in the bigquery-etl repository, looker-hub or in looker-spoke-default.

For modifications to schemas in restricted namespaces (see CODEOWNERS):

┆Issue is synchronized with this Jira Task

@kik-kik kik-kik added the bug Something isn't working label May 18, 2024
@kik-kik kik-kik self-assigned this May 18, 2024
@kik-kik kik-kik requested a review from scholtzan May 18, 2024 09:13
@dataops-ci-bot

This comment has been minimized.

Sign in to view
sean-rose
sean-rose reviewed May 18, 2024
View reviewed changes
Copy link
Contributor

@sean-rose sean-rose left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should the metrics_distribution_id field you added to the org_mozilla_ios_firefox_stable.baseline_v1.schema.json test files in #5611 be removed, since the field is only present in Fenix?

sql_generators/glean_usage/templates/baseline_clients_daily_v1.query.sql Outdated Show resolved Hide resolved
@kik-kik kik-kik force-pushed the fix/baseline_clients_daily-only-get-distribution_id-for-fenix branch 2 times, most recently from fc4d5ea to c809303 Compare May 21, 2024 10:20
@kik-kik kik-kik force-pushed the fix/baseline_clients_daily-only-get-distribution_id-for-fenix branch from c809303 to 01ffce0 Compare May 21, 2024 10:21
@dataops-ci-bot
Copy link

Integration report for "fix: baseline_clients_daily to only grab distribution_id for fenix app"

sql.diff

Click to expand! 
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/event_monitoring_live_v1/materialized_view.sql	2024-05-21 10:23:17.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/event_monitoring_live_v1/materialized_view.sql	2024-05-21 10:25:25.000000000 +0000
@@ -60,7 +60,7 @@
         client_info.app_display_version AS version,
         ping_info
       FROM
-        `moz-fx-data-shared-prod.bedrock_live.events_v1`
+        `moz-fx-data-shared-prod.bedrock_live.non_interaction_v1`
       UNION ALL
       SELECT
         submission_timestamp,
@@ -70,7 +70,7 @@
         client_info.app_display_version AS version,
         ping_info
       FROM
-        `moz-fx-data-shared-prod.bedrock_live.non_interaction_v1`
+        `moz-fx-data-shared-prod.bedrock_live.events_v1`
     )
   CROSS JOIN
     UNNEST(events) AS event,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml	2024-05-21 10:22:42.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml	2024-05-21 10:32:25.000000000 +0000
@@ -1,49 +1,49 @@
 fields:
-- mode: NULLABLE
-  name: submission_date
+- name: submission_date
   type: DATE
-- mode: NULLABLE
-  name: source
+  mode: NULLABLE
+- name: source
   type: STRING
-- mode: NULLABLE
-  name: event_type
+  mode: NULLABLE
+- name: event_type
   type: STRING
-- mode: NULLABLE
-  name: form_factor
+  mode: NULLABLE
+- name: form_factor
   type: STRING
-- mode: NULLABLE
-  name: country
+  mode: NULLABLE
+- name: country
   type: STRING
-- mode: NULLABLE
-  name: subdivision1
+  mode: NULLABLE
+- name: subdivision1
   type: STRING
-- mode: NULLABLE
-  name: advertiser
+  mode: NULLABLE
+- name: advertiser
   type: STRING
-- mode: NULLABLE
-  name: release_channel
+  mode: NULLABLE
+- name: release_channel
   type: STRING
-- mode: NULLABLE
-  name: position
+  mode: NULLABLE
+- name: position
   type: INTEGER
-- mode: NULLABLE
-  name: provider
+  mode: NULLABLE
+- name: provider
   type: STRING
-- mode: NULLABLE
-  name: match_type
+  mode: NULLABLE
+- name: match_type
   type: STRING
-- mode: NULLABLE
-  name: normalized_os
+  mode: NULLABLE
+- name: normalized_os
   type: STRING
-- mode: NULLABLE
-  name: suggest_data_sharing_enabled
+  mode: NULLABLE
+- name: suggest_data_sharing_enabled
   type: BOOLEAN
-- mode: NULLABLE
-  name: event_count
+  mode: NULLABLE
+- name: event_count
   type: INTEGER
-- mode: NULLABLE
-  name: user_count
+  mode: NULLABLE
+- name: user_count
   type: INTEGER
-- mode: NULLABLE
-  name: query_type
+  mode: NULLABLE
+- name: query_type
   type: STRING
+  mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml	2024-05-21 10:22:42.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml	2024-05-21 10:32:25.000000000 +0000
@@ -1,40 +1,40 @@
 fields:
-- mode: NULLABLE
-  name: submission_date
+- name: submission_date
   type: DATE
-- mode: NULLABLE
-  name: form_factor
+  mode: NULLABLE
+- name: form_factor
   type: STRING
-- mode: NULLABLE
-  name: country
+  mode: NULLABLE
+- name: country
   type: STRING
-- mode: NULLABLE
-  name: advertiser
+  mode: NULLABLE
+- name: advertiser
   type: STRING
-- mode: NULLABLE
-  name: normalized_os
+  mode: NULLABLE
+- name: normalized_os
   type: STRING
-- mode: NULLABLE
-  name: release_channel
+  mode: NULLABLE
+- name: release_channel
   type: STRING
-- mode: NULLABLE
-  name: position
+  mode: NULLABLE
+- name: position
   type: INTEGER
-- mode: NULLABLE
-  name: provider
+  mode: NULLABLE
+- name: provider
   type: STRING
-- mode: NULLABLE
-  name: match_type
+  mode: NULLABLE
+- name: match_type
   type: STRING
-- mode: NULLABLE
-  name: suggest_data_sharing_enabled
+  mode: NULLABLE
+- name: suggest_data_sharing_enabled
   type: BOOLEAN
-- mode: NULLABLE
-  name: impression_count
+  mode: NULLABLE
+- name: impression_count
   type: INTEGER
-- mode: NULLABLE
-  name: click_count
+  mode: NULLABLE
+- name: click_count
   type: INTEGER
-- mode: NULLABLE
-  name: query_type
+  mode: NULLABLE
+- name: query_type
   type: STRING
+  mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml	2024-05-21 10:22:42.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml	2024-05-21 10:32:37.000000000 +0000
@@ -26,6 +26,9 @@
 - name: adjust_network
   type: STRING
   mode: NULLABLE
+- name: install_source
+  type: STRING
+  mode: NULLABLE
 - name: retained_week_2
   type: BOOLEAN
   mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml	2024-05-21 10:22:42.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml	2024-05-21 10:32:37.000000000 +0000
@@ -48,6 +48,10 @@
   description: 'The type of source of a client installation.
 
     '
+- name: install_source
+  type: STRING
+  mode: NULLABLE
+  description: null
 - name: new_profiles
   type: INTEGER
   mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_tasks_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_tasks_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_tasks_derived/event_monitoring_live_v1/materialized_view.sql	2024-05-21 10:23:17.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_tasks_derived/event_monitoring_live_v1/materialized_view.sql	2024-05-21 10:25:25.000000000 +0000
@@ -50,7 +50,7 @@
         client_info.app_display_version AS version,
         ping_info
       FROM
-        `moz-fx-data-shared-prod.firefox_desktop_background_tasks_live.events_v1`
+        `moz-fx-data-shared-prod.firefox_desktop_background_tasks_live.background_tasks_v1`
       UNION ALL
       SELECT
         submission_timestamp,
@@ -60,7 +60,7 @@
         client_info.app_display_version AS version,
         ping_info
       FROM
-        `moz-fx-data-shared-prod.firefox_desktop_background_tasks_live.background_tasks_v1`
+        `moz-fx-data-shared-prod.firefox_desktop_background_tasks_live.events_v1`
     )
   CROSS JOIN
     UNNEST(events) AS event,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/event_monitoring_live_v1/materialized_view.sql	2024-05-21 10:23:18.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/event_monitoring_live_v1/materialized_view.sql	2024-05-21 10:25:26.000000000 +0000
@@ -60,7 +60,7 @@
         client_info.app_display_version AS version,
         ping_info
       FROM
-        `moz-fx-data-shared-prod.firefox_desktop_live.prototype_no_code_events_v1`
+        `moz-fx-data-shared-prod.firefox_desktop_live.newtab_v1`
       UNION ALL
       SELECT
         submission_timestamp,
@@ -70,7 +70,7 @@
         client_info.app_display_version AS version,
         ping_info
       FROM
-        `moz-fx-data-shared-prod.firefox_desktop_live.newtab_v1`
+        `moz-fx-data-shared-prod.firefox_desktop_live.prototype_no_code_events_v1`
       UNION ALL
       SELECT
         submission_timestamp,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/google_ads/ad_groups/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/google_ads/ad_groups/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/google_ads/ad_groups/schema.yaml	2024-05-21 10:23:17.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/google_ads/ad_groups/schema.yaml	2024-05-21 10:31:02.000000000 +0000
@@ -25,3 +25,16 @@
   mode: NULLABLE
   description: The name of the ad group. We take the most recently reported name for
     the ad group ID.
+- name: ad_group_segments
+  type: RECORD
+  mode: REPEATED
+  fields:
+  - name: key
+    type: STRING
+    mode: NULLABLE
+    description: Field name, e.g. region or country.
+  - name: value
+    type: STRING
+    mode: NULLABLE
+    description: Field value, e.g. "NA" or "US".
+  description: Segments that are parsed from the ad group name, e.g. region or country.
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/internet_outages/global_outages_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/internet_outages/global_outages_v1/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/internet_outages/global_outages_v1/schema.yaml	2024-05-21 10:22:42.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/internet_outages/global_outages_v1/schema.yaml	2024-05-21 10:33:03.000000000 +0000
@@ -1,49 +1,49 @@
 fields:
-- mode: NULLABLE
-  name: country
+- name: country
   type: STRING
-- mode: NULLABLE
-  name: city
+  mode: NULLABLE
+- name: city
   type: STRING
-- mode: NULLABLE
-  name: datetime
+  mode: NULLABLE
+- name: datetime
   type: TIMESTAMP
-- mode: NULLABLE
-  name: proportion_undefined
+  mode: NULLABLE
+- name: proportion_undefined
   type: FLOAT
-- mode: NULLABLE
-  name: proportion_timeout
+  mode: NULLABLE
+- name: proportion_timeout
   type: FLOAT
-- mode: NULLABLE
-  name: proportion_abort
+  mode: NULLABLE
+- name: proportion_abort
   type: FLOAT
-- mode: NULLABLE
-  name: proportion_unreachable
+  mode: NULLABLE
+- name: proportion_unreachable
   type: FLOAT
-- mode: NULLABLE
-  name: proportion_terminated
+  mode: NULLABLE
+- name: proportion_terminated
   type: FLOAT
-- mode: NULLABLE
-  name: proportion_channel_open
+  mode: NULLABLE
+- name: proportion_channel_open
   type: FLOAT
-- mode: NULLABLE
-  name: avg_dns_success_time
+  mode: NULLABLE
+- name: avg_dns_success_time
   type: FLOAT
-- mode: NULLABLE
-  name: missing_dns_success
+  mode: NULLABLE
+- name: missing_dns_success
   type: FLOAT
-- mode: NULLABLE
-  name: avg_dns_failure_time
+  mode: NULLABLE
+- name: avg_dns_failure_time
   type: FLOAT
-- mode: NULLABLE
-  name: missing_dns_failure
+  mode: NULLABLE
+- name: missing_dns_failure
   type: FLOAT
-- mode: NULLABLE
-  name: count_dns_failure
+  mode: NULLABLE
+- name: count_dns_failure
   type: FLOAT
-- mode: NULLABLE
-  name: ssl_error_prop
+  mode: NULLABLE
+- name: ssl_error_prop
   type: FLOAT
-- mode: NULLABLE
-  name: avg_tls_handshake_time
+  mode: NULLABLE
+- name: avg_tls_handshake_time
   type: FLOAT
+  mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/monitoring/airflow_trigger/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/monitoring/airflow_trigger/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/monitoring/airflow_trigger/schema.yaml	2024-05-21 10:23:18.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/monitoring/airflow_trigger/schema.yaml	2024-05-21 10:31:07.000000000 +0000
@@ -6,22 +6,12 @@
 - name: classpath
   type: STRING
   mode: NULLABLE
-  description: null
 - name: created_date
   type: TIMESTAMP
   mode: NULLABLE
 - name: kwargs
   type: STRING
   mode: NULLABLE
-  description: null
 - name: triggerer_id
   type: INTEGER
   mode: NULLABLE
-- name: trigger_id
-  type: INTEGER
-  mode: NULLABLE
-  description: null
-- name: created_at
-  type: TIMESTAMP
-  mode: NULLABLE
-  description: null
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/monitoring_derived/event_monitoring_aggregates_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/monitoring_derived/event_monitoring_aggregates_v1/query.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/monitoring_derived/event_monitoring_aggregates_v1/query.sql	2024-05-21 10:23:18.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/monitoring_derived/event_monitoring_aggregates_v1/query.sql	2024-05-21 10:27:24.000000000 +0000
@@ -55,7 +55,7 @@
       client_info.app_display_version AS version,
       ping_info
     FROM
-      `moz-fx-data-shared-prod.firefox_desktop_stable.prototype_no_code_events_v1`
+      `moz-fx-data-shared-prod.firefox_desktop_stable.newtab_v1`
     UNION ALL
     SELECT
       submission_timestamp,
@@ -65,7 +65,7 @@
       client_info.app_display_version AS version,
       ping_info
     FROM
-      `moz-fx-data-shared-prod.firefox_desktop_stable.newtab_v1`
+      `moz-fx-data-shared-prod.firefox_desktop_stable.prototype_no_code_events_v1`
     UNION ALL
     SELECT
       submission_timestamp,
@@ -572,7 +572,7 @@
       client_info.app_display_version AS version,
       ping_info
     FROM
-      `moz-fx-data-shared-prod.org_mozilla_ios_firefox_stable.events_v1`
+      `moz-fx-data-shared-prod.org_mozilla_ios_firefox_stable.first_session_v1`
     UNION ALL
     SELECT
       submission_timestamp,
@@ -592,7 +592,7 @@
       client_info.app_display_version AS version,
       ping_info
     FROM
-      `moz-fx-data-shared-prod.org_mozilla_ios_firefox_stable.first_session_v1`
+      `moz-fx-data-shared-prod.org_mozilla_ios_firefox_stable.events_v1`
   )
 CROSS JOIN
   UNNEST(events) AS event,
@@ -663,7 +663,7 @@
       client_info.app_display_version AS version,
       ping_info
     FROM
-      `moz-fx-data-shared-prod.org_mozilla_ios_firefoxbeta_stable.events_v1`
+      `moz-fx-data-shared-prod.org_mozilla_ios_firefoxbeta_stable.first_session_v1`
     UNION ALL
     SELECT
       submission_timestamp,
@@ -683,7 +683,7 @@
       client_info.app_display_version AS version,
       ping_info
     FROM
-      `moz-fx-data-shared-prod.org_mozilla_ios_firefoxbeta_stable.first_session_v1`
+      `moz-fx-data-shared-prod.org_mozilla_ios_firefoxbeta_stable.events_v1`
   )
 CROSS JOIN
   UNNEST(events) AS event,
@@ -754,7 +754,7 @@
       client_info.app_display_version AS version,
       ping_info
     FROM
-      `moz-fx-data-shared-prod.org_mozilla_ios_fennec_stable.events_v1`
+      `moz-fx-data-shared-prod.org_mozilla_ios_fennec_stable.first_session_v1`
     UNION ALL
     SELECT
       submission_timestamp,
@@ -774,7 +774,7 @@
       client_info.app_display_version AS version,
       ping_info
     FROM
-      `moz-fx-data-shared-prod.org_mozilla_ios_fennec_stable.first_session_v1`
+      `moz-fx-data-shared-prod.org_mozilla_ios_fennec_stable.events_v1`
   )
 CROSS JOIN
   UNNEST(events) AS event,
@@ -1555,7 +1555,7 @@
       client_info.app_display_version AS version,
       ping_info
     FROM
-      `moz-fx-data-shared-prod.mozillavpn_stable.vpnsession_v1`
+      `moz-fx-data-shared-prod.mozillavpn_stable.main_v1`
     UNION ALL
     SELECT
       submission_timestamp,
@@ -1565,7 +1565,7 @@
       client_info.app_display_version AS version,
       ping_info
     FROM
-      `moz-fx-data-shared-prod.mozillavpn_stable.main_v1`
+      `moz-fx-data-shared-prod.mozillavpn_stable.vpnsession_v1`
     UNION ALL
     SELECT
       submission_timestamp,
@@ -1646,7 +1646,7 @@
       client_info.app_display_version AS version,
       ping_info
     FROM
-      `moz-fx-data-shared-prod.org_mozilla_firefox_vpn_stable.vpnsession_v1`
+      `moz-fx-data-shared-prod.org_mozilla_firefox_vpn_stable.main_v1`
     UNION ALL
     SELECT
       submission_timestamp,
@@ -1656,7 +1656,7 @@
       client_info.app_display_version AS version,
       ping_info
     FROM
-      `moz-fx-data-shared-prod.org_mozilla_firefox_vpn_stable.main_v1`
+      `moz-fx-data-shared-prod.org_mozilla_firefox_vpn_stable.vpnsession_v1`
     UNION ALL
     SELECT
       submission_timestamp,
@@ -1737,7 +1737,7 @@
       client_info.app_display_version AS version,
       ping_info
     FROM
-      `moz-fx-data-shared-prod.org_mozilla_ios_firefoxvpn_stable.vpnsession_v1`
+      `moz-fx-data-shared-prod.org_mozilla_ios_firefoxvpn_stable.main_v1`
     UNION ALL
     SELECT
       submission_timestamp,
@@ -1747,7 +1747,7 @@
       client_info.app_display_version AS version,
       ping_info
     FROM
-      `moz-fx-data-shared-prod.org_mozilla_ios_firefoxvpn_stable.main_v1`
+      `moz-fx-data-shared-prod.org_mozilla_ios_firefoxvpn_stable.vpnsession_v1`
     UNION ALL
     SELECT
       submission_timestamp,
@@ -1828,7 +1828,7 @@
       client_info.app_display_version AS version,
       ping_info
     FROM
-      `moz-fx-data-shared-prod.org_mozilla_ios_firefoxvpn_network_extension_stable.vpnsession_v1`
+      `moz-fx-data-shared-prod.org_mozilla_ios_firefoxvpn_network_extension_stable.main_v1`
     UNION ALL
     SELECT
       submission_timestamp,
@@ -1838,7 +1838,7 @@
       client_info.app_display_version AS version,
       ping_info
     FROM
-      `moz-fx-data-shared-prod.org_mozilla_ios_firefoxvpn_network_extension_stable.main_v1`
+      `moz-fx-data-shared-prod.org_mozilla_ios_firefoxvpn_network_extension_stable.vpnsession_v1`
     UNION ALL
     SELECT
       submission_timestamp,
@@ -2000,7 +2000,7 @@
       client_info.app_display_version AS version,
       ping_info
     FROM
-      `moz-fx-data-shared-prod.bedrock_stable.events_v1`
+      `moz-fx-data-shared-prod.bedrock_stable.non_interaction_v1`
     UNION ALL
     SELECT
       submission_timestamp,
@@ -2010,7 +2010,7 @@
       client_info.app_display_version AS version,
       ping_info
     FROM
-      `moz-fx-data-shared-prod.bedrock_stable.non_interaction_v1`
+      `moz-fx-data-shared-prod.bedrock_stable.events_v1`
   )
 CROSS JOIN
   UNNEST(events) AS event,
@@ -2081,7 +2081,7 @@
       client_info.app_display_version AS version,
       ping_info
     FROM
-      `moz-fx-data-shared-prod.viu_politica_stable.video_index_v1`
+      `moz-fx-data-shared-prod.viu_politica_stable.main_events_v1`
     UNION ALL
     SELECT
       submission_timestamp,
@@ -2091,7 +2091,7 @@
       client_info.app_display_version AS version,
       ping_info
     FROM
-      `moz-fx-data-shared-prod.viu_politica_stable.main_events_v1`
+      `moz-fx-data-shared-prod.viu_politica_stable.video_index_v1`
   )
 CROSS JOIN
   UNNEST(events) AS event,
@@ -2162,7 +2162,7 @@
       client_info.app_display_version AS version,
       ping_info
     FROM
-      `moz-fx-data-shared-prod.firefox_desktop_background_tasks_stable.events_v1`
+      `moz-fx-data-shared-prod.firefox_desktop_background_tasks_stable.background_tasks_v1`
     UNION ALL
     SELECT
       submission_timestamp,
@@ -2172,7 +2172,7 @@
       client_info.app_display_version AS version,
       ping_info
     FROM
-      `moz-fx-data-shared-prod.firefox_desktop_background_tasks_stable.background_tasks_v1`
+      `moz-fx-data-shared-prod.firefox_desktop_background_tasks_stable.events_v1`
   )
 CROSS JOIN
   UNNEST(events) AS event,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/mozilla_vpn/subscriptions/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/mozilla_vpn/subscriptions/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/mozilla_vpn/subscriptions/schema.yaml	2024-05-21 10:23:18.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/mozilla_vpn/subscriptions/schema.yaml	2024-05-21 10:31:17.000000000 +0000
@@ -20,3 +20,45 @@
 - name: type
   type: STRING
   mode: NULLABLE
+- name: provider
+  type: STRING
+  mode: NULLABLE
+- name: apple_receipt
+  type: RECORD
+  mode: NULLABLE
+  fields:
+  - name: environment
+    type: STRING
+    mode: NULLABLE
+  - name: active_period
+    type: RECORD
+    mode: NULLABLE
+    fields:
+    - name: start_date
+      type: DATE
+      mode: NULLABLE
+    - name: end_date
+      type: DATE
+      mode: NULLABLE
+    - name: start_time
+      type: TIMESTAMP
+      mode: NULLABLE
+    - name: end_time
+      type: TIMESTAMP
+      mode: NULLABLE
+    - name: interval
+      type: STRING
+      mode: NULLABLE
+    - name: interval_count
+      type: INTEGER
+      mode: NULLABLE
+  - name: trial_period
+    type: RECORD
+    mode: NULLABLE
+    fields:
+    - name: start_time
+      type: TIMESTAMP
+      mode: NULLABLE
+    - name: end_time
+      type: TIMESTAMP
+      mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/mozillavpn_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/mozillavpn_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/mozillavpn_derived/event_monitoring_live_v1/materialized_view.sql	2024-05-21 10:23:17.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/mozillavpn_derived/event_monitoring_live_v1/materialized_view.sql	2024-05-21 10:25:25.000000000 +0000
@@ -50,7 +50,7 @@
         client_info.app_display_version AS version,
         ping_info
       FROM
-        `moz-fx-data-shared-prod.mozillavpn_live.vpnsession_v1`
+        `moz-fx-data-shared-prod.mozillavpn_live.main_v1`
       UNION ALL
       SELECT
         submission_timestamp,
@@ -60,7 +60,7 @@
         client_info.app_display_version AS version,
         ping_info
       FROM
-        `moz-fx-data-shared-prod.mozillavpn_live.main_v1`
+        `moz-fx-data-shared-prod.mozillavpn_live.vpnsession_v1`
       UNION ALL
       SELECT
         submission_timestamp,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_fenix/geckoview_version/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_fenix/geckoview_version/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_fenix/geckoview_version/schema.yaml	2024-05-21 10:22:42.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_fenix/geckoview_version/schema.yaml	2024-05-21 10:31:21.000000000 +0000
@@ -1,7 +1,10 @@
 fields:
-- type: DATETIME
-  name: build_hour
-- type: INTEGER
-  name: geckoview_major_version
-- type: INTEGER
-  name: n_pings
+- name: build_hour
+  type: DATETIME
+  mode: NULLABLE
+- name: geckoview_major_version
+  type: INTEGER
+  mode: NULLABLE
+- name: n_pings
+  type: INTEGER
+  mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_fenix_derived/baseline_clients_daily_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_fenix_derived/baseline_clients_daily_v1/query.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_fenix_derived/baseline_clients_daily_v1/query.sql	2024-05-21 10:23:18.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_fenix_derived/baseline_clients_daily_v1/query.sql	2024-05-21 10:25:35.000000000 +0000
@@ -23,7 +23,7 @@
     normalized_channel,
     normalized_os,
     normalized_os_version,
-    CAST(NULL AS STRING) AS distribution_id,
+    metrics.string.metrics_distribution_id AS distribution_id,
     metadata.geo.subdivision1 AS geo_subdivision,
   FROM
     `org_mozilla_fenix_stable.baseline_v1`
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_fenix_nightly_derived/baseline_clients_daily_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_fenix_nightly_derived/baseline_clients_daily_v1/query.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_fenix_nightly_derived/baseline_clients_daily_v1/query.sql	2024-05-21 10:23:18.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_fenix_nightly_derived/baseline_clients_daily_v1/query.sql	2024-05-21 10:25:35.000000000 +0000
@@ -23,7 +23,7 @@
     normalized_channel,
     normalized_os,
     normalized_os_version,
-    CAST(NULL AS STRING) AS distribution_id,
+    metrics.string.metrics_distribution_id AS distribution_id,
     metadata.geo.subdivision1 AS geo_subdivision,
   FROM
     `org_mozilla_fenix_nightly_stable.baseline_v1`
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_fennec_aurora_derived/baseline_clients_daily_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_fennec_aurora_derived/baseline_clients_daily_v1/query.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_fennec_aurora_derived/baseline_clients_daily_v1/query.sql	2024-05-21 10:23:18.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_fennec_aurora_derived/baseline_clients_daily_v1/query.sql	2024-05-21 10:25:36.000000000 +0000
@@ -23,7 +23,7 @@
     normalized_channel,
     normalized_os,
     normalized_os_version,
-    CAST(NULL AS STRING) AS distribution_id,
+    metrics.string.metrics_distribution_id AS distribution_id,
     metadata.geo.subdivision1 AS geo_subdivision,
   FROM
     `org_mozilla_fennec_aurora_stable.baseline_v1`
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_firefox_beta_derived/baseline_clients_daily_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_firefox_beta_derived/baseline_clients_daily_v1/query.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_firefox_beta_derived/baseline_clients_daily_v1/query.sql	2024-05-21 10:23:18.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_firefox_beta_derived/baseline_clients_daily_v1/query.sql	2024-05-21 10:25:37.000000000 +0000
@@ -23,7 +23,7 @@
     normalized_channel,
     normalized_os,
     normalized_os_version,
-    CAST(NULL AS STRING) AS distribution_id,
+    metrics.string.metrics_distribution_id AS distribution_id,
     metadata.geo.subdivision1 AS geo_subdivision,
   FROM
     `org_mozilla_firefox_beta_stable.baseline_v1`
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_firefox_derived/baseline_clients_daily_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_firefox_derived/baseline_clients_daily_v1/query.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_firefox_derived/baseline_clients_daily_v1/query.sql	2024-05-21 10:23:18.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_firefox_derived/baseline_clients_daily_v1/query.sql	2024-05-21 10:25:38.000000000 +0000
@@ -23,7 +23,7 @@
     normalized_channel,
     normalized_os,
     normalized_os_version,
-    CAST(NULL AS STRING) AS distribution_id,
+    metrics.string.metrics_distribution_id AS distribution_id,
     metadata.geo.subdivision1 AS geo_subdivision,
   FROM
     `org_mozilla_firefox_stable.baseline_v1`
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_firefox_vpn_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_firefox_vpn_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_firefox_vpn_derived/event_monitoring_live_v1/materialized_view.sql	2024-05-21 10:23:18.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_firefox_vpn_derived/event_monitoring_live_v1/materialized_view.sql	2024-05-21 10:25:27.000000000 +0000
@@ -50,7 +50,7 @@
         client_info.app_display_version AS version,
         ping_info
       FROM
-        `moz-fx-data-shared-prod.org_mozilla_firefox_vpn_live.vpnsession_v1`
+        `moz-fx-data-shared-prod.org_mozilla_firefox_vpn_live.main_v1`
       UNION ALL
       SELECT
         submission_timestamp,
@@ -60,7 +60,7 @@
         client_info.app_display_version AS version,
         ping_info
       FROM
-        `moz-fx-data-shared-prod.org_mozilla_firefox_vpn_live.main_v1`
+        `moz-fx-data-shared-prod.org_mozilla_firefox_vpn_live.vpnsession_v1`
       UNION ALL
       SELECT
         submission_timestamp,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_ios_fennec_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_ios_fennec_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_ios_fennec_derived/event_monitoring_live_v1/materialized_view.sql	2024-05-21 10:23:18.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_ios_fennec_derived/event_monitoring_live_v1/materialized_view.sql	2024-05-21 10:25:29.000000000 +0000
@@ -50,7 +50,7 @@
         client_info.app_display_version AS version,
         ping_info
       FROM
-        `moz-fx-data-shared-prod.org_mozilla_ios_fennec_live.events_v1`
+        `moz-fx-data-shared-prod.org_mozilla_ios_fennec_live.first_session_v1`
       UNION ALL
       SELECT
         submission_timestamp,
@@ -70,7 +70,7 @@
         client_info.app_display_version AS version,
         ping_info
       FROM
-        `moz-fx-data-shared-prod.org_mozilla_ios_fennec_live.first_session_v1`
+        `moz-fx-data-shared-prod.org_mozilla_ios_fennec_live.events_v1`
     )
   CROSS JOIN
     UNNEST(events) AS event,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_ios_firefoxbeta_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_ios_firefoxbeta_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_ios_firefoxbeta_derived/event_monitoring_live_v1/materialized_view.sql	2024-05-21 10:23:18.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_ios_firefoxbeta_derived/event_monitoring_live_v1/materialized_view.sql	2024-05-21 10:25:30.000000000 +0000
@@ -50,7 +50,7 @@
         client_info.app_display_version AS version,
         ping_info
       FROM
-        `moz-fx-data-shared-prod.org_mozilla_ios_firefoxbeta_live.events_v1`
+        `moz-fx-data-shared-prod.org_mozilla_ios_firefoxbeta_live.first_session_v1`
       UNION ALL
       SELECT
         submission_timestamp,
@@ -70,7 +70,7 @@
         client_info.app_display_version AS version,
         ping_info
       FROM
-        `moz-fx-data-shared-prod.org_mozilla_ios_firefoxbeta_live.first_session_v1`
+        `moz-fx-data-shared-prod.org_mozilla_ios_firefoxbeta_live.events_v1`
     )
   CROSS JOIN
     UNNEST(events) AS event,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_ios_firefox_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_ios_firefox_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_ios_firefox_derived/event_monitoring_live_v1/materialized_view.sql	2024-05-21 10:23:18.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_ios_firefox_derived/event_monitoring_live_v1/materialized_view.sql	2024-05-21 10:25:30.000000000 +0000
@@ -50,7 +50,7 @@
         client_info.app_display_version AS version,
         ping_info
       FROM
-        `moz-fx-data-shared-prod.org_mozilla_ios_firefox_live.events_v1`
+        `moz-fx-data-shared-prod.org_mozilla_ios_firefox_live.first_session_v1`
       UNION ALL
       SELECT
         submission_timestamp,
@@ -70,7 +70,7 @@
         client_info.app_display_version AS version,
         ping_info
       FROM
-        `moz-fx-data-shared-prod.org_mozilla_ios_firefox_live.first_session_v1`
+        `moz-fx-data-shared-prod.org_mozilla_ios_firefox_live.events_v1`
     )
   CROSS JOIN
     UNNEST(events) AS event,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_ios_firefoxvpn_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_ios_firefoxvpn_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_ios_firefoxvpn_derived/event_monitoring_live_v1/materialized_view.sql	2024-05-21 10:23:18.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_ios_firefoxvpn_derived/event_monitoring_live_v1/materialized_view.sql	2024-05-21 10:25:30.000000000 +0000
@@ -50,7 +50,7 @@
         client_info.app_display_version AS version,
         ping_info
       FROM
-        `moz-fx-data-shared-prod.org_mozilla_ios_firefoxvpn_live.vpnsession_v1`
+        `moz-fx-data-shared-prod.org_mozilla_ios_firefoxvpn_live.main_v1`
       UNION ALL
       SELECT
         submission_timestamp,
@@ -60,7 +60,7 @@
         client_info.app_display_version AS version,
         ping_info
       FROM
-        `moz-fx-data-shared-prod.org_mozilla_ios_firefoxvpn_live.main_v1`
+        `moz-fx-data-shared-prod.org_mozilla_ios_firefoxvpn_live.vpnsession_v1`
       UNION ALL
       SELECT
         submission_timestamp,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_ios_firefoxvpn_network_extension_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_ios_firefoxvpn_network_extension_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_ios_firefoxvpn_network_extension_derived/event_monitoring_live_v1/materialized_view.sql	2024-05-21 10:23:18.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_ios_firefoxvpn_network_extension_derived/event_monitoring_live_v1/materialized_view.sql	2024-05-21 10:25:30.000000000 +0000
@@ -50,7 +50,7 @@
         client_info.app_display_version AS version,
         ping_info
       FROM
-        `moz-fx-data-shared-prod.org_mozilla_ios_firefoxvpn_network_extension_live.vpnsession_v1`
+        `moz-fx-data-shared-prod.org_mozilla_ios_firefoxvpn_network_extension_live.main_v1`
       UNION ALL
       SELECT
         submission_timestamp,
@@ -60,7 +60,7 @@
         client_info.app_display_version AS version,
         ping_info
       FROM
-        `moz-fx-data-shared-prod.org_mozilla_ios_firefoxvpn_network_extension_live.main_v1`
+        `moz-fx-data-shared-prod.org_mozilla_ios_firefoxvpn_network_extension_live.vpnsession_v1`
       UNION ALL
       SELECT
         submission_timestamp,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/search_derived/mobile_search_clients_daily_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/search_derived/mobile_search_clients_daily_v1/query.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/search_derived/mobile_search_clients_daily_v1/query.sql	2024-05-21 10:22:42.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/search_derived/mobile_search_clients_daily_v1/query.sql	2024-05-21 10:25:23.000000000 +0000
@@ -1,6 +1,4 @@
--- Query generated by ./bqetl generate search
--- This file doesn't get overwritten by the generator. The generator output needs
--- to be written to this file manually.
+-- Query generated by bigquery-etl/search/mobile_search_clients_daily.py
 --
 -- Older versions separate source and engine with an underscore instead of period
 -- Return array of form [source, engine] if key is valid, empty array otherwise
@@ -498,7 +496,7 @@
     metrics.counter.browser_total_uri_count,
     client_info.locale,
   FROM
-    org_mozilla_ios_klar.metrics AS org_mozilla_klar_metrics
+    org_mozilla_ios_klar.metrics AS org_mozilla_ios_klar_metrics
 ),
 fenix_baseline AS (
   SELECT
@@ -874,7 +872,6 @@
     ANY_VALUE(sample_id) AS sample_id,
     udf.map_mode_last(ARRAY_CONCAT_AGG(experiments)) AS experiments,
     SUM(total_uri_count) AS total_uri_count,
-    CAST(NULL AS STRING) AS normalized_engine
   FROM
     combined_search_clients
   WHERE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/static/ca_postal_districts_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/static/ca_postal_districts_v1/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/static/ca_postal_districts_v1/schema.yaml	2024-05-21 10:22:42.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/static/ca_postal_districts_v1/schema.yaml	2024-05-21 10:31:56.000000000 +0000
@@ -1,9 +1,7 @@
 fields:
 - name: postal_district_code
   type: STRING
-  mode: REQUIRED
-  description: One-character Canadian postal district code.
+  mode: NULLABLE
 - name: province_code
   type: STRING
   mode: NULLABLE
-  description: Two-character Canadian province/territory code (if any).
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/static/country_codes_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/static/country_codes_v1/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/static/country_codes_v1/schema.yaml	2024-05-21 10:22:42.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/static/country_codes_v1/schema.yaml	2024-05-21 10:31:56.000000000 +0000
@@ -1,47 +1,28 @@
 fields:
 - name: name
-  description: Official country name per ISO 3166
   type: STRING
-  mode: REQUIRED
+  mode: NULLABLE
 - name: code
-  description: ISO 3166 alpha-2 country code
   type: STRING
-  mode: REQUIRED
+  mode: NULLABLE
 - name: code_3
-  description: ISO 3166 alpha-3 country code
   type: STRING
-  mode: REQUIRED
+  mode: NULLABLE
 - name: region_name
-  description: Region name. These are based on the UN Statistics Division standard
-    country or area codes for statistical use (M49), but with the "Americas" region
-    split into "North America" and "South America".
   type: STRING
-  mode: REQUIRED
+  mode: NULLABLE
 - name: subregion_name
-  description: Sub-region name. These are based on UN Statistics Division standard
-    country or area codes for statistical use (M49), but with the "Latin America and the
-    Caribbean" and "Sub-Saharan Africa" sub-regions split into more specific
-    sub-regions.
   type: STRING
-  mode: REQUIRED
+  mode: NULLABLE
 - name: pocket_available_on_newtab
-  description: Whether Pocket is available on the newtab page in this country. Note
-    that Pocket might only be available in certain locales/languages within a country.
-  type: BOOL
-  mode: REQUIRED
+  type: BOOLEAN
+  mode: NULLABLE
 - name: mozilla_vpn_available
-  description: Whether Mozilla VPN is available in this country.
-  type: BOOL
-  mode: REQUIRED
+  type: BOOLEAN
+  mode: NULLABLE
 - name: sponsored_tiles_available_on_newtab
-  description: Whether sponsored tiles are available on the newtab page in this country.
-    Note that Pocket might only be available in certain locales/languages within a
-    country.
-  type: BOOL
-  mode: REQUIRED
+  type: BOOLEAN
+  mode: NULLABLE
 - name: ads_value_tier
-  description: Lowercase label detailing the monetary value tier that Mozilla Ads
-    assign to that region based on market size and our existing products, e.g., tier
-    1, tier 2, etc.
   type: STRING
-  mode: REQUIRED
+  mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/static/country_names_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/static/country_names_v1/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/static/country_names_v1/schema.yaml	2024-05-21 10:22:42.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/static/country_names_v1/schema.yaml	2024-05-21 10:31:56.000000000 +0000
@@ -1,10 +1,7 @@
 fields:
 - name: name
-  description: An alias for a country's name (including misspellings and alternate
-    encodings).
   type: STRING
-  mode: REQUIRED
+  mode: NULLABLE
 - name: code
-  description: ISO 3166 alpha-2 country code
   type: STRING
-  mode: REQUIRED
+  mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/static/data_incidents_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/static/data_incidents_v1/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/static/data_incidents_v1/schema.yaml	2024-05-21 10:22:42.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/static/data_incidents_v1/schema.yaml	2024-05-21 10:31:56.000000000 +0000
@@ -1,22 +1,22 @@
 fields:
-- mode: NULLABLE
-  name: start_date
+- name: start_date
   type: DATE
-- mode: NULLABLE
-  name: end_date
+  mode: NULLABLE
+- name: end_date
   type: DATE
-- mode: NULLABLE
-  name: incident
+  mode: NULLABLE
+- name: incident
   type: STRING
-- mode: NULLABLE
-  name: description
+  mode: NULLABLE
+- name: description
   type: STRING
-- mode: NULLABLE
-  name: bug
+  mode: NULLABLE
+- name: bug
   type: STRING
-- mode: NULLABLE
-  name: product
+  mode: NULLABLE
+- name: product
   type: STRING
-- mode: NULLABLE
-  name: version
+  mode: NULLABLE
+- name: version
   type: STRING
+  mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/static/iana_tls_cipher_suites/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/static/iana_tls_cipher_suites/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/static/iana_tls_cipher_suites/schema.yaml	2024-05-21 10:22:42.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/static/iana_tls_cipher_suites/schema.yaml	2024-05-21 10:31:56.000000000 +0000
@@ -1,27 +1,16 @@
 fields:
-- mode: NULLABLE
-  description: Hex value assigned to the TLS cipher, in format like "0x00,0x84"; note
-    some values are ranges or contain wildcards
-  name: value
+- name: value
   type: STRING
-- mode: NULLABLE
-  description: Human-readable name of the TLS cipher
-  name: description
+  mode: NULLABLE
+- name: description
   type: STRING
-- mode: NULLABLE
-  description: Any TLS cipher suite that is specified for use with DTLS MUST define
-    limits on the use of the associated AEAD function that preserves margins for both
-    confidentiality and integrity, as specified in [RFC-ietf-tls-dtls13-43]
-  name: dtls_ok
+  mode: NULLABLE
+- name: dtls_ok
   type: BOOLEAN
-- mode: NULLABLE
-  description: Whether the TLS cipher is recommended by the IETF. If an item is not
-    marked as "recommended", it does not necessarily mean that it is flawed; rather,
-    it indicates that the item either has not been through the IETF consensus process,
-    has limited applicability, or is intended only for specific use cases
-  name: recommended
+  mode: NULLABLE
+- name: recommended
   type: BOOLEAN
-- mode: NULLABLE
-  description: RFCs or associated reference material for the TLS cipher
-  name: reference
+  mode: NULLABLE
+- name: reference
   type: STRING
+  mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/static/language_codes_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/static/language_codes_v1/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/static/language_codes_v1/schema.yaml	2024-05-21 10:22:42.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/static/language_codes_v1/schema.yaml	2024-05-21 10:31:56.000000000 +0000
@@ -1,17 +1,13 @@
 fields:
 - name: code_3
-  description: ISO 639 alpha-3 language code.
   type: STRING
-  mode: REQUIRED
+  mode: NULLABLE
 - name: code_2
-  description: ISO 639 alpha-2 language code (if any).
   type: STRING
   mode: NULLABLE
 - name: name
-  description: Language name.
   type: STRING
-  mode: REQUIRED
+  mode: NULLABLE
 - name: other_names
-  description: Other names for the language (if any).
   type: STRING
   mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/static/monitoring_distinct_docids_notes_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/static/monitoring_distinct_docids_notes_v1/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/static/monitoring_distinct_docids_notes_v1/schema.yaml	2024-05-21 10:22:42.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/static/monitoring_distinct_docids_notes_v1/schema.yaml	2024-05-21 10:31:56.000000000 +0000
@@ -1,19 +1,19 @@
 fields:
-- mode: NULLABLE
-  name: start_date
+- name: start_date
   type: DATE
-- mode: NULLABLE
-  name: end_date
+  mode: NULLABLE
+- name: end_date
   type: DATE
-- mode: NULLABLE
-  name: document_namespace
+  mode: NULLABLE
+- name: document_namespace
   type: STRING
-- mode: NULLABLE
-  name: document_type
+  mode: NULLABLE
+- name: document_type
   type: STRING
-- mode: NULLABLE
-  name: notes
+  mode: NULLABLE
+- name: notes
   type: STRING
-- mode: NULLABLE
-  name: bug
+  mode: NULLABLE
+- name: bug
   type: STRING
+  mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/static/monitoring_missing_columns_notes_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/static/monitoring_missing_columns_notes_v1/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/static/monitoring_missing_columns_notes_v1/schema.yaml	2024-05-21 10:22:42.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/static/monitoring_missing_columns_notes_v1/schema.yaml	2024-05-21 10:31:56.000000000 +0000
@@ -1,25 +1,25 @@
 fields:
-- mode: NULLABLE
-  name: start_date
+- name: start_date
   type: DATE
-- mode: NULLABLE
-  name: end_date
+  mode: NULLABLE
+- name: end_date
   type: DATE
-- mode: NULLABLE
-  name: document_namespace
+  mode: NULLABLE
+- name: document_namespace
   type: STRING
-- mode: NULLABLE
-  name: document_type
+  mode: NULLABLE
+- name: document_type
   type: STRING
-- mode: NULLABLE
-  name: document_version
+  mode: NULLABLE
+- name: document_version
   type: STRING
-- mode: NULLABLE
-  name: path
+  mode: NULLABLE
+- name: path
   type: STRING
-- mode: NULLABLE
-  name: notes
+  mode: NULLABLE
+- name: notes
   type: STRING
-- mode: NULLABLE
-  name: bug
+  mode: NULLABLE
+- name: bug
   type: STRING
+  mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/static/monitoring_missing_document_namespaces_notes_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/static/monitoring_missing_document_namespaces_notes_v1/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/static/monitoring_missing_document_namespaces_notes_v1/schema.yaml	2024-05-21 10:22:42.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/static/monitoring_missing_document_namespaces_notes_v1/schema.yaml	2024-05-21 10:31:56.000000000 +0000
@@ -1,22 +1,22 @@
 fields:
-- mode: NULLABLE
-  name: start_date
+- name: start_date
   type: DATE
-- mode: NULLABLE
-  name: end_date
+  mode: NULLABLE
+- name: end_date
   type: DATE
-- mode: NULLABLE
-  name: document_namespace
+  mode: NULLABLE
+- name: document_namespace
   type: STRING
-- mode: NULLABLE
-  name: document_type
+  mode: NULLABLE
+- name: document_type
   type: STRING
-- mode: NULLABLE
-  name: document_version
+  mode: NULLABLE
+- name: document_version
   type: STRING
-- mode: NULLABLE
-  name: notes
+  mode: NULLABLE
+- name: notes
   type: STRING
-- mode: NULLABLE
-  name: bug
+  mode: NULLABLE
+- name: bug
   type: STRING
+  mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/static/monitoring_schema_errors_notes_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/static/monitoring_schema_errors_notes_v1/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/static/monitoring_schema_errors_notes_v1/schema.yaml	2024-05-21 10:22:42.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/static/monitoring_schema_errors_notes_v1/schema.yaml	2024-05-21 10:31:56.000000000 +0000
@@ -1,22 +1,22 @@
 fields:
-- mode: NULLABLE
-  name: start_date
+- name: start_date
   type: DATE
-- mode: NULLABLE
-  name: end_date
+  mode: NULLABLE
+- name: end_date
   type: DATE
-- mode: NULLABLE
-  name: document_namespace
+  mode: NULLABLE
+- name: document_namespace
   type: STRING
-- mode: NULLABLE
-  name: document_type
+  mode: NULLABLE
+- name: document_type
   type: STRING
-- mode: NULLABLE
-  name: path
+  mode: NULLABLE
+- name: path
   type: STRING
-- mode: NULLABLE
-  name: notes
+  mode: NULLABLE
+- name: notes
   type: STRING
-- mode: NULLABLE
-  name: bug
+  mode: NULLABLE
+- name: bug
   type: STRING
+  mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/static/normal_distribution/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/static/normal_distribution/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/static/normal_distribution/schema.yaml	2024-05-21 10:22:42.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/static/normal_distribution/schema.yaml	2024-05-21 10:31:56.000000000 +0000
@@ -1,7 +1,7 @@
 fields:
 - name: score
   type: NUMERIC
-  mode: REQUIRED
+  mode: NULLABLE
 - name: value
   type: NUMERIC
-  mode: REQUIRED
+  mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/static/us_zip_code_prefixes_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/static/us_zip_code_prefixes_v1/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/static/us_zip_code_prefixes_v1/schema.yaml	2024-05-21 10:22:42.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/static/us_zip_code_prefixes_v1/schema.yaml	2024-05-21 10:31:56.000000000 +0000
@@ -1,9 +1,7 @@
 fields:
 - name: zip_code_prefix
   type: STRING
-  mode: REQUIRED
-  description: Three-digit US ZIP code prefix.
+  mode: NULLABLE
 - name: state_code
   type: STRING
   mode: NULLABLE
-  description: Two-character US state/territory code (if any).
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry/releases_latest/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry/releases_latest/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry/releases_latest/schema.yaml	2024-05-21 10:23:18.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry/releases_latest/schema.yaml	2024-05-21 10:32:02.000000000 +0000
@@ -2,21 +2,18 @@
 - name: date
   type: DATE
   mode: NULLABLE
-  description: null
 - name: product
   type: STRING
   mode: NULLABLE
 - name: category
   type: STRING
   mode: NULLABLE
-  description: null
 - name: channel
   type: STRING
   mode: NULLABLE
 - name: build_number
   type: INTEGER
   mode: NULLABLE
-  description: null
 - name: release_date
   type: DATE
   mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/viu_politica_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/viu_politica_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/viu_politica_derived/event_monitoring_live_v1/materialized_view.sql	2024-05-21 10:23:18.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/viu_politica_derived/event_monitoring_live_v1/materialized_view.sql	2024-05-21 10:25:25.000000000 +0000
@@ -50,7 +50,7 @@
         client_info.app_display_version AS version,
         ping_info
       FROM
-        `moz-fx-data-shared-prod.viu_politica_live.video_index_v1`
+        `moz-fx-data-shared-prod.viu_politica_live.main_events_v1`
       UNION ALL
       SELECT
         submission_timestamp,
@@ -60,7 +60,7 @@
         client_info.app_display_version AS version,
         ping_info
       FROM
-        `moz-fx-data-shared-prod.viu_politica_live.main_events_v1`
+        `moz-fx-data-shared-prod.viu_politica_live.video_index_v1`
     )
   CROSS JOIN
     UNNEST(events) AS event,

Link to full diff

@kik-kik kik-kik merged commit f6ea4f7 into main May 21, 2024
20 of 21 checks passed
@kik-kik kik-kik deleted the fix/baseline_clients_daily-only-get-distribution_id-for-fenix branch May 21, 2024 10:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sean-rose sean-rose sean-rose left review comments

@wwyc wwyc wwyc approved these changes

@scholtzan scholtzan Awaiting requested review from scholtzan

Assignees

@kik-kik kik-kik

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@kik-kik @dataops-ci-bot @sean-rose @wwyc