feat(DENG-3462): add activity fields to support KPI metrics to baseline_last_seen views #5434

kik-kik · 2024-04-25T13:45:11Z

feat(DENG-3462): add activity fields to support KPI metrics to `baseline_last_seen` views

This approach aims to add fields to enable us to calculate KPI metrics based on activity. The original proposal was to have a "main" view which would union all the tables and add these field (see: Create telemetry.active_users view ).

This PR takes a slightly different approach and instead introduces those additional fields in the upstream view (baseline_clients_last_seen).

The following fields are being added to support our KPI metric calculations:

activity_segment - categorise a client based on their activity "levels"
is_dau - sent ping and considered active on current date
is_wau - sent ping and considered active in the last 7 days
is_mau - sent ping and considered active in the last 28 days
is_daily_user - client a baseline sent ping on current date
is_weekly_user - client a baseline sent ping in the 7 days
is_monthly_user - client a baseline sent ping in the last 28 days

Checklist for reviewer:

Commits should reference a bug or github issue, if relevant (if a bug is referenced, the pull request should include the bug number in the title).
If the PR comes from a fork, trigger integration CI tests by running the Push to upstream workflow and provide the <username>:<branch> of the fork as parameter. The parameter will also show up
in the logs of the manual-trigger-required-for-fork CI task together with more detailed instructions.
If adding a new field to a query, ensure that the schema and dependent downstream schemas have been updated.
When adding a new derived dataset, ensure that data is not available already (fully or partially) and recommend extending an existing dataset in favor of creating new ones. Data can be available in the bigquery-etl repository, looker-hub or in looker-spoke-default.

For modifications to schemas in restricted namespaces (see CODEOWNERS):

Follow the change control procedure

┆Issue is synchronized with this Jira Task

kwindau · 2024-04-25T14:10:41Z

@kik-kik do all the fields have the columns it is referencing? distribution ID, isp_name, etc?

sql_generators/glean_usage/templates/baseline_clients_last_seen.view.sql

sql_generators/glean_usage/templates/baseline_clients_daily.view.sql

sql_generators/glean_usage/templates/baseline_clients_last_seen.view.sql

lucia-vargas-a

Some comments and the need to account for Desktop, because the Glean template also generates desktop's view.

lucia-vargas-a

This part of the description is incorrect This is a re-implementation of https://github.com/mozilla/bigquery-etl/pull/5396.
I'd probably suggest "This PR moves the logic to calculate growth metrics from a single view for all apps to the clients_last_seen` views per app.".

sql_generators/glean_usage/templates/baseline_clients_last_seen.view.sql

lucia-vargas-a

Comment on adding activity segments.

sql_generators/glean_usage/templates/cross_channel.view.sql

lucia-vargas-a

bits28 udf requires the IFNULL validation

kwindau

Looks good to me!

kik-kik · 2024-05-06T10:02:06Z

Blocked by: #5497

scholtzan · 2024-05-07T18:59:56Z

To get this merged we'll likely need to skip stage deploys. Some of the queries that got changed depend on certain views, and some views depend on queries that got changed.
The stage deploys first deploy all queries, then all views. So in some cases query schemas cannot be determined as the upstream views haven't been deployed yet.

Stage deploys can be skipped using the skip-stage-deploys CI parameter and setting it to false when triggering CI manually

lucia-vargas-a

Only a comment on the products_to_include_extra_activity_fields

sql_generators/glean_usage/templates/baseline_clients_last_seen.view.sql

bochocki

Blocked pending discussion.

dataops-ci-bot · 2024-05-21T17:34:17Z

Integration report for "feat: add activity fields to support KPI metrics to last_seen views"

`sql.diff`

Click to expand!

diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/dags/bqetl_glean_usage.py /tmp/workspace/generated-sql/dags/bqetl_glean_usage.py
--- /tmp/workspace/main-generated-sql/dags/bqetl_glean_usage.py	2024-05-21 17:21:01.000000000 +0000
+++ /tmp/workspace/generated-sql/dags/bqetl_glean_usage.py	2024-05-21 17:33:31.000000000 +0000
@@ -153,6 +153,18 @@
         pool="DATA_ENG_EXTERNALTASKSENSOR",
     )
 
+    accounts_backend_derived__clients_last_seen_joined__v1 = bigquery_etl_query(
+        task_id="accounts_backend_derived__clients_last_seen_joined__v1",
+        destination_table="clients_last_seen_joined_v1",
+        dataset_id="accounts_backend_derived",
+        project_id="moz-fx-data-shared-prod",
+        owner="ascholtz@mozilla.com",
+        email=["ascholtz@mozilla.com", "telemetry-alerts@mozilla.com"],
+        date_partition_parameter="submission_date",
+        depends_on_past=True,
+        task_group=task_group_accounts_backend,
+    )
+
     accounts_backend_derived__events_stream__v1 = bigquery_etl_query(
         task_id="accounts_backend_derived__events_stream__v1",
         destination_table="events_stream_v1",
@@ -208,6 +220,18 @@
         task_group=task_group_accounts_cirrus,
     )
 
+    accounts_cirrus_derived__clients_last_seen_joined__v1 = bigquery_etl_query(
+        task_id="accounts_cirrus_derived__clients_last_seen_joined__v1",
+        destination_table="clients_last_seen_joined_v1",
+        dataset_id="accounts_cirrus_derived",
+        project_id="moz-fx-data-shared-prod",
+        owner="ascholtz@mozilla.com",
+        email=["ascholtz@mozilla.com", "telemetry-alerts@mozilla.com"],
+        date_partition_parameter="submission_date",
+        depends_on_past=True,
+        task_group=task_group_accounts_cirrus,
+    )
+
     accounts_cirrus_derived__events_stream__v1 = bigquery_etl_query(
         task_id="accounts_cirrus_derived__events_stream__v1",
         destination_table="events_stream_v1",
@@ -238,6 +262,18 @@
         task_group=task_group_accounts_cirrus,
     )
 
+    accounts_frontend_derived__clients_last_seen_joined__v1 = bigquery_etl_query(
+        task_id="accounts_frontend_derived__clients_last_seen_joined__v1",
+        destination_table="clients_last_seen_joined_v1",
+        dataset_id="accounts_frontend_derived",
+        project_id="moz-fx-data-shared-prod",
+        owner="ascholtz@mozilla.com",
+        email=["ascholtz@mozilla.com", "telemetry-alerts@mozilla.com"],
+        date_partition_parameter="submission_date",
+        depends_on_past=True,
+        task_group=task_group_accounts_frontend,
+    )
+
     accounts_frontend_derived__events_stream__v1 = bigquery_etl_query(
         task_id="accounts_frontend_derived__events_stream__v1",
         destination_table="events_stream_v1",
@@ -256,6 +292,18 @@
         task_group=task_group_accounts_frontend,
     )
 
+    bedrock_derived__clients_last_seen_joined__v1 = bigquery_etl_query(
+        task_id="bedrock_derived__clients_last_seen_joined__v1",
+        destination_table="clients_last_seen_joined_v1",
+        dataset_id="bedrock_derived",
+        project_id="moz-fx-data-shared-prod",
+        owner="ascholtz@mozilla.com",
+        email=["ascholtz@mozilla.com", "telemetry-alerts@mozilla.com"],
+        date_partition_parameter="submission_date",
+        depends_on_past=True,
+        task_group=task_group_bedrock,
+    )
+
     bedrock_derived__events_stream__v1 = bigquery_etl_query(
         task_id="bedrock_derived__events_stream__v1",
         destination_table="events_stream_v1",
@@ -274,6 +322,18 @@
         task_group=task_group_bedrock,
     )
 
+    bergamot_derived__clients_last_seen_joined__v1 = bigquery_etl_query(
+        task_id="bergamot_derived__clients_last_seen_joined__v1",
+        destination_table="clients_last_seen_joined_v1",
+        dataset_id="bergamot_derived",
+        project_id="moz-fx-data-shared-prod",
+        owner="ascholtz@mozilla.com",
+        email=["ascholtz@mozilla.com", "telemetry-alerts@mozilla.com"],
+        date_partition_parameter="submission_date",
+        depends_on_past=True,
+        task_group=task_group_bergamot,
+    )
+
     burnham_derived__baseline_clients_daily__v1 = bigquery_etl_query(
         task_id="burnham_derived__baseline_clients_daily__v1",
         destination_table="baseline_clients_daily_v1",
@@ -1565,6 +1625,18 @@
         task_group=task_group_pine,
     )
 
+    debug_ping_view_derived__clients_last_seen_joined__v1 = bigquery_etl_query(
+        task_id="debug_ping_view_derived__clients_last_seen_joined__v1",
+        destination_table="clients_last_seen_joined_v1",
+        dataset_id="debug_ping_view_derived",
+        project_id="moz-fx-data-shared-prod",
+        owner="ascholtz@mozilla.com",
+        email=["ascholtz@mozilla.com", "telemetry-alerts@mozilla.com"],
+        date_partition_parameter="submission_date",
+        depends_on_past=True,
+        task_group=task_group_debug_ping_view,
+    )
+
     debug_ping_view_derived__events_stream__v1 = bigquery_etl_query(
         task_id="debug_ping_view_derived__events_stream__v1",
         destination_table="events_stream_v1",
@@ -1706,6 +1778,18 @@
         task_group=task_group_firefox_desktop_background_defaultagent,
     )
 
+    firefox_desktop_background_defaultagent_derived__clients_last_seen_joined__v1 = bigquery_etl_query(
+        task_id="firefox_desktop_background_defaultagent_derived__clients_last_seen_joined__v1",
+        destination_table="clients_last_seen_joined_v1",
+        dataset_id="firefox_desktop_background_defaultagent_derived",
+        project_id="moz-fx-data-shared-prod",
+        owner="ascholtz@mozilla.com",
+        email=["ascholtz@mozilla.com", "telemetry-alerts@mozilla.com"],
+        date_partition_parameter="submission_date",
+        depends_on_past=True,
+        task_group=task_group_firefox_desktop_background_defaultagent,
+    )
+
     firefox_desktop_background_defaultagent_derived__events_stream__v1 = bigquery_etl_query(
         task_id="firefox_desktop_background_defaultagent_derived__events_stream__v1",
         destination_table="events_stream_v1",
@@ -1773,6 +1857,18 @@
         task_group=task_group_firefox_desktop_background_tasks,
     )
 
+    firefox_desktop_background_tasks_derived__clients_last_seen_joined__v1 = bigquery_etl_query(
+        task_id="firefox_desktop_background_tasks_derived__clients_last_seen_joined__v1",
+        destination_table="clients_last_seen_joined_v1",
+        dataset_id="firefox_desktop_background_tasks_derived",
+        project_id="moz-fx-data-shared-prod",
+        owner="ascholtz@mozilla.com",
+        email=["ascholtz@mozilla.com", "telemetry-alerts@mozilla.com"],
+        date_partition_parameter="submission_date",
+        depends_on_past=True,
+        task_group=task_group_firefox_desktop_background_tasks,
+    )
+
     firefox_desktop_background_tasks_derived__events_stream__v1 = bigquery_etl_query(
         task_id="firefox_desktop_background_tasks_derived__events_stream__v1",
         destination_table="events_stream_v1",
@@ -2193,6 +2289,18 @@
         task_group=task_group_firefox_reality_pc,
     )
 
+    firefox_translations_derived__clients_last_seen_joined__v1 = bigquery_etl_query(
+        task_id="firefox_translations_derived__clients_last_seen_joined__v1",
+        destination_table="clients_last_seen_joined_v1",
+        dataset_id="firefox_translations_derived",
+        project_id="moz-fx-data-shared-prod",
+        owner="ascholtz@mozilla.com",
+        email=["ascholtz@mozilla.com", "telemetry-alerts@mozilla.com"],
+        date_partition_parameter="submission_date",
+        depends_on_past=True,
+        task_group=task_group_firefox_translations,
+    )
+
     firefox_translations_derived__events_stream__v1 = bigquery_etl_query(
         task_id="firefox_translations_derived__events_stream__v1",
         destination_table="events_stream_v1",
@@ -2333,6 +2441,18 @@
         task_group=task_group_focus_ios,
     )
 
+    glean_dictionary_derived__clients_last_seen_joined__v1 = bigquery_etl_query(
+        task_id="glean_dictionary_derived__clients_last_seen_joined__v1",
+        destination_table="clients_last_seen_joined_v1",
+        dataset_id="glean_dictionary_derived",
+        project_id="moz-fx-data-shared-prod",
+        owner="ascholtz@mozilla.com",
+        email=["ascholtz@mozilla.com", "telemetry-alerts@mozilla.com"],
+        date_partition_parameter="submission_date",
+        depends_on_past=True,
+        task_group=task_group_glean_dictionary,
+    )
+
     glean_dictionary_derived__events_stream__v1 = bigquery_etl_query(
         task_id="glean_dictionary_derived__events_stream__v1",
         destination_table="events_stream_v1",
@@ -2559,6 +2679,18 @@
         task_group=task_group_mach,
     )
 
+    mdn_yari_derived__clients_last_seen_joined__v1 = bigquery_etl_query(
+        task_id="mdn_yari_derived__clients_last_seen_joined__v1",
+        destination_table="clients_last_seen_joined_v1",
+        dataset_id="mdn_yari_derived",
+        project_id="moz-fx-data-shared-prod",
+        owner="ascholtz@mozilla.com",
+        email=["ascholtz@mozilla.com", "telemetry-alerts@mozilla.com"],
+        date_partition_parameter="submission_date",
+        depends_on_past=True,
+        task_group=task_group_mdn_yari,
+    )
+
     mdn_yari_derived__events_stream__v1 = bigquery_etl_query(
         task_id="mdn_yari_derived__events_stream__v1",
         destination_table="events_stream_v1",
@@ -2614,6 +2746,18 @@
         task_group=task_group_monitor_cirrus,
     )
 
+    monitor_cirrus_derived__clients_last_seen_joined__v1 = bigquery_etl_query(
+        task_id="monitor_cirrus_derived__clients_last_seen_joined__v1",
+        destination_table="clients_last_seen_joined_v1",
+        dataset_id="monitor_cirrus_derived",
+        project_id="moz-fx-data-shared-prod",
+        owner="ascholtz@mozilla.com",
+        email=["ascholtz@mozilla.com", "telemetry-alerts@mozilla.com"],
+        date_partition_parameter="submission_date",
+        depends_on_past=True,
+        task_group=task_group_monitor_cirrus,
+    )
+
     monitor_cirrus_derived__events_stream__v1 = bigquery_etl_query(
         task_id="monitor_cirrus_derived__events_stream__v1",
         destination_table="events_stream_v1",
@@ -2644,6 +2788,18 @@
         task_group=task_group_monitor_cirrus,
     )
 
+    monitor_frontend_derived__clients_last_seen_joined__v1 = bigquery_etl_query(
+        task_id="monitor_frontend_derived__clients_last_seen_joined__v1",
+        destination_table="clients_last_seen_joined_v1",
+        dataset_id="monitor_frontend_derived",
+        project_id="moz-fx-data-shared-prod",
+        owner="ascholtz@mozilla.com",
+        email=["ascholtz@mozilla.com", "telemetry-alerts@mozilla.com"],
+        date_partition_parameter="submission_date",
+        depends_on_past=True,
+        task_group=task_group_monitor_frontend,
+    )
+
     monitor_frontend_derived__events_stream__v1 = bigquery_etl_query(
         task_id="monitor_frontend_derived__events_stream__v1",
         destination_table="events_stream_v1",
@@ -2794,6 +2950,18 @@
         task_group=task_group_mach,
     )
 
+    mozilla_vpn_derived__clients_last_seen_joined__v1 = bigquery_etl_query(
+        task_id="mozilla_vpn_derived__clients_last_seen_joined__v1",
+        destination_table="clients_last_seen_joined_v1",
+        dataset_id="mozilla_vpn_derived",
+        project_id="moz-fx-data-shared-prod",
+        owner="ascholtz@mozilla.com",
+        email=["ascholtz@mozilla.com", "telemetry-alerts@mozilla.com"],
+        date_partition_parameter="submission_date",
+        depends_on_past=True,
+        task_group=task_group_mozilla_vpn,
+    )
+
     mozilla_vpn_derived__metrics_clients_daily__v1 = bigquery_etl_query(
         task_id="mozilla_vpn_derived__metrics_clients_daily__v1",
         destination_table="metrics_clients_daily_v1",
@@ -2845,6 +3013,20 @@
         )
     )
 
+    mozillavpn_backend_cirrus_derived__clients_last_seen_joined__v1 = (
+        bigquery_etl_query(
+            task_id="mozillavpn_backend_cirrus_derived__clients_last_seen_joined__v1",
+            destination_table="clients_last_seen_joined_v1",
+            dataset_id="mozillavpn_backend_cirrus_derived",
+            project_id="moz-fx-data-shared-prod",
+            owner="ascholtz@mozilla.com",
+            email=["ascholtz@mozilla.com", "telemetry-alerts@mozilla.com"],
+            date_partition_parameter="submission_date",
+            depends_on_past=True,
+            task_group=task_group_mozillavpn_backend_cirrus,
+        )
+    )
+
     mozillavpn_backend_cirrus_derived__events_stream__v1 = bigquery_etl_query(
         task_id="mozillavpn_backend_cirrus_derived__events_stream__v1",
         destination_table="events_stream_v1",
@@ -4746,6 +4928,18 @@
         task_group=task_group_pine,
     )
 
+    pine_derived__clients_last_seen_joined__v1 = bigquery_etl_query(
+        task_id="pine_derived__clients_last_seen_joined__v1",
+        destination_table="clients_last_seen_joined_v1",
+        dataset_id="pine_derived",
+        project_id="moz-fx-data-shared-prod",
+        owner="ascholtz@mozilla.com",
+        email=["ascholtz@mozilla.com", "telemetry-alerts@mozilla.com"],
+        date_partition_parameter="submission_date",
+        depends_on_past=True,
+        task_group=task_group_pine,
+    )
+
     pine_derived__events_stream__v1 = bigquery_etl_query(
         task_id="pine_derived__events_stream__v1",
         destination_table="events_stream_v1",
@@ -4812,6 +5006,18 @@
         task_group=task_group_reference_browser,
     )
 
+    treeherder_derived__clients_last_seen_joined__v1 = bigquery_etl_query(
+        task_id="treeherder_derived__clients_last_seen_joined__v1",
+        destination_table="clients_last_seen_joined_v1",
+        dataset_id="treeherder_derived",
+        project_id="moz-fx-data-shared-prod",
+        owner="ascholtz@mozilla.com",
+        email=["ascholtz@mozilla.com", "telemetry-alerts@mozilla.com"],
+        date_partition_parameter="submission_date",
+        depends_on_past=True,
+        task_group=task_group_treeherder,
+    )
+
     treeherder_derived__events_stream__v1 = bigquery_etl_query(
         task_id="treeherder_derived__events_stream__v1",
         destination_table="events_stream_v1",
@@ -4830,6 +5036,18 @@
         task_group=task_group_treeherder,
     )
 
+    viu_politica_derived__clients_last_seen_joined__v1 = bigquery_etl_query(
+        task_id="viu_politica_derived__clients_last_seen_joined__v1",
+        destination_table="clients_last_seen_joined_v1",
+        dataset_id="viu_politica_derived",
+        project_id="moz-fx-data-shared-prod",
+        owner="ascholtz@mozilla.com",
+        email=["ascholtz@mozilla.com", "telemetry-alerts@mozilla.com"],
+        date_partition_parameter="submission_date",
+        depends_on_past=True,
+        task_group=task_group_viu_politica,
+    )
+
     viu_politica_derived__events_stream__v1 = bigquery_etl_query(
         task_id="viu_politica_derived__events_stream__v1",
         destination_table="events_stream_v1",
@@ -4872,6 +5090,10 @@
         accounts_cirrus_derived__baseline_clients_daily__v1
     )
 
+    accounts_cirrus_derived__clients_last_seen_joined__v1.set_upstream(
+        checks__fail_accounts_cirrus_derived__baseline_clients_last_seen__v1
+    )
+
     accounts_cirrus_derived__events_stream__v1.set_upstream(
         wait_for_copy_deduplicate_all
     )
@@ -5570,6 +5792,10 @@
         firefox_desktop_background_defaultagent_derived__baseline_clients_daily__v1
     )
 
+    firefox_desktop_background_defaultagent_derived__clients_last_seen_joined__v1.set_upstream(
+        checks__fail_firefox_desktop_background_defaultagent_derived__baseline_clients_last_seen__v1
+    )
+
     firefox_desktop_background_defaultagent_derived__events_stream__v1.set_upstream(
         wait_for_copy_deduplicate_all
     )
@@ -5598,6 +5824,10 @@
         firefox_desktop_background_tasks_derived__baseline_clients_daily__v1
     )
 
+    firefox_desktop_background_tasks_derived__clients_last_seen_joined__v1.set_upstream(
+        checks__fail_firefox_desktop_background_tasks_derived__baseline_clients_last_seen__v1
+    )
+
     firefox_desktop_background_tasks_derived__events_stream__v1.set_upstream(
         wait_for_copy_deduplicate_all
     )
@@ -5922,6 +6152,10 @@
         monitor_cirrus_derived__baseline_clients_daily__v1
     )
 
+    monitor_cirrus_derived__clients_last_seen_joined__v1.set_upstream(
+        monitor_cirrus_derived__baseline_clients_last_seen__v1
+    )
+
     monitor_cirrus_derived__events_stream__v1.set_upstream(
         wait_for_copy_deduplicate_all
     )
@@ -5980,6 +6214,22 @@
 
     mozilla_mach_derived__events_stream__v1.set_upstream(wait_for_copy_deduplicate_all)
 
+    mozilla_vpn_derived__clients_last_seen_joined__v1.set_upstream(
+        mozillavpn_derived__baseline_clients_last_seen__v1
+    )
+
+    mozilla_vpn_derived__clients_last_seen_joined__v1.set_upstream(
+        org_mozilla_firefox_vpn_derived__baseline_clients_last_seen__v1
+    )
+
+    mozilla_vpn_derived__clients_last_seen_joined__v1.set_upstream(
+        org_mozilla_ios_firefoxvpn_derived__baseline_clients_last_seen__v1
+    )
+
+    mozilla_vpn_derived__clients_last_seen_joined__v1.set_upstream(
+        org_mozilla_ios_firefoxvpn_network_extension_derived__baseline_clients_last_seen__v1
+    )
+
     mozilla_vpn_derived__metrics_clients_daily__v1.set_upstream(
         wait_for_copy_deduplicate_all
     )
@@ -6004,6 +6254,10 @@
         mozillavpn_backend_cirrus_derived__baseline_clients_daily__v1
     )
 
+    mozillavpn_backend_cirrus_derived__clients_last_seen_joined__v1.set_upstream(
+        mozillavpn_backend_cirrus_derived__baseline_clients_last_seen__v1
+    )
+
     mozillavpn_backend_cirrus_derived__events_stream__v1.set_upstream(
         wait_for_copy_deduplicate_all
     )
@@ -6710,6 +6964,10 @@
         pine_derived__baseline_clients_daily__v1
     )
 
+    pine_derived__clients_last_seen_joined__v1.set_upstream(
+        checks__fail_pine_derived__baseline_clients_last_seen__v1
+    )
+
     pine_derived__events_stream__v1.set_upstream(wait_for_copy_deduplicate_all)
 
     pine_derived__metrics_clients_daily__v1.set_upstream(wait_for_copy_deduplicate_all)
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_backend: clients_last_seen_joined
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_backend_derived: clients_last_seen_joined_v1
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus: clients_last_seen_joined
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived: clients_last_seen_joined_v1
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend: clients_last_seen_joined
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend_derived: clients_last_seen_joined_v1
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/bedrock: clients_last_seen_joined
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived: clients_last_seen_joined_v1
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/bergamot: clients_last_seen_joined
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod: bergamot_derived
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/debug_ping_view: clients_last_seen_joined
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/debug_ping_view_derived: clients_last_seen_joined_v1
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_defaultagent: clients_last_seen_joined
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_defaultagent_derived: clients_last_seen_joined_v1
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_tasks: clients_last_seen_joined
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_tasks_derived: clients_last_seen_joined_v1
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_translations: clients_last_seen_joined
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_translations_derived: clients_last_seen_joined_v1
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/glean_dictionary: clients_last_seen_joined
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/glean_dictionary_derived: clients_last_seen_joined_v1
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/mdn_yari: clients_last_seen_joined
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/mdn_yari_derived: clients_last_seen_joined_v1
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/monitor_cirrus: clients_last_seen_joined
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/monitor_cirrus_derived: clients_last_seen_joined_v1
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/monitor_frontend: clients_last_seen_joined
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/monitor_frontend_derived: clients_last_seen_joined_v1
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/mozilla_vpn: clients_last_seen_joined
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/mozillavpn_backend_cirrus: clients_last_seen_joined
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/mozillavpn_backend_cirrus_derived: clients_last_seen_joined_v1
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/mozilla_vpn_derived: clients_last_seen_joined_v1
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/pine: clients_last_seen_joined
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/pine_derived: clients_last_seen_joined_v1
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/treeherder: clients_last_seen_joined
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/treeherder_derived: clients_last_seen_joined_v1
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/viu_politica: clients_last_seen_joined
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/viu_politica_derived: clients_last_seen_joined_v1
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_backend/clients_last_seen_joined/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_backend/clients_last_seen_joined/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_backend/clients_last_seen_joined/metadata.yaml	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_backend/clients_last_seen_joined/metadata.yaml	2024-05-21 17:30:34.000000000 +0000
@@ -0,0 +1,13 @@
+friendly_name: Clients Last Seen Joined
+description: |-
+  Please provide a description for the query
+owners: []
+labels: {}
+bigquery: null
+workgroup_access:
+- role: roles/bigquery.dataViewer
+  members:
+  - workgroup:mozilla-confidential
+references:
+  view.sql:
+  - moz-fx-data-shared-prod.accounts_backend_derived.clients_last_seen_joined_v1
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_backend/clients_last_seen_joined/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_backend/clients_last_seen_joined/view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_backend/clients_last_seen_joined/view.sql	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_backend/clients_last_seen_joined/view.sql	2024-05-21 17:21:09.000000000 +0000
@@ -0,0 +1,7 @@
+CREATE OR REPLACE VIEW
+  `moz-fx-data-shared-prod.accounts_backend.clients_last_seen_joined`
+AS
+SELECT
+  *
+FROM
+  `moz-fx-data-shared-prod.accounts_backend_derived.clients_last_seen_joined_v1`
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_backend_derived/clients_last_seen_joined_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_backend_derived/clients_last_seen_joined_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_backend_derived/clients_last_seen_joined_v1/metadata.yaml	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_backend_derived/clients_last_seen_joined_v1/metadata.yaml	2024-05-21 17:30:29.000000000 +0000
@@ -0,0 +1,40 @@
+friendly_name: Clients Last Seen Joined
+description: |-
+  Join on baseline and metrics views, but uses a LEFT JOIN
+  such that a given client will not appear in the result if
+  it is only represented in the metrics-based table.
+
+  NOTE:
+  In rare cases the union can result in the same client_id entry from different channels.
+  In such cases, the resulting data will only include the entry for the client_id
+  with the oldest first_seen_date value.
+owners:
+- ascholtz@mozilla.com
+labels:
+  incremental: true
+  schedule: daily
+  dag: bqetl_glean_usage
+  owner1: ascholtz
+scheduling:
+  dag_name: bqetl_glean_usage
+  task_group: accounts_backend
+  depends_on_past: true
+bigquery:
+  time_partitioning:
+    type: day
+    field: submission_date
+    require_partition_filter: true
+    expiration_days: null
+  range_partitioning: null
+  clustering:
+    fields:
+    - normalized_channel
+    - sample_id
+workgroup_access:
+- role: roles/bigquery.dataViewer
+  members:
+  - workgroup:mozilla-confidential
+references:
+  query.sql:
+  - moz-fx-data-shared-prod.accounts_backend.baseline_clients_last_seen
+  - moz-fx-data-shared-prod.accounts_backend.metrics_clients_last_seen
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_backend_derived/clients_last_seen_joined_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_backend_derived/clients_last_seen_joined_v1/query.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_backend_derived/clients_last_seen_joined_v1/query.sql	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_backend_derived/clients_last_seen_joined_v1/query.sql	2024-05-21 17:21:09.000000000 +0000
@@ -0,0 +1,51 @@
+WITH baseline AS (
+  SELECT
+    * EXCEPT (
+      -- These were added as part of https://mozilla-hub.atlassian.net/browse/DENG-3462
+      -- to enable KPI metric calculations. This exclusion is to avoid potential compatibility
+      -- issues downstream from this query.
+      activity_segment,
+      is_dau,
+      is_wau,
+      is_mau,
+      is_daily_user,
+      is_weekly_user,
+      is_monthly_user,
+    )
+  FROM
+    `moz-fx-data-shared-prod.accounts_backend.baseline_clients_last_seen`
+  WHERE
+    submission_date = @submission_date
+),
+metrics AS (
+  SELECT
+    *
+  FROM
+    `moz-fx-data-shared-prod.accounts_backend.metrics_clients_last_seen`
+  WHERE
+    -- The join between baseline and metrics pings is based on submission_date with a 1 day delay,
+    -- since metrics pings usually arrive within 1 day after their logical activity period.
+    submission_date = DATE_ADD(@submission_date, INTERVAL 1 DAY)
+)
+SELECT
+  baseline.client_id,
+  baseline.sample_id,
+  baseline.submission_date,
+  baseline.normalized_channel,
+  * EXCEPT (submission_date, normalized_channel, client_id, sample_id)
+FROM
+  baseline
+LEFT JOIN
+  metrics
+  ON baseline.client_id = metrics.client_id
+  AND baseline.sample_id = metrics.sample_id
+  AND (
+    baseline.normalized_channel = metrics.normalized_channel
+    OR (baseline.normalized_channel IS NULL AND metrics.normalized_channel IS NULL)
+  )
+-- In some rare cases we end up with the same client_id in multiple channels
+-- In those cases, this union can result in client_id duplicates.
+-- This logic ensures that the resulting table only includes the client from the channel
+-- we've seen first.
+QUALIFY
+  ROW_NUMBER() OVER (PARTITION BY client_id ORDER BY first_seen_date ASC) = 1
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus/baseline_clients_last_seen/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus/baseline_clients_last_seen/view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus/baseline_clients_last_seen/view.sql	2024-05-21 17:18:13.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus/baseline_clients_last_seen/view.sql	2024-05-21 17:20:39.000000000 +0000
@@ -14,6 +14,31 @@
   `moz-fx-data-shared-prod`.udf.pos_of_trailing_set_bit(
     days_seen_session_end_bits
   ) AS days_since_seen_session_end,
-  *
+  *,
+  -- Activity fields to support metrics built on top of activity
+  CASE
+    WHEN BIT_COUNT(days_active_bits)
+      BETWEEN 1
+      AND 6
+      THEN 'infrequent_user'
+    WHEN BIT_COUNT(days_active_bits)
+      BETWEEN 7
+      AND 13
+      THEN 'casual_user'
+    WHEN BIT_COUNT(days_active_bits)
+      BETWEEN 14
+      AND 20
+      THEN 'regular_user'
+    WHEN BIT_COUNT(days_active_bits) >= 21
+      THEN 'core_user'
+    ELSE 'other'
+  END AS activity_segment,
+  IFNULL(mozfun.bits28.days_since_seen(days_active_bits) = 0, FALSE) AS is_dau,
+  IFNULL(mozfun.bits28.days_since_seen(days_active_bits) < 7, FALSE) AS is_wau,
+  IFNULL(mozfun.bits28.days_since_seen(days_active_bits) < 28, FALSE) AS is_mau,
+  -- Metrics based on pings sent
+  IFNULL(mozfun.bits28.days_since_seen(days_seen_bits) = 0, FALSE) AS is_daily_user,
+  IFNULL(mozfun.bits28.days_since_seen(days_seen_bits) < 7, FALSE) AS is_weekly_user,
+  IFNULL(mozfun.bits28.days_since_seen(days_seen_bits) < 28, FALSE) AS is_monthly_user,
 FROM
   `moz-fx-data-shared-prod.accounts_cirrus_derived.baseline_clients_last_seen_v1`
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus/clients_last_seen_joined/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus/clients_last_seen_joined/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus/clients_last_seen_joined/metadata.yaml	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus/clients_last_seen_joined/metadata.yaml	2024-05-21 17:30:35.000000000 +0000
@@ -0,0 +1,13 @@
+friendly_name: Clients Last Seen Joined
+description: |-
+  Please provide a description for the query
+owners: []
+labels: {}
+bigquery: null
+workgroup_access:
+- role: roles/bigquery.dataViewer
+  members:
+  - workgroup:mozilla-confidential
+references:
+  view.sql:
+  - moz-fx-data-shared-prod.accounts_cirrus_derived.clients_last_seen_joined_v1
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus/clients_last_seen_joined/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus/clients_last_seen_joined/view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus/clients_last_seen_joined/view.sql	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus/clients_last_seen_joined/view.sql	2024-05-21 17:21:12.000000000 +0000
@@ -0,0 +1,7 @@
+CREATE OR REPLACE VIEW
+  `moz-fx-data-shared-prod.accounts_cirrus.clients_last_seen_joined`
+AS
+SELECT
+  *
+FROM
+  `moz-fx-data-shared-prod.accounts_cirrus_derived.clients_last_seen_joined_v1`
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/clients_last_seen_joined_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/clients_last_seen_joined_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/clients_last_seen_joined_v1/metadata.yaml	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/clients_last_seen_joined_v1/metadata.yaml	2024-05-21 17:30:37.000000000 +0000
@@ -0,0 +1,40 @@
+friendly_name: Clients Last Seen Joined
+description: |-
+  Join on baseline and metrics views, but uses a LEFT JOIN
+  such that a given client will not appear in the result if
+  it is only represented in the metrics-based table.
+
+  NOTE:
+  In rare cases the union can result in the same client_id entry from different channels.
+  In such cases, the resulting data will only include the entry for the client_id
+  with the oldest first_seen_date value.
+owners:
+- ascholtz@mozilla.com
+labels:
+  incremental: true
+  schedule: daily
+  dag: bqetl_glean_usage
+  owner1: ascholtz
+scheduling:
+  dag_name: bqetl_glean_usage
+  task_group: accounts_cirrus
+  depends_on_past: true
+bigquery:
+  time_partitioning:
+    type: day
+    field: submission_date
+    require_partition_filter: true
+    expiration_days: null
+  range_partitioning: null
+  clustering:
+    fields:
+    - normalized_channel
+    - sample_id
+workgroup_access:
+- role: roles/bigquery.dataViewer
+  members:
+  - workgroup:mozilla-confidential
+references:
+  query.sql:
+  - moz-fx-data-shared-prod.accounts_cirrus.baseline_clients_last_seen
+  - moz-fx-data-shared-prod.accounts_cirrus.metrics_clients_last_seen
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/clients_last_seen_joined_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/clients_last_seen_joined_v1/query.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/clients_last_seen_joined_v1/query.sql	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/clients_last_seen_joined_v1/query.sql	2024-05-21 17:21:12.000000000 +0000
@@ -0,0 +1,51 @@
+WITH baseline AS (
+  SELECT
+    * EXCEPT (
+      -- These were added as part of https://mozilla-hub.atlassian.net/browse/DENG-3462
+      -- to enable KPI metric calculations. This exclusion is to avoid potential compatibility
+      -- issues downstream from this query.
+      activity_segment,
+      is_dau,
+      is_wau,
+      is_mau,
+      is_daily_user,
+      is_weekly_user,
+      is_monthly_user,
+    )
+  FROM
+    `moz-fx-data-shared-prod.accounts_cirrus.baseline_clients_last_seen`
+  WHERE
+    submission_date = @submission_date
+),
+metrics AS (
+  SELECT
+    *
+  FROM
+    `moz-fx-data-shared-prod.accounts_cirrus.metrics_clients_last_seen`
+  WHERE
+    -- The join between baseline and metrics pings is based on submission_date with a 1 day delay,
+    -- since metrics pings usually arrive within 1 day after their logical activity period.
+    submission_date = DATE_ADD(@submission_date, INTERVAL 1 DAY)
+)
+SELECT
+  baseline.client_id,
+  baseline.sample_id,
+  baseline.submission_date,
+  baseline.normalized_channel,
+  * EXCEPT (submission_date, normalized_channel, client_id, sample_id)
+FROM
+  baseline
+LEFT JOIN
+  metrics
+  ON baseline.client_id = metrics.client_id
+  AND baseline.sample_id = metrics.sample_id
+  AND (
+    baseline.normalized_channel = metrics.normalized_channel
+    OR (baseline.normalized_channel IS NULL AND metrics.normalized_channel IS NULL)
+  )
+-- In some rare cases we end up with the same client_id in multiple channels
+-- In those cases, this union can result in client_id duplicates.
+-- This logic ensures that the resulting table only includes the client from the channel
+-- we've seen first.
+QUALIFY
+  ROW_NUMBER() OVER (PARTITION BY client_id ORDER BY first_seen_date ASC) = 1
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend/clients_last_seen_joined/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend/clients_last_seen_joined/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend/clients_last_seen_joined/metadata.yaml	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend/clients_last_seen_joined/metadata.yaml	2024-05-21 17:30:33.000000000 +0000
@@ -0,0 +1,13 @@
+friendly_name: Clients Last Seen Joined
+description: |-
+  Please provide a description for the query
+owners: []
+labels: {}
+bigquery: null
+workgroup_access:
+- role: roles/bigquery.dataViewer
+  members:
+  - workgroup:mozilla-confidential
+references:
+  view.sql:
+  - moz-fx-data-shared-prod.accounts_frontend_derived.clients_last_seen_joined_v1
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend/clients_last_seen_joined/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend/clients_last_seen_joined/view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend/clients_last_seen_joined/view.sql	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend/clients_last_seen_joined/view.sql	2024-05-21 17:21:13.000000000 +0000
@@ -0,0 +1,7 @@
+CREATE OR REPLACE VIEW
+  `moz-fx-data-shared-prod.accounts_frontend.clients_last_seen_joined`
+AS
+SELECT
+  *
+FROM
+  `moz-fx-data-shared-prod.accounts_frontend_derived.clients_last_seen_joined_v1`
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend_derived/clients_last_seen_joined_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend_derived/clients_last_seen_joined_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend_derived/clients_last_seen_joined_v1/metadata.yaml	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend_derived/clients_last_seen_joined_v1/metadata.yaml	2024-05-21 17:30:37.000000000 +0000
@@ -0,0 +1,40 @@
+friendly_name: Clients Last Seen Joined
+description: |-
+  Join on baseline and metrics views, but uses a LEFT JOIN
+  such that a given client will not appear in the result if
+  it is only represented in the metrics-based table.
+
+  NOTE:
+  In rare cases the union can result in the same client_id entry from different channels.
+  In such cases, the resulting data will only include the entry for the client_id
+  with the oldest first_seen_date value.
+owners:
+- ascholtz@mozilla.com
+labels:
+  incremental: true
+  schedule: daily
+  dag: bqetl_glean_usage
+  owner1: ascholtz
+scheduling:
+  dag_name: bqetl_glean_usage
+  task_group: accounts_frontend
+  depends_on_past: true
+bigquery:
+  time_partitioning:
+    type: day
+    field: submission_date
+    require_partition_filter: true
+    expiration_days: null
+  range_partitioning: null
+  clustering:
+    fields:
+    - normalized_channel
+    - sample_id
+workgroup_access:
+- role: roles/bigquery.dataViewer
+  members:
+  - workgroup:mozilla-confidential
+references:
+  query.sql:
+  - moz-fx-data-shared-prod.accounts_frontend.baseline_clients_last_seen
+  - moz-fx-data-shared-prod.accounts_frontend.metrics_clients_last_seen
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend_derived/clients_last_seen_joined_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend_derived/clients_last_seen_joined_v1/query.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend_derived/clients_last_seen_joined_v1/query.sql	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend_derived/clients_last_seen_joined_v1/query.sql	2024-05-21 17:21:13.000000000 +0000
@@ -0,0 +1,51 @@
+WITH baseline AS (
+  SELECT
+    * EXCEPT (
+      -- These were added as part of https://mozilla-hub.atlassian.net/browse/DENG-3462
+      -- to enable KPI metric calculations. This exclusion is to avoid potential compatibility
+      -- issues downstream from this query.
+      activity_segment,
+      is_dau,
+      is_wau,
+      is_mau,
+      is_daily_user,
+      is_weekly_user,
+      is_monthly_user,
+    )
+  FROM
+    `moz-fx-data-shared-prod.accounts_frontend.baseline_clients_last_seen`
+  WHERE
+    submission_date = @submission_date
+),
+metrics AS (
+  SELECT
+    *
+  FROM
+    `moz-fx-data-shared-prod.accounts_frontend.metrics_clients_last_seen`
+  WHERE
+    -- The join between baseline and metrics pings is based on submission_date with a 1 day delay,
+    -- since metrics pings usually arrive within 1 day after their logical activity period.
+    submission_date = DATE_ADD(@submission_date, INTERVAL 1 DAY)
+)
+SELECT
+  baseline.client_id,
+  baseline.sample_id,
+  baseline.submission_date,
+  baseline.normalized_channel,
+  * EXCEPT (submission_date, normalized_channel, client_id, sample_id)
+FROM
+  baseline
+LEFT JOIN
+  metrics
+  ON baseline.client_id = metrics.client_id
+  AND baseline.sample_id = metrics.sample_id
+  AND (
+    baseline.normalized_channel = metrics.normalized_channel
+    OR (baseline.normalized_channel IS NULL AND metrics.normalized_channel IS NULL)
+  )
+-- In some rare cases we end up with the same client_id in multiple channels
+-- In those cases, this union can result in client_id duplicates.
+-- This logic ensures that the resulting table only includes the client from the channel
+-- we've seen first.
+QUALIFY
+  ROW_NUMBER() OVER (PARTITION BY client_id ORDER BY first_seen_date ASC) = 1
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/bedrock/clients_last_seen_joined/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/bedrock/clients_last_seen_joined/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/bedrock/clients_last_seen_joined/metadata.yaml	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/bedrock/clients_last_seen_joined/metadata.yaml	2024-05-21 17:30:36.000000000 +0000
@@ -0,0 +1,13 @@
+friendly_name: Clients Last Seen Joined
+description: |-
+  Please provide a description for the query
+owners: []
+labels: {}
+bigquery: null
+workgroup_access:
+- role: roles/bigquery.dataViewer
+  members:
+  - workgroup:mozilla-confidential
+references:
+  view.sql:
+  - moz-fx-data-shared-prod.bedrock_derived.clients_last_seen_joined_v1
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/bedrock/clients_last_seen_joined/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/bedrock/clients_last_seen_joined/view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/bedrock/clients_last_seen_joined/view.sql	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/bedrock/clients_last_seen_joined/view.sql	2024-05-21 17:21:06.000000000 +0000
@@ -0,0 +1,7 @@
+CREATE OR REPLACE VIEW
+  `moz-fx-data-shared-prod.bedrock.clients_last_seen_joined`
+AS
+SELECT
+  *
+FROM
+  `moz-fx-data-shared-prod.bedrock_derived.clients_last_seen_joined_v1`
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/clients_last_seen_joined_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/clients_last_seen_joined_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/clients_last_seen_joined_v1/metadata.yaml	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/clients_last_seen_joined_v1/metadata.yaml	2024-05-21 17:30:37.000000000 +0000
@@ -0,0 +1,40 @@
+friendly_name: Clients Last Seen Joined
+description: |-
+  Join on baseline and metrics views, but uses a LEFT JOIN
+  such that a given client will not appear in the result if
+  it is only represented in the metrics-based table.
+
+  NOTE:
+  In rare cases the union can result in the same client_id entry from different channels.
+  In such cases, the resulting data will only include the entry for the client_id
+  with the oldest first_seen_date value.
+owners:
+- ascholtz@mozilla.com
+labels:
+  incremental: true
+  schedule: daily
+  dag: bqetl_glean_usage
+  owner1: ascholtz
+scheduling:
+  dag_name: bqetl_glean_usage
+  task_group: bedrock
+  depends_on_past: true
+bigquery:
+  time_partitioning:
+    type: day
+    field: submission_date
+    require_partition_filter: true
+    expiration_days: null
+  range_partitioning: null
+  clustering:
+    fields:
+    - normalized_channel
+    - sample_id
+workgroup_access:
+- role: roles/bigquery.dataViewer
+  members:
+  - workgroup:mozilla-confidential
+references:
+  query.sql:
+  - moz-fx-data-shared-prod.bedrock.baseline_clients_last_seen
+  - moz-fx-data-shared-prod.bedrock.metrics_clients_last_seen
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/clients_last_seen_joined_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/clients_last_seen_joined_v1/query.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/clients_last_seen_joined_v1/query.sql	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/clients_last_seen_joined_v1/query.sql	2024-05-21 17:21:06.000000000 +0000
@@ -0,0 +1,51 @@
+WITH baseline AS (
+  SELECT
+    * EXCEPT (
+      -- These were added as part of https://mozilla-hub.atlassian.net/browse/DENG-3462
+      -- to enable KPI metric calculations. This exclusion is to avoid potential compatibility
+      -- issues downstream from this query.
+      activity_segment,
+      is_dau,
+      is_wau,
+      is_mau,
+      is_daily_user,
+      is_weekly_user,
+      is_monthly_user,
+    )
+  FROM
+    `moz-fx-data-shared-prod.bedrock.baseline_clients_last_seen`
+  WHERE
+    submission_date = @submission_date
+),
+metrics AS (
+  SELECT
+    *
+  FROM
+    `moz-fx-data-shared-prod.bedrock.metrics_clients_last_seen`
+  WHERE
+    -- The join between baseline and metrics pings is based on submission_date with a 1 day delay,
+    -- since metrics pings usually arrive within 1 day after their logical activity period.
+    submission_date = DATE_ADD(@submission_date, INTERVAL 1 DAY)
+)
+SELECT
+  baseline.client_id,
+  baseline.sample_id,
+  baseline.submission_date,
+  baseline.normalized_channel,
+  * EXCEPT (submission_date, normalized_channel, client_id, sample_id)
+FROM
+  baseline
+LEFT JOIN
+  metrics
+  ON baseline.client_id = metrics.client_id
+  AND baseline.sample_id = metrics.sample_id
+  AND (
+    baseline.normalized_channel = metrics.normalized_channel
+    OR (baseline.normalized_channel IS NULL AND metrics.normalized_channel IS NULL)
+  )
+-- In some rare cases we end up with the same client_id in multiple channels
+-- In those cases, this union can result in client_id duplicates.
+-- This logic ensures that the resulting table only includes the client from the channel
+-- we've seen first.
+QUALIFY
+  ROW_NUMBER() OVER (PARTITION BY client_id ORDER BY first_seen_date ASC) = 1
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/event_monitoring_live_v1/materialized_view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/event_monitoring_live_v1/materialized_view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/event_monitoring_live_v1/materialized_view.sql	2024-05-21 17:18:13.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/bedrock_derived/event_monitoring_live_v1/materialized_view.sql	2024-05-21 17:20:20.000000000 +0000
@@ -50,7 +50,7 @@
         client_info.app_display_version AS version,
         ping_info
       FROM
-        `moz-fx-data-shared-prod.bedrock_live.interaction_v1`
+        `moz-fx-data-shared-prod.bedrock_live.events_v1`
       UNION ALL
       SELECT
         submission_timestamp,
@@ -60,7 +60,7 @@
         client_info.app_display_version AS version,
         ping_info
       FROM
-        `moz-fx-data-shared-prod.bedrock_live.events_v1`
+        `moz-fx-data-shared-prod.bedrock_live.interaction_v1`
       UNION ALL
       SELECT
         submission_timestamp,
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/bergamot/clients_last_seen_joined/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/bergamot/clients_last_seen_joined/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/bergamot/clients_last_seen_joined/metadata.yaml	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/bergamot/clients_last_seen_joined/metadata.yaml	2024-05-21 17:30:38.000000000 +0000
@@ -0,0 +1,13 @@
+friendly_name: Clients Last Seen Joined
+description: |-
+  Please provide a description for the query
+owners: []
+labels: {}
+bigquery: null
+workgroup_access:
+- role: roles/bigquery.dataViewer
+  members:
+  - workgroup:mozilla-confidential
+references:
+  view.sql:
+  - moz-fx-data-shared-prod.bergamot_derived.clients_last_seen_joined_v1
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/bergamot/clients_last_seen_joined/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/bergamot/clients_last_seen_joined/view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/bergamot/clients_last_seen_joined/view.sql	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/bergamot/clients_last_seen_joined/view.sql	2024-05-21 17:20:55.000000000 +0000
@@ -0,0 +1,7 @@
+CREATE OR REPLACE VIEW
+  `moz-fx-data-shared-prod.bergamot.clients_last_seen_joined`
+AS
+SELECT
+  *
+FROM
+  `moz-fx-data-shared-prod.bergamot_derived.clients_last_seen_joined_v1`
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/bergamot_derived/clients_last_seen_joined_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/bergamot_derived/clients_last_seen_joined_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/bergamot_derived/clients_last_seen_joined_v1/metadata.yaml	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/bergamot_derived/clients_last_seen_joined_v1/metadata.yaml	2024-05-21 17:30:39.000000000 +0000
@@ -0,0 +1,40 @@
+friendly_name: Clients Last Seen Joined
+description: |-
+  Join on baseline and metrics views, but uses a LEFT JOIN
+  such that a given client will not appear in the result if
+  it is only represented in the metrics-based table.
+
+  NOTE:
+  In rare cases the union can result in the same client_id entry from different channels.
+  In such cases, the resulting data will only include the entry for the client_id
+  with the oldest first_seen_date value.
+owners:
+- ascholtz@mozilla.com
+labels:
+  incremental: true
+  schedule: daily
+  dag: bqetl_glean_usage
+  owner1: ascholtz
+scheduling:
+  dag_name: bqetl_glean_usage
+  task_group: bergamot
+  depends_on_past: true
+bigquery:
+  time_partitioning:
+    type: day
+    field: submission_date
+    require_partition_filter: true
+    expiration_days: null
+  range_partitioning: null
+  clustering:
+    fields:
+    - normalized_channel
+    - sample_id
+workgroup_access:
+- role: roles/bigquery.dataViewer
+  members:
+  - workgroup:mozilla-confidential
+references:
+  query.sql:
+  - moz-fx-data-shared-prod.bergamot.baseline_clients_last_seen
+  - moz-fx-data-shared-prod.bergamot.metrics_clients_last_seen
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/bergamot_derived/clients_last_seen_joined_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/bergamot_derived/clients_last_seen_joined_v1/query.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/bergamot_derived/clients_last_seen_joined_v1/query.sql	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/bergamot_derived/clients_last_seen_joined_v1/query.sql	2024-05-21 17:20:55.000000000 +0000
@@ -0,0 +1,51 @@
+WITH baseline AS (
+  SELECT
+    * EXCEPT (
+      -- These were added as part of https://mozilla-hub.atlassian.net/browse/DENG-3462
+      -- to enable KPI metric calculations. This exclusion is to avoid potential compatibility
+      -- issues downstream from this query.
+      activity_segment,
+      is_dau,
+      is_wau,
+      is_mau,
+      is_daily_user,
+      is_weekly_user,
+      is_monthly_user,
+    )
+  FROM
+    `moz-fx-data-shared-prod.bergamot.baseline_clients_last_seen`
+  WHERE
+    submission_date = @submission_date
+),
+metrics AS (
+  SELECT
+    *
+  FROM
+    `moz-fx-data-shared-prod.bergamot.metrics_clients_last_seen`
+  WHERE
+    -- The join between baseline and metrics pings is based on submission_date with a 1 day delay,
+    -- since metrics pings usually arrive within 1 day after their logical activity period.
+    submission_date = DATE_ADD(@submission_date, INTERVAL 1 DAY)
+)
+SELECT
+  baseline.client_id,
+  baseline.sample_id,
+  baseline.submission_date,
+  baseline.normalized_channel,
+  * EXCEPT (submission_date, normalized_channel, client_id, sample_id)
+FROM
+  baseline
+LEFT JOIN
+  metrics
+  ON baseline.client_id = metrics.client_id
+  AND baseline.sample_id = metrics.sample_id
+  AND (
+    baseline.normalized_channel = metrics.normalized_channel
+    OR (baseline.normalized_channel IS NULL AND metrics.normalized_channel IS NULL)
+  )
+-- In some rare cases we end up with the same client_id in multiple channels
+-- In those cases, this union can result in client_id duplicates.
+-- This logic ensures that the resulting table only includes the client from the channel
+-- we've seen first.
+QUALIFY
+  ROW_NUMBER() OVER (PARTITION BY client_id ORDER BY first_seen_date ASC) = 1
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/bergamot_derived/dataset_metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/bergamot_derived/dataset_metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/bergamot_derived/dataset_metadata.yaml	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/bergamot_derived/dataset_metadata.yaml	2024-05-21 17:20:55.000000000 +0000
@@ -0,0 +1,10 @@
+friendly_name: Bergamot Derived
+description: |-
+  Derived data for bergamot_derived
+dataset_base_acl: derived
+user_facing: false
+labels: {}
+workgroup_access:
+- role: roles/bigquery.dataViewer
+  members:
+  - workgroup:mozilla-confidential
\ No newline at end of file
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham/baseline_clients_last_seen/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham/baseline_clients_last_seen/view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham/baseline_clients_last_seen/view.sql	2024-05-21 17:18:13.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham/baseline_clients_last_seen/view.sql	2024-05-21 17:20:39.000000000 +0000
@@ -14,6 +14,31 @@
   `moz-fx-data-shared-prod`.udf.pos_of_trailing_set_bit(
     days_seen_session_end_bits
   ) AS days_since_seen_session_end,
-  *
+  *,
+  -- Activity fields to support metrics built on top of activity
+  CASE
+    WHEN BIT_COUNT(days_active_bits)
+      BETWEEN 1
+      AND 6
+      THEN 'infrequent_user'
+    WHEN BIT_COUNT(days_active_bits)
+      BETWEEN 7
+      AND 13
+      THEN 'casual_user'
+    WHEN BIT_COUNT(days_active_bits)
+      BETWEEN 14
+      AND 20
+      THEN 'regular_user'
+    WHEN BIT_COUNT(days_active_bits) >= 21
+      THEN 'core_user'
+    ELSE 'other'
+  END AS activity_segment,
+  IFNULL(mozfun.bits28.days_since_seen(days_active_bits) = 0, FALSE) AS is_dau,
+  IFNULL(mozfun.bits28.days_since_seen(days_active_bits) < 7, FALSE) AS is_wau,
+  IFNULL(mozfun.bits28.days_since_seen(days_active_bits) < 28, FALSE) AS is_mau,
+  -- Metrics based on pings sent
+  IFNULL(mozfun.bits28.days_since_seen(days_seen_bits) = 0, FALSE) AS is_daily_user,
+  IFNULL(mozfun.bits28.days_since_seen(days_seen_bits) < 7, FALSE) AS is_weekly_user,
+  IFNULL(mozfun.bits28.days_since_seen(days_seen_bits) < 28, FALSE) AS is_monthly_user,
 FROM
   `moz-fx-data-shared-prod.burnham_derived.baseline_clients_last_seen_v1`
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/clients_last_seen_joined_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/clients_last_seen_joined_v1/query.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/clients_last_seen_joined_v1/query.sql	2024-05-21 17:18:13.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/clients_last_seen_joined_v1/query.sql	2024-05-21 17:20:57.000000000 +0000
@@ -1,6 +1,17 @@
 WITH baseline AS (
   SELECT
-    *
+    * EXCEPT (
+      -- These were added as part of https://mozilla-hub.atlassian.net/browse/DENG-3462
+      -- to enable KPI metric calculations. This exclusion is to avoid potential compatibility
+      -- issues downstream from this query.
+      activity_segment,
+      is_dau,
+      is_wau,
+      is_mau,
+      is_daily_user,
+      is_weekly_user,
+      is_monthly_user,
+    )
   FROM
     `moz-fx-data-shared-prod.burnham.baseline_clients_last_seen`
   WHERE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml	2024-05-21 17:17:35.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml	2024-05-21 17:25:33.000000000 +0000
@@ -1,49 +1,49 @@
 fields:
-- mode: NULLABLE
-  name: submission_date
+- name: submission_date
   type: DATE
-- mode: NULLABLE
-  name: source
+  mode: NULLABLE
+- name: source
   type: STRING
-- mode: NULLABLE
-  name: event_type
+  mode: NULLABLE
+- name: event_type
   type: STRING
-- mode: NULLABLE
-  name: form_factor
+  mode: NULLABLE
+- name: form_factor
   type: STRING
-- mode: NULLABLE
-  name: country
+  mode: NULLABLE
+- name: country
   type: STRING
-- mode: NULLABLE
-  name: subdivision1
+  mode: NULLABLE
+- name: subdivision1
   type: STRING
-- mode: NULLABLE
-  name: advertiser
+  mode: NULLABLE
+- name: advertiser
   type: STRING
-- mode: NULLABLE
-  name: release_channel
+  mode: NULLABLE
+- name: release_channel
   type: STRING
-- mode: NULLABLE
-  name: position
+  mode: NULLABLE
+- name: position
   type: INTEGER
-- mode: NULLABLE
-  name: provider
+  mode: NULLABLE
+- name: provider
   type: STRING
-- mode: NULLABLE
-  name: match_type
+  mode: NULLABLE
+- name: match_type
   type: STRING
-- mode: NULLABLE
-  name: normalized_os
+  mode: NULLABLE
+- name: normalized_os
   type: STRING
-- mode: NULLABLE
-  name: suggest_data_sharing_enabled
+  mode: NULLABLE
+- name: suggest_data_sharing_enabled
   type: BOOLEAN
-- mode: NULLABLE
-  name: event_count
+  mode: NULLABLE
+- name: event_count
   type: INTEGER
-- mode: NULLABLE
-  name: user_count
+  mode: NULLABLE
+- name: user_count
   type: INTEGER
-- mode: NULLABLE
-  name: query_type
+  mode: NULLABLE
+- name: query_type
   type: STRING
+  mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml	2024-05-21 17:17:35.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml	2024-05-21 17:25:33.000000000 +0000
@@ -1,40 +1,40 @@
 fields:
-- mode: NULLABLE
-  name: submission_date
+- name: submission_date
   type: DATE
-- mode: NULLABLE
-  name: form_factor
+  mode: NULLABLE
+- name: form_factor
   type: STRING
-- mode: NULLABLE
-  name: country
+  mode: NULLABLE
+- name: country
   type: STRING
-- mode: NULLABLE
-  name: advertiser
+  mode: NULLABLE
+- name: advertiser
   type: STRING
-- mode: NULLABLE
-  name: normalized_os
+  mode: NULLABLE
+- name: normalized_os
   type: STRING
-- mode: NULLABLE
-  name: release_channel
+  mode: NULLABLE
+- name: release_channel
   type: STRING
-- mode: NULLABLE
-  name: position
+  mode: NULLABLE
+- name: position
   type: INTEGER
-- mode: NULLABLE
-  name: provider
+  mode: NULLABLE
+- name: provider
   type: STRING
-- mode: NULLABLE
-  name: match_type
+  mode: NULLABLE
+- name: match_type
   type: STRING
-- mode: NULLABLE
-  name: suggest_data_sharing_enabled
+  mode: NULLABLE
+- name: suggest_data_sharing_enabled
   type: BOOLEAN
-- mode: NULLABLE
-  name: impression_count
+  mode: NULLABLE
+- name: impression_count
   type: INTEGER
-- mode: NULLABLE
-  name: click_count
+  mode: NULLABLE
+- name: click_count
   type: INTEGER
-- mode: NULLABLE
-  name: query_type
+  mode: NULLABLE
+- name: query_type
   type: STRING
+  mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/debug_ping_view/clients_last_seen_joined/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/debug_ping_view/clients_last_seen_joined/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/debug_ping_view/clients_last_seen_joined/metadata.yaml	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/debug_ping_view/clients_last_seen_joined/metadata.yaml	2024-05-21 17:30:36.000000000 +0000
@@ -0,0 +1,13 @@
+friendly_name: Clients Last Seen Joined
+description: |-
+  Please provide a description for the query
+owners: []
+labels: {}
+bigquery: null
+workgroup_access:
+- role: roles/bigquery.dataViewer
+  members:
+  - workgroup:mozilla-confidential
+references:
+  view.sql:
+  - moz-fx-data-shared-prod.debug_ping_view_derived.clients_last_seen_joined_v1
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/debug_ping_view/clients_last_seen_joined/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/debug_ping_view/clients_last_seen_joined/view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/debug_ping_view/clients_last_seen_joined/view.sql	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/debug_ping_view/clients_last_seen_joined/view.sql	2024-05-21 17:21:14.000000000 +0000
@@ -0,0 +1,7 @@
+CREATE OR REPLACE VIEW
+  `moz-fx-data-shared-prod.debug_ping_view.clients_last_seen_joined`
+AS
+SELECT
+  *
+FROM
+  `moz-fx-data-shared-prod.debug_ping_view_derived.clients_last_seen_joined_v1`
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/debug_ping_view_derived/clients_last_seen_joined_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/debug_ping_view_derived/clients_last_seen_joined_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/debug_ping_view_derived/clients_last_seen_joined_v1/metadata.yaml	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/debug_ping_view_derived/clients_last_seen_joined_v1/metadata.yaml	2024-05-21 17:30:37.000000000 +0000
@@ -0,0 +1,40 @@
+friendly_name: Clients Last Seen Joined
+description: |-
+  Join on baseline

⚠️ Only part of the diff is displayed.

Link to full diff

kik-kik · 2024-05-21T19:10:57Z

Closing in favor of #5594

kik-kik requested review from lucia-vargas-a and kwindau April 25, 2024 13:45

kik-kik self-assigned this Apr 25, 2024

bochocki reviewed Apr 25, 2024

View reviewed changes

sql_generators/glean_usage/templates/baseline_clients_last_seen.view.sql Outdated Show resolved Hide resolved

bochocki reviewed Apr 25, 2024

View reviewed changes

sql_generators/glean_usage/templates/baseline_clients_last_seen.view.sql Outdated Show resolved Hide resolved

bochocki reviewed Apr 25, 2024

View reviewed changes

sql_generators/glean_usage/templates/baseline_clients_last_seen.view.sql Outdated Show resolved Hide resolved

lucia-vargas-a reviewed Apr 25, 2024

View reviewed changes

sql_generators/glean_usage/templates/baseline_clients_daily.view.sql Outdated Show resolved Hide resolved