Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update clients_last_seen_joined query and scheduler #5401

Conversation

lucia-vargas-a
Copy link
Contributor

@lucia-vargas-a lucia-vargas-a commented Apr 19, 2024

Resolves DENG-3475

Checklist for reviewer:

  • Commits should reference a bug or github issue, if relevant (if a bug is referenced, the pull request should include the bug number in the title).
  • If the PR comes from a fork, trigger integration CI tests by running the Push to upstream workflow and provide the <username>:<branch> of the fork as parameter. The parameter will also show up
    in the logs of the manual-trigger-required-for-fork CI task together with more detailed instructions.
  • If adding a new field to a query, ensure that the schema and dependent downstream schemas have been updated.
  • When adding a new derived dataset, ensure that data is not available already (fully or partially) and recommend extending an existing dataset in favor of creating new ones. Data can be available in the bigquery-etl repository, looker-hub or in looker-spoke-default.

For modifications to schemas in restricted namespaces (see CODEOWNERS):

┆Issue is synchronized with this Jira Task

@dataops-ci-bot

This comment has been minimized.

@@ -15,7 +15,7 @@ scheduling:
dag_name: bqetl_glean_usage
task_group: {{ app_name }}
depends_on_past: true
date_partition_offset: -1
date_partition_offset: -2
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does this need an additional day of lag if it's only looking one day ahead?

@dataops-ci-bot
Copy link

Integration report for "One day offset."

sql.diff

Click to expand!
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/dags/bqetl_analytics_aggregations.py /tmp/workspace/generated-sql/dags/bqetl_analytics_aggregations.py
--- /tmp/workspace/main-generated-sql/dags/bqetl_analytics_aggregations.py	2024-04-19 22:58:53.000000000 +0000
+++ /tmp/workspace/generated-sql/dags/bqetl_analytics_aggregations.py	2024-04-19 23:10:28.000000000 +0000
@@ -290,7 +290,7 @@
 
     active_users_aggregates_attribution_v1 = bigquery_etl_query(
         task_id="active_users_aggregates_attribution_v1",
-        destination_table="active_users_aggregates_attribution_v1",
+        destination_table='active_users_aggregates_attribution_v1${{ macros.ds_format(macros.ds_add(ds, -1), "%Y-%m-%d", "%Y%m%d") }}',
         dataset_id="telemetry_derived",
         project_id="moz-fx-data-shared-prod",
         owner="lvargas@mozilla.com",
@@ -299,13 +299,14 @@
             "lvargas@mozilla.com",
             "telemetry-alerts@mozilla.com",
         ],
-        date_partition_parameter="submission_date",
+        date_partition_parameter=None,
         depends_on_past=False,
+        parameters=["submission_date:DATE:{{macros.ds_add(ds, -1)}}"],
     )
 
     active_users_aggregates_device_v1 = bigquery_etl_query(
         task_id="active_users_aggregates_device_v1",
-        destination_table="active_users_aggregates_device_v1",
+        destination_table='active_users_aggregates_device_v1${{ macros.ds_format(macros.ds_add(ds, -1), "%Y-%m-%d", "%Y%m%d") }}',
         dataset_id="telemetry_derived",
         project_id="moz-fx-data-shared-prod",
         owner="lvargas@mozilla.com",
@@ -314,8 +315,9 @@
             "lvargas@mozilla.com",
             "telemetry-alerts@mozilla.com",
         ],
-        date_partition_parameter="submission_date",
+        date_partition_parameter=None,
         depends_on_past=False,
+        parameters=["submission_date:DATE:{{macros.ds_add(ds, -1)}}"],
     )
 
     with TaskGroup(
@@ -334,7 +336,7 @@
 
     active_users_aggregates_v1 = bigquery_etl_query(
         task_id="active_users_aggregates_v1",
-        destination_table="active_users_aggregates_v1",
+        destination_table='active_users_aggregates_v1${{ macros.ds_format(macros.ds_add(ds, -1), "%Y-%m-%d", "%Y%m%d") }}',
         dataset_id="telemetry_derived",
         project_id="moz-fx-data-shared-prod",
         owner="lvargas@mozilla.com",
@@ -343,8 +345,9 @@
             "lvargas@mozilla.com",
             "telemetry-alerts@mozilla.com",
         ],
-        date_partition_parameter="submission_date",
+        date_partition_parameter=None,
         depends_on_past=False,
+        parameters=["submission_date:DATE:{{macros.ds_add(ds, -1)}}"],
     )
 
     checks__warn_fenix_derived__active_users_aggregates__v3 = bigquery_dq_check(
@@ -453,7 +456,7 @@
 
     fenix_active_users_aggregates = bigquery_etl_query(
         task_id="fenix_active_users_aggregates",
-        destination_table="active_users_aggregates_v3",
+        destination_table='active_users_aggregates_v3${{ macros.ds_format(macros.ds_add(ds, -1), "%Y-%m-%d", "%Y%m%d") }}',
         dataset_id="fenix_derived",
         project_id="moz-fx-data-shared-prod",
         owner="lvargas@mozilla.com",
@@ -462,8 +465,9 @@
             "lvargas@mozilla.com",
             "telemetry-alerts@mozilla.com",
         ],
-        date_partition_parameter="submission_date",
+        date_partition_parameter=None,
         depends_on_past=False,
+        parameters=["submission_date:DATE:{{macros.ds_add(ds, -1)}}"],
     )
 
     firefox_desktop_active_users_aggregates = bigquery_etl_query(
@@ -483,7 +487,7 @@
 
     firefox_ios_active_users_aggregates = bigquery_etl_query(
         task_id="firefox_ios_active_users_aggregates",
-        destination_table="active_users_aggregates_v3",
+        destination_table='active_users_aggregates_v3${{ macros.ds_format(macros.ds_add(ds, -1), "%Y-%m-%d", "%Y%m%d") }}',
         dataset_id="firefox_ios_derived",
         project_id="moz-fx-data-shared-prod",
         owner="lvargas@mozilla.com",
@@ -492,13 +496,14 @@
             "lvargas@mozilla.com",
             "telemetry-alerts@mozilla.com",
         ],
-        date_partition_parameter="submission_date",
+        date_partition_parameter=None,
         depends_on_past=False,
+        parameters=["submission_date:DATE:{{macros.ds_add(ds, -1)}}"],
     )
 
     focus_android_active_users_aggregates = bigquery_etl_query(
         task_id="focus_android_active_users_aggregates",
-        destination_table="active_users_aggregates_v3",
+        destination_table='active_users_aggregates_v3${{ macros.ds_format(macros.ds_add(ds, -1), "%Y-%m-%d", "%Y%m%d") }}',
         dataset_id="focus_android_derived",
         project_id="moz-fx-data-shared-prod",
         owner="lvargas@mozilla.com",
@@ -507,13 +512,14 @@
             "lvargas@mozilla.com",
             "telemetry-alerts@mozilla.com",
         ],
-        date_partition_parameter="submission_date",
+        date_partition_parameter=None,
         depends_on_past=False,
+        parameters=["submission_date:DATE:{{macros.ds_add(ds, -1)}}"],
     )
 
     focus_ios_active_users_aggregates = bigquery_etl_query(
         task_id="focus_ios_active_users_aggregates",
-        destination_table="active_users_aggregates_v3",
+        destination_table='active_users_aggregates_v3${{ macros.ds_format(macros.ds_add(ds, -1), "%Y-%m-%d", "%Y%m%d") }}',
         dataset_id="focus_ios_derived",
         project_id="moz-fx-data-shared-prod",
         owner="lvargas@mozilla.com",
@@ -522,13 +528,14 @@
             "lvargas@mozilla.com",
             "telemetry-alerts@mozilla.com",
         ],
-        date_partition_parameter="submission_date",
+        date_partition_parameter=None,
         depends_on_past=False,
+        parameters=["submission_date:DATE:{{macros.ds_add(ds, -1)}}"],
     )
 
     klar_ios_active_users_aggregates = bigquery_etl_query(
         task_id="klar_ios_active_users_aggregates",
-        destination_table="active_users_aggregates_v3",
+        destination_table='active_users_aggregates_v3${{ macros.ds_format(macros.ds_add(ds, -1), "%Y-%m-%d", "%Y%m%d") }}',
         dataset_id="klar_ios_derived",
         project_id="moz-fx-data-shared-prod",
         owner="lvargas@mozilla.com",
@@ -537,13 +544,14 @@
             "lvargas@mozilla.com",
             "telemetry-alerts@mozilla.com",
         ],
-        date_partition_parameter="submission_date",
+        date_partition_parameter=None,
         depends_on_past=False,
+        parameters=["submission_date:DATE:{{macros.ds_add(ds, -1)}}"],
     )
 
     telemetry_derived__cohort_daily_statistics__v1 = bigquery_etl_query(
         task_id="telemetry_derived__cohort_daily_statistics__v1",
-        destination_table="cohort_daily_statistics_v1",
+        destination_table='cohort_daily_statistics_v1${{ macros.ds_format(macros.ds_add(ds, -1), "%Y-%m-%d", "%Y%m%d") }}',
         dataset_id="telemetry_derived",
         project_id="moz-fx-data-shared-prod",
         owner="anicholson@mozilla.com",
@@ -553,8 +561,9 @@
             "lvargas@mozilla.com",
             "telemetry-alerts@mozilla.com",
         ],
-        date_partition_parameter="activity_date",
+        date_partition_parameter=None,
         depends_on_past=False,
+        parameters=["activity_date:DATE:{{macros.ds_add(ds, -1)}}"],
     )
 
     telemetry_derived__desktop_cohort_daily_retention__v1 = bigquery_etl_query(
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/dags/bqetl_ctxsvc_derived.py /tmp/workspace/generated-sql/dags/bqetl_ctxsvc_derived.py
--- /tmp/workspace/main-generated-sql/dags/bqetl_ctxsvc_derived.py	2024-04-19 22:58:53.000000000 +0000
+++ /tmp/workspace/generated-sql/dags/bqetl_ctxsvc_derived.py	2024-04-19 23:10:27.000000000 +0000
@@ -114,7 +114,7 @@
 
     contextual_services_derived__adm_forecasting__v1 = bigquery_etl_query(
         task_id="contextual_services_derived__adm_forecasting__v1",
-        destination_table="adm_forecasting_v1",
+        destination_table='adm_forecasting_v1${{ macros.ds_format(macros.ds_add(ds, -1), "%Y-%m-%d", "%Y%m%d") }}',
         dataset_id="contextual_services_derived",
         project_id="moz-fx-data-shared-prod",
         owner="skahmann@mozilla.com",
@@ -124,8 +124,9 @@
             "telemetry-alerts@mozilla.com",
             "wstuckey@mozilla.com",
         ],
-        date_partition_parameter="submission_date",
+        date_partition_parameter=None,
         depends_on_past=False,
+        parameters=["submission_date:DATE:{{macros.ds_add(ds, -1)}}"],
     )
 
     contextual_services_derived__event_aggregates__v1 = bigquery_etl_query(
@@ -230,7 +231,7 @@
 
     contextual_services_derived__suggest_revenue_levers_daily__v1 = bigquery_etl_query(
         task_id="contextual_services_derived__suggest_revenue_levers_daily__v1",
-        destination_table="suggest_revenue_levers_daily_v1",
+        destination_table='suggest_revenue_levers_daily_v1${{ macros.ds_format(macros.ds_add(ds, -1), "%Y-%m-%d", "%Y%m%d") }}',
         dataset_id="contextual_services_derived",
         project_id="moz-fx-data-shared-prod",
         owner="skahmann@mozilla.com",
@@ -240,8 +241,9 @@
             "telemetry-alerts@mozilla.com",
             "wstuckey@mozilla.com",
         ],
-        date_partition_parameter="submission_date",
+        date_partition_parameter=None,
         depends_on_past=False,
+        parameters=["submission_date:DATE:{{macros.ds_add(ds, -1)}}"],
     )
 
     contextual_services_derived__adm_forecasting__v1.set_upstream(
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/dags/bqetl_data_observability_test_data_copy.py /tmp/workspace/generated-sql/dags/bqetl_data_observability_test_data_copy.py
--- /tmp/workspace/main-generated-sql/dags/bqetl_data_observability_test_data_copy.py	2024-04-19 22:58:53.000000000 +0000
+++ /tmp/workspace/generated-sql/dags/bqetl_data_observability_test_data_copy.py	2024-04-19 23:10:29.000000000 +0000
@@ -117,13 +117,14 @@
 
     fenix_derived__clients_last_seen_joined__v1 = bigquery_etl_query(
         task_id="fenix_derived__clients_last_seen_joined__v1",
-        destination_table="clients_last_seen_joined_v1",
+        destination_table='clients_last_seen_joined_v1${{ macros.ds_format(macros.ds_add(ds, -1), "%Y-%m-%d", "%Y%m%d") }}',
         dataset_id="fenix_derived",
         project_id="data-observability-dev",
         owner="kik@mozilla.com",
         email=["akommasani@mozilla.com", "ascholtz@mozilla.com", "kik@mozilla.com"],
-        date_partition_parameter="submission_date",
+        date_partition_parameter=None,
         depends_on_past=True,
+        parameters=["submission_date:DATE:{{macros.ds_add(ds, -1)}}"],
     )
 
     fenix_derived__event_types__v1 = bigquery_etl_query(
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/dags/bqetl_glean_usage.py /tmp/workspace/generated-sql/dags/bqetl_glean_usage.py
--- /tmp/workspace/main-generated-sql/dags/bqetl_glean_usage.py	2024-04-19 22:58:53.000000000 +0000
+++ /tmp/workspace/generated-sql/dags/bqetl_glean_usage.py	2024-04-19 23:10:31.000000000 +0000
@@ -358,13 +358,14 @@
 
     burnham_derived__clients_last_seen_joined__v1 = bigquery_etl_query(
         task_id="burnham_derived__clients_last_seen_joined__v1",
-        destination_table="clients_last_seen_joined_v1",
+        destination_table='clients_last_seen_joined_v1${{ macros.ds_format(macros.ds_add(ds, -1), "%Y-%m-%d", "%Y%m%d") }}',
         dataset_id="burnham_derived",
         project_id="moz-fx-data-shared-prod",
         owner="ascholtz@mozilla.com",
         email=["ascholtz@mozilla.com", "telemetry-alerts@mozilla.com"],
-        date_partition_parameter="submission_date",
+        date_partition_parameter=None,
         depends_on_past=True,
+        parameters=["submission_date:DATE:{{macros.ds_add(ds, -1)}}"],
         task_group=task_group_burnham,
     )
 
@@ -778,13 +779,14 @@
 
     fenix_derived__clients_last_seen_joined__v1 = bigquery_etl_query(
         task_id="fenix_derived__clients_last_seen_joined__v1",
-        destination_table="clients_last_seen_joined_v1",
+        destination_table='clients_last_seen_joined_v1${{ macros.ds_format(macros.ds_add(ds, -1), "%Y-%m-%d", "%Y%m%d") }}',
         dataset_id="fenix_derived",
         project_id="moz-fx-data-shared-prod",
         owner="ascholtz@mozilla.com",
         email=["ascholtz@mozilla.com", "telemetry-alerts@mozilla.com"],
-        date_partition_parameter="submission_date",
+        date_partition_parameter=None,
         depends_on_past=True,
+        parameters=["submission_date:DATE:{{macros.ds_add(ds, -1)}}"],
         task_group=task_group_fenix,
     )
 
@@ -1033,13 +1035,14 @@
 
     firefox_desktop_background_update_derived__clients_last_seen_joined__v1 = bigquery_etl_query(
         task_id="firefox_desktop_background_update_derived__clients_last_seen_joined__v1",
-        destination_table="clients_last_seen_joined_v1",
+        destination_table='clients_last_seen_joined_v1${{ macros.ds_format(macros.ds_add(ds, -1), "%Y-%m-%d", "%Y%m%d") }}',
         dataset_id="firefox_desktop_background_update_derived",
         project_id="moz-fx-data-shared-prod",
         owner="ascholtz@mozilla.com",
         email=["ascholtz@mozilla.com", "telemetry-alerts@mozilla.com"],
-        date_partition_parameter="submission_date",
+        date_partition_parameter=None,
         depends_on_past=True,
+        parameters=["submission_date:DATE:{{macros.ds_add(ds, -1)}}"],
         task_group=task_group_firefox_desktop_background_update,
     )
 
@@ -1123,13 +1126,14 @@
 
     firefox_desktop_derived__clients_last_seen_joined__v1 = bigquery_etl_query(
         task_id="firefox_desktop_derived__clients_last_seen_joined__v1",
-        destination_table="clients_last_seen_joined_v1",
+        destination_table='clients_last_seen_joined_v1${{ macros.ds_format(macros.ds_add(ds, -1), "%Y-%m-%d", "%Y%m%d") }}',
         dataset_id="firefox_desktop_derived",
         project_id="moz-fx-data-shared-prod",
         owner="ascholtz@mozilla.com",
         email=["ascholtz@mozilla.com", "telemetry-alerts@mozilla.com"],
-        date_partition_parameter="submission_date",
+        date_partition_parameter=None,
         depends_on_past=True,
+        parameters=["submission_date:DATE:{{macros.ds_add(ds, -1)}}"],
         task_group=task_group_firefox_desktop,
     )
 
@@ -1176,13 +1180,14 @@
 
     firefox_echo_show_derived__clients_last_seen_joined__v1 = bigquery_etl_query(
         task_id="firefox_echo_show_derived__clients_last_seen_joined__v1",
-        destination_table="clients_last_seen_joined_v1",
+        destination_table='clients_last_seen_joined_v1${{ macros.ds_format(macros.ds_add(ds, -1), "%Y-%m-%d", "%Y%m%d") }}',
         dataset_id="firefox_echo_show_derived",
         project_id="moz-fx-data-shared-prod",
         owner="ascholtz@mozilla.com",
         email=["ascholtz@mozilla.com", "telemetry-alerts@mozilla.com"],
-        date_partition_parameter="submission_date",
+        date_partition_parameter=None,
         depends_on_past=True,
+        parameters=["submission_date:DATE:{{macros.ds_add(ds, -1)}}"],
         task_group=task_group_firefox_echo_show,
     )
 
@@ -1212,13 +1217,14 @@
 
     firefox_fire_tv_derived__clients_last_seen_joined__v1 = bigquery_etl_query(
         task_id="firefox_fire_tv_derived__clients_last_seen_joined__v1",
-        destination_table="clients_last_seen_joined_v1",
+        destination_table='clients_last_seen_joined_v1${{ macros.ds_format(macros.ds_add(ds, -1), "%Y-%m-%d", "%Y%m%d") }}',
         dataset_id="firefox_fire_tv_derived",
         project_id="moz-fx-data-shared-prod",
         owner="ascholtz@mozilla.com",
         email=["ascholtz@mozilla.com", "telemetry-alerts@mozilla.com"],
-        date_partition_parameter="submission_date",
+        date_partition_parameter=None,
         depends_on_past=True,
+        parameters=["submission_date:DATE:{{macros.ds_add(ds, -1)}}"],
         task_group=task_group_firefox_fire_tv,
     )
 
@@ -1248,13 +1254,14 @@
 
     firefox_ios_derived__clients_last_seen_joined__v1 = bigquery_etl_query(
         task_id="firefox_ios_derived__clients_last_seen_joined__v1",
-        destination_table="clients_last_seen_joined_v1",
+        destination_table='clients_last_seen_joined_v1${{ macros.ds_format(macros.ds_add(ds, -1), "%Y-%m-%d", "%Y%m%d") }}',
         dataset_id="firefox_ios_derived",
         project_id="moz-fx-data-shared-prod",
         owner="ascholtz@mozilla.com",
         email=["ascholtz@mozilla.com", "telemetry-alerts@mozilla.com"],
-        date_partition_parameter="submission_date",
+        date_partition_parameter=None,
         depends_on_past=True,
+        parameters=["submission_date:DATE:{{macros.ds_add(ds, -1)}}"],
         task_group=task_group_firefox_ios,
     )
 
@@ -1312,13 +1319,14 @@
 
     firefox_reality_derived__clients_last_seen_joined__v1 = bigquery_etl_query(
         task_id="firefox_reality_derived__clients_last_seen_joined__v1",
-        destination_table="clients_last_seen_joined_v1",
+        destination_table='clients_last_seen_joined_v1${{ macros.ds_format(macros.ds_add(ds, -1), "%Y-%m-%d", "%Y%m%d") }}',
         dataset_id="firefox_reality_derived",
         project_id="moz-fx-data-shared-prod",
         owner="ascholtz@mozilla.com",
         email=["ascholtz@mozilla.com", "telemetry-alerts@mozilla.com"],
-        date_partition_parameter="submission_date",
+        date_partition_parameter=None,
         depends_on_past=True,
+        parameters=["submission_date:DATE:{{macros.ds_add(ds, -1)}}"],
         task_group=task_group_firefox_reality,
     )
 
@@ -1348,13 +1356,14 @@
 
     firefox_reality_pc_derived__clients_last_seen_joined__v1 = bigquery_etl_query(
         task_id="firefox_reality_pc_derived__clients_last_seen_joined__v1",
-        destination_table="clients_last_seen_joined_v1",
+        destination_table='clients_last_seen_joined_v1${{ macros.ds_format(macros.ds_add(ds, -1), "%Y-%m-%d", "%Y%m%d") }}',
         dataset_id="firefox_reality_pc_derived",
         project_id="moz-fx-data-shared-prod",
         owner="ascholtz@mozilla.com",
         email=["ascholtz@mozilla.com", "telemetry-alerts@mozilla.com"],
-        date_partition_parameter="submission_date",
+        date_partition_parameter=None,
         depends_on_past=True,
+        parameters=["submission_date:DATE:{{macros.ds_add(ds, -1)}}"],
         task_group=task_group_firefox_reality_pc,
     )
 
@@ -1401,13 +1410,14 @@
 
     focus_android_derived__clients_last_seen_joined__v1 = bigquery_etl_query(
         task_id="focus_android_derived__clients_last_seen_joined__v1",
-        destination_table="clients_last_seen_joined_v1",
+        destination_table='clients_last_seen_joined_v1${{ macros.ds_format(macros.ds_add(ds, -1), "%Y-%m-%d", "%Y%m%d") }}',
         dataset_id="focus_android_derived",
         project_id="moz-fx-data-shared-prod",
         owner="ascholtz@mozilla.com",
         email=["ascholtz@mozilla.com", "telemetry-alerts@mozilla.com"],
-        date_partition_parameter="submission_date",
+        date_partition_parameter=None,
         depends_on_past=True,
+        parameters=["submission_date:DATE:{{macros.ds_add(ds, -1)}}"],
         task_group=task_group_focus_android,
     )
 
@@ -1459,13 +1469,14 @@
 
     focus_ios_derived__clients_last_seen_joined__v1 = bigquery_etl_query(
         task_id="focus_ios_derived__clients_last_seen_joined__v1",
-        destination_table="clients_last_seen_joined_v1",
+        destination_table='clients_last_seen_joined_v1${{ macros.ds_format(macros.ds_add(ds, -1), "%Y-%m-%d", "%Y%m%d") }}',
         dataset_id="focus_ios_derived",
         project_id="moz-fx-data-shared-prod",
         owner="ascholtz@mozilla.com",
         email=["ascholtz@mozilla.com", "telemetry-alerts@mozilla.com"],
-        date_partition_parameter="submission_date",
+        date_partition_parameter=None,
         depends_on_past=True,
+        parameters=["submission_date:DATE:{{macros.ds_add(ds, -1)}}"],
         task_group=task_group_focus_ios,
     )
 
@@ -1540,13 +1551,14 @@
 
     klar_android_derived__clients_last_seen_joined__v1 = bigquery_etl_query(
         task_id="klar_android_derived__clients_last_seen_joined__v1",
-        destination_table="clients_last_seen_joined_v1",
+        destination_table='clients_last_seen_joined_v1${{ macros.ds_format(macros.ds_add(ds, -1), "%Y-%m-%d", "%Y%m%d") }}',
         dataset_id="klar_android_derived",
         project_id="moz-fx-data-shared-prod",
         owner="ascholtz@mozilla.com",
         email=["ascholtz@mozilla.com", "telemetry-alerts@mozilla.com"],
-        date_partition_parameter="submission_date",
+        date_partition_parameter=None,
         depends_on_past=True,
+        parameters=["submission_date:DATE:{{macros.ds_add(ds, -1)}}"],
         task_group=task_group_klar_android,
     )
 
@@ -1576,13 +1588,14 @@
 
     klar_ios_derived__clients_last_seen_joined__v1 = bigquery_etl_query(
         task_id="klar_ios_derived__clients_last_seen_joined__v1",
-        destination_table="clients_last_seen_joined_v1",
+        destination_table='clients_last_seen_joined_v1${{ macros.ds_format(macros.ds_add(ds, -1), "%Y-%m-%d", "%Y%m%d") }}',
         dataset_id="klar_ios_derived",
         project_id="moz-fx-data-shared-prod",
         owner="ascholtz@mozilla.com",
         email=["ascholtz@mozilla.com", "telemetry-alerts@mozilla.com"],
-        date_partition_parameter="submission_date",
+        date_partition_parameter=None,
         depends_on_past=True,
+        parameters=["submission_date:DATE:{{macros.ds_add(ds, -1)}}"],
         task_group=task_group_klar_ios,
     )
 
@@ -1640,13 +1653,14 @@
 
     lockwise_android_derived__clients_last_seen_joined__v1 = bigquery_etl_query(
         task_id="lockwise_android_derived__clients_last_seen_joined__v1",
-        destination_table="clients_last_seen_joined_v1",
+        destination_table='clients_last_seen_joined_v1${{ macros.ds_format(macros.ds_add(ds, -1), "%Y-%m-%d", "%Y%m%d") }}',
         dataset_id="lockwise_android_derived",
         project_id="moz-fx-data-shared-prod",
         owner="ascholtz@mozilla.com",
         email=["ascholtz@mozilla.com", "telemetry-alerts@mozilla.com"],
-        date_partition_parameter="submission_date",
+        date_partition_parameter=None,
         depends_on_past=True,
+        parameters=["submission_date:DATE:{{macros.ds_add(ds, -1)}}"],
         task_group=task_group_lockwise_android,
     )
 
@@ -1676,13 +1690,14 @@
 
     lockwise_ios_derived__clients_last_seen_joined__v1 = bigquery_etl_query(
         task_id="lockwise_ios_derived__clients_last_seen_joined__v1",
-        destination_table="clients_last_seen_joined_v1",
+        destination_table='clients_last_seen_joined_v1${{ macros.ds_format(macros.ds_add(ds, -1), "%Y-%m-%d", "%Y%m%d") }}',
         dataset_id="lockwise_ios_derived",
         project_id="moz-fx-data-shared-prod",
         owner="ascholtz@mozilla.com",
         email=["ascholtz@mozilla.com", "telemetry-alerts@mozilla.com"],
-        date_partition_parameter="submission_date",
+        date_partition_parameter=None,
         depends_on_past=True,
+        parameters=["submission_date:DATE:{{macros.ds_add(ds, -1)}}"],
         task_group=task_group_lockwise_ios,
     )
 
@@ -1712,13 +1727,14 @@
 
     mach_derived__clients_last_seen_joined__v1 = bigquery_etl_query(
         task_id="mach_derived__clients_last_seen_joined__v1",
-        destination_table="clients_last_seen_joined_v1",
+        destination_table='clients_last_seen_joined_v1${{ macros.ds_format(macros.ds_add(ds, -1), "%Y-%m-%d", "%Y%m%d") }}',
         dataset_id="mach_derived",
         project_id="moz-fx-data-shared-prod",
         owner="ascholtz@mozilla.com",
         email=["ascholtz@mozilla.com", "telemetry-alerts@mozilla.com"],
-        date_partition_parameter="submission_date",
+        date_partition_parameter=None,
         depends_on_past=True,
+        parameters=["submission_date:DATE:{{macros.ds_add(ds, -1)}}"],
         task_group=task_group_mach,
     )
 
@@ -2149,13 +2165,14 @@
 
     mozphab_derived__clients_last_seen_joined__v1 = bigquery_etl_query(
         task_id="mozphab_derived__clients_last_seen_joined__v1",
-        destination_table="clients_last_seen_joined_v1",
+        destination_table='clients_last_seen_joined_v1${{ macros.ds_format(macros.ds_add(ds, -1), "%Y-%m-%d", "%Y%m%d") }}',
         dataset_id="mozphab_derived",
         project_id="moz-fx-data-shared-prod",
         owner="ascholtz@mozilla.com",
         email=["ascholtz@mozilla.com", "telemetry-alerts@mozilla.com"],
-        date_partition_parameter="submission_date",
+        date_partition_parameter=None,
         depends_on_past=True,
+        parameters=["submission_date:DATE:{{macros.ds_add(ds, -1)}}"],
         task_group=task_group_mozphab,
     )
 
@@ -2202,13 +2219,14 @@
 
     mozregression_derived__clients_last_seen_joined__v1 = bigquery_etl_query(
         task_id="mozregression_derived__clients_last_seen_joined__v1",
-        destination_table="clients_last_seen_joined_v1",
+        destination_table='clients_last_seen_joined_v1${{ macros.ds_format(macros.ds_add(ds, -1), "%Y-%m-%d", "%Y%m%d") }}',
         dataset_id="mozregression_derived",
         project_id="moz-fx-data-shared-prod",
         owner="ascholtz@mozilla.com",
         email=["ascholtz@mozilla.com", "telemetry-alerts@mozilla.com"],
-        date_partition_parameter="submission_date",
+        date_partition_parameter=None,
         depends_on_past=True,
+        parameters=["submission_date:DATE:{{macros.ds_add(ds, -1)}}"],
         task_group=task_group_mozregression,
     )
 
@@ -4374,13 +4392,14 @@
 
     reference_browser_derived__clients_last_seen_joined__v1 = bigquery_etl_query(
         task_id="reference_browser_derived__clients_last_seen_joined__v1",
-        destination_table="clients_last_seen_joined_v1",
+        destination_table='clients_last_seen_joined_v1${{ macros.ds_format(macros.ds_add(ds, -1), "%Y-%m-%d", "%Y%m%d") }}',
         dataset_id="reference_browser_derived",
         project_id="moz-fx-data-shared-prod",
         owner="ascholtz@mozilla.com",
         email=["ascholtz@mozilla.com", "telemetry-alerts@mozilla.com"],
-        date_partition_parameter="submission_date",
+        date_partition_parameter=None,
         depends_on_past=True,
+        parameters=["submission_date:DATE:{{macros.ds_add(ds, -1)}}"],
         task_group=task_group_reference_browser,
     )
 
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/dags/bqetl_kpis_shredder.py /tmp/workspace/generated-sql/dags/bqetl_kpis_shredder.py
--- /tmp/workspace/main-generated-sql/dags/bqetl_kpis_shredder.py	2024-04-19 22:58:53.000000000 +0000
+++ /tmp/workspace/generated-sql/dags/bqetl_kpis_shredder.py	2024-04-19 23:10:28.000000000 +0000
@@ -132,14 +132,15 @@
 
     fenix_active_users_aggregates_for_deletion_requests = bigquery_etl_query(
         task_id="fenix_active_users_aggregates_for_deletion_requests",
-        destination_table="active_users_aggregates_deletion_request_v1",
+        destination_table='active_users_aggregates_deletion_request_v1${{ macros.ds_format(macros.ds_add(ds, -1), "%Y-%m-%d", "%Y%m%d") }}',
         dataset_id="fenix_derived",
         project_id="moz-fx-data-shared-prod",
         owner="lvargas@mozilla.com",
         email=["lvargas@mozilla.com", "telemetry-alerts@mozilla.com"],
-        date_partition_parameter="partition_date",
+        date_partition_parameter=None,
         depends_on_past=False,
-        parameters=[
+        parameters=["partition_date:DATE:{{macros.ds_add(ds, -1)}}"]
+        + [
             "end_date:DATE:{{macros.ds_add(ds, 27)}}",
             "start_date:DATE:{{macros.ds_add(ds, 27-28*4)}}",
         ],
@@ -162,14 +163,15 @@
 
     firefox_ios_active_users_aggregates_for_deletion_requests = bigquery_etl_query(
         task_id="firefox_ios_active_users_aggregates_for_deletion_requests",
-        destination_table="active_users_aggregates_deletion_request_v1",
+        destination_table='active_users_aggregates_deletion_request_v1${{ macros.ds_format(macros.ds_add(ds, -1), "%Y-%m-%d", "%Y%m%d") }}',
         dataset_id="firefox_ios_derived",
         project_id="moz-fx-data-shared-prod",
         owner="lvargas@mozilla.com",
         email=["lvargas@mozilla.com", "telemetry-alerts@mozilla.com"],
-        date_partition_parameter="partition_date",
+        date_partition_parameter=None,
         depends_on_past=False,
-        parameters=[
+        parameters=["partition_date:DATE:{{macros.ds_add(ds, -1)}}"]
+        + [
             "end_date:DATE:{{macros.ds_add(ds, 27)}}",
             "start_date:DATE:{{macros.ds_add(ds, 27-28*4)}}",
         ],
@@ -177,14 +179,15 @@
 
     focus_ios_active_users_aggregates_for_deletion_requests = bigquery_etl_query(
         task_id="focus_ios_active_users_aggregates_for_deletion_requests",
-        destination_table="active_users_aggregates_deletion_request_v1",
+        destination_table='active_users_aggregates_deletion_request_v1${{ macros.ds_format(macros.ds_add(ds, -1), "%Y-%m-%d", "%Y%m%d") }}',
         dataset_id="focus_ios_derived",
         project_id="moz-fx-data-shared-prod",
         owner="lvargas@mozilla.com",
         email=["lvargas@mozilla.com", "telemetry-alerts@mozilla.com"],
-        date_partition_parameter="partition_date",
+        date_partition_parameter=None,
         depends_on_past=False,
-        parameters=[
+        parameters=["partition_date:DATE:{{macros.ds_add(ds, -1)}}"]
+        + [
             "end_date:DATE:{{macros.ds_add(ds, 27)}}",
             "start_date:DATE:{{macros.ds_add(ds, 27-28*4)}}",
         ],
@@ -192,14 +195,15 @@
 
     klar_ios_active_users_aggregates_for_deletion_requests = bigquery_etl_query(
         task_id="klar_ios_active_users_aggregates_for_deletion_requests",
-        destination_table="active_users_aggregates_deletion_request_v1",
+        destination_table='active_users_aggregates_deletion_request_v1${{ macros.ds_format(macros.ds_add(ds, -1), "%Y-%m-%d", "%Y%m%d") }}',
         dataset_id="klar_ios_derived",
         project_id="moz-fx-data-shared-prod",
         owner="lvargas@mozilla.com",
         email=["lvargas@mozilla.com", "telemetry-alerts@mozilla.com"],
-        date_partition_parameter="partition_date",
+        date_partition_parameter=None,
         depends_on_past=False,
-        parameters=[
+        parameters=["partition_date:DATE:{{macros.ds_add(ds, -1)}}"]
+        + [
             "end_date:DATE:{{macros.ds_add(ds, 27)}}",
             "start_date:DATE:{{macros.ds_add(ds, 27-28*4)}}",
         ],
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/dags/bqetl_newtab.py /tmp/workspace/generated-sql/dags/bqetl_newtab.py
--- /tmp/workspace/main-generated-sql/dags/bqetl_newtab.py	2024-04-19 22:58:53.000000000 +0000
+++ /tmp/workspace/generated-sql/dags/bqetl_newtab.py	2024-04-19 23:10:27.000000000 +0000
@@ -76,7 +76,7 @@
 
     telemetry_derived__newtab_clients_daily__v1 = bigquery_etl_query(
         task_id="telemetry_derived__newtab_clients_daily__v1",
-        destination_table="newtab_clients_daily_v1",
+        destination_table='newtab_clients_daily_v1${{ macros.ds_format(macros.ds_add(ds, -1), "%Y-%m-%d", "%Y%m%d") }}',
         dataset_id="telemetry_derived",
         project_id="moz-fx-data-shared-prod",
         owner="mbowerman@mozilla.com",
@@ -85,30 +85,33 @@
             "mbowerman@mozilla.com",
             "telemetry-alerts@mozilla.com",
         ],
-        date_partition_parameter="submission_date",
+        date_partition_parameter=None,
         depends_on_past=False,
+        parameters=["submission_date:DATE:{{macros.ds_add(ds, -1)}}"],
     )
 
     telemetry_derived__newtab_interactions__v1 = bigquery_etl_query(
         task_id="telemetry_derived__newtab_interactions__v1",
-        destination_table="newtab_interactions_v1",
+        destination_table='newtab_interactions_v1${{ macros.ds_format(macros.ds_add(ds, -1), "%Y-%m-%d", "%Y%m%d") }}',
         dataset_id="telemetry_derived",
         project_id="moz-fx-data-shared-prod",
         owner="anicholson@mozilla.com",
         email=["anicholson@mozilla.com", "telemetry-alerts@mozilla.com"],
-        date_partition_parameter="submission_date",
+        date_partition_parameter=None,
         depends_on_past=False,
+        parameters=["submission_date:DATE:{{macros.ds_add(ds, -1)}}"],
     )
 
     telemetry_derived__newtab_visits__v1 = bigquery_etl_query(
         task_id="telemetry_derived__newtab_visits__v1",
-        destination_table="newtab_visits_v1",
+        destination_table='newtab_visits_v1${{ macros.ds_format(macros.ds_add(ds, -1), "%Y-%m-%d", "%Y%m%d") }}',
         dataset_id="telemetry_derived",
         project_id="moz-fx-data-shared-prod",
         owner="anicholson@mozilla.com",
         email=["anicholson@mozilla.com", "telemetry-alerts@mozilla.com"],
-        date_partition_parameter="submission_date",
+        date_partition_parameter=None,
         depends_on_past=False,
+        parameters=["submission_date:DATE:{{macros.ds_add(ds, -1)}}"],
     )
 
     telemetry_derived__newtab_clients_daily__v1.set_upstream(
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/dags/bqetl_search_dashboard.py /tmp/workspace/generated-sql/dags/bqetl_search_dashboard.py
--- /tmp/workspace/main-generated-sql/dags/bqetl_search_dashboard.py	2024-04-19 22:58:53.000000000 +0000
+++ /tmp/workspace/generated-sql/dags/bqetl_search_dashboard.py	2024-04-19 23:10:26.000000000 +0000
@@ -157,7 +157,7 @@
 
     search_derived__search_revenue_levers_daily__v1 = bigquery_etl_query(
         task_id="search_derived__search_revenue_levers_daily__v1",
-        destination_table="search_revenue_levers_daily_v1",
+        destination_table='search_revenue_levers_daily_v1${{ macros.ds_format(macros.ds_add(ds, -1), "%Y-%m-%d", "%Y%m%d") }}',
         dataset_id="search_derived",
         project_id="moz-fx-data-shared-prod",
         owner="xluo@mozilla.com",
@@ -167,8 +167,9 @@
             "telemetry-alerts@mozilla.com",
             "xluo@mozilla.com",
         ],
-        date_partition_parameter="submission_date",
+        date_partition_parameter=None,
         depends_on_past=False,
+        parameters=["submission_date:DATE:{{macros.ds_add(ds, -1)}}"],
     )
 
     search_derived__desktop_search_aggregates_by_userstate__v1.set_upstream(
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/dags/bqetl_sponsored_tiles_clients_daily.py /tmp/workspace/generated-sql/dags/bqetl_sponsored_tiles_clients_daily.py
--- /tmp/workspace/main-generated-sql/dags/bqetl_sponsored_tiles_clients_daily.py	2024-04-19 22:58:53.000000000 +0000
+++ /tmp/workspace/generated-sql/dags/bqetl_sponsored_tiles_clients_daily.py	2024-04-19 23:10:27.000000000 +0000
@@ -100,7 +100,7 @@
 
     sponsored_tiles_clients_daily_v1 = bigquery_etl_query(
         task_id="sponsored_tiles_clients_daily_v1",
-        destination_table="sponsored_tiles_clients_daily_v1",
+        destination_table='sponsored_tiles_clients_daily_v1${{ macros.ds_format(macros.ds_add(ds, -1), "%Y-%m-%d", "%Y%m%d") }}',
         dataset_id="telemetry_derived",
         project_id="moz-fx-data-shared-prod",
         owner="skahmann@mozilla.com",
@@ -109,8 +109,9 @@
             "skahmann@mozilla.com",
             "telemetry-alerts@mozilla.com",
         ],
-        date_partition_parameter="submission_date",
+        date_partition_parameter=None,
         depends_on_past=False,
+        parameters=["submission_date:DATE:{{macros.ds_add(ds, -1)}}"],
     )
 
     sponsored_tiles_clients_daily_v1.set_upstream(
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/dags/bqetl_unified.py /tmp/workspace/generated-sql/dags/bqetl_unified.py
--- /tmp/workspace/main-generated-sql/dags/bqetl_unified.py	2024-04-19 22:58:53.000000000 +0000
+++ /tmp/workspace/generated-sql/dags/bqetl_unified.py	2024-04-19 23:10:27.000000000 +0000
@@ -154,7 +154,7 @@
 
     checks__fail_telemetry_derived__unified_metrics__v1 = bigquery_dq_check(
         task_id="checks__fail_telemetry_derived__unified_metrics__v1",
-        source_table="unified_metrics_v1",
+        source_table='unified_metrics_v1${{ macros.ds_format(macros.ds_add(ds, -1), "%Y-%m-%d", "%Y%m%d") }}',
         dataset_id="telemetry_derived",
         project_id="moz-fx-data-shared-prod",
         is_dq_check_fail=True,
@@ -166,7 +166,7 @@
             "telemetry-alerts@mozilla.com",
         ],
         depends_on_past=False,
-        parameters=["submission_date:DATE:{{ds}}"],
+        parameters=["submission_date:DATE:{{macros.ds_add(ds, -1)}}"],
         retries=0,
     )
 
@@ -213,7 +213,7 @@
 
     checks__warn_telemetry_derived__unified_metrics__v1 = bigquery_dq_check(
         task_id="checks__warn_telemetry_derived__unified_metrics__v1",
-        source_table="unified_metrics_v1",
+        source_table='unified_metrics_v1${{ macros.ds_format(macros.ds_add(ds, -1), "%Y-%m-%d", "%Y%m%d") }}',
         dataset_id="telemetry_derived",
         project_id="moz-fx-data-shared-prod",
         is_dq_check_fail=False,
@@ -225,7 +225,7 @@
             "telemetry-alerts@mozilla.com",
         ],
         depends_on_past=False,
-        parameters=["submission_date:DATE:{{ds}}"],
+        parameters=["submission_date:DATE:{{macros.ds_add(ds, -1)}}"],
         retries=0,
     )
 
@@ -245,7 +245,7 @@
 
     telemetry_derived__rolling_cohorts__v1 = bigquery_etl_query(
         task_id="telemetry_derived__rolling_cohorts__v1",
-        destination_table="rolling_cohorts_v1",
+        destination_table='rolling_cohorts_v1${{ macros.ds_format(macros.ds_add(ds, -1), "%Y-%m-%d", "%Y%m%d") }}',
         dataset_id="telemetry_derived",
         project_id="moz-fx-data-shared-prod",
         owner="anicholson@mozilla.com",
@@ -256,8 +256,9 @@
             "lvargas@mozilla.com",
             "telemetry-alerts@mozilla.com",
         ],
-        date_partition_parameter="cohort_date",
+        date_partition_parameter=None,
         depends_on_past=False,
+        parameters=["cohort_date:DATE:{{macros.ds_add(ds, -1)}}"],
     )
 
     with TaskGroup(
@@ -276,7 +277,7 @@
 
     telemetry_derived__unified_metrics__v1 = bigquery_etl_query(
         task_id="telemetry_derived__unified_metrics__v1",
-        destination_table="unified_metrics_v1",
+        destination_table='unified_metrics_v1${{ macros.ds_format(macros.ds_add(ds, -1), "%Y-%m-%d", "%Y%m%d") }}',
         dataset_id="telemetry_derived",
         project_id="moz-fx-data-shared-prod",
         owner="loines@mozilla.com",
@@ -286,8 +287,9 @@
             "lvargas@mozilla.com",
             "telemetry-alerts@mozilla.com",
         ],
-        date_partition_parameter="submission_date",
+        date_partition_parameter=None,
         depends_on_past=False,
+        parameters=["submission_date:DATE:{{macros.ds_add(ds, -1)}}"],
     )
 
     with TaskGroup(
Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_fennec_aurora/history_sync: schema.yaml
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/clients_last_seen_joined_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/clients_last_seen_joined_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/clients_last_seen_joined_v1/metadata.yaml	2024-04-19 22:56:13.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/clients_last_seen_joined_v1/metadata.yaml	2024-04-19 23:07:42.000000000 +0000
@@ -14,6 +14,7 @@
   dag_name: bqetl_glean_usage
   task_group: burnham
   depends_on_past: true
+  date_partition_offset: -1
 bigquery:
   time_partitioning:
     type: day
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml	2024-04-19 22:55:38.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml	2024-04-19 23:05:12.000000000 +0000
@@ -1,49 +1,49 @@
 fields:
-- mode: NULLABLE
-  name: submission_date
+- name: submission_date
   type: DATE
-- mode: NULLABLE
-  name: source
+  mode: NULLABLE
+- name: source
   type: STRING
-- mode: NULLABLE
-  name: event_type
+  mode: NULLABLE
+- name: event_type
   type: STRING
-- mode: NULLABLE
-  name: form_factor
+  mode: NULLABLE
+- name: form_factor
   type: STRING
-- mode: NULLABLE
-  name: country
+  mode: NULLABLE
+- name: country
   type: STRING
-- mode: NULLABLE
-  name: subdivision1
+  mode: NULLABLE
+- name: subdivision1
   type: STRING
-- mode: NULLABLE
-  name: advertiser
+  mode: NULLABLE
+- name: advertiser
   type: STRING
-- mode: NULLABLE
-  name: release_channel
+  mode: NULLABLE
+- name: release_channel
   type: STRING
-- mode: NULLABLE
-  name: position
+  mode: NULLABLE
+- name: position
   type: INTEGER
-- mode: NULLABLE
-  name: provider
+  mode: NULLABLE
+- name: provider
   type: STRING
-- mode: NULLABLE
-  name: match_type
+  mode: NULLABLE
+- name: match_type
   type: STRING
-- mode: NULLABLE
-  name: normalized_os
+  mode: NULLABLE
+- name: normalized_os
   type: STRING
-- mode: NULLABLE
-  name: suggest_data_sharing_enabled
+  mode: NULLABLE
+- name: suggest_data_sharing_enabled
   type: BOOLEAN
-- mode: NULLABLE
-  name: event_count
+  mode: NULLABLE
+- name: event_count
   type: INTEGER
-- mode: NULLABLE
-  name: user_count
+  mode: NULLABLE
+- name: user_count
   type: INTEGER
-- mode: NULLABLE
-  name: query_type
+  mode: NULLABLE
+- name: query_type
   type: STRING
+  mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml	2024-04-19 22:55:38.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml	2024-04-19 23:05:11.000000000 +0000
@@ -1,40 +1,40 @@
 fields:
-- mode: NULLABLE
-  name: submission_date
+- name: submission_date
   type: DATE
-- mode: NULLABLE
-  name: form_factor
+  mode: NULLABLE
+- name: form_factor
   type: STRING
-- mode: NULLABLE
-  name: country
+  mode: NULLABLE
+- name: country
   type: STRING
-- mode: NULLABLE
-  name: advertiser
+  mode: NULLABLE
+- name: advertiser
   type: STRING
-- mode: NULLABLE
-  name: normalized_os
+  mode: NULLABLE
+- name: normalized_os
   type: STRING
-- mode: NULLABLE
-  name: release_channel
+  mode: NULLABLE
+- name: release_channel
   type: STRING
-- mode: NULLABLE
-  name: position
+  mode: NULLABLE
+- name: position
   type: INTEGER
-- mode: NULLABLE
-  name: provider
+  mode: NULLABLE
+- name: provider
   type: STRING
-- mode: NULLABLE
-  name: match_type
+  mode: NULLABLE
+- name: match_type
   type: STRING
-- mode: NULLABLE
-  name: suggest_data_sharing_enabled
+  mode: NULLABLE
+- name: suggest_data_sharing_enabled
   type: BOOLEAN
-- mode: NULLABLE
-  name: impression_count
+  mode: NULLABLE
+- name: impression_count
   type: INTEGER
-- mode: NULLABLE
-  name: click_count
+  mode: NULLABLE
+- name: click_count
   type: INTEGER
-- mode: NULLABLE
-  name: query_type
+  mode: NULLABLE
+- name: query_type
   type: STRING
+  mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml	2024-04-19 22:55:38.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml	2024-04-19 23:03:46.000000000 +0000
@@ -26,6 +26,9 @@
 - name: adjust_network
   type: STRING
   mode: NULLABLE
+- name: install_source
+  type: STRING
+  mode: NULLABLE
 - name: retained_week_2
   type: BOOLEAN
   mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml	2024-04-19 22:55:38.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml	2024-04-19 23:03:47.000000000 +0000
@@ -48,6 +48,10 @@
   description: 'The type of source of a client installation.
 
     '
+- name: install_source
+  type: STRING
+  mode: NULLABLE
+  description: null
 - name: new_profiles
   type: INTEGER
   mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/clients_last_seen_joined_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/clients_last_seen_joined_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/clients_last_seen_joined_v1/metadata.yaml	2024-04-19 22:56:12.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/clients_last_seen_joined_v1/metadata.yaml	2024-04-19 23:07:45.000000000 +0000
@@ -14,6 +14,7 @@
   dag_name: bqetl_glean_usage
   task_group: fenix
   depends_on_past: true
+  date_partition_offset: -1
 bigquery:
   time_partitioning:
     type: day
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_update_derived/clients_last_seen_joined_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_update_derived/clients_last_seen_joined_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_update_derived/clients_last_seen_joined_v1/metadata.yaml	2024-04-19 22:56:12.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_background_update_derived/clients_last_seen_joined_v1/metadata.yaml	2024-04-19 23:07:48.000000000 +0000
@@ -14,6 +14,7 @@
   dag_name: bqetl_glean_usage
   task_group: firefox_desktop_background_update
   depends_on_past: true
+  date_partition_offset: -1
 bigquery:
   time_partitioning:
     type: day
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/clients_last_seen_joined_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/clients_last_seen_joined_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/clients_last_seen_joined_v1/metadata.yaml	2024-04-19 22:56:12.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived/clients_last_seen_joined_v1/metadata.yaml	2024-04-19 23:07:42.000000000 +0000
@@ -14,6 +14,7 @@
   dag_name: bqetl_glean_usage
   task_group: firefox_desktop
   depends_on_past: true
+  date_partition_offset: -1
 bigquery:
   time_partitioning:
     type: day
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_echo_show_derived/clients_last_seen_joined_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_echo_show_derived/clients_last_seen_joined_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_echo_show_derived/clients_last_seen_joined_v1/metadata.yaml	2024-04-19 22:56:11.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_echo_show_derived/clients_last_seen_joined_v1/metadata.yaml	2024-04-19 23:07:49.000000000 +0000
@@ -14,6 +14,7 @@
   dag_name: bqetl_glean_usage
   task_group: firefox_echo_show
   depends_on_past: true
+  date_partition_offset: -1
 bigquery:
   time_partitioning:
     type: day
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_fire_tv_derived/clients_last_seen_joined_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_fire_tv_derived/clients_last_seen_joined_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_fire_tv_derived/clients_last_seen_joined_v1/metadata.yaml	2024-04-19 22:56:12.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_fire_tv_derived/clients_last_seen_joined_v1/metadata.yaml	2024-04-19 23:07:49.000000000 +0000
@@ -14,6 +14,7 @@
   dag_name: bqetl_glean_usage
   task_group: firefox_fire_tv
   depends_on_past: true
+  date_partition_offset: -1
 bigquery:
   time_partitioning:
     type: day
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/clients_last_seen_joined_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/clients_last_seen_joined_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/clients_last_seen_joined_v1/metadata.yaml	2024-04-19 22:56:12.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/clients_last_seen_joined_v1/metadata.yaml	2024-04-19 23:07:45.000000000 +0000
@@ -14,6 +14,7 @@
   dag_name: bqetl_glean_usage
   task_group: firefox_ios
   depends_on_past: true
+  date_partition_offset: -1
 bigquery:
   time_partitioning:
     type: day
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/funnel_retention_clients_week_2_v1/checks.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/funnel_retention_clients_week_2_v1/checks.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/funnel_retention_clients_week_2_v1/checks.sql	2024-04-19 22:55:39.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/funnel_retention_clients_week_2_v1/checks.sql	2024-04-19 22:56:22.000000000 +0000
@@ -1,9 +1,5 @@
-{#
--- Disabled for now due to known duplication issue in Fenix data, see:
--- https://bugzilla.mozilla.org/show_bug.cgi?id=1887708
--- #warn
--- {{ is_unique(["client_id"]) }}
-#}
+#warn
+{{ is_unique(["client_id"]) }}
 
 #warn
 {{ min_row_count(1, "submission_date = @submission_date") }}
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/funnel_retention_clients_week_4_v1/checks.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/funnel_retention_clients_week_4_v1/checks.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/funnel_retention_clients_week_4_v1/checks.sql	2024-04-19 22:55:39.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/funnel_retention_clients_week_4_v1/checks.sql	2024-04-19 22:56:22.000000000 +0000
@@ -1,9 +1,5 @@
-{#
--- Disabled for now due to known duplication issue in Fenix data, see: 
--- [bug-1887708](https://bugzilla.mozilla.org/show_bug.cgi?id=1887708)
--- #warn
--- {{ is_unique(["client_id"]) }}
-#}
+#warn
+{{ is_unique(["client_id"]) }}
 
 #warn
 {{ min_row_count(1, "submission_date = @submission_date") }}
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_reality_derived/clients_last_seen_joined_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_reality_derived/clients_last_seen_joined_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_reality_derived/clients_last_seen_joined_v1/metadata.yaml	2024-04-19 22:56:12.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_reality_derived/clients_last_seen_joined_v1/metadata.yaml	2024-04-19 23:07:49.000000000 +0000
@@ -14,6 +14,7 @@
   dag_name: bqetl_glean_usage
   task_group: firefox_reality
   depends_on_past: true
+  date_partition_offset: -1
 bigquery:
   time_partitioning:
     type: day
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_reality_pc_derived/clients_last_seen_joined_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_reality_pc_derived/clients_last_seen_joined_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_reality_pc_derived/clients_last_seen_joined_v1/metadata.yaml	2024-04-19 22:56:11.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_reality_pc_derived/clients_last_seen_joined_v1/metadata.yaml	2024-04-19 23:07:49.000000000 +0000
@@ -14,6 +14,7 @@
   dag_name: bqetl_glean_usage
   task_group: firefox_reality_pc
   depends_on_past: true
+  date_partition_offset: -1
 bigquery:
   time_partitioning:
     type: day
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/clients_last_seen_joined_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/clients_last_seen_joined_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/clients_last_seen_joined_v1/metadata.yaml	2024-04-19 22:56:12.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_android_derived/clients_last_seen_joined_v1/metadata.yaml	2024-04-19 23:07:42.000000000 +0000
@@ -14,6 +14,7 @@
   dag_name: bqetl_glean_usage
   task_group: focus_android
   depends_on_past: true
+  date_partition_offset: -1
 bigquery:
   time_partitioning:
     type: day
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived/clients_last_seen_joined_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived/clients_last_seen_joined_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived/clients_last_seen_joined_v1/metadata.yaml	2024-04-19 22:56:12.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/focus_ios_derived/clients_last_seen_joined_v1/metadata.yaml	2024-04-19 23:07:41.000000000 +0000
@@ -14,6 +14,7 @@
   dag_name: bqetl_glean_usage
   task_group: focus_ios
   depends_on_past: true
+  date_partition_offset: -1
 bigquery:
   time_partitioning:
     type: day
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/internet_outages/global_outages_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/internet_outages/global_outages_v1/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/internet_outages/global_outages_v1/schema.yaml	2024-04-19 22:55:38.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/internet_outages/global_outages_v1/schema.yaml	2024-04-19 23:05:13.000000000 +0000
@@ -1,49 +1,49 @@
 fields:
-- mode: NULLABLE
-  name: country
+- name: country
   type: STRING
-- mode: NULLABLE
-  name: city
+  mode: NULLABLE
+- name: city
   type: STRING
-- mode: NULLABLE
-  name: datetime
+  mode: NULLABLE
+- name: datetime
   type: TIMESTAMP
-- mode: NULLABLE
-  name: proportion_undefined
+  mode: NULLABLE
+- name: proportion_undefined
   type: FLOAT
-- mode: NULLABLE
-  name: proportion_timeout
+  mode: NULLABLE
+- name: proportion_timeout
   type: FLOAT
-- mode: NULLABLE
-  name: proportion_abort
+  mode: NULLABLE
+- name: proportion_abort
   type: FLOAT
-- mode: NULLABLE
-  name: proportion_unreachable
+  mode: NULLABLE
+- name: proportion_unreachable
   type: FLOAT
-- mode: NULLABLE
-  name: proportion_terminated
+  mode: NULLABLE
+- name: proportion_terminated
   type: FLOAT
-- mode: NULLABLE
-  name: proportion_channel_open
+  mode: NULLABLE
+- name: proportion_channel_open
   type: FLOAT
-- mode: NULLABLE
-  name: avg_dns_success_time
+  mode: NULLABLE
+- name: avg_dns_success_time
   type: FLOAT
-- mode: NULLABLE
-  name: missing_dns_success
+  mode: NULLABLE
+- name: missing_dns_success
   type: FLOAT
-- mode: NULLABLE
-  name: avg_dns_failure_time
+  mode: NULLABLE
+- name: avg_dns_failure_time
   type: FLOAT
-- mode: NULLABLE
-  name: missing_dns_failure
+  mode: NULLABLE
+- name: missing_dns_failure
   type: FLOAT
-- mode: NULLABLE
-  name: count_dns_failure
+  mode: NULLABLE
+- name: count_dns_failure
   type: FLOAT
-- mode: NULLABLE
-  name: ssl_error_prop
+  mode: NULLABLE
+- name: ssl_error_prop
   type: FLOAT
-- mode: NULLABLE
-  name: avg_tls_handshake_time
+  mode: NULLABLE
+- name: avg_tls_handshake_time
   type: FLOAT
+  mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/klar_android_derived/clients_last_seen_joined_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_android_derived/clients_last_seen_joined_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/klar_android_derived/clients_last_seen_joined_v1/metadata.yaml	2024-04-19 22:56:13.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_android_derived/clients_last_seen_joined_v1/metadata.yaml	2024-04-19 23:07:49.000000000 +0000
@@ -14,6 +14,7 @@
   dag_name: bqetl_glean_usage
   task_group: klar_android
   depends_on_past: true
+  date_partition_offset: -1
 bigquery:
   time_partitioning:
     type: day
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/klar_ios_derived/clients_last_seen_joined_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_ios_derived/clients_last_seen_joined_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/klar_ios_derived/clients_last_seen_joined_v1/metadata.yaml	2024-04-19 22:56:12.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/klar_ios_derived/clients_last_seen_joined_v1/metadata.yaml	2024-04-19 23:07:42.000000000 +0000
@@ -14,6 +14,7 @@
   dag_name: bqetl_glean_usage
   task_group: klar_ios
   depends_on_past: true
+  date_partition_offset: -1
 bigquery:
   time_partitioning:
     type: day
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/lockwise_android_derived/clients_last_seen_joined_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/lockwise_android_derived/clients_last_seen_joined_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/lockwise_android_derived/clients_last_seen_joined_v1/metadata.yaml	2024-04-19 22:56:12.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/lockwise_android_derived/clients_last_seen_joined_v1/metadata.yaml	2024-04-19 23:07:49.000000000 +0000
@@ -14,6 +14,7 @@
   dag_name: bqetl_glean_usage
   task_group: lockwise_android
   depends_on_past: true
+  date_partition_offset: -1
 bigquery:
   time_partitioning:
     type: day
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/lockwise_ios_derived/clients_last_seen_joined_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/lockwise_ios_derived/clients_last_seen_joined_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/lockwise_ios_derived/clients_last_seen_joined_v1/metadata.yaml	2024-04-19 22:56:12.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/lockwise_ios_derived/clients_last_seen_joined_v1/metadata.yaml	2024-04-19 23:07:49.000000000 +0000
@@ -14,6 +14,7 @@
   dag_name: bqetl_glean_usage
   task_group: lockwise_ios
   depends_on_past: true
+  date_partition_offset: -1
 bigquery:
   time_partitioning:
     type: day
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/mach_derived/clients_last_seen_joined_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/mach_derived/clients_last_seen_joined_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/mach_derived/clients_last_seen_joined_v1/metadata.yaml	2024-04-19 22:56:11.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/mach_derived/clients_last_seen_joined_v1/metadata.yaml	2024-04-19 23:07:49.000000000 +0000
@@ -14,6 +14,7 @@
   dag_name: bqetl_glean_usage
   task_group: mach
   depends_on_past: true
+  date_partition_offset: -1
 bigquery:
   time_partitioning:
     type: day
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/mozphab_derived/clients_last_seen_joined_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/mozphab_derived/clients_last_seen_joined_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/mozphab_derived/clients_last_seen_joined_v1/metadata.yaml	2024-04-19 22:56:11.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/mozphab_derived/clients_last_seen_joined_v1/metadata.yaml	2024-04-19 23:07:48.000000000 +0000
@@ -14,6 +14,7 @@
   dag_name: bqetl_glean_usage
   task_group: mozphab
   depends_on_past: true
+  date_partition_offset: -1
 bigquery:
   time_partitioning:
     type: day
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/mozregression_derived/clients_last_seen_joined_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/mozregression_derived/clients_last_seen_joined_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/mozregression_derived/clients_last_seen_joined_v1/metadata.yaml	2024-04-19 22:56:13.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/mozregression_derived/clients_last_seen_joined_v1/metadata.yaml	2024-04-19 23:07:49.000000000 +0000
@@ -14,6 +14,7 @@
   dag_name: bqetl_glean_usage
   task_group: mozregression
   depends_on_past: true
+  date_partition_offset: -1
 bigquery:
   time_partitioning:
     type: day
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_fennec_aurora/history_sync/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_fennec_aurora/history_sync/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_fennec_aurora/history_sync/schema.yaml	2024-04-19 22:56:12.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_fennec_aurora/history_sync/schema.yaml	1970-01-01 00:00:00.000000000 +0000
@@ -1,591 +0,0 @@
-fields:
-- name: additional_properties
-  type: STRING
-  mode: NULLABLE
-  description: A JSON string containing any payload properties not present in the
-    schema
-- name: client_info
-  type: RECORD
-  mode: NULLABLE
-  fields:
-  - name: android_sdk_version
-    type: STRING
-    mode: NULLABLE
-    description: The optional Android specific SDK version of the software running
-      on this hardware device.
-  - name: app_build
-    type: STRING
-    mode: NULLABLE
-    description: The build identifier generated by the CI system (e.g. "1234/A").
-      For language bindings that provide automatic detection for this value, (e.g.
-      Android/Kotlin), in the unlikely event that the build identifier can not be
-      retrieved from the OS, it is set to "inaccessible". For other language bindings,
-      if the value was not provided through configuration, this metric gets set to
-      `Unknown`.
-  - name: app_channel
-    type: STRING
-    mode: NULLABLE
-    description: The channel the application is being distributed on.
-  - name: app_display_version
-    type: STRING
-    mode: NULLABLE
-    description: The user visible version string (e.g. "1.0.3").  In the unlikely
-      event that the display v

⚠️ Only part of the diff is displayed.

Link to full diff

@lucia-vargas-a
Copy link
Contributor Author

Closing based on decision to revert the change that generated duplicates. See Jira task.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants