Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: webrecorder/browsertrix
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: v1.14.0
Choose a base ref
...
head repository: webrecorder/browsertrix
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: v1.14.1
Choose a head ref
  • 5 commits
  • 18 files changed
  • 5 contributors

Commits on Feb 26, 2025

  1. Add "Copy <item type> ID" to a bunch of menus (#2426)

    Addresses feedback from here
    https://discord.com/channels/895426029194207262/910966759165657161/1344367205004873819
    by @tw4l.
    
    Add "Copy <item type> ID" to a bunch of menus, including all list and
    detail pages, as well as all other item/crawl/page lists.
    
    | Screenshots |
    |--------|
    | <img width="323" alt="Screenshot 2025-02-26 at 3 56 48 PM"
    src="https://github.com/user-attachments/assets/32044c47-65f3-4e80-8f39-df5fd2101324"
    /> |
    | <img width="246" alt="Screenshot 2025-02-26 at 4 02 06 PM"
    src="https://github.com/user-attachments/assets/8f2d6272-f450-4923-b5c9-751a2eea9a26"
    /> |
    | <img width="419" alt="Screenshot 2025-02-26 at 4 02 55 PM"
    src="https://github.com/user-attachments/assets/0c005a33-055d-4fb7-a79e-9bedae57b785"
    /> |
    | <img width="1104" alt="Screenshot 2025-02-26 at 1 57 01 PM"
    src="https://github.com/user-attachments/assets/7ee43400-1b30-4c78-89a0-3ddb89ef90ca"
    /> |
    | <img width="292" alt="Screenshot 2025-02-26 at 4 01 10 PM"
    src="https://github.com/user-attachments/assets/929f7870-aa83-4f3c-947a-efad377e0b49"
    /> |
    | <img width="240" alt="Screenshot 2025-02-26 at 4 03 19 PM"
    src="https://github.com/user-attachments/assets/45bff838-f741-45ce-b1a7-a8cfefa9656b"
    /> |
    
    ---------
    
    Co-authored-by: Henry Wilkinson <henry@wilkinson.graphics>
    emma-sg and Shrinks99 authored Feb 26, 2025
    Copy the full SHA
    00e85c3 View commit details

Commits on Feb 27, 2025

  1. ingress: only set ssl-redirect if using tls (#2432)

    otherwise, http path should be accessible. Can be used when TLS
    termination handled outside of ingress.
    ikreymer authored Feb 27, 2025
    Copy the full SHA
    6766843 View commit details
  2. Prevent btrix helper from doing anything to k8s contexts other than d…

    …ocker-desktop (#2431)
    
    The `./btrix` development helper shouldn't be used for anything other
    than local dev, which this commit helps to enforce.
    
    When running any command, if the k8s context is anything other than
    `docker-desktop` the script will now shut down immediately without doing
    anything and print the message: "Attempting to modify context other than
    docker-desktop not supported. Quitting."
    tw4l authored Feb 27, 2025
    Copy the full SHA
    da77b06 View commit details
  3. Add superadmin endpoint to readd scheduled workflow cronjobs (#2430)

    Adds new superadmin-only `POST /orgs/all/crawlconfigs/reAddCronjobs`
    endpoint to update/recreate scheduled workflow cronjobs across all orgs.
    tw4l authored Feb 27, 2025
    Copy the full SHA
    3dc8c82 View commit details
  4. version: bump to 1.14.1

    ikreymer committed Feb 27, 2025
    Copy the full SHA
    376c998 View commit details
26 changes: 26 additions & 0 deletions backend/btrixcloud/crawlconfigs.py
Original file line number Diff line number Diff line change
@@ -36,6 +36,7 @@
CrawlerChannel,
CrawlerChannels,
StartedResponse,
SuccessResponse,
CrawlConfigAddedResponse,
CrawlConfigSearchValues,
CrawlConfigUpdateResponse,
@@ -1036,6 +1037,21 @@ def get_warc_prefix(self, org: Organization, crawlconfig: CrawlConfig) -> str:
prefix = org.slug + "-" + name
return prefix[:80]

async def re_add_all_scheduled_cron_jobs(self):
"""Re-add all scheduled workflow cronjobs"""
match_query = {"schedule": {"$nin": ["", None]}, "inactive": {"$ne": True}}
async for config_dict in self.crawl_configs.find(match_query):
config = CrawlConfig.from_dict(config_dict)
try:
await self.crawl_manager.update_scheduled_job(config)
print(f"Updated cronjob for scheduled workflow {config.id}", flush=True)
# pylint: disable=broad-except
except Exception as err:
print(
f"Error updating cronjob for scheduled workflow {config.id}: {err}",
flush=True,
)


# ============================================================================
# pylint: disable=too-many-locals
@@ -1272,6 +1288,16 @@ async def make_inactive(cid: UUID, org: Organization = Depends(org_crawl_dep)):

return await ops.do_make_inactive(crawlconfig)

@app.post("/orgs/all/crawlconfigs/reAddCronjobs", response_model=SuccessResponse)
async def re_add_all_scheduled_cron_jobs(
user: User = Depends(user_dep),
):
if not user.is_superuser:
raise HTTPException(status_code=403, detail="Not Allowed")

asyncio.create_task(ops.re_add_all_scheduled_cron_jobs())
return {"success": True}

org_ops.router.include_router(router)

return ops
2 changes: 1 addition & 1 deletion backend/btrixcloud/version.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
"""current version"""

__version__ = "1.14.0"
__version__ = "1.14.1"
5 changes: 5 additions & 0 deletions btrix
Original file line number Diff line number Diff line change
@@ -120,6 +120,11 @@ CONTEXT=$(cat ~/.kube/config | grep "current-context:" | sed "s/current-context:
MICROK8S="-microk8s"
WAIT="-wait"

if [ $CONTEXT != "docker-desktop" ]; then
echo "Attempting to modify context other than docker-desktop not supported. Quitting."
exit 1
fi

if [[ $1 = "setup" ]]; then
setupLocalConfig
fi
2 changes: 1 addition & 1 deletion chart/Chart.yaml
Original file line number Diff line number Diff line change
@@ -5,7 +5,7 @@ type: application
icon: https://webrecorder.net/assets/icon.png

# Browsertrix and Chart Version
version: v1.14.0
version: v1.14.1

dependencies:
- name: btrix-admin-logging
8 changes: 5 additions & 3 deletions chart/templates/ingress.yaml
Original file line number Diff line number Diff line change
@@ -10,18 +10,20 @@ metadata:
{{- if .Values.ingress.useOldClassAnnotation }}
kubernetes.io/ingress.class: {{ .Values.ingress_class | default "nginx" }}
{{- end }}
nginx.ingress.kubernetes.io/ssl-redirect: "true"
nginx.ingress.kubernetes.io/proxy-body-size: "0"
nginx.ingress.kubernetes.io/proxy-request-buffering: "off"
# for larger uploads to not timeout
nginx.ingress.kubernetes.io/proxy-read-timeout: "300"
nginx.ingress.kubernetes.io/proxy-buffering: "off"
nginx.ingress.kubernetes.io/upstream-vhost: "{{ .Values.ingress.host }}"
{{- if .Values.ingress.tls }}
cert-manager.io/cluster-issuer: {{ .Values.ingress.custom_cluster_issuer | default "cert-main" }}
{{- end }}
nginx.ingress.kubernetes.io/upstream-vhost: "{{ .Values.ingress.host }}"
nginx.ingress.kubernetes.io/configuration-snippet: |
proxy_set_header X-Forwarded-Proto {{ .Values.ingress.tls | ternary "https" "http" }};
nginx.ingress.kubernetes.io/ssl-redirect: "true"
{{- else }}
nginx.ingress.kubernetes.io/ssl-redirect: "false"
{{- end }}

spec:
{{- if not .Values.ingress.useOldClassAnnotation }}
4 changes: 2 additions & 2 deletions chart/values.yaml
Original file line number Diff line number Diff line change
@@ -103,7 +103,7 @@ replica_deletion_delay_days: 0

# API Image
# =========================================
backend_image: "docker.io/webrecorder/browsertrix-backend:1.14.0"
backend_image: "docker.io/webrecorder/browsertrix-backend:1.14.1"
backend_pull_policy: "Always"

backend_password_secret: "PASSWORD!"
@@ -161,7 +161,7 @@ backend_avg_memory_threshold: 95

# Nginx Image
# =========================================
frontend_image: "docker.io/webrecorder/browsertrix-frontend:1.14.0"
frontend_image: "docker.io/webrecorder/browsertrix-frontend:1.14.1"
frontend_pull_policy: "Always"

frontend_cpu: "10m"
2 changes: 1 addition & 1 deletion frontend/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "browsertrix-frontend",
"version": "1.14.0",
"version": "1.14.1",
"main": "index.ts",
"license": "AGPL-3.0-or-later",
"dependencies": {
5 changes: 4 additions & 1 deletion frontend/src/components/ui/menu-item-link.ts
Original file line number Diff line number Diff line change
@@ -47,7 +47,10 @@ export class MenuItemLink extends TailwindElement {
download=${this.download}
aria-disabled=${this.disabled}
@click=${(e: MouseEvent) => {
if (this.disabled || this.loading) return;
if (this.disabled || this.loading) {
e.preventDefault();
return;
}
if (this.download) {
const dropdown = this.shadowRoot!.host.closest<
42 changes: 29 additions & 13 deletions frontend/src/pages/org/archived-item-detail/archived-item-detail.ts
Original file line number Diff line number Diff line change
@@ -630,6 +630,26 @@ export class ArchivedItemDetail extends BtrixElement {
<sl-divider></sl-divider>
`,
)}
${when(
isSuccessfullyFinished(this.item),
() => html`
<btrix-menu-item-link
href=${`/api/orgs/${this.orgId}/all-crawls/${this.itemId}/download?auth_bearer=${authToken}`}
download
>
<sl-icon name="cloud-download" slot="prefix"></sl-icon>
${msg("Download Item")}
${this.item?.fileSize
? html` <btrix-badge
slot="suffix"
class="font-monostyle text-xs text-neutral-500"
>${this.localize.bytes(this.item.fileSize)}</btrix-badge
>`
: nothing}
</btrix-menu-item-link>
<sl-divider></sl-divider>
`,
)}
${when(
this.itemType === "crawl",
() => html`
@@ -659,19 +679,15 @@ export class ArchivedItemDetail extends BtrixElement {
<sl-icon name="tags" slot="prefix"></sl-icon>
${msg("Copy Tags")}
</sl-menu-item>
${when(
isSuccessfullyFinished(this.item),
() => html`
<sl-divider></sl-divider>
<btrix-menu-item-link
href=${`/api/orgs/${this.orgId}/all-crawls/${this.itemId}/download?auth_bearer=${authToken}`}
download
>
<sl-icon name="cloud-download" slot="prefix"></sl-icon>
${msg("Download Item")}
</btrix-menu-item-link>
`,
)}
<sl-menu-item
@click=${() =>
ClipboardController.copyToClipboard(
this.item?.id ?? this.itemId ?? "",
)}
>
<sl-icon name="copy" slot="prefix"></sl-icon>
${msg("Copy Item ID")}
</sl-menu-item>
${when(
this.isCrawler,
() => html`
27 changes: 27 additions & 0 deletions frontend/src/pages/org/archived-item-detail/ui/qa.ts
Original file line number Diff line number Diff line change
@@ -16,6 +16,7 @@ import queryString from "query-string";
import { BtrixElement } from "@/classes/BtrixElement";
import { type Dialog } from "@/components/ui/dialog";
import type { PageChangeEvent } from "@/components/ui/pagination";
import { ClipboardController } from "@/controllers/clipboard";
import { iconFor as iconForPageReview } from "@/features/qa/page-list/helpers";
import * as pageApproval from "@/features/qa/page-list/helpers/approval";
import type { SelectDetail } from "@/features/qa/qa-run-dropdown";
@@ -392,6 +393,13 @@ export class ArchivedItemDetailQA extends BtrixElement {
</btrix-menu-item-link>
<sl-divider></sl-divider>
`}
<sl-menu-item
@click=${() => ClipboardController.copyToClipboard(run.id)}
>
<sl-icon name="copy" slot="prefix"></sl-icon>
${msg("Copy Run ID")}
</sl-menu-item>
<sl-divider></sl-divider>
<sl-menu-item
@click=${() => {
this.deleting = run.id;
@@ -750,12 +758,16 @@ export class ArchivedItemDetailQA extends BtrixElement {
"[clickable-start] minmax(12rem, auto)",
"minmax(min-content, 12rem)",
"minmax(min-content, 12rem) [clickable-end]",
"min-content",
].join(" ")}"
>
<btrix-table-head>
<btrix-table-header-cell>${msg("Page")}</btrix-table-header-cell>
<btrix-table-header-cell>${msg("Approval")}</btrix-table-header-cell>
<btrix-table-header-cell>${msg("Comments")}</btrix-table-header-cell>
<btrix-table-header-cell class="px-0">
<span class="sr-only">${msg("Row actions")}</span>
</btrix-table-header-cell>
</btrix-table-head>
<btrix-table-body class="rounded border">
${this.pages?.items.map(
@@ -809,6 +821,21 @@ export class ArchivedItemDetailQA extends BtrixElement {
${msg("None")}
</span>`}
</btrix-table-cell>
<btrix-table-cell class="p-0">
<div class="col action">
<btrix-overflow-dropdown>
<sl-menu>
<sl-menu-item
@click=${() =>
ClipboardController.copyToClipboard(page.id)}
>
<sl-icon name="copy" slot="prefix"></sl-icon>
${msg("Copy Page ID")}
</sl-menu-item>
</sl-menu>
</btrix-overflow-dropdown>
</div>
</btrix-table-cell>
</btrix-table-row>
`,
)}
46 changes: 27 additions & 19 deletions frontend/src/pages/org/archived-items.ts
Original file line number Diff line number Diff line change
@@ -598,6 +598,26 @@ export class CrawlsList extends BtrixElement {
<sl-divider></sl-divider>
`,
)}
${when(
isSuccessfullyFinished(item),
() => html`
<btrix-menu-item-link
href=${`/api/orgs/${this.orgId}/all-crawls/${item.id}/download?auth_bearer=${authToken}`}
download
>
<sl-icon name="cloud-download" slot="prefix"></sl-icon>
${msg("Download Item")}
${item.fileSize
? html` <btrix-badge
slot="suffix"
class="font-monostyle text-xs text-neutral-500"
>${this.localize.bytes(item.fileSize)}</btrix-badge
>`
: nothing}
</btrix-menu-item-link>
<sl-divider></sl-divider>
`,
)}
${item.type === "crawl"
? html`
<sl-menu-item
@@ -615,14 +635,9 @@ export class CrawlsList extends BtrixElement {
<sl-icon name="copy" slot="prefix"></sl-icon>
${msg("Copy Workflow ID")}
</sl-menu-item>
<sl-menu-item
@click=${() => ClipboardController.copyToClipboard(item.id)}
>
<sl-icon name="copy" slot="prefix"></sl-icon>
${msg("Copy Archived Item ID")}
</sl-menu-item>
`
: nothing}
<sl-menu-item
@click=${() =>
ClipboardController.copyToClipboard(item.tags.join(", "))}
@@ -631,19 +646,12 @@ export class CrawlsList extends BtrixElement {
<sl-icon name="tags" slot="prefix"></sl-icon>
${msg("Copy Tags")}
</sl-menu-item>
${when(
isSuccessfullyFinished(item),
() => html`
<sl-divider></sl-divider>
<btrix-menu-item-link
href=${`/api/orgs/${this.orgId}/all-crawls/${item.id}/download?auth_bearer=${authToken}`}
download
>
<sl-icon name="cloud-download" slot="prefix"></sl-icon>
${msg("Download Item")}
</btrix-menu-item-link>
`,
)}
<sl-menu-item
@click=${() => ClipboardController.copyToClipboard(item.id)}
>
<sl-icon name="copy" slot="prefix"></sl-icon>
${msg("Copy Item ID")}
</sl-menu-item>
${when(
this.isCrawler && (item.type !== "crawl" || !isActive(item)),
() => html`
10 changes: 9 additions & 1 deletion frontend/src/pages/org/browser-profiles-detail.ts
Original file line number Diff line number Diff line change
@@ -10,6 +10,7 @@ import type { Profile, ProfileWorkflow } from "./types";

import { BtrixElement } from "@/classes/BtrixElement";
import type { Dialog } from "@/components/ui/dialog";
import { ClipboardController } from "@/controllers/clipboard";
import type { BrowserConnectionChange } from "@/features/browser-profiles/profile-browser";
import { pageNav } from "@/layouts/pageHeader";
import { isApiError } from "@/utils/api";
@@ -432,12 +433,19 @@ export class BrowserProfilesDetail extends BtrixElement {
${msg("Duplicate Profile")}
</sl-menu-item>
<sl-divider></sl-divider>
<sl-menu-item
@click=${() => ClipboardController.copyToClipboard(this.profileId)}
>
<sl-icon name="copy" slot="prefix"></sl-icon>
${msg("Copy Profile ID")}
</sl-menu-item>
<sl-divider></sl-divider>
<sl-menu-item
style="--sl-color-neutral-700: var(--danger)"
@click=${() => void this.deleteProfile()}
>
<sl-icon slot="prefix" name="trash3"></sl-icon>
${msg("Delete")}
${msg("Delete Profile")}
</sl-menu-item>
</sl-menu>
</sl-dropdown>
11 changes: 10 additions & 1 deletion frontend/src/pages/org/browser-profiles-list.ts
Original file line number Diff line number Diff line change
@@ -15,6 +15,7 @@ import {
SortDirection,
type SortValues,
} from "@/components/ui/table/table-header-cell";
import { ClipboardController } from "@/controllers/clipboard";
import { pageHeader } from "@/layouts/pageHeader";
import type {
APIPaginatedList,
@@ -325,14 +326,22 @@ export class BrowserProfilesList extends BtrixElement {
<sl-icon slot="prefix" name="files"></sl-icon>
${msg("Duplicate Profile")}
</sl-menu-item>
<sl-divider></sl-divider>
<sl-menu-item
@click=${() => ClipboardController.copyToClipboard(data.id)}
>
<sl-icon name="copy" slot="prefix"></sl-icon>
${msg("Copy Profile ID")}
</sl-menu-item>
<sl-divider></sl-divider>
<sl-menu-item
style="--sl-color-neutral-700: var(--danger)"
@click=${() => {
void this.deleteProfile(data);
}}
>
<sl-icon slot="prefix" name="trash3"></sl-icon>
${msg("Delete")}
${msg("Delete Profile")}
</sl-menu-item>
</sl-menu>
</btrix-overflow-dropdown>
Loading