Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[D1] initial wrangler d1 export #5425

Merged
merged 5 commits into from
Mar 29, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/wicked-balloons-compare.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"wrangler": minor
---

feat: implement `wrangler d1 export`
2 changes: 2 additions & 0 deletions packages/wrangler/src/__tests__/d1/d1.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ describe("d1", () => {
wrangler d1 delete <name> Delete D1 database
wrangler d1 backup Interact with D1 Backups
wrangler d1 execute <database> Executed command or SQL file
wrangler d1 export <name> Export the contents or schema of your database as a .sql file
wrangler d1 time-travel Use Time Travel to restore, fork or copy a database at a specific point-in-time.
wrangler d1 migrations Interact with D1 Migrations

Expand Down Expand Up @@ -60,6 +61,7 @@ describe("d1", () => {
wrangler d1 delete <name> Delete D1 database
wrangler d1 backup Interact with D1 Backups
wrangler d1 execute <database> Executed command or SQL file
wrangler d1 export <name> Export the contents or schema of your database as a .sql file
wrangler d1 time-travel Use Time Travel to restore, fork or copy a database at a specific point-in-time.
wrangler d1 migrations Interact with D1 Migrations

Expand Down
126 changes: 126 additions & 0 deletions packages/wrangler/src/d1/export.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
import fs from "node:fs/promises";
import { fetch } from "undici";
import { printWranglerBanner } from "..";
import { fetchResult } from "../cfetch";
import { readConfig } from "../config";
import { UserError } from "../errors";
import { logger } from "../logger";
import { requireAuth } from "../user";
import { Name } from "./options";
import { getDatabaseByNameOrBinding } from "./utils";
import type { Config } from "../config";
import type {
CommonYargsArgv,
StrictYargsOptionsToInterface,
} from "../yargs-types";
import type { Database } from "./types";

export function Options(yargs: CommonYargsArgv) {
return Name(yargs)

Check warning on line 19 in packages/wrangler/src/d1/export.ts

View check run for this annotation

Codecov / codecov/patch

packages/wrangler/src/d1/export.ts#L18-L19

Added lines #L18 - L19 were not covered by tests
.option("local", {
type: "boolean",
describe: "Export from your local DB you use with wrangler dev",
conflicts: "remote",
})
.option("remote", {
type: "boolean",
describe: "Export from your live D1",
conflicts: "local",
})
.option("no-schema", {
type: "boolean",
describe: "Only output table contents, not the DB schema",
conflicts: "no-data",
})
.option("no-data", {
type: "boolean",
describe:
"Only output table schema, not the contents of the DBs themselves",
conflicts: "no-schema",
})
.option("table", {
type: "string",
describe: "Specify which tables to include in export",
})
.option("output", {
type: "string",
describe: "Which .sql file to output to",
demandOption: true,
});
}

type HandlerOptions = StrictYargsOptionsToInterface<typeof Options>;
export const Handler = async (args: HandlerOptions): Promise<void> => {
const { local, remote, name, output, noSchema, noData, table } = args;
await printWranglerBanner();
const config = readConfig(args.config, args);

Check warning on line 56 in packages/wrangler/src/d1/export.ts

View check run for this annotation

Codecov / codecov/patch

packages/wrangler/src/d1/export.ts#L54-L56

Added lines #L54 - L56 were not covered by tests

if (local)
throw new UserError(

Check warning on line 59 in packages/wrangler/src/d1/export.ts

View check run for this annotation

Codecov / codecov/patch

packages/wrangler/src/d1/export.ts#L59

Added line #L59 was not covered by tests
`Local imports/exports will be coming in a future version of Wrangler.`
);
if (!remote)
throw new UserError(`You must specify either --local or --remote`);

Check warning on line 63 in packages/wrangler/src/d1/export.ts

View check run for this annotation

Codecov / codecov/patch

packages/wrangler/src/d1/export.ts#L63

Added line #L63 was not covered by tests

// Allow multiple --table x --table y flags or none
const tables: string[] = table
? Array.isArray(table)
? table
: [table]
: [];

Check warning on line 70 in packages/wrangler/src/d1/export.ts

View check run for this annotation

Codecov / codecov/patch

packages/wrangler/src/d1/export.ts#L67-L70

Added lines #L67 - L70 were not covered by tests

const result = await exportRemotely(

Check warning on line 72 in packages/wrangler/src/d1/export.ts

View check run for this annotation

Codecov / codecov/patch

packages/wrangler/src/d1/export.ts#L72

Added line #L72 was not covered by tests
config,
name,
output,
tables,
noSchema,
noData
);
return result;

Check warning on line 80 in packages/wrangler/src/d1/export.ts

View check run for this annotation

Codecov / codecov/patch

packages/wrangler/src/d1/export.ts#L80

Added line #L80 was not covered by tests
};

type ExportMetadata = {
signedUrl: string;
};

async function exportRemotely(

Check warning on line 87 in packages/wrangler/src/d1/export.ts

View check run for this annotation

Codecov / codecov/patch

packages/wrangler/src/d1/export.ts#L87

Added line #L87 was not covered by tests
config: Config,
name: string,
output: string,
tables: string[],
noSchema?: boolean,
noData?: boolean
) {
const accountId = await requireAuth(config);
const db: Database = await getDatabaseByNameOrBinding(

Check warning on line 96 in packages/wrangler/src/d1/export.ts

View check run for this annotation

Codecov / codecov/patch

packages/wrangler/src/d1/export.ts#L94-L96

Added lines #L94 - L96 were not covered by tests
config,
accountId,
name
);

logger.log(`🌀 Executing on remote database ${name} (${db.uuid}):`);
logger.log(`🌀 Creating export...`);
const metadata = await fetchResult<ExportMetadata>(

Check warning on line 104 in packages/wrangler/src/d1/export.ts

View check run for this annotation

Codecov / codecov/patch

packages/wrangler/src/d1/export.ts#L102-L104

Added lines #L102 - L104 were not covered by tests
`/accounts/${accountId}/d1/database/${db.uuid}/export`,
{
method: "POST",
headers: {
"Content-Type": "application/json",
},
body: JSON.stringify({
outputFormat: "file",
dumpOptions: {
noSchema,
noData,
tables,
},
}),
}
);

logger.log(`🌀 Downloading SQL to ${output}`);
const contents = await fetch(metadata.signedUrl);

Check warning on line 123 in packages/wrangler/src/d1/export.ts

View check run for this annotation

Codecov / codecov/patch

packages/wrangler/src/d1/export.ts#L122-L123

Added lines #L122 - L123 were not covered by tests
await fs.writeFile(output, contents.body || "");
logger.log(`Done!`);

Check warning on line 125 in packages/wrangler/src/d1/export.ts

View check run for this annotation

Codecov / codecov/patch

packages/wrangler/src/d1/export.ts#L125

Added line #L125 was not covered by tests
}
7 changes: 7 additions & 0 deletions packages/wrangler/src/d1/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ import * as Backups from "./backups";
import * as Create from "./create";
import * as Delete from "./delete";
import * as Execute from "./execute";
import * as Export from "./export";
import * as Info from "./info";
import * as Insights from "./insights";
import * as List from "./list";
Expand Down Expand Up @@ -85,6 +86,12 @@ export function d1(yargs: CommonYargsArgv) {
Execute.Options,
Execute.Handler
)
.command(
"export <name>",
"Export the contents or schema of your database as a .sql file",
Export.Options,
Export.Handler
)
.command(
"time-travel",
"Use Time Travel to restore, fork or copy a database at a specific point-in-time.",
Expand Down