feat: add strategies for command discovery

[mdonnalley]

This PR introduces the idea of "command discovery strategies". Three strategies are added: pattern, explicit, and single - explicit is the only net-new strategy being added here.

Command Discovery Strategies

Support three strategies for command discovery:

1. pattern - this is the current behavior that finds commands based on glob patterns
2. explicit - find commands that are exported from a specified file. This is to support developers who wish to bundle their CLI into a single file.
3. single - CLI contains a single command executed by top-level bin

pattern Strategy

For pattern, plugins could continue to do this:

{
  "oclif": {
    "commands": "./dist/commands",
  }
}

which will tell oclif to look for commands in that directory (this is skipped if an oclif.manifest.json is present)

Alternatively, plugins could set this configuration:

{
  "oclif": {
    "commands": {
      "strategy": "pattern",
      "target": "./dist/commands"
    }
  }
}

And they could even add globPatterns to override the glob patterns that oclif uses when searching for command files:

{
  "oclif": {
    "commands": {
      "strategy": "pattern",
      "target": "./dist/commands",
      "globPatterns": [
         "**/*.+(js|cjs|mjs|ts|tsx|mts|cts)",
        "!**/*.+(d.*|test.*|spec.*|helpers.*)?(x)"
      ]
    }
  }
}

explicit Strategy

For explicit, plugins would add a new file (e.g. src/commands.ts) and then add this configuration to the package.json

{
  "oclif": {
    "commands": {
      "strategy": "explicit",
      "target": "./dist/index.js",
      "identifier": "COMMANDS",
    }
  }
}

src/index.ts would then need to have an export with the same name as the identifier (if not set, it defaults to default) that's an object of command names to command classes, e.g.

import Hello from './commands/hello'
import HelloWorld from './commands/hello/world'

export const COMMANDS = {
  hello: Hello,
  'hello:world': HelloWorld,
  howdy: Hello, // alias the `hello` command to `howdy`
}

The explicit strategy is useful to those who can't rely on file paths because they've bundled their code (oclif/oclif#653) but it can also be used if you simply prefer to be more explicit about your commands instead of relying on oclif "magically" finding commands from the file system.

It can also be leveraged to create or modify commands at runtime (e.g. add flags to a command based on an API spec - see oclif + dynamic commands section below).

Unfortunately, there is no notable performance improvement for development since most of the time is spent auto-transpiling typescript with ts-node

single Strategy

This strategy is for single command CLIs, i.e. CLIs whose bin invokes a single command.

The current way to achieve this to set this in the package.json

{
  "oclif": {
    "default": ".",
    "commands": "./dist",
  }
}

The default tells oclif to use . as the command id when no command is found and commands tells oclif where to find the command file. In the example, ./dist will resolve to to ./dist/index.js

The default property has always been a less-than-ideal workaround and will be deprecated in favor of these settings:

{
  "oclif": {
    "commands": {
      "strategy": "single",
      "target": "./dist"
    }
  }
}

Note about oclif.manifest.json

For all strategies, the oclif.manifest.json will be used to load the commands instead of the default behavior of the strategy.

oclif + Bundling

Forewarning: we do not plan to support bundling given the endless number of tools + configurations that could be used. But if you choose to use a bundler, like esbuild there are a couple hard requirements - you must have a package.json in your root directory and a bin/run or bin/run.js bin script. This means that you will not be able to successfully bundle your entire CLI (src code, package.json, node_modules, etc) into a single file.

If you still want to bundle your CLI or plugin into a single file you will need to do the following (see example repo):

1. Use the explicit strategy (see description above). Be sure to set the target to your bundled file and the identifier to the name of the object that contains your commands
2. Update your hooks to include an identifier and target.

Traditionally, hooks have been defined like this:

"oclif": {
    "hooks": {
      "init": "./dist/hooks/init.js"
    }
}

But in order for them to work once they been bundled into a single file, you'll need to instead define them like this:

"oclif": {
    "hooks": {
      "init": {
        "target": "./dist/index.js",
        "identifier": "INIT_HOOK"
      }
    }
}

That configuration is essentially telling oclif to look for an INIT_HOOK export inside of ./dist/index.js

oclif + Dynamic Commands

You can use the explicit strategy if you want to manipulate or create commands at runtime. Please note that such usage of the explicit strategy cannot be used with an oclif.manifest.json.

Example:

// src/index.ts
import {Command, Flags} from '@oclif/core'

import Hello from './commands/hello'
import HelloWorld from './commands/hello/world'

const dynamicCommands: Record<string, Command.Class> = {}
if (process.env.DYNAMIC_COMMAND) {
  dynamicCommands[process.env.DYNAMIC_COMMAND] = class extends Command {
    static flags = {
      flag1: Flags.boolean({description: 'flag1 description'}),
    }

    async run(): Promise<void> {
      const {flags} = await this.parse(this.constructor as Command.Class)
      this.log('hello from', this.id, flags)
    }
  }
}

export const COMMANDS = {
  hello: Hello,
  'hello:world': HelloWorld,
  ...dynamicCommands,
}

❯ DYNAMIC_COMMAND=foo:bar:baz bin/run.js foo bar baz --flag1
hello from foo:bar:baz { flag1: true }

QA

pattern strategy (defined as string)

• oclif generate my-cli
• rm -rf dist
• yarn link @oclif/core
• bin/dev.js hello world

pattern strategy (defined as object)

• oclif generate my-cli
• rm -rf dist
• yarn link @oclif/core
• change oclif.commands in pjson to:

{
  "strategy": "pattern",
  "target": "./dist/commands"
}

• bin/dev.js hello world
• add world.helpers.ts to src/commands
• update oclif.commands in pjson to:

{
  "strategy": "pattern",
  "target": "./dist/commands",
   "globPatterns": [
      "**/*.+(js|cjs|mjs|ts|tsx|mts|cts)",
      "!**/*.+(d.*|test.*|spec.*|helpers.*)?(x)"
   ]
}

• bin/dev.js hello world

explicit strategy

• oclif generate my-cli
• rm -rf dist
• yarn link @oclif/core
• change oclif.commands in pjson to:

{
  "strategy": "explicit",
  "target": "./dist/commands.js",
   "identifier": "COMMANDS",
}

• create src/commands.js and put this in it:

import Hello from './commands/hello'
import HelloWorld from './commands/hello/world'

export const COMMANDS = {
  hello: Hello,
  'hello:world': HelloWorld,
  howdy: Hello, // alias the `hello` command to `howdy`
}

• bin/dev.js hello world
• bin/dev.js howdy
• Remove identifier from config and change export const COMMANDS to export default const
• bin/dev.js hello world
• bin/dev.js howdy

explicit strategy in bundled plugin

• clone https://github.com/oclif/plugin-test-esbuild
• yarn
• bin/dev.js esbuild

This plugin has an init hook so you should see output from that as well as output from the command

Single command cli using default

• oclif generate my-cli
• rm -rf dist
• yarn link @oclif/core
• export a command from src/index.ts
• update oclif section of pjson to:

{
  "commands": "./dist",
  "default": "."
}

• bin/dev.js

single strategy

• oclif generate my-cli
• rm -rf dist
• yarn link @oclif/core
• export a command from src/index.ts
• update oclif.commands in pjson to:

{
  "strategy": "single",
  "target": "./dist/index.js",
}

• bin/dev.js

sf integration

For this you will need to:

• create QA release of sf that uses @oclif/core@dev and oclif@dev
• sf plugins install @oclif/plugin-test-esbuild
• sf esbuild

Work Items and Issues

Closes #943
Fixes oclif/oclif#653
Fixes oclif/oclif#1053
Fixes #965
Fixes #979
Fixes #995
@W-15005785@

[mshanemc]

why are these separate check/throws for the same condition/error?

[mdonnalley]

Just a vestige of a previous implementation - I'll clean it up

[mshanemc]

to simplify types and undefined checks further down, what if it always returned a CommandDiscovery (and the return from undefined sets it to the default behavior?

[mdonnalley]

There are instances where there's not any command discovery, i.e. plugins that only have hooks but no commands (e.g. plugin-not-found)

Setting to the default would probably be fine, since it just wouldn't find any commands but then you're incurring the cost of searching for commands when we already know that none will be found

[mshanemc]

I know you tested my "2 keys pointing at one file" manually, but what about adding it to the fixture so we can't break that behavior once people start depending on it?

[mshanemc]

approve, one type-only suggestion

[mshanemc]

QA:

1. published a QA release
2. used yarn resolutions to make everything in plugin-marketplace to go to this oclif/core (confirmed with yarn why)
3. linked that into sf via plugins link . [intentionally putting a plugin with new strategies in a CLI based on an oclif that doesn't know about them]

---

✅ as is (no pjson.oclif changes): works fine

"commands": {
      "strategy": "pattern",
      "target": "./lib/commands"
    },

🟡 command executes, but it also displayed this before.

(node:74010) TypeError Plugin: @salesforce/cli: orig.startsWith is not a function
module: @oclif/core@3.19.1
plugin: @salesforce/cli
root: /Users/shane.mclaughlin/.local/share/sf/client/2.29.1-374fd9f

I feel like this is likely an artifact of plugin linking doing tsnode things?

---

ok, let's publish the plugin via npm and manually install it into sf instead of linking.

"as is" @salesforce/plugin-marketplace@1.0.23-qa.0
✅ sf plugins discover

---

@salesforce/plugin-marketplace@1.0.23-qa.1 includes

"commands": {
      "strategy": "pattern",
      "target": "./lib/commands"
    },

❌
TypeError: orig.startsWith is not a function

`@salesforce/plugin-marketplace@1.0.23-qa.2` uses explicit https://github.com/salesforcecli/plugin-marketplace/commit/a178568829f21301b354bb82fd0372e806dbce63

❌ 
TypeError: orig.startsWith is not a function

I'm not goign to able to QA these using sf, I guess.  Unless I bump its oclif?  Maybe via yarn link?

---

`@salesforce/plugin-marketplace@1.0.23-qa.2`  with `oclif/core` yarn-linked into sf
✅ found the explicit stuff
✅ got warning about plugin version not matching `sf`'s version

---
go gack to `@salesforce/plugin-marketplace@1.0.23-qa.1`
✅ works fine
✅ got warning about plugin version not matching `sf`'s version

---
not tested:

- target patterns 
- single command cli

[mshanemc]

QA cont

1. move the command to commands/plugin/discover/index.ts
2. move all the /shared stuff into discover
   strategy is using pattern/globPatterns

"commands": {
      "strategy": "pattern",
      "target": "./lib/commands",
      "globPatterns": [
        "**/index.+(js|cjs|mjs|ts|tsx|mts|cts)"
      ]
    },

❌ when building with @oclif/plugin-command-snapshot the snapshot plugin is looking in the folders.

Error: No return type found for file /Users/shane.mclaughlin/eng/salesforcecli/plugin-marketplace/src/commands/plugins/discover/discoverQuery.ts

I think that traces to https://github.com/oclif/plugin-command-snapshot/blob/19a332cfc2773632e469378acc62dab816805cf7/src/commands/schema/generate.ts#L104 which isn't going to be aware of these new globs.

---

published as @salesforce/plugin-marketplace@1.0.23-qa.3

when installed into a CLI that has this oclif yarn-linked in
✅ works as expected.

---

plugins:link into an sf that includes this oclif
✅ works fine
✅ sf discover plugins (tax-free)
✅ sf discover (it found the plugin and let me run it from the list of commands)
✅ sf plugins sicdover recommends the correct command

⚠️ not tested:

[mshanemc]

Review afterthought: have you considered how bundling would work with hooks? As is, I think hooks are defined in pjson with filepaths. ex:

"hooks": {
      "sf-doctor-@salesforce/plugin-deploy-retrieve": "./lib/hooks/diagnostics"
    },

Wouldn't you need the hook to point to something inside the bundled file instead of an entire file?

[mshanemc]

QA cont

bad target like

"hooks": {
      "init": [
        {
          "target": "./dist/bad.js",
          "identifier": "hook2"
        }
      ]
    }

✅ /❓ runs without error. DEBUG=* is needed to shows the MODULE_NOT_FOUND stuff.
✅ hook also doesn't run or throw if the target is supplied but not the identifier.

---

export const COMMANDS = {
  'bad:cmd': 'foo',

using bin/run.js bad:cmd results in an uglier node error.

➜  plugin-test-esbuild git:(main) ✗ ./bin/run.js bad cmd
(node:77116) Error Plugin: @oclif/plugin-test-esbuild: command bad:cmd not found
module: @oclif/core@3.18.1
task: cacheCommand
plugin: @oclif/plugin-test-esbuild
root: /Users/shane.mclaughlin/eng/oclif/plugin-test-esbuild
See more details with DEBUG=*
(Use `node --trace-warnings ...` to show where the warning was created)
I'm the other hook
Greetings! from plugin-test-esbuild init hook
 ›   Error: command bad:cmd not found

other oclif and sf plugins provide a nicer error

➜  plugin-plugins git:(main) ✗ ./bin/run.js bad:cmd
 ›   Error: command bad:cmd not found

[mshanemc]

export const COMMANDS = {
  esbuild: ESBuild,
  hello: Hello,
  'hello:alias': HelloWorld,
  'hello:world': HelloWorld,
}

with "flexibleTaxonomy": true,

✅ honors tax-free

[mshanemc]

if you filter first you don't have to deal with undefineds. You can also modify the method to always return an array since the 2 consumers are doing ?? [] after it anyway.

Suggested change

	private async getCommandIdsFromTarget(): Promise<string[] | undefined> {
	const commandsFromExport = await this.loadCommandsFromTarget()
	if (commandsFromExport) {
	return (
	Object.entries(commandsFromExport)
	.map(([id, cmd]) => {
	if (!ensureCommandClass(cmd)) return
	return id
	})
	// eslint-disable-next-line unicorn/prefer-native-coercion-functions
	.filter((f): f is string => Boolean(f))
	)
	}
	}
	private async getCommandIdsFromTarget(): Promise<string[]> {
	return Object.entries((await this.loadCommandsFromTarget()) ?? [])
	.filter(([, cmd]) => ensureCommandClass(cmd))
	.map(([id]) => id)
	}

[mshanemc]

QA:

plugins-plugins 4.2.6-0.dev into an sf that has yarn-linked this oclif/core
✅ installs the signups plugin, runs help and org shape list

sf plugins link . in https://github.com/oclif/plugin-test-esbuild

❌  plugin-test-esbuild git:(main) sf plugins link .
 ›   Warning: Plugin @oclif/plugin-plugins (4.2.6-dev.0) differs from the version specified by sf (4.2.5)
Linking plugin .... !
    TypeError: orig.startsWith is not a function

✅ successfully installed plugin-test-esbuild. hellos work, including hooks from dependency @oclif/plugin-test-esm-1

[mshanemc]

QA cont [using only plugin-test-esbuild and its bin/run.js]
✅ install @oclif/plugin-commands
✅ commands shows commands including from other plugins (esm1)
✅ can see help for esm1
✅ install more plugins
@oclif/plugin-not-found 3.0.13
@oclif/plugin-search 1.0.18
@oclif/plugin-which 3.1.2

❌ ./bin/run.js search , then select any command
? Search for a command hello world
TypeError: orig.startsWith is not a function

✅ which works (as expected, always says that it's the root plugin)

✅ ./bin/run.js bad:cmd => plugin-not-found is working correctly, and answer the prompt y works!

›   Warning: bad cmd is not a bundle command.
Did you mean esbuild? [y/n]: y
hello I am a bundled (esbuild) plugin from /Users/shane.mclaughlin/eng/oclif/plugin-test-esbuild! 

[mshanemc]

QA cont

✅ linking an oclif plugin works fine

➜ plugin-test-esbuild git:(main) ./bin/run.js version --verbose
› Warning: @oclif/plugin-version is a linked ESM module and cannot be auto-transpiled. Existing compiled source will be used instead.
Greetings! from plugin-test-esbuild init hook
Greetings! from plugin-test-esm-1 init hook
CLI Version:
@oclif/plugin-test-esbuild/0.4.3

Architecture:
darwin-arm64

Node Version:
node-v20.9.0

Plugin Version:
@oclif/plugin-commands 3.1.7 (user)
@oclif/plugin-not-found 3.0.13 (user)
@oclif/plugin-search 1.0.18 (user)
@oclif/plugin-test-esbuild 0.4.3 (core)
@oclif/plugin-version 2.0.13 (link) /Users/shane.mclaughlin/eng/oclif/plugin-version
@oclif/plugin-which 3.1.2 (user)

OS and Version:
Darwin 23.3.0

Shell:
zsh

Root Path:
/Users/shane.mclaughlin/eng/oclif/plugin-test-esbuild

[mshanemc]

QA: install https://github.com/oclif/plugin-autocomplete from github

✅ autocomplete command runs. Not tested: using the generated autocomplete (since using bin/run for testing)