Review the structure of the docs to reflect Kubeflow 1.9

[hbelmiro]

Based on #3712 (comment), we need to review the structure of the docs.

The new structured is defined in the comment bellow.

[hbelmiro]

/assign @hbelmiro

[hbelmiro]

@StefanoFioravanzo what do you think about the following structure?

• Overview (Section)
  • Introduction
  • Installation
  • Getting Started
• How tos/User guides (Section)
  • Run a pipeline (there are two sections that seems duplicated)
    • Run a Pipeline
    • Run a basic pipeline using KFP Dashboard and SDK
  • Compile a Pipeline
  • Execute KFP pipelines locally
  • Interact with KFP via the CLI
  • Create components (Section)
    • Lightweight Python Components
    • Compose components into pipelines - Pipeline Basics
    • Containerized Python Components
    • Container Components
    • Special Case: Importer Components
    • Additional Functionality
  • Load and Share Components
  • Create pipelines with control flow
  • Pass small amounts of data between components (Parameters)
  • Create, use, pass, and track ML artifacts (Artifacts)
  • Build a more advanced ML pipeline (from https://www.kubeflow.org/docs/components/pipelines/v2/installation/quickstart/)
  • Use caching
  • Server Configuration
  • Migrate from KFP SDK v1
  • Author tasks with platform-specific functionality
• Reference (Section)
  • Community and Support
  • Version Compatibility
  • Pipelines API Reference
  • Pipelines SDK Reference

[StefanoFioravanzo]

@hbelmiro this looks pretty good! Are you planning on submitting a PR with this refactoring? Few questions:

• Did you take every single guide from the current structure, or is there anything that was left out?
• Current we have v1 and v2 as the two top-level sections. Are you planning to remove them? If yes, where does the v1 content go?
• It could make sense to promote Migrate from KFP SDK v1 to a top-level page. WDYT?

[hbelmiro]

@StefanoFioravanzo

Are you planning on submitting a PR with this refactoring?

Yes, once we get aligned with the structure.
I plan to move the existing pages to the new structure in a first PR and then make other changes, if needed, in follow-up PRs. It will make reviews easier and unlock other changes.

Current we have v1 and v2 as the two top-level sections. Are you planning to remove them? If yes, where does the v1 content go?

Actually, not. I didn't analyze v1. The suggested structure is only for v2.
What do you think of setting v1 as a stretch goal? We can prioritize v2 and when v2 is good we enhance v1.

It could make sense to promote Migrate from KFP SDK v1 to a top-level page. WDYT?

It could, but I don't see the need. Also, it concerns me that we can be opening a precedent and having everything mixed again in the future.

[StefanoFioravanzo]

I plan to move the existing pages to the new structure in a first PR and then make other changes, if needed, in follow-up PRs. It will make reviews easier and unlock other changes.

Totally agree!

What do you think of setting v1 as a stretch goal? We can prioritize v2 and when v2 is good we enhance v1.

I would rather suggest an approach that moves the docs away from this structure

/pipelines/
  /v1
  /v2

to something like this

/pipelines/
  /<unfolded v2 content>
  /legacy-v1/

But we can work on this in a separate PR. I don't think it makes sense to spend precious hours in refactoring the v1 structure. We can keep it as-is and move it under that legacy section. If needed we can promote some guides from v1 to v2.

It could, but I don't see the need. Also, it concerns me that we can be opening a precedent and having everything mixed again in the future.

It's a very good point. I was just wondering if that was a good place. It is a "user guide", although not exactly a "how-to". But it's ok, we can refine in the future if the "user guide" section needs more splitting.

[hbelmiro]

I would rather suggest an approach that moves the docs away from this structure

/pipelines/
  /v1
  /v2

to something like this

/pipelines/
  /<unfolded v2 content>
  /legacy-v1/

But we can work on this in a separate PR. I don't think it makes sense to spend precious hours in refactoring the v1 structure. We can keep it as-is and move it under that legacy section. If needed we can promote some guides from v1 to v2.

Perfect. I think the same way.

It would be nice to have @milosjava's inputs also on the new structure.

Updated structure

• Overview (Section)
  • Introduction
  • Installation
  • Getting Started
• How tos/User guides (Section)
  • Run a pipeline (there are two sections that seems duplicated)
    • Run a Pipeline
    • Run a basic pipeline using KFP Dashboard and SDK
  • Compile a Pipeline
  • Execute KFP pipelines locally
  • Interact with KFP via the CLI
  • Create components (Section)
    • Lightweight Python Components
    • Compose components into pipelines - Pipeline Basics
    • Containerized Python Components
    • Container Components
    • Special Case: Importer Components
    • Additional Functionality
  • Load and Share Components
  • Create pipelines with control flow
  • Pass small amounts of data between components (Parameters)
  • Create, use, pass, and track ML artifacts (Artifacts)
  • Build a more advanced ML pipeline (from https://www.kubeflow.org/docs/components/pipelines/v2/installation/quickstart/)
  • Use caching
  • Server Configuration
  • Migrate from KFP SDK v1
  • Author tasks with platform-specific functionality
• Reference (Section)
  • Community and Support
  • Version Compatibility
  • Pipelines API Reference
  • Pipelines SDK Reference
• Legacy-v1
  • [The entire existing v1 tree]

[hbelmiro]

Also including @diegolovison in the discussion.

[milosjava]

@hbelmiro documentation is quite good. One thing maybe to improve would be to give more info in section control-flow , regarding the part when it says: " not yet supported by the KFP orchestration backend, but may be supported by other orchestration backends". Do we have some docs regarding "orchestration backends"?

[HumairAK]

cc @chensun / @zijianjoy

[HumairAK]

In my experience, Operational and User driven docs are separated into their own sections

Currently the docs do not separate these personas, and we should really start doing that, a user of kubeflow may not care about how to install kubeflow, or how to configure object storage, minio, aws, etc. where as operations will

I suggest making a few adjustments like so:

* Overview
  *  Introduction
  *  Getting Started
* Operator Manual
  * Installation
  * Server Configuration
* User Guides
  * ...same as before (except server configuration moved to operations)

There's a lot that can added to this section moving forward to make managing kubeflow pipeline deployments easier