Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Registered Model Alias Endpoints and Update RegisteredModel/ModelVersion Payloads #8055

Merged
merged 12 commits into from
Mar 23, 2023

Conversation

arpitjasa-db
Copy link
Collaborator

Related Issues/PRs

#8001

What changes are proposed in this pull request?

Add SetRegisteredModelAlias, DeleteRegisteredModelAlias, and GetModelVersionByAlias methods to file_store, rest_store, and sqlalchemy_store. Also update RegisteredModel payload to have an aliases field that returns a map of aliases to version numbers, and ModelVersion payload to have an aliases field that returns a list of aliases associated with the version.

How is this patch tested?

  • Existing unit/integration tests
  • New unit/integration tests
  • Manual tests (describe details, including test results, below)

Does this PR change the documentation?

  • No. You can skip the rest of this section.
  • Yes. Make sure the changed pages / sections render correctly in the documentation preview.

Release Notes

Is this a user-facing change?

  • No. You can skip the rest of this section.
  • Yes. Give a description of this change to be included in the release notes for MLflow users.

Add SetRegisteredModelAlias, DeleteRegisteredModelAlias, and GetModelVersionByAlias methods to file_store, rest_store, and sqlalchemy_store. Also update RegisteredModel payload to have an aliases field that returns a map of aliases to version numbers, and ModelVersion payload to have an aliases field that returns a list of aliases associated with the version.

What component(s), interfaces, languages, and integrations does this PR affect?

Components

  • area/artifacts: Artifact stores and artifact logging
  • area/build: Build and test infrastructure for MLflow
  • area/docs: MLflow documentation pages
  • area/examples: Example code
  • area/model-registry: Model Registry service, APIs, and the fluent client calls for Model Registry
  • area/models: MLmodel format, model serialization/deserialization, flavors
  • area/recipes: Recipes, Recipe APIs, Recipe configs, Recipe Templates
  • area/projects: MLproject format, project running backends
  • area/scoring: MLflow Model server, model deployment tools, Spark UDFs
  • area/server-infra: MLflow Tracking server backend
  • area/tracking: Tracking Service, tracking client APIs, autologging

Interface

  • area/uiux: Front-end, user experience, plotting, JavaScript, JavaScript dev server
  • area/docker: Docker use across MLflow's components, such as MLflow Projects and MLflow Models
  • area/sqlalchemy: Use of SQLAlchemy in the Tracking Service or Model Registry
  • area/windows: Windows support

Language

  • language/r: R APIs and clients
  • language/java: Java APIs and clients
  • language/new: Proposals for new client languages

Integrations

  • integrations/azure: Azure and Azure ML integrations
  • integrations/sagemaker: SageMaker integrations
  • integrations/databricks: Databricks integrations

How should the PR be classified in the release notes? Choose one:

  • rn/breaking-change - The PR will be mentioned in the "Breaking Changes" section
  • rn/none - No description will be included. The PR will be mentioned only by the PR number in the "Small Bugfixes and Documentation Updates" section
  • rn/feature - A new user-facing feature worth mentioning in the release notes
  • rn/bug-fix - A user-facing bug fix worth mentioning in the release notes
  • rn/documentation - A user-facing documentation change worth mentioning in the release notes

@mlflow-automation
Copy link
Collaborator

mlflow-automation commented Mar 17, 2023

Documentation preview for 091d1e4 will be available here when this CircleCI job completes successfully.

More info

@github-actions github-actions bot added area/model-registry Model registry, model registry APIs, and the fluent client calls for model registry area/sqlalchemy Use of SQL alchemy in tracking service or model registry rn/feature Mention under Features in Changelogs. labels Mar 17, 2023
Copy link
Collaborator Author

@arpitjasa-db arpitjasa-db left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some open questions on the direction of this PR

@@ -1,12 +1,14 @@
from mlflow.entities.model_registry.registered_model import RegisteredModel
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should I add docs to this PR or in a follow-up PR? It's already rather large and don't want to make it too big, but will follow best practices.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd leave that for a follow up ;)

@BenWilson2
Copy link
Member

Looks like you'll have to override these methods in mlflow/store/_unity_catalog/registry/rest_store.py (even with a placeholder of raising a NotImplementedError for the time being)

Copy link
Member

@harupy harupy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some comments, LGTM once they are resolved!

@BenWilson2
Copy link
Member

For this next block of failing tests, I'd try:

  1. ON DELETE CASCADE (as you mentioned to me - it's a good thing to try and it SHOULD fix it)
  2. If that isn't working, the manual approach will be to handle reverse dependency operations within the DAG of foreign keys (delete the tail ref, then work back to get to the upstream definition)

…Version Payloads

Signed-off-by: Arpit Jasapara <arpit.jasapara@databricks.com>
Signed-off-by: Arpit Jasapara <arpit.jasapara@databricks.com>
Signed-off-by: Arpit Jasapara <arpit.jasapara@databricks.com>
Signed-off-by: Arpit Jasapara <arpit.jasapara@databricks.com>
Signed-off-by: Arpit Jasapara <arpit.jasapara@databricks.com>
Signed-off-by: Arpit Jasapara <arpit.jasapara@databricks.com>
Signed-off-by: Arpit Jasapara <arpit.jasapara@databricks.com>
Signed-off-by: Arpit Jasapara <arpit.jasapara@databricks.com>
Signed-off-by: Arpit Jasapara <arpit.jasapara@databricks.com>
Copy link
Member

@BenWilson2 BenWilson2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Looks like there are just 2 test-suite related failures. Great work!

Signed-off-by: Arpit Jasapara <arpit.jasapara@databricks.com>
Signed-off-by: Arpit Jasapara <arpit.jasapara@databricks.com>
Signed-off-by: Arpit Jasapara <arpit.jasapara@databricks.com>
Copy link
Collaborator

@kriscon-db kriscon-db left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Member

@harupy harupy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

re-LGTM

@arpitjasa-db arpitjasa-db merged commit e4ac7cc into mlflow:master Mar 23, 2023
@arpitjasa-db arpitjasa-db deleted the aliases branch March 23, 2023 01:52
BenWilson2 pushed a commit to BenWilson2/mlflow that referenced this pull request Mar 23, 2023
…Version Payloads (mlflow#8055)

* Add Registered Model Alias Endpoints and Update RegisteredModel/ModelVersion Payloads

Signed-off-by: Arpit Jasapara <arpit.jasapara@databricks.com>

* Applied comments

Signed-off-by: Arpit Jasapara <arpit.jasapara@databricks.com>

* Added to unity catalog

Signed-off-by: Arpit Jasapara <arpit.jasapara@databricks.com>

* Fix tests

Signed-off-by: Arpit Jasapara <arpit.jasapara@databricks.com>

* Fixed tests

Signed-off-by: Arpit Jasapara <arpit.jasapara@databricks.com>

* switch order to make the sql overlords happy

Signed-off-by: Arpit Jasapara <arpit.jasapara@databricks.com>

* Added delete cascade

Signed-off-by: Arpit Jasapara <arpit.jasapara@databricks.com>

* Update latest_schema

Signed-off-by: Arpit Jasapara <arpit.jasapara@databricks.com>

* fixed tests

Signed-off-by: Arpit Jasapara <arpit.jasapara@databricks.com>

* Fixed tests

Signed-off-by: Arpit Jasapara <arpit.jasapara@databricks.com>

* Fixed tests

Signed-off-by: Arpit Jasapara <arpit.jasapara@databricks.com>

* Add validation tests

Signed-off-by: Arpit Jasapara <arpit.jasapara@databricks.com>

---------

Signed-off-by: Arpit Jasapara <arpit.jasapara@databricks.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/model-registry Model registry, model registry APIs, and the fluent client calls for model registry area/sqlalchemy Use of SQL alchemy in tracking service or model registry rn/feature Mention under Features in Changelogs.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants