Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add new arguments-v3 schema #1641

Merged
merged 6 commits into from
Mar 14, 2025
Merged

Add new arguments-v3 schema #1641

merged 6 commits into from
Mar 14, 2025

Conversation

Viicos
Copy link
Member

@Viicos Viicos commented Feb 20, 2025

Change Summary

Fixes pydantic/pydantic#11240

The new arguments schema allow to validate mappings (both from JSON and Python) differently (see issue), but still validates ArgsKwargs instances in Python mode as it does with the existing arguments schema.

Trying to incorporate tests into the existing 'arguments' schema tests ended up being too complicated as the core schemas and error locs are different between core schemas.

Thus, I went with completely new tests, this time with one test file per parameter kind. Existing misc. 'arguments' schema tests were mostly copied into test_general.py/test_alias.py. Whenever possible, tests were split into multiple ones to avoid over-parametrizing, as together with Sydney we found it unclear when expected can be the successful validation result or the validation error.

Related issue number

Checklist

  • Unit tests for the changes exist
  • Documentation reflects the changes where applicable
  • Pydantic tests pass with this pydantic-core (except for expected changes)
  • My PR is ready to review, please add a comment including the phrase "please review" to assign reviewers

Sorry, something went wrong.

Copy link

codspeed-hq bot commented Feb 20, 2025

CodSpeed Performance Report

Merging #1641 will not alter performance

Comparing new-arguments-schema (bcbf637) with main (53bdfa6)

Summary

✅ 157 untouched benchmarks

@Viicos Viicos force-pushed the new-arguments-schema branch 2 times, most recently from 2aadb68 to 8148d18 Compare February 27, 2025 12:00

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
@Viicos Viicos force-pushed the new-arguments-schema branch from 4c08142 to 1555034 Compare February 27, 2025 17:13
@Viicos Viicos changed the title Add new arguments schema Add new arguments-v3 schema Feb 27, 2025
@Viicos Viicos marked this pull request as ready for review February 27, 2025 18:27
@sydney-runkle
Copy link
Contributor

I think it makes sense to have @samuelcolvin and/or @dmontagu review here, given the relevance to pydantic-ai?

@Viicos Viicos force-pushed the new-arguments-schema branch 2 times, most recently from 95d68a4 to 3997db9 Compare March 11, 2025 14:47
@Viicos Viicos force-pushed the new-arguments-schema branch from 3997db9 to b6a2ca1 Compare March 11, 2025 14:59
Copy link
Member

@samuelcolvin samuelcolvin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

generally looks good, although I admit I haven't had time to read every line.

Comment on lines +50 to +52
name: String,
mode: ParameterMode,
lookup_key_collection: LookupKeyCollection,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

btw, if we are looking to reduce allocations I think we should consider seeing if we can deduplicate name and lookup_key_collection, I imagine frequently they hold the same data, copied.

Not for this PR but maybe an interesting thing to put in an issue / follow-up?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will create a follow-up issue

if let Some(args) = args_kwargs.args() {
let len = args.len();
if len > self.positional_params_count {
if let Some(var_args_param) = self.parameters.iter().find(|p| p.mode == ParameterMode::VarArgs) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we validated the structure of the parameters at build time, we might be able to just know this statically without having to iterate to find it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So you mean having an extra field on the validator struct, as an Option<Parameter>?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking so, yes. Might even want something like

positional_args: Vec<Parameter>,
var_args: Option<Parameter>,
keyword_only_args: Vec<Parameter>,
var_kwargs: Option<Parameter>,

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.. but also fine to leave as follow up.

Comment on lines +517 to +523
for (index, item) in args.iter().enumerate().skip(self.positional_params_count) {
errors.push(ValLineError::new_with_loc(
ErrorTypeDefaults::UnexpectedPositionalArgument,
item,
index,
));
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think extra handling may be relevant here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would align with the extra behavior for kwargs, but the current arguments schema doesn't apply anything. I'll leave it as is for now and mention in it in the follow up issue.

Copy link
Contributor

@davidhewitt davidhewitt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM for a first implementation, with a bunch of possible incremental follow ups later. 👍

@Viicos
Copy link
Member Author

Viicos commented Mar 13, 2025

Created #1669 as a follow-up issue.

@Viicos Viicos merged commit 1defe62 into main Mar 14, 2025
29 checks passed
@Viicos Viicos deleted the new-arguments-schema branch March 14, 2025 15:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

New JSON validation mode for callables
4 participants