Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Python 3.11 date.fromisoformat() allows extra formats #1076

Conversation

jvtm
Copy link
Contributor

@jvtm jvtm commented Mar 28, 2023

Python 3.11 and later allow additional ISO8601 formats in datetime module ISO8601 parsing. These formats are not RFC3339 section 5.6 compliant.

Especially datetime.date.fromisoformat() now allows strings like:

  • 20230328 (2023-03-28)
  • 2022W527 (2023-01-01)
  • 2023-W01 (2023-01-02)
  • 2023-W13-2 (2023-03-28)

Fix by doing a regular expression check before passing the value to datetime module. This made the original .isascii() check unnecessary.

See:

Tests covering the invalid values to be sent to json-schema-org/JSON-Schema-Test-Suite

Fixes #1056.


📚 Documentation preview 📚: https://python-jsonschema--1076.org.readthedocs.build/en/1076/

@jvtm
Copy link
Contributor Author

jvtm commented Mar 28, 2023

Example diff to test files (similar change needed for most of the other versions)

--- a/json/tests/draft2020-12/optional/format/date.json
+++ b/json/tests/draft2020-12/optional/format/date.json
@@ -220,6 +220,26 @@
                 "description": "invalid non-ASCII '৪' (a Bengali 4)",
                 "data": "1963-06-1৪",
                 "valid": false
+            },
+            {
+                "description": "ISO8601 / non-RFC3339: YYYYMMDD without dashes (2023-03-28)",
+                "data": "20230328",
+                "valid": false
+            },
+            {
+                "description": "ISO8601 / non-RFC3339: week number implicit day of week (2023-01-02)",
+                "data": "2023-W01",
+                "valid": false
+            },
+            {
+                "description": "ISO8601 / non-RFC3339: week number with day of week (2023-03-28)",
+                "data": "2023-W13-2",
+                "valid": false
+            },
+            {
+                "description": "ISO8601 / non-RFC3339: week number + day of week + rollover to next year (2023-01-01)",
+                "data": "2022W527",
+                "valid": false
             }
         ]
     }

@jvtm jvtm marked this pull request as ready for review March 28, 2023 07:14
jvtm added a commit to jvtm/JSON-Schema-Test-Suite that referenced this pull request Mar 28, 2023
Python 3.11 `datetime` module allows additional ISO8601 date, time and
date-time formats that are not RFC3339 5.6 compliant.

This change adds tests for invalid `date` values that are at the moment
allowed by at least Python `jsonschema` library as valid `date` sttrings.

Looks like `date-time` and `time` test cases already cover at least some
values that are valid by ISO8601 but not by RFC3339 section 5.6.

See:
 * https://docs.python.org/3/whatsnew/3.11.html#datetime
 * python/cpython@1303f8c927
 * https://docs.python.org/3.11/library/datetime.html#datetime.date.fromisoformat
 * https://www.rfc-editor.org/rfc/rfc3339#section-5.6

PR sent to `jsonschema` library python-jsonschema/jsonschema#1076.
@jvtm jvtm marked this pull request as draft March 28, 2023 07:54
jvtm added a commit to jvtm/JSON-Schema-Test-Suite that referenced this pull request Mar 28, 2023
Python 3.11 `datetime` module allows additional ISO8601 date, time and
date-time formats that are not RFC3339 5.6 compliant.

This change adds tests for invalid `date` values that are at the moment
allowed by at least Python `jsonschema` library as valid `date` sttrings.

Looks like `date-time` and `time` test cases already cover at least some
values that are valid by ISO8601 but not by RFC3339 section 5.6.

See:
 * https://docs.python.org/3/whatsnew/3.11.html#datetime
 * python/cpython@1303f8c927
 * https://docs.python.org/3.11/library/datetime.html#datetime.date.fromisoformat
 * https://www.rfc-editor.org/rfc/rfc3339#section-5.6

PR sent to `jsonschema` library python-jsonschema/jsonschema#1076.
Python 3.11 and later allow additional ISO8601 formats in `datetime` module
ISO8601 parsing. These formats are not RFC3339 section 5.6 compliant.

Especially `datetime.date.fromisoformat()` now allows strings like:
 * `20230328` (2023-03-28)
 * `2022W527` (2023-01-01)
 * `2023-W01` (2023-01-02)
 * `2023-W13-2` (2023-03-28)

Fix by doing a regular expression check before passing the value to `datetime`
module. This made the original `.isascii()` check unnecessary.

See:
 * https://docs.python.org/3/whatsnew/3.11.html#datetime
 * python/cpython@1303f8c927
 * https://docs.python.org/3.11/library/datetime.html#datetime.date.fromisoformat
 * https://www.rfc-editor.org/rfc/rfc3339#section-5.6

Tests covering the invalid values to be sent to json-schema-org/JSON-Schema-Test-Suite

Fixes python-jsonschema#1056.
@jvtm jvtm force-pushed the python311-date-fromisoformat-allows-extra-formats branch from 956ff9f to 781f8cd Compare March 28, 2023 08:04
@jvtm jvtm marked this pull request as ready for review March 28, 2023 08:10
@Julian
Copy link
Member

Julian commented Mar 28, 2023

Well done, looks great, pushing alongside an update to the (vendored) suite. Thanks again, appreciated!

@Julian Julian merged commit 8b8f355 into python-jsonschema:main Mar 28, 2023
119 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

String date format check allows unexpected formats with Python 3.11 and later
2 participants