[stepfunctions] CustomState loses explicit null fields from parsed JSON

[erik-sab]

CustomState https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws-stepfunctions.CustomState.html
looses explicit null fields from parsed JSON.

Reproduction Steps

JSON provided bellow

    "Enable_Termination_Protection": {
      "Type": "Task",
      "Resource": "arn:aws:states:::elasticmapreduce:setClusterTerminationProtection",
      "Parameters": {
        "ClusterId.$": "$.ClusterId",
        "TerminationProtected": true
      },
      "ResultPath": null,
      "Next": "rollup"
    },

generates next output parsed as CustomStep and added to the flow

    "Enable_Termination_Protection": {
      "Next": "rollup",
      "Resource": "arn:aws:states:::elasticmapreduce:setClusterTerminationProtection",
      "Type": "Task",
      "Parameters": {
        "ClusterId.$": "$.ClusterId",
        "TerminationProtected": true
      }

"ResultPath": null is missing in the result

Environment

• CLI Version : 1.47.0
• Framework Version: 1.47.0
• Node.js Version: v12.18.1
• OS : Linux
• Language (Version): Java 11

Other

Worarround for this particular case is to provide any fake result part, which is not null e.g. $.devnull

---

This is 🐛 Bug Report

[shivlaks]

@erik-telesoftas are you able to use the special value DISCARD as we would with other states?
I believe this concern of the null not translating to other languages did come up, I'll take a look at the Java experience and see what we can do to address this bug!

[erik-sab]

Using DISCARD does not work either.

It creates "ResultPath":"DISCARD" and stack creation fails with an error:

6:00:19 AM | UPDATE_FAILED        | AWS::StepFunctions::StateMachine | Foo...ateMachineDC19C52A
Invalid State Machine Definition: 'SCHEMA_VALIDATION_FAILED: Value is not a Reference Path: Reference path didn't start with '$' at /States/Enable_Termination_Protection/ResultPath, SCHEMA_VALIDATION_FAILED:
 Value is not a Reference Path: Reference path didn't start with '$' at /States/Disable_Termination_Protection/ResultPath' 
(Service: AWSStepFunctions; Status Code: 400; Error Code: InvalidDefinition; Request ID: 7ce1216a-0f91-4577-a5ce-f2a47071fe82; Proxy: null)

[cwensel]

I'm seeing the same behavior if I pull in a custom state machine json (rendered by SAM).

The node tree passed to CfnStateMachine.definition does have a property with a null value, it just never makes it back out.

[jungle-gecko]

Same issue on following environment:
CDK Version: 2.28.0 (build ba233f0)
Python Version: v3.7.10
OS : Amazon Linux 2

Looks like this issue in SFN Data Science SDK: aws/aws-step-functions-data-science-sdk-python#45

[chismom-aws]

Also experiencing this issue, specifically when using service integrations:
CDK Version: 2.27.0
Python Version: v3.7.10
OS: MacOS Big Sur

[JeremyPatrol]

Same issue.

CDK Version: 2.31.1
Python Version: v3.10.2
OS: MacOS Monterey

[jonathan5p]

Any updates on this issue?

[ANHPearce]

Bump

Not seeing any progress on this in 3 years and still having the issue. Also not seeing a lot of updates for Step Functions in general in CDK.

You guys still working on this at all or should I be looking for a different deployment tool for my workflows?

[mrgrain]

@ANHPearce Can you post your code that is not working? Both using null and JsonPath.DISCARD works for me.
I think this was fixed in #24593

import * as cdk from 'aws-cdk-lib/core';
import * as sfn from 'aws-cdk-lib/aws-stepfunctions';

const app = new cdk.App();
const stack = new cdk.Stack(app, 'StepFunctionsNullStack', {
  env: { region: 'eu-west-1' }
});

const customA = new sfn.CustomState(stack, "JsonPath.DISCARD", {
  stateJson: {
    Type: "Task",
    Resource:
      "arn:aws:states:::elasticmapreduce:setClusterTerminationProtection",
    Parameters: {
      "ClusterId.$": "$.ClusterId",
      TerminationProtected: true,
    },
    ResultPath: sfn.JsonPath.DISCARD,
    Next: "rollup",
  },
});
const customB = new sfn.CustomState(stack, "null", {
  stateJson: {
    Type: "Task",
    Resource:
      "arn:aws:states:::elasticmapreduce:setClusterTerminationProtection",
    Parameters: {
      "ClusterId.$": "$.ClusterId",
      TerminationProtected: true,
    },
    ResultPath: null,
    Next: "rollup",
  },
});

new sfn.StateMachine(stack, "StateMachine", {
  definition: sfn.Chain.start(customA).next(customB),
});

StateMachine2E01A3A5:
    Type: AWS::StepFunctions::StateMachine
    Properties:
      DefinitionString: '{"StartAt":"JsonPath.DISCARD","States":{"JsonPath.DISCARD":{"Next":"rollup","Type":"Task","Resource":"arn:aws:states:::elasticmapreduce:setClusterTerminationProtection","Parameters":{"ClusterId.$":"$.ClusterId","TerminationProtected":true},"ResultPath":null},"null":{"End":true,"Type":"Task","Resource":"arn:aws:states:::elasticmapreduce:setClusterTerminationProtection","Parameters":{"ClusterId.$":"$.ClusterId","TerminationProtected":true},"ResultPath":null,"Next":"rollup"}}}'
      RoleArn:
        Fn::GetAtt:
          - StateMachineRoleB840431D
          - Arn
    DependsOn:
      - StateMachineRoleB840431D

[mrgrain]

Closing this for now. As far as I can tell this has been fixed in #24593.

If you still encounter this issue, please create a NEW issue, but feel free to reference this one.
A new issue gives us the chance to do triage the issue correctly.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.