aws-opensearchservice: cannot disable logging

[SigniantJoe]

Describe the bug

Enabling logging on OpenSearch clusters will populate LogPublishingOptions in the output template, but disabling logging leaves LogPublishingOptions empty so the cluster will retain the previous logging config. I don't see any documentation saying that enabling logging is a one-way trip, so I think this is a bug.

Specifically, I'm talking about these logging props:

new es.Domain(scope, id, {
 ...
 logging: {
    slowSearchLogEnabled: false,
    appLogEnabled: false,
    slowIndexLogEnabled: false,
  },
})

Expected Behavior

    "LogPublishingOptions": {
        "ES_APPLICATION_LOGS": {
            "Enabled": false
        },
        "SEARCH_SLOW_LOGS": {
            "Enabled": false
        },
        "INDEX_SLOW_LOGS": {
            "Enabled": false
        }
    },

Current Behavior

    "LogPublishingOptions": {},

Reproduction Steps

const { Stack, Duration } = require('aws-cdk-lib');
const es = require('aws-cdk-lib/aws-opensearchservice');

class PocStack extends Stack {
  constructor(scope, id, props) {
    super(scope, id, props);

    const cluster = new es.Domain(this, 'OpenSearchCluster', {
      domainName: 'test-opensearch-cluster',
      version: es.EngineVersion.OPENSEARCH_2_11,
      capacity: {
        dataNodes: 1,
        dataNodeInstanceType: "t4g.medium.search",
      },
      ebs: {
        volumeSize: 10,
      },
      logging: {
        appLogEnabled: true,
      }
    });
  }
}

module.exports = { PocStack }

Possible Solution

If you don't need the current behaviour, then have it perform the expected behaviour from above.

If there is a good reason it works that way or you want backwards-compatibility, then maybe have logging props like appLogEnabled default to some non-boolean value (e.g. -1). If left default (not specified by user), then LogPublishingOptions can be empty. If the user explicitly specifies either true or false, then populate the LogPublishingOptions to enable or disable logging as appropriate.

Additional Information/Context

An environment hit the limit on cloudwatch resource policies, so I'm working on a change related to the new (much appreciated) suppressLogsResourcePolicy prop. The way things are currently, I'll have to manually (shudder) disable logging on a cluster to clear up policy space before I can deploy the new multi-cluster logging resource policy.

CDK CLI Version

2.128.0

Framework Version

No response

Node.js Version

v18.18.2

OS

OSX

Language

TypeScript

Language Version

No response

Other information

No response

[pahud]

I am not sure if this is a bug from cloudformation but we should note this in the CDK doc. Thank you for the report.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

[SigniantJoe]

@pahud

I think perhaps there was a miscommunication here, as I feel the PR doesn't accurately describe the situation and actually increases confusion. The PR creates the assumption that it is possible to disable OpenSearch logging if you set the parameters explicitly to false, but the CDK in its current form does not behave that way.

Take a look at

aws-cdk/packages/aws-cdk-lib/aws-opensearchservice/lib/domain.ts

Line 1771 in 3906f09

if (this.appLogGroup) {

for example. If true, it populates logPublishingOptions. If false, it does not populate the config.

Cloudformation needs THOSE logPublishingOptions to contain "enabled: false" for it to have any effect

[pahud]

@SigniantJoe

Do you mean if I specify this way

new opensearch.Domain(this, 'Domain', {
	version: opensearch.EngineVersion.OPENSEARCH_2_11,
	logging: {
		slowSearchLogEnabled: false,
		appLogEnabled: false,
		slowIndexLogEnabled: false,
	},
})

it should synthesize to

"LogPublishingOptions": {
        "ES_APPLICATION_LOGS": {
            "Enabled": false
        },
        "SEARCH_SLOW_LOGS": {
            "Enabled": false
        },
        "INDEX_SLOW_LOGS": {
            "Enabled": false
        }
    },

but we are actually getting

"LogPublishingOptions": {},

If yes, I agree with you.

My question is - if I have enabled it like

new opensearch.Domain(this, 'Domain', {
	version: opensearch.EngineVersion.OPENSEARCH_2_11,
	logging: {
		slowSearchLogEnabled: true,
		appLogEnabled: true,
		slowIndexLogEnabled: true,
	},
})

Now I need to disable them and update to

new opensearch.Domain(this, 'Domain', {
	version: opensearch.EngineVersion.OPENSEARCH_2_11,
})

Will they be disabled from the enabled state? If not, I think #29202 is still relevant?

[pahud]

Reopen this issue as it's still relevant.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

[SigniantJoe]

@pahud you are correct in your latest interpretation about getting empty logPublishingOptions when it should be populated. This means that if you deploy with logging enabled then try to deploy with logging disabled, logging will remain enabled.

I actually made my own PR with unit tests for this, which I think explains the issue: #29205 It's currently stuck waiting on me to updating snapshots, which I'm looking into