Config: Managed rules can no longer be updated

[kbalk]

I was curious as to whether the list of AWS managed rules needed to be updated and when I ran the script pull_down_aws_managed_rules.py in the scripts directory, it produced errors;

ERROR:  Unknown label: 'Resource Types', line: '**Resource Types:** AWS::ACM::Certificate'
ERROR:  Unknown label: 'Resource Types', line: '**Resource Types:** AWS::ACM::Certificate'
ERROR:  Unknown label: 'Resource Types', line: '**Resource Types:** AWS::ElasticLoadBalancingV2::LoadBalancer'
...

I checked the git repo used by the script to retrieve the config info and the git repo was archived on June 15, 2023. AWS doesn't want to maintain a separate list of this info per this article.

The rules that we have now are a year old and we could update/add to those rules as needed, but that's a little ugly.

I'll look into pulling this info from some other source. I think I originally tried pulling those rules from a non-github source and it wasn't so easy or was missing info that was available in the github repo.

[kbalk]

I looked at the PR for originally adding the pull_down_aws_managed_rules.py and in one of the comments you mentioned checking boto3 to see how they handled managed rules. I'll see how they're addressing the issue now.

[bblommers]

I looked at the PR for originally adding the pull_down_aws_managed_rules.py and in one of the comments you mentioned checking boto3 to see how they handled managed rules. I'll see how they're addressing the issue now.

I presume that was without me actually doing any research - as it doesn't seem straightforward.

Maybe we should parse the HTML docs instead:
https://docs.aws.amazon.com/config/latest/developerguide/managed-rules-by-aws-config.html

[kbalk]

I looked at the PR for originally adding the pull_down_aws_managed_rules.py and in one of the comments you mentioned checking boto3 to see how they handled managed rules. I'll see how they're addressing the issue now.

I presume that was without me actually doing any research - as it doesn't seem straightforward.

Maybe we should parse the HTML docs instead: https://docs.aws.amazon.com/config/latest/developerguide/managed-rules-by-aws-config.html

Yup, I was referring to your suggestion to look at boto3. And yup, the HTML will have to be parsed. I know I looked at doing that originally and chose not to, but I'm not sure why. Although some reasons I can think of is that the HTML is more likely to change than the markdown docs and there's more extraneous stuff in the HTML so that means pulling down more info than needed.

I apologize for the delay in working on this problem ... I was hoping to get started on it this weekend.

[kbalk]

Related PR: #6823