Created by M.Moon
A simple lambda for ensuring your CloudWatch logs don't get out of control. Because you shouldn't have to feel frantic about your monthly AWS spend.
This lambda aims to:
- Cleanup your CloudWatch log groups
- Lower your monthly AWS spend
- Automate account maintenance (in regards to your logs)
This lambda is written in Python 3
The cloud is cheap but that isn't an excuse to blow your yearly budget just to 'have something around'.
Chances are you have a lot of underutilized logs sitting in Cloudwatch, this simple lambda runs on a schedule to ensure all log groups across any specified region in your account adhere to the same expiration policy.
Without specification, CloudWatch Logs are set to expire after 10 years. If all of your infrastructure is in AWS and you're using CloudWatch Logs heavily (even if it's just a pitstop before you push your logs elsewhere) chances are you're paying quite a bit to store logs you aren't using. Your spend might be low now, but in 3 years you're going to be paying quite a bit just to house logs you (probably) don't care about anymore.
Cloudwatch-Log-Manager is a lambda that runs on a schedule. It runs through all of your CloudWatch Log Groups in each specified region and checks to see if each group has the desired expiration age. If the log group already has the expected age, it passes, if not, it sets the desired retention policy on the group.
The general recommendation is 180 or 365 days. Without updating the generic setting of this lambda is 180 days.
Pretty simple.
Setting up is easy just follow these steps:
- Clone this directory
- Add your desired amount of days to store your logs and add the regions you have active logs in to /dist/local_config
- Run
python account_setup.pyfrom the directory you cloned this in - Kick your feet up and worry about one less thing (alcoholic beverage optional)
The file 'account_setup.py' is a setup script that does the following:
- Creates a ZIP off the
/distdirectory - Creates IAM role for the Lambda
- Creates IAM policy for the role
- Creates CloudWatch Event that is set to run every 14 days
- Creates Lambda with the CloudWatch Event as the trigger
You can change the frequency of the lambda in the main function of 'account_setup' function.