Visualizing your AWS environment in a single pane of glass
At Sigma Computing we love using our own product to visualize security and infrastructure data. If you haven’t checked out our other posts in this series please do so.
One of the issues that we struggle with is, how do you know what is happening across all the AWS services you consume in your account? There is no single pane of glass today that will tell you, so we started creating one around a few common AWS services. Visualized here (in order) we have AWS Cloudtrail, Guard Duty, S3 Access Logs and Cloudfront. You can visualize any AWS service where you can consume logs for that usage, so expect another dashboard with a few more sources added soon.
Cloudtrail is the service that logs event history for your account. It allows you to audit all commands made through the console, APIs, SDKs, etc. In our first row of visualizations we are looking at what users/services are making calls, what IPs those calls are coming from and how those calls are being made. In the second row we are looking at which AWS accounts are getting the most events, in what region(s) those events are being recieved, and the type and which source of those events. This gives us a quick look dashboard of what is happening in our account over the last 7 days and allows us to quickly spot potential anomalies.
In the second section we just have a simple table of Guard Duty alerts so we can quickly see any potential issues across out accounts. Guard Duty is AWS’s threat detection service that continuously monitors your account for malicious activity across a number of services.
Next we are visualizing S3 Access Logs and we actually had to use the power of Sigma to visualize these properly. S3’s log format is different than some other AWS services as it uses a [06/Feb/2019:00:00:38 +0000] date format, where Sigma uses a standard ISO 8601 format. No problem though in Sigma we can run the following and transform the date:
DateTrunc(“hour”, CallDatetime(“try_to_timestamp”, [COLUMN/TIME], “[DD/MON/YYYY:HH24:MI:SS +0000]”))
This section allows us to see what IPs are making the most calls to our buckets, how much transfer is being done, which buckets are generating the most logs, the user-agents accessing buckets and the status code.
Lastly we added a section for Cloudfront, which is AWS’s Content Delivery Network. This section is similar to the dashboard we did for Cloudflare. We can see how much traffic this Cloudfront distribution is pushing, which PoPs are being hit, which host is generating the traffic, top IPs, cache response and which SSL Protocol is being used.
So there you have it: an extensible AWS dashboard where you can visualize the AWS services you consume in a single pane of glass. You can add whatever services you use as long as you can consume logs for those services.
