This post covers how to perform logging within AWS Lambda. Lambda has built-in integration to Cloudwatch logs, making it a default choice for logs, but the way a distributed system like lambda performs logging, is quite different from how you'd do in a monolithic app.

For the brave folks still reading this -- let's dive in.

Lambda Logging: 101

When building lambda functions, it's always a good idea to sprinkle logging statements throughout your code -- particularly in areas where something important happens. This is just generally good advice regardless of whether you're using Lambda.

A lambda function is a like a little island, surrounded by network. Unlike Fargate containers, of EC2 instances, they do not have EFS, EBS or some other fast storage support. Everything that goes into a lambda, goes in via the network interface (and network only).

Last week I took the AWS Certified Security - Specialty exam -- and I passed with a score of 930 (Woohoo!!)

In this post I cover why I took it, what I did to pass, my overall exam experience, and some tips I learnt along the way.

So let's go.

Why?

Why would anybody pay good money, subject themselves to hours of studying, only to end up sitting in a cold exam room for hours answering many multiple choice questions!

And the reward for that work is an unsigned PDF file claiming you're 'certified', and 'privilege' access to buy AWS branded notebooks and water bottles!! Unless those water bottles come with a reserved instance for Microsoft SQL server in Bahrain, I'm not interested.

But, jokes asides, I did this for fun and profit, and fortunately I really did enjoy the preparing for this exam. It exposed me to AWS services that I barely knew -- and forced me to level-up my skills even on those that I knew.

The exam has a massive focus on VPC, KMS, IAM, S3, EC2, Cloudtrail and Cloudwatch. While lightly touching Guardduty, Macie, Config, Inspector, Lambda, Cloudfront, WAF, System Manager and AWS Shield.

You need to catch you breath just reading through that list!

But for those diligently keeping count -- you'd notice that the majority of those services are serverless -- meaning the exam combined my two technological love-affairs ... security and serverless!

I wasn't lying when I said it was fun. So what about the profit.

I'm not sure how good this would be for my career (I literally got the cert last week), but for $300, it's is relatively cheap, with a tonne of practical value. So trying to get an ROI on this, isn't going to be hard.

For comparison, the CCSP certification cost nearly twice as much, is highly theoretical and requires professional experience.

The results also help me validate my past years of working on serverless projects, proving I wasn't just some rando posting useless hobby projects on GitHub. Instead, I'm now a certified AWS professional, posting useless hobby projects on GitHub (it's all about how you market it!)

So now that we've covered the why, let's move onto how.

GitHub actions is the new kid on the workflow block.

It allows users to orchestrate workflows using familiar git commands like push & pull requests, and un-familiar GitHub events like gollum, issue creation and milestone closures.

In this post, we'll use GitHub actions to orchestrate a build pipeline that will deploy lambda functions using the Serverless framework. There's a lot of tutorials that cover the basics of doing this, but we'll dive deeper and cover off framework plugins and deploying to different environments using feature branches.

In my honest (and truly humble) opinion, VPCs don't make much sense in a serverless architecture -- it's not that they don't add value, it's that the value the add isn't worth the complexity you incur.

After all, you can't log into a lambda function, there are no inward connections allowed. And it isn't a persistent environment, some functions may timeout after just 2-3 seconds. Sure, network level security is still worthy pursuit, but for serverless, tightly managing IAM roles and looking after your software supply chain for vulnerabilities would be better value for your money.

But if you've got a fleet of EC2s already deployed in a VPC, and your Lambda function needs access them. Then you have no choice but to deploy that function in a VPC as well. Or, if your org requires full network logging of all your workloads, then you'll also need VPC (and their flow logs) to comply with such requests.

Don't get me wrong, there is value in having your functions in a VPC, just probably not as much as you think.

Put that aside though, let's dive into the wonderful world of Lambda functions and VPCs

Working Example

First, imagine we deploy a simple VPC with 4 subnets.

1. A Public Subnet with a Nat Gateway inside it.
2. A Private Subnet which routes all traffic through that NAT Gateway
3. A Private Subnet without internet (only local routing)
4. A Private Subnet without internet but with a SSM VPCe inside it

Let's label these subnets (1), (2) ,(3) and (4) for simplicity.

Now we write some Lambda functions, and deploy each of them to each subnet. The functions have an attached security group that allows all outgoing connections, and similarly each subnet has very liberal NACLs that allow incoming and outgoing connections.

Then we create a gateway S3 VPC-endpoint (VPCe), and route subnet (4) to it.

Finally, we enable private DNS on the entire VPC. And then outside the subnet we create a bucket and an System Manager Parameter Store Parameter (AWS really need better terms for these things).

The final network looks like this:

Amazon KMS is one of the most integrated AWS services, but probably also the least understood. Most developers know about it, and what it can do, but never really fully realize the potential of the service. So here's a rundown of the innards of the KMS service.

What is KMS?

KMS (Key Management Service) is an AWS offering that allows us to create, manage and use cryptographic keys. Like everything else in AWS, it's highly available, provided via API, and charged on a per-use basis.

The system, which was introduced on the first day of the 2020 school session yesterday, takes only two seconds to scan a pupil’s face before his personal information, such as full name, pupil number and class, is stored into the school’s database

https://www.nst.com.my/news/nation/2020/01/552737/sk-taman-perling-1-uses-facial-recognition-scanners-mark-pupils

A school in Johor became the first in the country to introduce facial recognition scanners to take pupils attendance -- hopefully it'll be the last.

This year I resolve to support the media that I like, i.e start paying for content I've been consuming for free all this time.

I believe that if we want better media, we need to start paying for it, and it's not a matter of quality content, a free and fair media, is an absolute necessity for democracy to operate -- after all, people can't make informed choices if they aren't informed.

Last week, I launched a new pipeline for Klayers to build Python3.8 Lambda layers in addition to Python3.7. For this, I needed a separate pipeline because not only is it a new runtime, but under the hood this Lambda uses a new Operating System (Amazon Linux 2 vs. Amazon Linux 1)

So I took the opportunity to make things right from an account hierarchy perspective. Klayers for Python3.7 lived in it's own separate account from all my other hobby projects on AWS -- but I kept all stages in it (default, dev and production). [note:Default is an odd-name, but it ties to the Terraform nomenclature]. This afforded some flexibility, but the account felt bloated from the weight of the different deployments -- even though they existed in different regions.

It made no sense to have default and dev on the same account as production -- especially since accounts were free. Having entirely separate accounts for prod & non-prod incurred no cost, and came with the benefit of additional free-tiers and tidier accounts with fewer resources in them -- but the benefits don't stop there.

Had a blast at PyConSG 2019, really cool to be in the presence of so many pythonistas. Would definitely recommend, especially since python is one of the more broadly used languages (AI, Blockchain, RPA, etc).

My talk was on AWS Lambda (naturally!).

Slide deck can be found in this GitHub Repo. Let me know what you think either in the comments here or on Youtube.