AI on keithrozario.com

Spacy in a Lambda

Mon, 22 Apr 2019 00:29:34 +0000

I've been really digging into Lambda Layers lately, and once you begin using layers you'll wonder how you got by without them.

Layers allow you to package just about anything into lambda, but in a modular way. So elements of your code that don't change much, can be packaged into layers, while keeping your actual lambda deployment for just the code that's changing.

It's akin to docker cache, where you keep the un-changing elements higher up in your docker file, separate from the code that always changes. The difference though, is that docker cache speeds up builds, while layers speeds up lambda deployments.

But layers aren't magic, and they're still limited by the AWS size limit, hence your entire function (including all it's layers) need to be no larger than 250MB (unzipped).

Which is tough for something like spaCy -- because its default installation size on a AWS Linux is ~400MB (or 492MB based on my quick installation on lambci for python3.7). So, in order to get spaCy working on a lambda, certain tweaks are going to be necessary.

Some have tried working around this problem by installing spaCy onto the lambda container on cold-start -- i.e. pull the data into lambda only when you have access to the 512MB in /tmp. Cool solution, but it almost completely fills out /tmp, and makes a cold-start even slower.

A more optimal solution would be to reduce the size of the spaCy installation and have it fit into a layer! Fortunately I found a GitHub issue after some googling that enables us to do exactly this.

It involves removing unnecessary language files, which spaCy lazy load in, If you're only interested in one language, you can simply remove the unnnecessary language files in the site-packages/spacy/lang directory.

After manually removing all non-English (en) language files, I managed to reduce the size of the spaCy package to 110MB, which fits very nicely into a lambda layer, in the end my lang directory only had the following files:

As a bonus, I also included the English en_core_web_sm-2.1.0 model, to make the lambda layer fully usable on its own .

Finally I published it as a publicly available layer, for anyone to consume. One of the amazing things about layers, is that once a layer is made, it can be shared across AWS for anyone to consume.

Internet of shitty things!

Mon, 24 Oct 2016 23:05:27 +0000

Brian Krebs is the most reputable name in CyberSecurity reporting, his krebsonsecurity website is the best source of ‘real’ journalism on the subject.

But reputation works both ways, the same thing that makes him popular in some circles, makes him unpopular in other. He’s had criminal hackers send him heroin in the mail and even have SWAT teams descend on his home with guns all blazing (in a phenomenon called swatting!). Reporting and exposing underground cyber-criminals comes at a price, you don’t piss of darknet crime lords without taking a few hits along the way.

The problem though is when those ‘few’ hits, turn into a hurricane of web traffic aimed at your server, because that’s exactly what descended on Krebs’ server late last week, when krebsonsecurity was hit by an epic DDOS attack

DDOS is an acronym for Distributed-Denial-of-Service, which basically means forcing so much web traffic to a single website that it eventually collapses–making it unable to provide services to the ‘real’ visitors of the site. All websites run on servers with finite capacity, DDOS attacks are about sending enough traffic to those servers that they eventually exceed that capacity.

But this DDOS was different, and krebsonsecurity will go down in history as the Hiroshima of this type of DDOS. But nuclear weapons only had Hiroshima and Nagasaki, krebsonsecurity will be the first in a Looooong line of DDOS attacks of this scale.

So what makes this attack so different as to merit it’s own class? Well 3 things.

Technology saves lives, but it isn't perfect

Sat, 02 Jul 2016 18:07:38 +0000

What do you do when the technology turns on you?

Or when the feature that’s built to save you, is the one that might just kill you?

There’s a stark similarity between the Takata airbag fiasco, that’s already taken 2 Malaysian lives, and the lady who died in self-driving Tesla.

Both involve the auto-industry and both are technology related, but together they represent a much deeper issue at hand–despite our noblest expectations, technology isn’t perfect–but it’s better than we had before.

We’ve all been trained by Hollywood to expect perfect technology, working all the time and in every scenario, but in reality technology sometimes fails, and newer technology fails more often.

Technology endures through failures, only by our good graces, but unless we grant that grace to it, we will not progress.

What should our response to a technical failure be?

Do we insist on removing ALL traces of the offending technology, or do we accept it as a price of progress, that the occasional failure is a tax we pay to get better technology.

But are some taxes just too high?

Society might accept failing antennas on an iPhone, or even bad Google searches, but an air-bag, that might blow a hole in your chest or a car that might crash you into a truck, might be too high of a price.

So is the tax for air-bags and self-driving cars just not worth the potential safety we get in return?