As Malaysians woke up today, to a brand new cabinet of Ministers, many have already begun expressing their dissatisfaction on the lineup. I know better than to wade into these politically charged discussions – but I will point out that my people have long been overlooked for Ministerial positions.

Who are ‘my people’ you ask…

Hackers.

Or if you prefer a less negative word – Geeks. But for the rest of this post, I’ll use the more accurate term of hacker to refer to technically savvy folks who subscribe to the hacker ethic.

Yes, we in the hacker community have long been overlooked for ministerial positions, and I for one, choose to speak out against this travesty. But before I delve into why I think we’ve not played a bigger part in politics, let me first make the case for why we need hackers in parliament.

Why we need hackers in parliament

But it’s not enough that policy makers merely understand technology, they also need to subscribe to the hacker ethic , and bring that ethic into the decisions they make.

What is the hacker ethic? Well I’m glad you asked.

The ethic has no hard definition, but it incorporates things like Sharing, Openness, Decentralization and Free access to computers, etc. The ethic further includes attitudes, like pure meritocracy, the idea that hackers should be judged for their hacking (and nothing else), not age, gender, degrees or even position in a hierarchy. So anytime you see some poor sod who claims to be a hacker, but puts CISSP, PMP, CEH at the end of their LinkedIn profile – you know they’re not really hackers.

You can see ethic played out at hacker conferences throughout the world, hackers are ever willing to share what they’ve built with anyone who’ll listen, and they’re accepting of anyone willing to learn, at any age bracket, without any education or formal training.

The Hacker perspective is an interesting one, and like all perspectives, may not always be right or appropriate, but it’s important for it to be present at the decision making process, if nothing more than to add to the diversity of thought.

So why aren’t there more hackers in decision making levels? Well let’s see what it takes to reach the decision making level in the first place.

[caption id=“attachment_6436” align=“aligncenter” width=“550”]

Boy, that’s a lot of RED![/caption]

Last week, MyNic suffered a massive outage taking out any website that had a .my domain, including local banks like maybank2u.com.my and even government websites hosted on .gov.my.

Here’s a great report on what happened from IANIX. I’m no DNSSEC expert, but here’s my laymen reading of what happened:

1. .my uses DNSSEC
2. Up to 11-Jun,.my used a DNSKEY with key tag:25992
3. For some reason, this key went missing on the 15-Jun, and was replaced with DNSKEY key tag:63366. Which is still a valid SEP for .my
4. Unfortunately, the DS record on root, was still pointing to key tag:25992
5. So DNSSEC starting failing
6. 15 hours later, instead of correcting the error, someone tried to switch off DNSSEC removing all the signatures (RRSIG)
7. But this didn't work, as the parent zone still had a DS entry that pointed to key tag:25992 and hence was still expecting DNSSEC to be turned on.
8. 5 hours after that, they added back the missing DNSKEY key tag:25992 (oh we found it!), but added invalid Signatures for all entries -- still failing.
9. Only 4 hours after that did they fix it, with the proper DS entry on root for DNSKEY key tag:63366and valid signatures.
10. That's a 24 hour outage on all .my domains.

I defend my fellow IT practitioners a lot on this blog, but this is a difficult one. Clearly this was the work of someone who didn’t know what they were doing, and refused to ask for help, instead tried one failed fix after another which made things worse. As my good friend Mark Twain would say – it’s like a Mouse trying to fix a pumpkin.

I don’t fully understand DNSSEC (it’s complicated), but I’m not in charge of a TLD. It’s unacceptable that someone could screw up this badly – and for that screw up to impact so many people, and all we got was a lousy press release.

The point is, it shouldn’t take 24 hours to resolve a DNSSEC issue, especially when it’s such a critical piece of infrastructure. I’ve gone through reports of similar DNSSEC failures, and in most cases recovery takes 1-5 hours. The .nasa.gov TLD had a similar issue, that was resolved in an hour, very rarely do we see a 24 hour outage, so what gives?

I look forward to an official report from MyNIC to our spanking new communications ministry, and for that to be shared to the public.

Ok, I’ve been pretty involved in the latest data breach, so here’s my side of the story.

At around 11pm last Friday, I got a query from Zurairi at The Malay Mail, asking for a second opinion on a strange email the newsdesk received from an ‘anonymous source’. The email was  regular vulnerability disclosure, but one that was full of details, attached with an enormous amount of data.

This wasn’t a two-liner tweet, this was a detailed email with outlined sub-sections. It covered why they were sending the email, what the vulnerable system was, how to exploit the vulnerability and finally (and most importantly!) a link to a Google Drive folder containing Gigabytes of data.

The email pointed to a Ministry of Education site called SAPSNKRA, used for parents to check on their children’s exam results. Quick Google searches reveal the site had security issues in the past including one blog site advising parents to proceed past the invalid certificate warning in firefox. But let’s get back to the breach.

My first reaction was to test the vulnerability, and sure enough, the site was vulnerable to SQL Injection, in exactly the manner specified by the email. So far email looked legitimate.

Next, I verified the data in the Google Drive folder, by downloading the gigabytes of text files, and checking the IC Numbers of children I knew.

I further cross-checked a few parents IC numbers against the electoral roll. Most children have some indicator of their fathers name embedded in their own, either through a surname or the full name of the father after the bin, binti, a/l or a/p. By keying in the fathers IC number, and cross-referencing the fathers name against what was in the breach, it was easy to see that the data was the real deal.

So I called back Zurairi and confirmed to him that the data was real, and that the site should be taken offline. I also contacted a buddy of mine over at MKN, to see if he could help, and Zurairi had independently raised a ticket with MyCert (a ticket??!!) and tried to contact the Education Minister via his aide.

Obviously neither Zurairi nor myself, or any of the other journalist I kept in touch with, could report on the story. The site was still vulnerable, and we didn’t want someone else breaching it.

The next morning, I emailed the anonymous source and asked them to take down the Google Drive, explaining that the breach was confirmed, and people were working to take down the site. Hence there was no reason to continue exposing all of that personal information on the internet.

They agreed, and wiped the drive clean, and shortly after I got confirmation that the SAPSNKRA website had been taken down. So with the site down, and the Google Drive wiped cleaned, it seemed the worst was behind us.

Danger averted…at least for now.

But, since Data breaches last forever, and this was a breach, we should talk about what data was in the system. Zurairi did a good job here, but here’s my more detail take on the issue.

Couple months back I started GovTLSAudit. A simple service that would scan  .gov.my domains, and report on their implementation of TLS. But the service seems to have benefits above and beyond that, specifically around having a list of a government sites that we can use to cross-check against other intel sources like Shodan (which we already do daily) and VirusTotal.

So here’s 3 times GovTLSAudit helped secure government websites.

That time Yayasan Islam Terengganu was used a phishing website

paypal-security-wmid0f4-110ll-pp16.yit.gov.my appleid.corn-security2016wmid7780f4-110ll-16.yit.gov.my paypal-security-wmid7110f4-110ll-pp16.yit.gov.my

I took to twitter to report my findings – I kinda like twitter for this, and the very next day Exabytes come back with a followup that they were fixing it. That’s good, because having a phishing campaign run on .gov.my infrastructure isn’t exactly what you’d like.

There’s a lot more details in the tweet about how I investigated this,– click here to follow the thread. A warning though – I regularly delete my old tweets. So get it while it’s there :).

If you’ve come here from a link on twitter – you’d see that the address bar still says login.astro.com.my, but the site is rendering this page from my blog. If not, click this link to see what I mean. You’ll get something like this:

Somehow I’ve managed to serve content from my site on an astro domain. Rest assured, I haven’t ‘hacked’ astro servers and uploaded my page, but I’ve performed an equally sinister attack called open redirect.

I previously wrote about how data breaches are like diamonds:

• They're not as rare as you think
• They're worth far more to you than to a thief
• They last forever

First off, Lowyat has already reported 3 big data breaches (at least by my count), and rest assured these won’t be the last. Data breaches will continue to happen, and just like diamonds, they’re not as rare people think they are – they happen all the time, get used to it.

In 2015, I was invited to a variety program on Astro to talk about cybersecurity.

This was just after Malaysian Airlines (MAS) had their DNS hijacked, but I was specifically told by the producer that I could NOT talk about the MAS hack, because MAS was a government linked company, and they couldn’t talk bad about GLCs.

Then half-way through the interview they asked me about government intervention, and I said something to the effect of “Governments are part of the problem and should refrain from censoring the internet”, that sound-bite never made it to TV because it was censored.

Gov TLS Audit finally has a website to complement the API.

I used the services of a guy from fiverr to code the site, it isn’t the best design in the world, but it’s good enough for now. The site allows you to query a site and view the historical details of a particular .gov.my website. The full list of .gov hostnames can be found here.

It also links to the full daily scan outputs (in csv, json and jsonl formats) if you wish to download and do more analysis. Please note, the csv output has some errors that I’ve not had time to fix, best you use the jsonl or json file, which don’t have errors, but also have much more details.

Two weeks ago, I rage-tweeted something regarding Malaysian politics that got a lot more viral than I liked (I’ve censored out the profanity for various reasons, most notably, there are teenagers who read this blog). It was a pointless collection of 200 characters, that somehow resonated with people enough to be shared across social media. Obviously, since it was me, the tweet was filled with a small collection of profanities, and laced with just the right amount of emotive content :)

But then things started getting bad.

Soon after I tweeted, I received messages from folks I hadn’t met in decades, showing me screenshots of their whatsapp group that had my tweet – my wife’s chinese speaking colleagues were showing it to her at work – I checked, and nearly 2,000 people retweeted it, which isn’t typical for me, and frankly speaking pretty scary.

As much as I’d like to have my content shared, the tweet in question is nothing but couple of crude words pieced together in a ‘rage-tweet’. And I understand that it emotionally resonates with folks who are angry, but if this the level of discourse we’re having on  Malaysian social media, we should be alarmed. Completely pointless rants being viralled is not how we ubah, it is the absolute opposite of how we ubah!

Research on the virality of articles from the New York Times showed that ‘angry’ content was more viral than any other, beating out awe, surprise and even practical value. The angrier the content, the more likely it would be shared. A rage-tweet is more likely to go viral than something like fuel-saving tips, even though the latter clearly is more valuable to readers.

At this point, I’d rant about how the media has a responsibility to look beyond clicks and ads, and to think about the impact of their content on society, but since I owned the tweet, I simply deleted it. Of course, I can’t stop the screen-shots being shared across whatsapp, but we do what we can.

Deleting your tweets

But while, what I tweet today, may be relevant and acceptable today, it may not be 2-3 years from now. Kinda like how Apu from the Simpsons was acceptable and non-offensive in the 90’s.

I’m ashamed to say it, but I once thought that Michael Learns to Rock was a great rock band, in context, thats acceptable for a 12 year old 2 decades ago, before even Napster or Limewire. Of course, as a adult in 2018, I’m thoroughly aware that AC/DC are the greatest rock band ever, and Michael Learns to Rock, well they’re not exactly Denmark’s best export.

And that’s the problem, twitter removes context  – it’s very easy to take a 140 character tweet from 5 years ago out of context. Nobody cares about context on a platform that limits users to 140 characters (or 280 characters since end 2017). Maybe you quoted an article from TheMalaysianInsider, which, guess what, no longer exist. Context is rather fluid on twitter, and it changes rapidly over weeks, let alone the years from your first tweet.

For example,  this tweet from Bersatu’s Wan Saiful:

No the internet shouldn't be censored. He doesn't know how to shut up. His views are old and racist. But freedom of speech is for everyone.

— Wan Saiful Wan Jan (@wansaiful) November 10, 2013

Gee, I wonder who he was talking about, a simple internet search will give you the answer, but that’s not the point.

Wan Saiful changed his opinion,  and he’s explained why, people should be allowed to change their mind.Freedom to change your opinion not just perfectly fine, it’s a per-requisite for progress.If we allow our tweet history to be a ball-and-chain that ties us to our old idealogy, how could we ever progress? Everybody changes their mind – and that’s OK.

The point is twitter should not be a historical archive – it should be current. A great place to have an informed discussion of current affairs, but not a place to keep old, out-dated and out of context material floating around.

Hence, I decided to delete all my tweets that were older than 90 days old, and here’s how.

Last Month, I embarked on a new project called GovTLS Audit, a simple(ish) program that would scan 1000+ government websites to check for their TLS implementation. The code would go through a list of hostnames, and scan each host for TLS implementation details like redirection properties, certificate details, http headers, even stiching together Shodan results into a single comprehensive data record. That record would inserted into a DynamoDB, and exposed via a rest endpoint.

Initially I ran the scans manually Sunday night, and then uploaded the output files to S3 Buckets, and ran the scripts to insert them into the DB.

But 2 weeks ago, I decided to Automate the Process, and the architecture of this simple project is complete(ish!). Nothing is ever complete, but this is a good checkpoint, for me to begin documenting the architecture of GovTLS Audit (sometimes called siteaudit), and for me to share.

What is GovTLS Audit

A different script then reads in the JSONL file and loads each row into database (DynamoDB), and then uploads the 3 files as one zip to an S3 bucket.

On the ‘server-side’ there are 3 lambda functions, all connected to an API Gateway Endpoint Resource.

• One that Queries the latest details for a site [/siteDetails]
• One that Queries the historical summaries for the site [/siteHistory]
• One that List all scan (zip files) in the S3 Bucket [/listScans]