When I started software development in 2020 I had a very different view on how software was designed and developed. It was heavily influenced by what I learnt in my under graduate course. Dataflow diagrams, High-level design diagrams, low-level diagrams, class diagrams and project structure was a general sense that I knew.

In early 2021 I was tasked to focus on the security aspects of my backup project including working on securing various secrets and the encryption architecture of files being backed up. With my traditional sense of software design and development, I added dataflow diagrams, high-level design diagrams and low-level design diagrams. I thought I went the extra mile by adding the class diagrams, database schema structures and other details into my document.

I happily scheduled a design review with the architect (my mentor) for the file encryption design. In the design review my mentor asked me a number of questions which I had no answers to. “Where is this encryption key stored?”, “What will you do if someone takes a memory dump and the encryption key is leaked?”, “If the encryption key is leaked, will you re-encrypt all the files?”, “How many encryption keys will you store?”, “How will you encrypt the encryption key, and who will encrypt that key?”. All these questions were difficult to answer, I had focused only on one part of the problem which was encrypting the file and storing the encryption key, I had missed all the other points.

My mentor then asked me to do something called as threat modelling and come back. This blog pretty much sums up the theory about what I learnt from there on. There is information about what I learnt, and how I built my own methodology for threat modelling.

Threats and Vulnerabilities

When I started to do some research about the topic, I realised that there is a difference between a what is considered to be a vulnerability and what is a threat. I had interchangeably used these terms before and had seen others do it too.

What is a Vulnerability?

A vulnerability is any weakness of a computer system which can be exploited to negatively impact the security policy of the system.

Example 1, Role Based Access Control is not implemented on the database containing user data. Any database user can create, update, or delete user records.

What is a Threat?

A threat is any negative impact on a computer system which can be introduced through a vulnerability.

Example 2., A rogue developer accesses the database to delete all data from the user's database.

If you notice here, the above threat is a result of Example 1’s vulnerability. Since, there was no RBAC implemented on the database there is a threat for a rogue developer to delete all data.

So now, how do I find out all off these vulnerabilities and corresponding threats, I thought that it was not humanly possible to do that by my next design review call. I find out later that there is a way to categorize common threats and vulnerabilities using something called as a STRIDE model.

What is Threat Modelling?

Threat modelling is a pre-emptive process to identify, communicate, and understand threats and mitigations. The result of threat modelling is a threat model.

What, why and who?

What is a Threat Model?

A threat model is a structured representation of all the information that affects the security of an application.

In essence, it is a view of the application and its environment through the lens of security.

Why Threat Model?

When you perform threat modeling, you begin to recognize what can go wrong in a system.

In an ideal scenario, threat modeling should take place as soon as the architecture is in place. However, modelling all threats at this time might not be ideal.

No matter when you end up performing the threat model, understand that the cost of resolving issues generally increases further along in the SDLC.

Who Should Threat Model?

You. Everyone. Anyone who is concerned about the privacy, safety, and security of their system.

This intrigued my interests and did a little more research to find out some theory behind threat modelling and what are the best practices. Since I was in the early part of my career, I used to spend a lot of time doing theoretical research and analysis for any task assigned to me. I found the threat modelling manifesto on my research journeys (pre-ChatGPT era)

Threat Modelling Manifesto

This was formed by a group of security researchers and CISOs across the globe. More information in the references. I learnt Threat Modelling can be as simple as asking 4 questions while to define a design or solution.

Ask Four Key Questions

What are we working on?
What can go wrong?
What are going to do about it?
Did we do a good enough job?

At that time I was working on securing different secrets so, here is an example.

Example

What are we working on?

The User-Agent will upload files to an S3 bucket designated to that user using AccessKeyID and SecretAccessKey of that user.

What can go wrong?

A memory dump of the user-agent can yield the AccessKeyID and SecretAccessKey which can be maliciously used by the user to get free storage on Wasabi.

What are going to do about it?

Create a wrapper on the remote API to fetch an STS token for the S3 client instead of storing the AccessKeyID and Secret Access Key on the user-agent.

Did we do a good enough job?

We have moved the threat away from the user-agent to the Remote API.
Securing the deployments from remote access and raising alerts on memory dumps or root commands can help us identify malicious behavior and remediate issues

The next section is a little interesting and I have designed it based on my experience as a security engineer. These steps are a mix of various sources stitched together with my experience.

3 Step Approach for Threat Modelling

Step 1: Decompose the Application

On a high-level we do the following things:

Identify entities, entry points, exit points and interactions between various components of the system.
Create a Dataflow diagram (DFD) highlighting different interactions and privilege boundaries.

Tool recommendation: Microsoft Threat Modelling Tool

Activities to decompose the application:

Start with a three-tier architecture
Identify external entities and interactions
Identify internal entities and interactions
Identify boundaries of the application.
Identify the trust zones (network layer, domain layer, interaction layer)
Identify 3rd-party components, integrations and zones.
Create a data-flow-diagram with the above information.

It can be as simple as a diagram like this:

Threat Modeling

In the above diagram you can see the dataflows, network boundaries and interactions between different components of the application.

Step 2: Categorize and Rank Threats

Step 2.1 Threat Categorization

A threat categorization such as STRIDE is useful in the identification of threats by classifying attacker goals such as:

Spoofing
Tampering
Repudiation
Information Disclosure
Denial of Service
Elevation of Privilege

Type	Description	Security Control
Spoofing	Threat action aimed at accessing and use of another user’s credentials, such as username and password.	Authentication
Tampering	Threat action intending to maliciously change or modify persistent data, such as records in a database, and the alteration of data in transit between two computers over an open network, such as the Internet.	Integrity
Repudiation	Threat action aimed at performing prohibited operations in a system that lacks the ability to trace the operations.	Non-Repudiation
Information disclosure	Threat action intending to read a file that one was not granted access to, or to read data in transit.	Confidentiality
Denial of service	Threat action attempting to deny access to valid users, such as by making a web server temporarily unavailable or unusable.	Availability
Elevation of privilege	Threat action intending to gain privileged access to resources in order to gain unauthorized access to information or to compromise a system.	Authorization

Step 2.2 Threat Ranking

Risk = (Probability of Threat) x (Cost to organization)

To be holistically analyzed with stake holders and understand the risk appetite.

This is a very deep topic, I haven’t understood this yet fully, but from what I know you must present the inherent threats and the impact of the threat (cost) to the business and the business decides what is a threat worth fixing.

An example is, there is a vulnerability that the master key of the application is stored with a single admin user. The threat is that the admin user can be malicious and delete the master key or use that to decrypt all secrets.

The business might consider calculate the risk to be low if the admin user is the CEO of the company or the business owner himself. The probability is near zero so we might not need to worry about engineering to fix the threat immediately. Now, there are other risks here “what if the CEO is not too secure with how he manages secrets?”, so the CEO will accept the risk. There are lots of scenarios for something as simple as a password. I will try to write another blog sometime later to speak about another topic called Shamir Secret Sharing which can solve the problem of distributed secrets. Before I digress more let’s move to step 3.

Step 3. Identify Countermeasures & Remediations

Common Threat types and Counter Measures

Threat Type	Mitigation Techniques
Spoofing Identity	1. Appropriate authentication 2. Protect secret data 3. Don’t store secrets
Tampering with data	1. Appropriate authorization 2. Hashes 3. MACs 4. Digital signatures 5. Tamper resistant protocols
Repudiation	1. Digital signatures 2. Timestamps 3. Audit trails
Information Disclosure	1. Authorization 2. Privacy-enhanced protocols 3. Encryption 4. Protect secrets 5. Don’t store secrets
Denial of Service	1. Appropriate authentication 2. Appropriate authorization 3. Filtering 4. Throttling 5. Quality of service
Elevation of privilege	1. Run with least privilege

At the end of the threat modelling activity, you will have a list of threats which you now have to sit and categorize. This is a difficult task since you need to see if your design fully mitigates the risk or not. For some threats, the cost of development might be high and they might eventually go into the bucket of non-mitigated threats. It’s good to document it so that there is a record of what were the threats identified and what were the mitigations put in place.

Threat Profiling

Non mitigated threats: Threats which have no countermeasures and represent vulnerabilities that can be fully exploited and cause an impact.
Partially mitigated threats: Threats partially mitigated by one or more countermeasures and can only partially be exploited to cause a limited impact.
- Note: A threat is categorized as partially mitigated only when the risk level is considerably reduced from its state as a non-mitigated threat.
Fully mitigated threats: These threats have appropriate countermeasures in place and do not expose vulnerabilities.

This is the theory behind threat modelling. In a future blog, I will take an example of threat modelling and walk through the 3 steps to help understand threat modelling better. I will also show the usage of tools like Microsoft Threat Modelling to build a threat model.