Security Testing

There are multiple different testing methods for security of an application:

Testing and Development Activities

Security tests only make sens if:

  • only release versions can be tested
  • the software does not change between tests

Peneration Testing

Penetration testing has 3 main classes:

  • White Box: Tester has access to credentials, documentation, source code, debug, …

  • Black Box: Tester has minimal information (e.g. only target IP range)

  • Grey Box: A mix between white and black box testing

  • Pen testing is mostly useless except the environment is exactly the same as the production environment.

  • Grey box testing is done mostly because information is not available.

  • Pen tests should not do flooding (only if explicitly allowed) and should be done in off-hours.

  • Pen tests can pretty much always break the system.

A penetration test fails if no vulnerabilities are found. The customer might feel secure but the system is still vulnerable, because the wrong things were tested (because of time or resource constraints). In the best case, pen tests should identify root-causes of vulnerabilities.

Fuzz-Testing

The idea of fuzz testing is, to test the application with (more or less) random input and observe behavior. Behavior deviation could be:

  • the software crashes
  • the software returns non-sensical results
  • the software returns errors
  • the software requires more time or resources to process the input
ProCons
Easy to automateNeeds significant setup
Can be used if source code is not availableMay crash the software (alerts target)
Allows easy estimate if more effort makes senseIs not a complete test

Structured vs. Unstructured Fuzzing

  • Structured Fuzzing: Input is generated based on a model of the input format
  • Unstructured Fuzzing: Input is generated randomly

Unstructured fuzzing is easier to implement but structured fuzzing is more efficient.

Tools for fuzzing include: American Fuzzy Lop

Code Inspection and Review

Code inspection and review is the process of manually checking the source code for quality and vulnerabilities. It is critically dependent on the skill of the reviewer and the time available.

Direct results:

  • bugs

Indirect results:

  • code quality
  • interface quality
  • skill-level of the developers

Code Scanners

Code scanners are automated tools which look for problems in the source code. They can be very simple and only look for patterns or highly sophisticated:

  • check data flow (follow input data to problematic system calls)
  • compare time of check and time of use (TOCTOU)
  • check unused variables
  • …

Execution Emulators

Provides report on code execution to find problems and vulnerabilities. Valgrind

Patching

Problems with patching:

  • might break existing functionality
  • might introduce new problems or fail to fix existing ones
  • must be done securely (e.g. signed updates)
  • might change operational characteristics (e.g. performance or resource usage)
  • might take a loooong time

Problems with not patching:

  • vulnerabilities accumulate
  • vulnerabilities become generally or well-known
  • “emergent properties” (meaning that the combination of vulnerabilities is easier to exploit than the individual vulnerabilities themselves)

Unavailability of Patches

  • Vendor does not provide patches
  • The product is end-of-life
  • It is not possible to patch the product (e.g. embedded systems)
  • Patches are delayed
  • Patches are available but break functionality
  • Patches are available but (re-)introduce vulnerabilities

Attackers monitor these problems and exploit them or intensify efforts to find exploits.

Backdoors

Backdoors are hidden (the one placing it does not tell anyone about it) ways to access a system. This breaks the security model of the system.

Backdoors can be placed in the code, in libraries, in the compiler, in the operating system, in the hardware, … Can also be placed in cryptography (create NOBUS backdoors). They are placed by insiders (e.g. developers, system administrators), state-actors (secret organizations, political groups), vendors, or other attackers.

Front Doors

Fontdoors are backdoors which are placed intentionally. This is also known as “lawfull interception” or “lawfull access”. They assume that people fall for simple manipulation.

Code Backdoors

Code backdoors are placed in the source code. They are locally/remotely accessible, hidden (undocumented) functionality to give access to third parties which has not been authorized by the owner.

Hiding techniques include:

  • meta-technique (hide them as common coding mistakes)
  • “accidentially” leave debug code active
  • omit workarounds for known vulanerabilites (or only partially fix them)
  • use violation of “least surprise”
  • race-conditions
  • bad initialization
  • use bad or misleading comments
  • intentionally obfuscate code or make it more complex/hard to read

Finding and fixing backdoors is usually more expensive than rewriting the code from scratch.

Cryptographic Backdoors

Uses cryptography to hide the backdoor. The backdoor is usually a key which is known to the attacker. The compromised algorithm cannot be distinguished from a secure one.

This is a “mathematically compomised design”.

Solution:

  • do not use alogrithms which are not well-known
  • assume and acknowledge that the possibility is there

Economics of Security

The key question is: Is it profitable to make insecure software? The sad answer is: Yes, it is.

Being insecure mostly seems more profitable (frequency of attack * cost of attack vs. cost of making software). People do not seem to care and preventing attacks it too expensive.

But, what could be done?

  • make developer liable (which standards are applied)
  • implement working product liability or insurance (which standards to apply? other engineering disciplines have this!)
  • require certifications (which standards are applied?)
  • require independent reviews (this requires a lot of time and resources)
  • improve eduction (this takes a lot of time and resources, is computer science too young?)