Known Exploit Resistance

Criteria: The product is protected from known software vulnerabilities that present a danger from attackers.

See the test in action:

• Smart lock
• Smart pressure cooker
• Smart baby monitor

Notes:

• This test contains only one criteria and only one indicator. However, the procedural overview in the Digital Standard separates out testing into three areas, Browsers, Apps, and Connected Devices.
• Common Vulnerabilities Exposures is a publicly available list of known software and firmware vulnerabilities, where each vulnerability is given a unique identifier and standardized description. It is operated by the MITRE Corporation, a U.S. non-profit that oversees federally funded research and development centers (FFRDCs). Each vulnerability is commonly known as a "CVE." In general, all well-known vulnerabilities will have CVEs. Searching the CVE list for the name of the piece of software you are analyzing, is the first step in testing against known vulnerabilities for software of all types.

Methodology for Assessing Each Indicator

1) The software is secure against known bugs and types of attacks.

Browsers

• Identify whether the product being evaluated has a browser component. For example, tablets, phones, and televisions may ship with browsers, whereas connected devices like fitness trackers, water sensors, and grills may not.
• Identify if the browser component is a recognizable web browser (e.g. Chrome, Firefox, Safari, etc.), a customized derivation of a recognizable web browser, or an unrecognized or custom web browser.
• You will be testing whether the product shipped with a vulnerable version of a browser, so you need to locate any CVEs affecting the installed browser. Search the CVE database using the name of the browser component you are testing (e.g. Chrome, Firefox, Safari, etc.), and the name of the manufacturer and product as keywords. Identify CVEs affecting the browser (read more about effectively searching CVEs here).
• For each CVE that you have identified as affecting the browser component you are testing, determine whether the CVE has been addressed in current versions of the software (e.g. the vulnerability has been patched in Chrome).
• Determine if the browser component you are testing is updated enough to address each identified CVE.
• If the browser is not updated, or it is unclear which version the browser component is based on, for each CVE that you have identified as affecting the browser determine whether the CVE's proof of concept code is posted publicly. Usually this will be linked from the CVE notice, or the documentation of the vulnerability linked from the CVE page.
• If the original proof of concept code for the CVE you are testing against is available, use it to test the browser for the issue defined in the vulnerability notice.
• If the CVE’s original proof of concept code is not available, devise custom code based on the detailed description of the vulnerability either in the CVE itself or in the reference linked from the CVE page, to test the browser for the issue identified in the vulnerability notice.
• Check if the browser is protected from the identified vulnerabilities.
• If the browser is protected from all the identified known exploits, mark PASS.
• If the browser is not protected from all identified known exploits, mark FAIL.
• If the product does not ship with a browser, mark NA.

Apps

• Search the CVE database using the app name as a keyword, and identify CVEs for the app.
• Identify whether the CVE's proof of concept code is posted publicly. Usually this will be linked from the CVE notice, or the documentation of the vulnerability linked from the CVE page.
• Review documentation from the app manufacturer to see if there has been a software update released that fixes any known vulnerabilities.
• Search online for any known/high-profile vulnerabilities for this class of app. For example, a “class of apps” could involve all media players on a platform, and the so-called “Stagefright” bug was a generic attack on a common Android media library, and had effects on many video apps.
• For each applicable CVE, identify whether the CVE's proof of concept code is posted publicly. Usually this will be linked from the CVE notice, or the documentation of the vulnerability linked from the CVE page.
• If the original proof of concept code for the CVE you are testing against is available, use it to test the app for the issue defined in the vulnerability notice.
• If the CVE’s original proof of concept code is not available, devise custom code based on the detailed description of the vulnerability either in the CVE itself or in the reference linked from the CVE page, to test the browser for the issue identified in the vulnerability notice.
• If the app proves to no longer be vulnerable to known and published vulnerabilities, mark PASS.
• If after running provided or custom proof of concept code it is identified that the app has any of the following, mark FAIL:
  • Unpatched known vulnerabilities specific to the application.
  • Unpatched known vulnerabilities specific to its app class.

Connected Devices

• Note: Where it is difficult to extract the firmware from a device for testing, there are more complex ways in which testers can attempt to extract the embedded codebase from a running chip, however these methods require specialized, expensive equipment that may not be available to non-expert testers. For our purposes we have not pursued the issue for this methodology. We will update this description if we have greater success when we test additional products.
• Establish whether a connected device has software that runs on the device itself. Often this is the code that controls the actual non-internet facing part of the device, for example the code that actually interacts with a physical lock, or turns off a water faucet. These embedded devices use low-level processors where the device’s operating code is written directly to a part of the chip—the product’s firmware.
• Retrieving firmware from a device:
  • Sometimes this step will not be possible, in many cases these types of difficulties indicate code quality (it is more secure against tampering).
  • Check the manufacturer's site. Often an IoT device will not be self updating and will require a manual update, which requires downloading a firmware package. Failing that, firmware can be pulled from the chip itself.
• If you can find the firmware, attempt to identify what language it is written in, and any other details that can be used as keywords when searching for relevant CVEs.
• Search the CVE database using information gained about the firmware as keywords and identify existing CVEs for the connected device firmware.
• For each applicable CVE, identify whether the CVE's proof of concept code is posted publicly. Usually this will be linked from the CVE notice, or the documentation of the vulnerability linked from the CVE page.
• If the original proof of concept code for the CVE you are testing against is available, use it to test the app for the issue defined in the vulnerability notice.
• If the CVE’s original proof of concept code is not available, devise custom code based on the detailed description of the vulnerability either in the CVE itself or in the reference linked from the CVE page, to test the browser for the issue identified in the vulnerability notice.
• If you can find and test the firmware, and it is not flagged as susceptible to know exploits, mark PASS.
• If after running provided or custom proof of concept code it is identified that the firmware has unpatched known vulnerabilities, mark FAIL.