Best Build Practices

Criteria: The software was built and developed according to the industry's best practices for security.

See this test in action:

• Smart lock
• Smart pressure cooker
• Smart baby monitor

Notes:

• This test is only considering the Android app that is used to interact with the product. We made this choice due to the open nature of Android, and the wide availability of free tools for inspecting Android app code.
• These indicators speak to quality and craftsmanship that goes into the software's coding. Namely, in making sure that the code follows known best practices around secure coding. The procedural overview for these indicators however suggest looking for things that are not an exact fit for analyzing mobile app code. Mainly, since Android apps are written in Java, which handles memory descriptions limits and cleaning up memory allocation (garbage collection), looking for things like how the code handles memory is less of concern than other aspects of good coding. However, if you are evaluating a program or app written in a memory unsafe language (one in which you must do your own memory management) such as C or C++, you absolutely should check the memory handling of that code.
• Many Internet of Things devices have software that runs on the device itself, often this is the code that controls the actual non-internet facing part of the "thing" (i.e. the code that actually interacts with a physical lock, or turns off a water faucet). These embedded devices use low-level processors where the device’s operating code is written directly to a part of the chip. You should endeavor to get a copy of this code. In testing the initial IoT device for this project, we made several different attempts to obtain it, but were ultimately unable to gain access. In many cases these types of difficulties indicate code quality (it is more secure against tampering). That said, there are more complex ways in which we could have attempted to extract the embedded codebase from a running chip; because these methods would have required specialized, expensive equipment, we have not pursued the issue for this methodology. We will update this description if we have greater success when we test additional products.
• For testing Android apps, you will need a copy of the app’s package file (APK). For downloading APK files, we use gplaycli, which is a Python tool for searching, downloading, and updating APK files from the official Google Playstore. Information on downloading and using gplaycli can be found on the project's GitHub page: https://github.com/matlink/gplaycli.
• In Android, apps must request permission to access data, or use hardware features of a device. Common examples of “permissions” include an app requesting access to a user’s contacts, call status, or use of the camera. In addition to requesting permission to access data and features from other parts of the system, an app may set its own permissions that define how other apps can access the data it keeps and the features it provides.

Methodology for Assessing Each Indicator

1) The product was built with effectively implemented safety features.

• Note: If possible to obtain firmware, or other SoC code, it should be analyzed as part of this indicator. This methodology currently only focuses on Android app analysis.
• Note: This indicator requires three separate elements that capture various aspects of good build practice, and therefore there are three opportunities for pass/fail.

Best build practice 1

• Obtain a copy of the Android app for the product using gplaycli.
• Obtain and install the Android Studio IDE from the Android project.
• In Android Studio, select "profile or debug APK” and select the downloaded APK file.
• In Android Studio Select the APK in the project tree window, and then select the "analyze" menu (either at the top, or with a right click), and select “inspect code.”
• The code inspection tool runs many tests against the entire project (APK), but we are primarily interested in the security warnings. To find those in the result of the scan go to "Menu -> Android -> Lint -> Security."
• If the app passes all standard security checks, mark PASS.
• If the app fails security linting mark FAIL.

Best build practice 2

• Inspect the “AndroidManifest.xml” file to see if signature-based permissions are enabled (e.g. android:protectionLevel="signature" is used in permission stanzas).
• If the app uses signature-based permissions, or does not require any special permissions, mark PASS.
• If the app requires special permissions, but does not use signature-based permissions mark FAIL.

Best build practice 3

• Inspect the “AndroidManifest.xml” file to see if data from the app can be sent to different apps. This will likely be flagged by the linter, but can also be checked by seeing if provider definitions set android:exported="false". It is also important to note whether or not the app may need to share data with another provider in order to function as intended.
• If the app protects data from being exported to other apps, mark PASS.
• If the app requires some data to be shared, but protects other data, mark PASS.
• If the app does not block data from being shared, and it is not necessary to function, mark FAIL.

2) The software does not make use of unsafe functions or libraries.

• Note: If possible to obtain firmware, or other SoC code, it should be analyzed as part of this indicator. This methodology currently only focuses on Android app analysis.
• Obtain a copy of the Android app for the product using gplaycli.
• Obtain and install the Android Studio IDE from the Android project.
• In Android Studio, select "Profile or debug APK” and select the downloaded APK file.
• Pull out data from the binary that speaks to coding hygiene.
  • Run the Android Studio code Linter, and note the number of errors and warnings in the “Corrections” section (“Menu -> Android -> Lint -> Corrections).
  • Also examine the results in the “General” section, and note the number of syntax errors, and other warnings.
  • If linting the app generates no errors or warnings for correctness, mark PASS.
  • If linting the app generates warnings, but no errors, mark PARTIAL PASS.
  • If linting the app generates errors for correctness and/or syntax errors, mark FAIL.

3) The software is not overly complex.

• Note: If possible to obtain firmware, or other SoC code, it should be analyzed as part of this indicator. This methodology currently only focuses on Android app analysis. While this indicator is geared toward keeping code lean, and thus easier to review, it may be the case that an is designed for a family of products, rather than the specific product being tested. In these cases the codebase may contain libraries and functionality that are not at all needed for basic functioning of the product, but are needed for other products in the family, for example a smart lock may have a companion smart doorbell, and the same app is used to interact with both devices. It is our evaluation that this approach makes sense, and it is likely better to have developers’ attention being spent on maintaining a single app well, rather than several similar apps tailored toward limited run products.
• Pull out data from the binary that speaks to code complexity.
  • Obtain a copy of the Android for the product using gplaycli.
  • Obtain and install the Android Studio IDE from the Android project.
  • In Android Studio, select "Profile or debug APK” and select the downloaded file.
  • In Android Studio a Java class tree can be found on the left side of the window (of a default configuration). Examine the class tree and look for how many there are, including those from third-party libraries, and whether those classes are needed for the functionality of the device being tested.
  • If there are not extraneous classes in the code, mark PASS.
  • If there is some extraneous code to support a sibling/variant product, mark PASS.
  • If there are unneeded libraries or classes, mark FAIL.