In comparison to Windows or even Linux, the public knowledge and guidance around digital forensics for Solaris / SunOS is rather thin.
During this engagement, we increased our knowledge of Solaris by a lot, investigating it for different attacker techniques. In this blog post, we want to share our experience investigating potential persistence using system processes, related to MITRE ATT&CK technique T1543.
As with most investigations, we had a lot of data at our hands. To reduce the amount of data to focus on, we took the following approaches:
First, we identified the install date of the machine and used information from the file system timeline to identify files that came from the install and files that were added or modified later. The latter category was then prioritized in the investigation.
One has to be aware that this method is vulnerable to timestomping attacks. Furthermore, older Solaris systems use a UFS filesystem, which has no birth timestamp to work with.
Second, we hashed configuration files and stacked the hashes (on a system itself as well as between systems). This allowed to reduce the number of files to investigate, as only one copy of a file per hash needed to be checked. The approach also highlighted files that had their default content modified.
Following are important data sources / forensic artifacts which we utilized for this aspect of the investigation.
While this is not a proper artifact, it is nonetheless an important information to collect for a machine.
Depending on the Solaris version, one can identify its installation date with at least one of the following ways:
The directory /var/sadm/system/logs/ contains log files from the installation. These log files have the install date in their name; as such, this info can be pulled from a file system timeline (used by us for Solaris 9 and Solaris 10)
Solaris still includes the traditional “init” system for running services. While this is deprecated for the modern Solaris versions, it is still available and usable.
The script files belonging to such services are stored in the directory /etc/init/.
Solaris also includes the “inetd” system for running services on demand. While this is deprecated for the modern Solaris versions, it is still available and usable.
The configuration file, showing which commands are run for which sockets, is stored at /etc/inetd.conf or /etc/inet/inetd.conf.
On modern Solaris versions (checked for Solaris 11.4), this configuration has no active entries by default.
Solaris also offers the “run command” system for running services based on the “run level” of the system. While this is deprecated for the modern Solaris versions, it is still available and usable.
The script files for the different “run levels” are stored in following directories:
Our practical experience showed that some duplication of scripts between different run levels can be expected.
Service Management Framework (SMF) is the currently default way of running services on Solaris. It came into use with Solaris 10 and is meant to replace the other systems.
Services are defined using XML files. These are normally stored under /lib/svc/manifest/ and /var/svc/manifest/. Each service definition states the executions for the service, the dependencies as well as environment settings. A service definition can directly define the commands to run for the service or reference a script file. These elements can be quickly found by filtering the configuration file for the string “exec=”. Service script files are normally stored under /lib/svc/method/.
For each service, a separate log file is kept under /var/svc/log/. This log shows service-specific messages, including when the service was started and stopped.
On a live system, the services can be enumerated using “svcs -a”. This will show running services, disabled services as well as services started through legacy systems. It also shows the start time of running services.
Solaris Zones are native solution for a kind of container, similar to FreeBSD jails. They were introduced with Solaris 10. Each zone on a system effectively represents another system that can be investigated.
Each zone creates its own file system hierarchy, anchored somewhere in the main file system hierarchy. This means that a file system timeline of the main system covers each zone stored on the system.
Zones are configured using XML files. Configuration templates are stored under /etc/zones/. Console output and system logs for each zone are stored under /var/log/zones/.
On a live system, the running zones can be checked with “zoneadm list -i -v“. The main system will also be shown with the output:
Solaris primarily uses SSH for remote access. It is vulnerable to the typical manipulation of the sshd configuration. The configuration is stored at /etc/ssh/sshd_config.
On modern versions of Solaris, the sshd service is managed through the SMF system, using the service configuration /lib/svc/manifest/network/ssh.xml and the service script /lib/svc/method/sshd.
All files and command output mentioned are plaintext and can be easily processed using typical command line utilities and ones favorite text editors.
We contributed to the tool Unix-like Artifacts Collector (UAC) so that the points mentioned above are covered by it. This enables investigation of these points regarding the technique T1543 using the smaller data package created by UAC.
Solaris offers multiple ways to manipulate system processes, including four different mechanisms of running services and a native solution for containers. By contributing to the UAC project, we make future investigations of Solaris more convenient.
We are looking forward to see what the community builds on top of the information provided here.
This blogpost has been conducted by Herbert Bärschneider and published on behalf of SEC Defence.