Profile paths that contain %SystemDrive% require special handling for artifact filters

[joachimmetz]

Windows XP user profile paths contain %SystemDrive% this requires an additional expansion when used in artifact filters

[ant1]

With ppa:gift/stable and plaso 20190131

$ log2timeline.py --artifact_filters OperaHistory OperaHistory.plaso xp-tdungan-c-drive.E01 
2019-02-02 10:24:17,862 [INFO] (MainProcess) PID:3480 <data_location> Determined data location: /usr/share/plaso
2019-02-02 10:24:17,932 [INFO] (MainProcess) PID:3480 <artifact_definitions> Determined artifact definitions path: /usr/share/artifacts
Checking availability and versions of dependencies.
[OK]


Source path	: /home/xxx/xp-tdungan-c-drive.E01
Source type	: storage media image
Artifact filters	: OperaHistory
Processing time	: 00:00:00

Processing started.
2019-02-02 10:24:32,300 [INFO] (MainProcess) PID:3480 <engine> Preprocessing detected operating systems: Windows NT
2019-02-02 10:24:32,301 [INFO] (MainProcess) PID:3480 <extraction_tool> Parser filter expression set to: winxp,win7
2019-02-02 10:24:36,420 [WARNING] (MainProcess) PID:3480 <artifact_filters> The path filter must be defined as an absolute path: "%SystemDrive%\Documents and Settings\Tim/Library/Opera//global_history.dat"
2019-02-02 10:24:36,423 [WARNING] (MainProcess) PID:3480 <artifact_filters> The path filter must be defined as an absolute path: "%SystemDrive%\Documents and Settings\LocalService/Library/Opera//global_history.dat"
2019-02-02 10:24:36,423 [WARNING] (MainProcess) PID:3480 <artifact_filters> The path filter must be defined as an absolute path: "\WINDOWS\system32\config\systemprofile/Library/Opera//global_history.dat"
2019-02-02 10:24:36,423 [WARNING] (MainProcess) PID:3480 <artifact_filters> The path filter must be defined as an absolute path: "%SystemDrive%\Documents and Settings\RSydow/Library/Opera//global_history.dat"
2019-02-02 10:24:36,423 [WARNING] (MainProcess) PID:3480 <artifact_filters> The path filter must be defined as an absolute path: "%SystemDrive%\Documents and Settings\tdungan/Library/Opera//global_history.dat"
2019-02-02 10:24:36,423 [WARNING] (MainProcess) PID:3480 <artifact_filters> The path filter must be defined as an absolute path: "%SystemDrive%\Documents and Settings\NetworkService/Library/Opera//global_history.dat"
2019-02-02 10:24:36,424 [WARNING] (MainProcess) PID:3480 <artifact_filters> The path filter must be defined as an absolute path: "%SystemDrive%\Documents and Settings\SRL-Helpdesk/Library/Opera//global_history.dat"
2019-02-02 10:24:36,424 [WARNING] (MainProcess) PID:3480 <artifact_filters> The path filter must be defined as an absolute path: "%SystemDrive%\Documents and Settings\vibranium/Library/Opera//global_history.dat"
2019-02-02 10:24:36,425 [WARNING] (MainProcess) PID:3480 <log2timeline> Unable to build filter specification: Error processing filters, no valid specifications built.

[joachimmetz]

So the artifact definition OperaHistory does not use %SystemDrive% but %%users.appdata%%

---
name: OperaHistory
doc: Opera browser history (global_history.dat).
sources:
- type: FILE
  attributes: {paths: ['%%users.homedir%%/Library/Opera//global_history.dat']}
  supported_os: [Darwin]
- type: FILE
  attributes: {paths: ['%%users.homedir%%/.opera/global_history.dat']}
  supported_os: [Linux]
- type: FILE
  attributes:
    paths:
      - '%%users.appdata%%\Opera\Opera\global_history.dat'
      - '%%users.appdata%%\Opera Software\Opera Stable\History'
    separator: '\'
  supported_os: [Windows]
supported_os: [Windows,Darwin,Linux]
labels: [Browser]
urls: ['http://www.forensicswiki.org/wiki/Opera']

Which is a variant of issue #2326

[joachimmetz]

However expansion replaces this with '%SystemDrive%'

2019-02-02 15:43:12,317 [WARNING] (MainProcess) PID:5858 <artifact_filters> The path filter must be defined as an absolute path: "%SystemDrive%\Documents and Settings\LocalService/Library/Opera//global_history.dat"

Looks that double expansion and/or normalization is needed for user profile paths.

[joachimmetz]

After adding changes to strip %SystemDrive% #2328

2019-02-02 16:49:03,929 [WARNING] (MainProcess) PID:7701 <artifact_filters> The path filter must be defined as an absolute path: "\Documents and Settings\vibranium/Library/Opera//global_history.dat"
2019-02-02 16:49:03,929 [WARNING] (MainProcess) PID:7701 <log2timeline> Unable to build filter specification: Error processing filters, no valid specifications built.

Return has the wrong level of indentation https://github.com/log2timeline/plaso/blob/master/plaso/engine/artifact_filters.py#L239 (yeah Python)

After fixing that:

2019-02-02 17:26:20,262 [WARNING] (MainProcess) PID:10511 <artifact_filters> The path filter must be defined as an absolute path: "%%users.appdata%%\Opera\Opera\global_history.dat"

Need expansion of %%users.appdata%% to properly function

[joachimmetz]

Tracking changes for %%users.appdata%% expansion in #2326

[ant1]

It seems that %%environ_systemdrive%% needs special handling too

How to reproduce:

log2timeline.py --artifact_filters NTFSMFTFiles NTFSMFTFiles.plaso /media/FOR508/xp-tdungan-10.3.58.7/xp-tdungan-c-drive/xp-tdungan-c-drive.E01

[joachimmetz]

Ack, any variable that is used needs to be supported. I've created #2343 to track this.

Also artifacts could like use some enforcement to ensure this set of variables is known ForensicArtifacts/artifacts#311