Add Action to publish to crates.io (#40)

mandiant · Dec 16, 2024 · ec1e17d · ec1e17d
1 parent 3f42f10
commit ec1e17d
Show file tree

Hide file tree

Showing 3 changed files with 88 additions and 87 deletions.
diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml
@@ -0,0 +1,19 @@
+on:
+  push:
+    tags:
+      - v[0-9]+.*
+
+name: Publish
+
+jobs:
+  crates_publish:
+    name: Publish (crates.io)
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: dtolnay/rust-toolchain@stable
+
+      - name: Logon to Crates
+        run: cargo login ${{ secrets.CRATES_IO_API_TOKEN }}
+      - name: "Publish"
+        run: cargo publish
diff --git a/Cargo.toml b/Cargo.toml
@@ -2,27 +2,31 @@
 name = "macos-unifiedlogs"
 version = "0.1.0"
 edition = "2021"
+license = "Apache-2.0"
+repository = "https://github.com/mandiant/macos-unifiedlogs"
+description = "A library to help parse macOS UnifiedLogs"
+keywords = ["forensics", "macOS", "unifiedlog"]
 
 # See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
 
 [dependencies]
 nom = "7.1.3"
-serde_json = "1.0.128"
-serde = { version = "1.0.210", features = ["derive"] }
+serde_json = "1.0.133"
+serde = { version = "1.0.215", features = ["derive"] }
 log = "0.4.22"
 lz4_flex = "0.11.3"
 byteorder = "1.5.0"
 plist = "1.7.0"
-regex = "1.10.6"
+regex = "1.11.1"
 base64 = "0.22.1"
 chrono = "0.4.38"
 
 [dev-dependencies]
 simplelog = "0.12.2"
-csv = "1.3.0"
+csv = "1.3.1"
 chrono = "0.4.38"
 criterion = "0.5.1"
-anyhow = "1.0.93"
+anyhow = "1.0.94"
 
 [[bench]]
 name = "high_sierra_benchmark"

diff --git a/README.md b/README.md
@@ -1,94 +1,72 @@
 # macos-unifiedlogs
-A simple Rust library that can help parse the macOS Unified Log files.  
 
-Unified Logs were introduced in macOS version 10.12 (Sierra, 2016). Part of Apple's goal to create a unified log format for all Apple products. They exist on macOS, iOS, watchOS, tvOS.
-The Unified Logs replace many of the old log formats Apple used. This simple library can be used to parse files.  
+A simple Rust library that can help parse the macOS Unified Log files.
+
+Unified Logs were introduced in macOS version 10.12 (Sierra, 2016). Part of
+Apple's goal to create a unified log format for all Apple products. They exist
+on macOS, iOS, watchOS, tvOS. The Unified Logs replace many of the old log
+formats Apple used. This library can be used to parse these log files.\
 Data that is currently extracted includes:
-* Process ID
-* Thread ID
-* Activity ID
-* Log Message
-* Timestamp (Intel and ARM supported)
-* Effective User ID (EUID)
-* Log Type
-* Event Type
-* Library
-* Subsystem
-* Category
-* Process
-* Raw message - Message extracted from UUID file
-* Message entries - Message parts from tracev3 file. Combines with Raw message to get the Log Message
-* Library UUID
-* Process UUID
-* Boot UUID
-* Timezone
+
+- Process ID
+- Thread ID
+- Activity ID
+- Log Message
+- Timestamp (Intel and ARM supported)
+- Effective User ID (EUID)
+- Log Type
+- Event Type
+- Library
+- Subsystem
+- Category
+- Process
+- Raw message - Message extracted from UUID file
+- Message entries - Message parts from tracev3 file. Combines with Raw message
+  to get the Log Message
+- Library UUID
+- Process UUID
+- Boot UUID
+- Timezone
 
 ## Running
-Three (3) simple example binaries are available in `examples`.  
-* `unifiedlog_parser` - Can parse all logs into a single CSV file. It can also be run on a live system. The resulting CSV file will likely be quite large 
-* `unifiedlog_parser_json` - Can parse all logs into JSON files. It can also be run on a live system. Each log file (tracev3 file) will correspond to a single JSON file. Depending on the logs, hundreds of JSON files may get created
-* `parse_tracev3` - Can parse a single tracev3 file without any timesync or uuidtext files, to a JSON file. However, without the uuidtext or timesync files the resulting JSON file will be heavily incomplete.  
 
-See `RUNNING.md` for overview of running the example binaries
-## Using as Library
-If you want to import this project into a Rust application add the following to you `Cargo.toml` file
-```
-macos-unifiedlogs = {git = "https://github.com/mandiant/macos-UnifiedLogs"}
-```
-If you want to pin to a specific commit
-```
-macos-unifiedlogs = {git = "https://github.com/mandiant/macos-UnifiedLogs", rev = "commit hash"}
-```
-See `Library.md` for overview of how to use the library. Simple example projects are also available to review and use
-## Status
-This library has been heavily tested on log data from macOS Sierra (10.12.5) to Monterey (12).  
-Its been tested against 100+ million log entries. However, due the complexity of the Unified Log format there are some limitations:
-1. No printf style error code lookup support. This library does not do any error code lookups for log messages. The native `log` command on macOS supports error code lookups when it encounters printf style `%m` messages.  
-    An example base log messsage: 'Failed to open file, error: %m'  
-    a. This Library outputs:
-    ```
-      Failed to open file, error: 1
-    ```
-    b. The macOS Log command outputs:
-    ```
-      Failed to open file, error: no such file or directory
-    ```
-    Here the error code 1 gets translated to the error string message
+Four example binaries are available.
+
+- `unifiedlog_parser` - Can parse all logs into a single CSV file. It can also
+  be run on a live system. The resulting CSV file will likely be quite large
+- `unifiedlog_iterator` - Can parse all logs into a single CSV file using an
+  iterator. May use less memory than the other example files. It can also be run
+  on a live system. The resulting CSV file will likely be quite large
+- `unifiedlog_parser_json` - Can parse all logs into JSON files. It can also be
+  run on a live system. Each log file will be parsed to a single JSON file.
+- `parse_tracev3` - Can parse a single tracev3 file without any timesync or
+  uuidtext files to JSON. However, without the uuidtext or timesync files the
+  resulting JSON file will be incomplete.
 
-2. No support for custom object decoders. The Unified Log format allows a developer to log abrirtary data to the logs. Apple also includes a handful of custom objects that developer can use to log raw data. An example list can be found in `man os_log`. However, it is not a complete list.
-```
-man os_log
-...
-     To format a log message, use a printf(3) format string.  You may also use the "%@" format specifier for use with Obj-C/CF/Swift objects, and %.*P which can be used to decode arbitrary binary data.  The logging system also supports custom decoding of values by denoting value types inline in the format %{value_type}d.  The built-in value type decoders are:
+## Limitations
 
-     Value type      Custom specifier         Example output
-     BOOL            %{BOOL}d                 YES
-     bool            %{bool}d                 true
-     darwin.errno    %{darwin.errno}d         [32: Broken pipe]
-     darwin.mode     %{darwin.mode}d          drwxr-xr-x
-     darwin.signal   %{darwin.signal}d        [sigsegv: Segmentation Fault]
-     time_t          %{time_t}d               2016-01-12 19:41:37
-     timeval         %{timeval}.*P            2016-01-12 19:41:37.774236
-     timespec        %{timespec}.*P           2016-01-12 19:41:37.2382382823
-     bytes           %{bytes}d                4.72 kB
-     iec-bytes       %{iec-bytes}d            4.61 KiB
-     bitrate         %{bitrate}d              123 kbps
-     iec-bitrate     %{iec-bitrate}d          118 Kibps
-     uuid_t          %{uuid_t}.16P            10742E39-0657-41F8-AB99-878C5EC2DCAA
-     sockaddr        %{network:sockaddr}.*P   fe80::f:86ff:fee9:5c16
-     in_addr         %{network:in_addr}d      127.0.0.1
-     in6_addr        %{network:in6_addr}.16P  fe80::f:86ff:fee9:5c16
-```
-Currently when the library encounters log messages that contain arbitrary binary data, it will base64 the data as a string.
-Support for custom decoders will hopefully be added in version 2 of the library.
+Its been tested against millions of log entries. However, due the complexity of
+the Unified Log format there are some limitations:
 
-3. No support for log messages that have custom object structures or protocol buffer data. 
-   Some logs contain binary plist files, custom object structures, or protocol buffer data. This library currently supports parsing binary plist data, but it does not support custom object structures, or protocol buffer data.
-   The custom object structures are similar to the custom decoders mentioned above. But there is no list of decoders  
-   Support for custom object structures will hopefully be added in version 2 of the library
+1. No printf style error code lookup support. This library does not do any error
+   code lookups for log messages. The native `log` command on macOS supports
+   error code lookups when it encounters printf style `%m` messages.\
+   An example base log messsage: 'Failed to open file, error: %m'\
+   a. This Library outputs:
+   ```
+   Failed to open file, error: 1
+   ```
+   b. The macOS Log command outputs:
+   ```
+   Failed to open file, error: no such file or directory
+   ```
+
+2. Support for most custom objects in log messages. However, some objects are
+   not supported. Unsupported objects will be base64 encoded
 
 # References
-https://github.com/ydkhatri/UnifiedLogReader  
-https://github.com/libyal/dtformats/blob/main/documentation/Apple%20Unified%20Logging%20and%20Activity%20Tracing%20formats.asciidoc  
-https://eclecticlight.co/2018/03/19/macos-unified-log-1-why-what-and-how/  
+
+https://github.com/ydkhatri/UnifiedLogReader\
+https://github.com/libyal/dtformats/blob/main/documentation/Apple%20Unified%20Logging%20and%20Activity%20Tracing%20formats.asciidoc\
+https://eclecticlight.co/2018/03/19/macos-unified-log-1-why-what-and-how/\
 https://www.crowdstrike.com/blog/how-to-leverage-apple-unified-log-for-incident-response/