encapp: V1.18 build and updates to README

All changes in protobuf are now in the prebuild apk.
New tests definitions added and README updated.

Signed-off-by: JohanBlome <[email protected]>

README.dev.md

@@ -153,6 +153,8 @@ $ adb shell cat /sdcard/codecs.txt
 
 ## 3. Regression test
 
+**NOTE: this is currently not working properly, it will be adressed.**
+
 encapp_verify.py will run through the tests defined in 'tests/' folder and try to verify:
 * bitrate
 * key frames
@@ -178,3 +180,13 @@ $ encapp_verify.py -i /tmp/KristenAndSara_1280x720_60.yuv -is 1280x720 -if 30 -o
 ```
 
 For a raw input both input and output resolution and fps needs to be specified even if raw buffer will not allow scaling (which surface encoding does).
+
+
+## 4. System/Unit testing
+All current tests can be run with:
+python3 -m pytest ~/code/Github/encapp_master/encapp/scripts/tests/
+
+* The unit tests can be run without a device connected.
+* The system tests require a device connected. To pass all tests there needs to be a h264 encodera and decoder. It also needs to have surface textures available.
+
+

README.md

@@ -48,44 +48,29 @@ $ adb shell appops set --uid com.facebook.encapp MANAGE_EXTERNAL_STORAGE allow
 (2) run the list command:
 ```
 $ ./scripts/encapp.py list
-...
---- List of supported encoders  ---
-
-        MediaCodec {
-            name: OMX.google.h264.encoder
-            type {
-                mime_type: video/avc
-                max_supported_instances: 32
-                color {
-                    format: 2135033992
-                    name: COLOR_FormatYUV420Flexible
-                }
-                color {
-                    format: 19
-                    name: COLOR_FormatYUV420Planar
-                }
-                ...
-            }
-        }
+c2.exynos.h263.encoder
+c2.exynos.h264.encoder
+c2.exynos.hevc.encoder
+....
+```
 
-*******
---- List of supported decoders  ---
-
-        MediaCodec {
-            name: OMX.google.h264.decoder
-            type {
-                mime_type: video/avc
-                max_supported_instances: 32
-                color {
-                    format: 2135033992
-                    name: COLOR_FormatYUV420Flexible
-                }
-                ...
-            }
-        }
+By default is will show all encoders in a list.
+To see the deoders, use:
 
+
+```
+$ ./scripts/encapp.py list --decoders
+c2.exynos.h263.decoder
+c2.exynos.h264.decoder
+c2.exynos.h264.decoder.secure
+....
 ```
 
+By default will be a json compatible file in the current directory called, codecs_XXX.
+**Important** the result will be cached. To force refresh use the "-nc" or "--no-cache" argument.
+More infor can be obtained by increasing the information level or setting it to "-1".
+
+
 Note: The `scripts/encapp.py` scripts will install a prebuild apk before running the test. If you have several devices attached to your host, you can use either the "`ANDROID_SERIAL`" environment variable or the "`--serial <serial>`" CLI option.
 
 
@@ -120,12 +105,13 @@ In the case of surface encoder from raw the source should be nv21 regardless of
 
 Now run the h264 encoder (`OMX.google.h264.encoder`):
 ```
-$ ./scripts/encapp.py run tests/bitrate_buffer.pbtxt --local-workdir /tmp/test -e input.filepath /tmp/akiyo_qcif.y4m
+$ ./scripts/encapp.py run tests/bitrate_buffer.pbtxt --local-workdir /tmp/test [ -e input.filepath /tmp/akiyo_qcif.y4m ]
 ...
 results collect: ['/tmp/test/encapp_<uuid>.json']
 ```
 
 Note that the json file is not the only result of the experiment copied to the directory specified using "--local-workdir":
+Also, the [-e ...] is already defined in the test description, this would override the test specified input.
 ```
 $ ls -Flasg /tmp/test/
 11140K -rw-r--r--.   1 ... 11404800 Jan 18 11:28 akiyo_qcif.y4m.176x144.29.97.yuv420p.yuv
@@ -148,13 +134,13 @@ Files include:
 Note that the default encoder value is a Google-provided, software, h264 encoder (`OMX.google.h264.encoder`). If you want to test one of the encoders (from the "`list`" command output), use the CLI. For example, some Qualcomm devices offer an h264 HW encoder called "`OMX.qcom.video.encoder.avc`". In order to test it, use:
 
 ```
-$ ./scripts/encapp.py run tests/bitrate_surface.pbtxt --local-workdir /tmp/test -e input.filepath /tmp/akiyo_qcif.y4m --codec OMX.qcom.video.encoder.avc -e input.pix_fmt nv21
+$ ./scripts/encapp.py run tests/bitrate_surface.pbtxt --local-workdir /tmp/test --codec OMX.qcom.video.encoder.avc
 ...
 results collect: ['/tmp/test/encapp_<uuid>.json']
 ```
 
-Note that we are changing the test from "`tests/bitrate_buffer.pbtxt`" to "`tests/bitrate_surface.pbtxt`". This is due to Buffer mode for HW encoders being broken at the moment.
-
+Note that we are changing the test from "`tests/bitrate_buffer.pbtxt`" to "`tests/bitrate_surface.pbtxt`". This encoding will use a txture as input.
+Also we used a hw encoder. If for example running on a Pixel8/9 try: "-c c2.exynos.h264.encoder" instead.
 
 ## 3.2. HD Video Encoding
 
@@ -193,9 +179,12 @@ $ ls -Flasg /tmp/test/
 For usability reasons (allow testing with generic video files), encapp allows using an encoded video as a source, instead of raw (yuv) video. In this case, encapp will choose one of its decoders (hardware decoders are prioritized), and decode the video to raw (yuv) before testing the encoder.
 
 ```
-$ ./scripts/encapp.py run tests/bitrate_transcoder.pbtxt --local-workdir /tmp/test -e input.filepath /tmp/akiyo_qcif.mp4
-...
-results collect: ['/tmp/test/encapp_<uuid>.json']
+$ /scripts/encapp.py run gtests/bitrate_buffer_transcoder.pbtxt
+```
+
+Or using a surface texture:
+```
+$ /scripts/encapp.py run gtests/bitrate_surface_transcoder.pbtxt
 ```
 
 Note that, in this case, we get the encoded video (`encapp_<uuid>.mp4`) and the original source video (`akiyo_qcif.mp4`).
@@ -216,7 +205,7 @@ Encapp also allows visualizing the video being decoded. We can run the previous
 The parameter for showing the video is `show` in the `input` section.
 
 ```
-$ ./scripts/encapp.py run tests/bitrate_transcoder_show.pbtxt --local-workdir /tmp/test -e input.filepath /tmp/akiyo_qcif.mp4
+$ ./scripts/encapp.py run tests/bitrate_surface_transcoder_show.pbtxt --local-workdir /tmp/test -e input.filepath /tmp/akiyo_qcif.mp4
 ...
 results collect: ['/tmp/test/encapp_<uuid>.json']
 ```
@@ -435,6 +424,25 @@ test {
 }
 '''
 
+Command line argument wil have precedence.
+
+## 5.4 Test setup
+
+Many of the parameters available on cli are also available in the protobuf description in the "TestSetup" message.
+This can be used to create device specific pbtxt file so not command line typing is needed. E.g:
+```
+$ ./scripts/encapp,py run important_test.pbtxt my_special_device.pbtxt
+```
+Where the "my_special_device.pbtxt" can contain the 
+* serial number (serial)
+* device work dir (device_workdir)
+* local workdir (local_workdir)
+* separate sources (separate_sources)
+    - This setting will split any test containing multiple source video file to the video files are run sequentially and removed before next file, helping memory constrained devices.
+* mediastore
+    - where to store temporary files
+* source directory (source_dir)
+    - where to look for input file. This makes it possible to only have the filename and not the whole filepath in the "input.filepath" setting.
 
 
 
@@ -465,6 +473,30 @@ $ encapp_quality.py --header --media MEDIA_FOLDER $(encapp_search.py)
 Since the json file only contains the name of the source for an encoding the source folder needs to be provided.
 The output will be a csv file (default name is 'quality.csv') containing vmaf, ssim, psnr and other relevant properties.
 
+# 6.1 Output naming
+The default output naming is a uuid with not connection to the actual test being run.
+However, "Common.output_filename" can be defined with placeholders to force a different naming scheme.
+All settings can in theory be used, e.g.
+```
+...
+ common {
+        id: "bitrate_buffer"
+        description: "Verify encoding bitrate - buffer"
+        output_filename: "[input.filepath].[configure.bitrate].[xxxx]-[xx]"
+    }
+    configure {
+        codec: "OMX.google.h264.encoder"
+        bitrate: "100kbps,200kbps"
+    }
+
+```
+The 'X' will give a random hex value (0-f), but of course andy number of random numbers can be added. They can be used to minimize potential naming conflicts.
+
+```
+$./scripts/encapp.py tests/bitrate_buffer_naming.pbtxt
+ok: test id: "bitrate_buffer" run_id: akiyo_qcif.100kbps.1af5-c9 result: ok
+ok: test id: "bitrate_buffer" run_id: akiyo_qcif.200kbps.6809-90 result: ok
+
 
 # 7. Requirements
 

README.tests.md

@@ -178,6 +178,9 @@ List of valid keys:
     * but python world will allow time too
   * example: "100"  // wait until 100 frames have been processed (in a different encoder or decoder) before starting this encoder
 
+* "output_filename": str
+  *  The default output naming is a uuid with not connection to the actual test being run.However, "output_filename" can be defined with placeholders to force a different naming scheme.
+  * example: "output_filename: "[input.filepath].[configure.bitrate].[xxxx]-[xx]"
 
 ## 3.3. Input Information
 
@@ -218,6 +221,21 @@ List of valid keys:
     * python world will allow time units too
   * example: 300  // play 300 frames at the original file framerate, then quit
 
+* "pursuit": int
+  * encapp will try to start this many codecs with the same configuration.
+
+* "realtime":
+  * unless realtime is set or video is displayed, encoding will run as fast as possible.
+
+* "stoptime_sec":
+  * same as playout but in sec time instead
+
+* "show":
+  * if possible sow the vidoe on the device screen
+
+* "device_decode":
+  * decode a compressed file on device and not on host.
+
 Shortcuts (host side only):
 * "resolution": str
   * compat way to express both "width" and "height"
@@ -463,6 +481,32 @@ Example:
 
 This test will encode the "foo.yuv" file using the h264 encoder. Once it is finished, it will do the same with the "bar.yuv" file.
 
+## 3.8 Test definition
+The TestSetup message containes settings related to setup of the test and not the actual test itself and inlcudes the following:
+
+* "device_workdir"
+  * Where data can be stored on the device
+
+* "local_workdir"
+  * Where to store output from the test
+
+* "serial"
+  * Serial id of the device to be tested
+
+* "device_cmd";
+  * Default is adb, set it to idb for Apple
+
+* "run_cmd"
+  * If a different command is needed to start the device app it can be defined here e.g. "appXYZ -r " + "DEF.pbtxt. Needs to be self contained, i.e. paths etc, defined in the protobuf
+
+* separate_sources;
+  * This setting will split any test containing multiple source video file to the video files are run sequentially and removed before next file, helping memory constrained devices.
+
+* mediastore
+  * where to store temporary files
+
+* "source_dir":
+  * where to look for input file. This makes it possible to only have the filename and not the whole filepath in the "input.filepath" setting.
 
 # 4. References
 

app/build.gradle

@@ -9,7 +9,7 @@ android {
         minSdkVersion 27
         targetSdkVersion 31
         versionCode 1
-        versionName "1.17"
+        versionName "1.18"
         setProperty("archivesBaseName", applicationId + "-v" + versionName)
     }
     buildTypes {

scripts/encapp_tool/adb_cmds.py

@@ -108,8 +108,7 @@ def remove_file(serial: str, file: str, debug: int) -> None:
     """
     if USE_IDB:
         # remove the output
-        print(f"Removing {counter}/{len(output_files)}", end="\r")
-        cmd = f"idb file rm {location}/{file} --udid {serial}  --bundle-id {IDB_BUNDLE_ID}"
+        cmd = f"idb file rm {file} --udid {serial}  --bundle-id {IDB_BUNDLE_ID}"
         run_cmd(cmd, debug)
     else:
         # remove the output

tests/bitrate_buffer_naming.pbtxt

@@ -0,0 +1,14 @@
+test {
+    input {
+        filepath: "/tmp/akiyo_qcif.y4m"
+    }
+    common {
+        id: "bitrate_buffer"
+        description: "Verify encoding bitrate - buffer"
+        output_filename: "[input.filepath].[configure.bitrate].[XXXX]-[XX]"
+    }
+    configure {
+        codec: "OMX.google.h264.encoder"
+        bitrate: "100kbps,200kbps"
+    }
+}

tests/bitrate_buffer_transcoder.pbtxt

@@ -0,0 +1,14 @@
+test {
+    input {
+        filepath: "/tmp/akiyo_qcif.mp4"
+        device_decode: true,
+    }
+    common {
+        id: "bitrate_buffer_transcoder",
+        description: "Verify encoding bitrate - buffer transcoding"
+    }
+    configure {
+        codec: "OMX.google.h264.encoder"
+        bitrate: "200 kbps"
+    }
+}

tests/bitrate_surface_transcoder.pbtxt

@@ -0,0 +1,15 @@
+test {
+    input {
+        filepath: "/tmp/akiyo_qcif.mp4"
+        device_decode: true,
+    }
+    common {
+        id: "bitrate_transcoder_surface"
+        description: "Verify encoding bitrate - transcode usng surface"
+    }
+    configure {
+        codec: "OMX.google.h264.encoder"
+        bitrate: "200 kbps"
+        surface: true
+    }
+}