Skip to content

Commit

Permalink
Add WearOS demo for audio tagging (#777)
Browse files Browse the repository at this point in the history
  • Loading branch information
csukuangfj authored Apr 17, 2024
1 parent bcd9e48 commit 69440e4
Show file tree
Hide file tree
Showing 46 changed files with 1,114 additions and 16 deletions.
174 changes: 174 additions & 0 deletions .github/workflows/apk-audio-tagging-wearos.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,174 @@
name: apk-audio-tagging-wearos

on:
push:
tags:
- '*'

workflow_dispatch:

concurrency:
group: apk-audio-tagging-wearos-${{ github.ref }}
cancel-in-progress: true

permissions:
contents: write

jobs:
apk_audio_tagging_wearos:
if: github.repository_owner == 'csukuangfj' || github.repository_owner == 'k2-fsa'
runs-on: ${{ matrix.os }}
name: apk for WearOS ${{ matrix.index }}/${{ matrix.total }}
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest]
total: ["1"]
index: ["0"]

steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0

# https://github.com/actions/setup-java
- uses: actions/setup-java@v4
with:
distribution: 'temurin' # See 'Supported distributions' for available options
java-version: '21'

- name: ccache
uses: hendrikmuhs/[email protected]
with:
key: ${{ matrix.os }}-android

- name: Display NDK HOME
shell: bash
run: |
echo "ANDROID_NDK_LATEST_HOME: ${ANDROID_NDK_LATEST_HOME}"
ls -lh ${ANDROID_NDK_LATEST_HOME}
- name: Install Python dependencies
shell: bash
run: |
python3 -m pip install --upgrade pip jinja2
- name: Setup build tool version variable
shell: bash
run: |
echo "---"
ls -lh /usr/local/lib/android/
echo "---"
ls -lh /usr/local/lib/android/sdk
echo "---"
ls -lh /usr/local/lib/android/sdk/build-tools
echo "---"
BUILD_TOOL_VERSION=$(ls /usr/local/lib/android/sdk/build-tools/ | tail -n 1)
echo "BUILD_TOOL_VERSION=$BUILD_TOOL_VERSION" >> $GITHUB_ENV
echo "Last build tool version is: $BUILD_TOOL_VERSION"
- name: Generate build script
shell: bash
run: |
cd scripts/apk
total=${{ matrix.total }}
index=${{ matrix.index }}
./generate-audio-tagging-apk-script.py --total $total --index $index
chmod +x build-apk-audio-tagging-wearos.sh
mv -v ./build-apk-audio-tagging-wearos.sh ../..
- name: build APK
shell: bash
run: |
export CMAKE_CXX_COMPILER_LAUNCHER=ccache
export PATH="/usr/lib/ccache:/usr/local/opt/ccache/libexec:$PATH"
cmake --version
export ANDROID_NDK=$ANDROID_NDK_LATEST_HOME
./build-apk-audio-tagging-wearos.sh
- name: Display APK
shell: bash
run: |
ls -lh ./apks/
du -h -d1 .
# https://github.com/marketplace/actions/sign-android-release
- uses: r0adkll/sign-android-release@v1
name: Sign app APK
with:
releaseDirectory: ./apks
signingKeyBase64: ${{ secrets.ANDROID_SIGNING_KEY }}
alias: ${{ secrets.ANDROID_SIGNING_KEY_ALIAS }}
keyStorePassword: ${{ secrets.ANDROID_SIGNING_KEY_STORE_PASSWORD }}
env:
BUILD_TOOLS_VERSION: ${{ env.BUILD_TOOL_VERSION }}

- name: Display APK for audio tagging after signing
shell: bash
run: |
ls -lh ./apks/
du -h -d1 .
- name: Rename APK for audio tagging after signing
shell: bash
run: |
cd apks
rm -fv signingKey.jks
rm -fv *.apk.idsig
rm -fv *-aligned.apk
all_apks=$(ls -1 *-signed.apk)
echo "----"
echo $all_apks
echo "----"
for apk in ${all_apks[@]}; do
n=$(echo $apk | sed -e s/-signed//)
mv -v $apk $n
done
cd ..
ls -lh ./apks/
du -h -d1 .
- name: Display APK after rename
shell: bash
run: |
ls -lh ./apks/
du -h -d1 .
- name: Publish to huggingface
env:
HF_TOKEN: ${{ secrets.HF_TOKEN }}
uses: nick-fields/retry@v3
with:
max_attempts: 20
timeout_seconds: 200
shell: bash
command: |
git config --global user.email "[email protected]"
git config --global user.name "Fangjun Kuang"
rm -rf huggingface
export GIT_LFS_SKIP_SMUDGE=1
git clone https://huggingface.co/csukuangfj/sherpa-onnx-apk huggingface
cd huggingface
git fetch
git pull
git merge -m "merge remote" --ff origin main
mkdir -p audio-tagging-wearos
cp -v ../apks/*.apk ./audio-tagging-wearos/
git status
git lfs track "*.apk"
git add .
git commit -m "add more apks"
git push https://csukuangfj:[email protected]/csukuangfj/sherpa-onnx-apk main
2 changes: 1 addition & 1 deletion .github/workflows/test-build-wheel.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ jobs:
export PATH=/c/hostedtoolcache/windows/Python/3.9.13/x64/bin:$PATH
export PATH=/c/hostedtoolcache/windows/Python/3.10.11/x64/bin:$PATH
export PATH=/c/hostedtoolcache/windows/Python/3.11.9/x64/bin:$PATH
export PATH=/c/hostedtoolcache/windows/Python/3.12.2/x64/bin:$PATH
export PATH=/c/hostedtoolcache/windows/Python/3.12.3/x64/bin:$PATH
which sherpa-onnx
sherpa-onnx --help
2 changes: 1 addition & 1 deletion .github/workflows/test-pip-install.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ jobs:
export PATH=/c/hostedtoolcache/windows/Python/3.9.13/x64/bin:$PATH
export PATH=/c/hostedtoolcache/windows/Python/3.10.11/x64/bin:$PATH
export PATH=/c/hostedtoolcache/windows/Python/3.11.9/x64/bin:$PATH
export PATH=/c/hostedtoolcache/windows/Python/3.12.2/x64/bin:$PATH
export PATH=/c/hostedtoolcache/windows/Python/3.12.3/x64/bin:$PATH
sherpa-onnx --help
sherpa-onnx-keyword-spotter --help
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ jobs:
ls -lh
echo "----------"
wget https://github.com/k2-fsa/sherpa-onnx/releases/download/asr-models/sherpa-onnx-streaming-paraformer-bilingual-zh-en.tar.bz2
curl -SL -O https://github.com/k2-fsa/sherpa-onnx/releases/download/asr-models/sherpa-onnx-streaming-paraformer-bilingual-zh-en.tar.bz2
tar xvf sherpa-onnx-streaming-paraformer-bilingual-zh-en.tar.bz2
rm sherpa-onnx-streaming-paraformer-bilingual-zh-en.tar.bz2
Expand Down
4 changes: 4 additions & 0 deletions android/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,3 +26,7 @@ for usage.

- [SherpaOnnxWebSocket](./SherpaOnnxWebSocket) It shows how to write a websocket
client for the Python streaming websocket server.

- [SherpaOnnxAudioTagging](./SherpaOnnxAudioTagging) It shows how to use audio tagging.

- [SherpaOnnxAudioTaggingWearOS](./SherpaOnnxAudioTagging) It shows how to use audio tagging on WearOS.
Original file line number Diff line number Diff line change
Expand Up @@ -103,14 +103,14 @@ class AudioTagging(
//
// See also
// https://k2-fsa.github.io/sherpa/onnx/audio-tagging/
fun getAudioTaggingConfig(type: Int): AudioTaggingConfig? {
fun getAudioTaggingConfig(type: Int, numThreads: Int=1): AudioTaggingConfig? {
when (type) {
0 -> {
val modelDir = "sherpa-onnx-zipformer-small-audio-tagging-2024-04-15"
return AudioTaggingConfig(
model = AudioTaggingModelConfig(
zipformer = OfflineZipformerAudioTaggingModelConfig(model = "$modelDir/model.int8.onnx"),
numThreads = 1,
numThreads = numThreads,
debug = true,
),
labels = "$modelDir/class_labels_indices.csv",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ package com.k2fsa.sherpa.onnx.audio.tagging
import android.content.res.AssetManager
import android.util.Log
import com.k2fsa.sherpa.onnx.AudioTagging
import com.k2fsa.sherpa.onnx.audio.tagging.wear.os.presentation.TAG
import com.k2fsa.sherpa.onnx.getAudioTaggingConfig

object Tagger {
Expand All @@ -11,14 +12,14 @@ object Tagger {
get() {
return _tagger!!
}
fun initTagger(assetManager: AssetManager? = null) {
fun initTagger(assetManager: AssetManager? = null, numThreads: Int = 1) {
synchronized(this) {
if (_tagger != null) {
return
}

Log.i(TAG, "Initializing audio tagger")
val config = getAudioTaggingConfig(type = 0)!!
val config = getAudioTaggingConfig(type = 0, numThreads=numThreads)!!
_tagger = AudioTagging(assetManager, config)
}
}
Expand Down
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
<resources>
<string name="app_name">SherpaOnnxAudioTagging</string>
</resources>
<string name="app_name">Audio Tagging</string>
</resources>
15 changes: 15 additions & 0 deletions android/SherpaOnnxAudioTaggingWearOs/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
*.iml
.gradle
/local.properties
/.idea/caches
/.idea/libraries
/.idea/modules.xml
/.idea/workspace.xml
/.idea/navEditor.xml
/.idea/assetWizardSettings.xml
.DS_Store
/build
/captures
.externalNativeBuild
.cxx
local.properties
1 change: 1 addition & 0 deletions android/SherpaOnnxAudioTaggingWearOs/app/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
/build
66 changes: 66 additions & 0 deletions android/SherpaOnnxAudioTaggingWearOs/app/build.gradle.kts
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
plugins {
id("com.android.application")
id("org.jetbrains.kotlin.android")
}

android {
namespace = "com.k2fsa.sherpa.onnx.audio.tagging.wear.os"
compileSdk = 34

defaultConfig {
applicationId = "com.k2fsa.sherpa.onnx.audio.tagging.wear.os"
minSdk = 26
targetSdk = 34
versionCode = 1
versionName = "1.0"
vectorDrawables {
useSupportLibrary = true
}

}

buildTypes {
release {
isMinifyEnabled = false
proguardFiles(
getDefaultProguardFile("proguard-android-optimize.txt"),
"proguard-rules.pro"
)
}
}
compileOptions {
sourceCompatibility = JavaVersion.VERSION_1_8
targetCompatibility = JavaVersion.VERSION_1_8
}
kotlinOptions {
jvmTarget = "1.8"
}
buildFeatures {
compose = true
}
composeOptions {
kotlinCompilerExtensionVersion = "1.5.1"
}
packaging {
resources {
excludes += "/META-INF/{AL2.0,LGPL2.1}"
}
}
}

dependencies {

implementation("com.google.android.gms:play-services-wearable:18.1.0")
implementation(platform("androidx.compose:compose-bom:2023.08.00"))
implementation("androidx.compose.ui:ui")
implementation("androidx.compose.ui:ui-tooling-preview")
implementation("androidx.wear.compose:compose-material:1.1.2")
implementation("androidx.wear.compose:compose-foundation:1.1.2")
implementation("androidx.activity:activity-compose:1.7.2")
implementation("androidx.core:core-splashscreen:1.0.1")
implementation("androidx.compose.material3:material3-android:1.2.1")
androidTestImplementation(platform("androidx.compose:compose-bom:2023.08.00"))
androidTestImplementation("androidx.compose.ui:ui-test-junit4")
debugImplementation("androidx.compose.ui:ui-tooling")
debugImplementation("androidx.compose.ui:ui-test-manifest")
}
8 changes: 8 additions & 0 deletions android/SherpaOnnxAudioTaggingWearOs/app/lint.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
<?xml version="1.0" encoding="UTF-8"?>
<lint>
<!-- Ignore the IconLocation for the Tile preview images -->
<issue id="IconLocation">
<ignore path="res/drawable/tile_preview.png" />
<ignore path="res/drawable-round/tile_preview.png" />
</issue>
</lint>
21 changes: 21 additions & 0 deletions android/SherpaOnnxAudioTaggingWearOs/app/proguard-rules.pro
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Add project specific ProGuard rules here.
# You can control the set of applied configuration files using the
# proguardFiles setting in build.gradle.
#
# For more details, see
# http://developer.android.com/guide/developing/tools/proguard.html

# If your project uses WebView with JS, uncomment the following
# and specify the fully qualified class name to the JavaScript interface
# class:
#-keepclassmembers class fqcn.of.javascript.interface.for.webview {
# public *;
#}

# Uncomment this to preserve the line number information for
# debugging stack traces.
#-keepattributes SourceFile,LineNumberTable

# If you keep the line number information, uncomment this to
# hide the original source file name.
#-renamesourcefileattribute SourceFile
Loading

0 comments on commit 69440e4

Please sign in to comment.