Add iOS Support for sherpa-onnx-go

[JatosDetking]

Description:
I am currently encountering an issue while attempting to build the sherpa-onnx-go library for iOS. The build process fails with the following error message:

bash
Copy code
(.venv) aleksandarsvinarov@Aleksandars-MacBook-Pro test % GOOS=ios GOARCH=arm64 CGO_ENABLED=1 CC=$(go env GOROOT)/misc/ios/clangwrap.sh go build -ldflags -w -v -tags ios -o "IOS.a" -buildmode=c-archive

package test
imports github.com/k2-fsa/sherpa-onnx-go/sherpa_onnx: build constraints exclude all Go files in /Users/aleksandarsvinarov/go/pkg/mod/github.com/k2-fsa/[email protected]/sherpa_onnx
Steps to Reproduce:
Set up the environment:
sh
Copy code
(.venv) aleksandarsvinarov@Aleksandars-MacBook-Pro test %
Run the following build command:
sh
Copy code
GOOS=ios GOARCH=arm64 CGO_ENABLED=1 CC=$(go env GOROOT)/misc/ios/clangwrap.sh go build -ldflags -w -v -tags ios -o "IOS.a" -buildmode=c-archive
Expected Behavior:
The library should build successfully for the iOS target architecture.

Actual Behavior:
The build process fails with the message:

bash
Copy code
package test
imports github.com/k2-fsa/sherpa-onnx-go/sherpa_onnx: build constraints exclude all Go files in /Users/aleksandarsvinarov/go/pkg/mod/github.com/k2-fsa/[email protected]/sherpa_onnx
Environment:
macOS (version details)
Go version: (version details)
sherpa-onnx-go version: v1.8.7
Additional Information:
It appears that there may be build constraints within the sherpa-onnx-go package that exclude files necessary for the iOS build. Any guidance or modifications to support iOS would be greatly appreciated.

Thank you for your attention to this issue.

[csukuangfj]

Please follow the instructions below strictly

1. Build sherpa-onnx for iOS

# Assume you put sherpa-onnx inside /tmp
# You can change it to any directory you want
cd /tmp

git clone https://github.com/k2-fsa/sherpa-onnx
cd sherpa-onnx
./build-ios.sh

cd scripts/go/_internal
ln -s /tmp/sherpa-onnx/build-ios ./

2. Create build_ios.go

cd /tmp/sherpa-onnx/scripts/go/_internal
touch build_ios.go

and put the following content inside build_ios.go:

//go:build darwin && arm64 && ios

package sherpa_onnx

// #cgo LDFLAGS: -L ${SRCDIR}/build-ios/sherpa-onnx.xcframework/ios-arm64 -L ${SRCDIR}/build-ios/ios-onnxruntime/onnxruntime.xcframework/ios-arm64 -lsherpa-onnx -lonnxruntime -Wl,-rpath,${SRCDIR}/lib/aarch64-apple-darwin
import "C"

3. Ready to build your go program

Suppose that you want to build
https://github.com/k2-fsa/sherpa-onnx/tree/master/scripts/go/_internal/streaming-decode-files

cd /tmp/sherpa-onnx/scripts/go/_internal/streaming-decode-files
go mod tidy

GOOS=ios GOARCH=arm64 CGO_ENABLED=1 CC=$(go env GOROOT)/misc/ios/clangwrap.sh go build -ldflags -w -v -tags ios -o "IOS.a" -buildmode=c-archive

In the end, you should get the following file

(py38) fangjuns-MacBook-Pro:streaming-decode-files fangjun$ ls -lh IOS.a
-rw-r--r--  1 fangjun  staff   2.3M Jul 16 21:42 IOS.a

(py38) fangjuns-MacBook-Pro:streaming-decode-files fangjun$ ar t ./IOS.a
__.SYMDEF SORTED
go.o
000000.o
000001.o
000002.o
000003.o
000004.o
000005.o
000006.o
000007.o
000008.o
000009.o
000010.o
000011.o
000012.o

(py38) fangjuns-MacBook-Pro:streaming-decode-files fangjun$ ar x ./IOS.a
(py38) fangjuns-MacBook-Pro:streaming-decode-files fangjun$ file *.o
000000.o: Mach-O 64-bit object arm64
000001.o: Mach-O 64-bit object arm64
000002.o: Mach-O 64-bit object arm64
000003.o: Mach-O 64-bit object arm64
000004.o: Mach-O 64-bit object arm64
000005.o: Mach-O 64-bit object arm64
000006.o: Mach-O 64-bit object arm64
000007.o: Mach-O 64-bit object arm64
000008.o: Mach-O 64-bit object arm64
000009.o: Mach-O 64-bit object arm64
000010.o: Mach-O 64-bit object arm64
000011.o: Mach-O 64-bit object arm64
000012.o: Mach-O 64-bit object arm64
go.o:     Mach-O 64-bit object arm64

By the way, could you tell us how to use IOS.a?

[JatosDetking]

I'm currently working on a project that involves integrating voice activity detection (VAD) using Go and Swift. Below is a sample of my Go code for the VAD functionality:

package main

import (
"C"
"bytes"
"encoding/binary"
"log"
"os"
"strings"

"streaming-decode-files/speexdsp"

sherpa "github.com/k2-fsa/sherpa-onnx-go/sherpa_onnx"
flag "github.com/spf13/pflag"
"github.com/youpy/go-wav"

)
import (
"fmt"

"github.com/gordonklaus/portaudio"

)

//export testVad
func testVad() {
log.SetFlags(log.LstdFlags | log.Lmicroseconds)

config := sherpa.VadModelConfig{}

// Please download silero_vad.onnx from
// https://github.com/snakers4/silero-vad/raw/master/src/silero_vad/data/silero_vad.onnx

config.SileroVad.Model = "./silero_vad.onnx"
config.SileroVad.Threshold = 0.5
config.SileroVad.MinSilenceDuration = 0.04
config.SileroVad.MinSpeechDuration = 0.04
config.SileroVad.WindowSize = 512
config.SampleRate = 16000
config.NumThreads = 1
config.Provider = "cpu"

var bufferSizeInSeconds float32 = 5

vad := sherpa.NewVoiceActivityDetector(&config, bufferSizeInSeconds)

err := portaudio.Initialize()
if err != nil {
	log.Fatalf("Failed to initialize PortAudio: %v", err)
}

// resampler, err := speexdsp.NewResampler(1, 48000, 16000, 5)
if err != nil {
	log.Fatalf("Failed to initialize resampler: %v", err)
}

stream, err := portaudio.OpenDefaultStream(1, 1, 48000, 1920, func(in, out []int16) {
	noice := make([]int16, 1920, 1920)
	copy(out, noice)

	// start := time.Now()

	outputFloat, err := speexdsp.ResampleSpeex(in, 48000, 16000)
	if err != nil {
		log.Fatal(err)
	}

	vad.AcceptWaveform(outputFloat)

	if vad.IsSpeech() {
		// duration := time.Since(start)
		// fmt.Printf("Execution time is: %v\n", duration)
	}

	for !vad.IsEmpty() {
		speechSegment := vad.Front()
		vad.Pop()

		duration := float32(len(speechSegment.Samples)) / float32(config.SampleRate)
		fmt.Println("Time:", duration, "s")
	}
})

if err != nil {
	log.Fatal(err)
}

err = stream.Start()
if err != nil {
	log.Fatal(err)
}
fmt.Println("playAndRecordsPortAudioVAD")

}
When I import the IOS.a library into my Swift application, I encounter several errors.

[csukuangfj]

When I import the IOS.a library into my Swift application, I encounter several errors.

You need to link with

-lc++

[csukuangfj]

Please have a look at
https://k2-fsa.github.io/sherpa/ncnn/ios/for-the-more-curious-swift.html

Even if it is for sherpa-ncnn,you can just change ncnn to onnx and everything applies also to sherpa-onnx.

[csukuangfj]

@JatosDetking Does it work for you now?

[JatosDetking]

Yes thanks for IOS works. Can you help me with MACOS form swift build from arm64?

[csukuangfj]

Can you help me with MACOS form swift build from arm64?

Could you describe in detail the issue you have?

Please post the complete logs and commands.