Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

libjpeg-turbo dependency with MacPorts #22

Open
igorsyl opened this issue Jan 17, 2017 · 8 comments
Open

libjpeg-turbo dependency with MacPorts #22

igorsyl opened this issue Jan 17, 2017 · 8 comments

Comments

@igorsyl
Copy link

igorsyl commented Jan 17, 2017 •
edited
Loading

The build instructions use brew for dependencies. I can't build using MacPorts dependencies however:

MBP:~/go-vncdriver% sudo port install libjpeg-turbo
--->  Fetching archive for libjpeg-turbo
--->  Attempting to fetch libjpeg-turbo-1.5.0_0.darwin_16.x86_64.tbz2 from https://packages.macports.org/libjpeg-turbo
--->  Attempting to fetch libjpeg-turbo-1.5.0_0.darwin_16.x86_64.tbz2.rmd160 from https://packages.macports.org/libjpeg-turbo
--->  Installing libjpeg-turbo @1.5.0_0
--->  Activating libjpeg-turbo @1.5.0_0
--->  Cleaning libjpeg-turbo
--->  Updating database of binaries
--->  Scanning binaries for linking errors
--->  No broken files found.

MBP:~/go-vncdriver% python build.py
Could not find libjpeg. HINT: try 'sudo apt-get install libjpeg-turbo8-dev' on Ubuntu or 'brew install libjpeg-turbo' on OSX
@tlbtlbtlb
Copy link
Contributor

You can tell the build script where to find it by setting LIBJPG in the environment. Like,

LIBJPG=/opt/local/lib/libjpeg-turbo.dylib python build.py

I'm not sure where Macports puts the library, but

port contents libjpeg-turbo

should tell you.

@igorsyl
Copy link
Author

igorsyl commented Jan 17, 2017 •
edited
Loading

Thanks, is there a similar variable for include path?

MBP:~/go-vncdriver% sudo LIBJPG=/opt/local/lib/libturbojpeg.dylib python build.py
Env info:

export GOPATH='go-vncdriver/.build'
export GO15VENDOREXPERIMENT='1'
export CGO_CFLAGS='-I/opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/numpy/core/include -I/opt/local/Library/Frameworks/Python.framework/Versions/3.6/include/python3.6m'
export CGO_LDFLAGS='/opt/local/lib/libturbojpeg.dylib -undefined dynamic_lookup'

Running build with OpenGL rendering.
Building with OpenGL: GOPATH=go-vncdriver/.build go build -buildmode=c-shared -o go_vncdriver.so github.com/openai/go-vncdriver. (Set GO_VNCDRIVER_NOGL to build without OpenGL.)
# github.com/openai/go-vncdriver/vendor/github.com/pixiv/go-libjpeg/jpeg
.build/src/github.com/openai/go-vncdriver/vendor/github.com/pixiv/go-libjpeg/jpeg/compress.go:6:10: fatal error: 'jpeglib.h' file not found
#include "jpeglib.h"
         ^
1 error generated.

```

@tlbtlbtlb
Copy link
Contributor

In that case, run the export and build commands manually, adding in a -I /opt/local/include/ to the CGO_CFLAGS.

ie from the go-vncdriver directory,

cd ..
export GOPATH="$PWD/go-vncdriver/.build"
export GO15VENDOREXPERIMENT='1'
export CGO_CFLAGS='-I/opt/local/include -I/opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/numpy/core/include -I/opt/local/Library/Frameworks/Python.framework/Versions/3.6/include/python3.6m'
export CGO_LDFLAGS='/opt/local/lib/libturbojpeg.dylib -undefined dynamic_lookup'
go build -buildmode=c-shared -o go_vncdriver.so github.com/openai/go-vncdriver

@tlbtlbtlb
Copy link
Contributor

Beware that Tensorflow doesn't yet support python 3.6, so it may be difficult to run agents in this configuration. Python 3.5 is better supported.

@dogayuksel
Copy link

Thank you @tlbtlbtlb for great insights. I have a similar problem, would appreciate if you could give me a direction to investigate further.

I also couldn't get go-vncdriver to build using libjpeg-turbo installed through macports.
I am trying it in a conda environment. I had to deactivate jpeg library as it conflicts with libjpeg-turbo.

go version go1.6 darwin/amd64

My environment variables look like this:

GOPATH=/Users/doga/working/go-vncdriver/.build
CGO_LDFLAGS=/opt/local/lib/libturbojpeg.dylib -undefined dynamic_lookup
CGO_CFLAGS=-I/opt/local/include -I/Users/doga/.conda/envs/universe-starter-kit/lib/python3.5/site-packages/numpy/core/include/ -I/Users/doga/.conda/envs/universe-starter-agent/include/python3.5m

My build command looks like this:

(universe-starter-agent) doga:go-vncdriver $ go build -v -buildmode=c-shared -o go_vncdriver.so github.com/openai/go-vncdriver

And I get the following error as a result:

github.com/openai/go-vncdriver/vendor/github.com/juju/errors
github.com/openai/go-vncdriver/vendor/github.com/op/go-logging
github.com/openai/go-vncdriver/flexflate
github.com/openai/go-vncdriver/vendor/github.com/pixiv/go-libjpeg/rgb
github.com/openai/go-vncdriver/vendor/github.com/go-gl/gl/v2.1/gl
github.com/openai/go-vncdriver/vendor/github.com/go-gl/glfw/v3.2/glfw
github.com/openai/go-vncdriver/vendor/github.com/pixiv/go-libjpeg/jpeg
github.com/openai/go-vncdriver/flexzlib
# github.com/openai/go-vncdriver/vendor/github.com/pixiv/go-libjpeg/jpeg
ld: warning: directory not found for option '-L/usr/local/opt/jpeg-turbo/lib'
ld: library not found for -ljpeg
clang: error: linker command failed with exit code 1 (use -v to see invocation)
# github.com/openai/go-vncdriver/vendor/github.com/go-gl/glfw/v3.2/glfw
failed MSpanList_Insert 0xb60e30 0x1cad8886e84b 0x0 0x0
fatal error: MSpanList_Insert

runtime stack:
runtime.throw(0x46e5d0, 0x10)
	/usr/local/go/src/runtime/panic.go:530 +0x90
runtime.(*mSpanList).insert(0x982d48, 0xb60e30)
	/usr/local/go/src/runtime/mheap.go:933 +0x293
runtime.(*mheap).freeSpanLocked(0x982540, 0xb60e30, 0x100, 0x0)
	/usr/local/go/src/runtime/mheap.go:809 +0x4be
runtime.(*mheap).grow(0x982540, 0x8, 0x0)
	/usr/local/go/src/runtime/mheap.go:675 +0x2a0
runtime.(*mheap).allocSpanLocked(0x982540, 0x1, 0xb60db8)
	/usr/local/go/src/runtime/mheap.go:553 +0x4e3
runtime.(*mheap).alloc_m(0x982540, 0x1, 0xa, 0xb60db8)
	/usr/local/go/src/runtime/mheap.go:437 +0x119
runtime.(*mheap).alloc.func1()
	/usr/local/go/src/runtime/mheap.go:502 +0x41
runtime.systemstack(0xc820481e58)
	/usr/local/go/src/runtime/asm_amd64.s:307 +0xab
runtime.(*mheap).alloc(0x982540, 0x1, 0x1000000000a, 0xeb4f)
	/usr/local/go/src/runtime/mheap.go:503 +0x63
runtime.(*mcentral).grow(0x983c70, 0x0)
	/usr/local/go/src/runtime/mcentral.go:209 +0x93
runtime.(*mcentral).cacheSpan(0x983c70, 0xc82060c6f0)
	/usr/local/go/src/runtime/mcentral.go:89 +0x47d
runtime.(*mcache).refill(0xaf4000, 0xa, 0xc82007a0c0)
	/usr/local/go/src/runtime/mcache.go:119 +0xcc
runtime.mallocgc.func2()
	/usr/local/go/src/runtime/malloc.go:642 +0x2b
runtime.systemstack(0xc82002a000)
	/usr/local/go/src/runtime/asm_amd64.s:291 +0x79
runtime.mstart()
	/usr/local/go/src/runtime/proc.go:1048

goroutine 1 [running]:
runtime.systemstack_switch()
	/usr/local/go/src/runtime/asm_amd64.s:245 fp=0xc8207b91e0 sp=0xc8207b91d8
runtime.mallocgc(0x90, 0x437240, 0x0, 0x19b3f0)
	/usr/local/go/src/runtime/malloc.go:643 +0x869 fp=0xc8207b92b8 sp=0xc8207b91e0
runtime.newobject(0x437240, 0x0)
	/usr/local/go/src/runtime/malloc.go:781 +0x42 fp=0xc8207b92e0 sp=0xc8207b92b8
cmd/compile/internal/gc.Nod(0x7800f85d00, 0x0, 0x0, 0x0)
	/usr/local/go/src/cmd/compile/internal/gc/subr.go:337 +0x27 fp=0xc8207b9300 sp=0xc8207b92e0
cmd/compile/internal/gc.temp(0xc82093e4b0, 0xc8207b9730)
	/usr/local/go/src/cmd/compile/internal/gc/gen.go:615 +0x2e fp=0xc8207b9330 sp=0xc8207b9300
cmd/compile/internal/gc.ordertemp(0xc82093e4b0, 0xc8207b9730, 0x196600, 0x20a431)
	/usr/local/go/src/cmd/compile/internal/gc/order.go:64 +0x2c fp=0xc8207b9370 sp=0xc8207b9330
cmd/compile/internal/gc.ordercopyexpr(0xc8205f1680, 0xc82093e4b0, 0xc8207b9730, 0x0, 0x3df01)
	/usr/local/go/src/cmd/compile/internal/gc/order.go:95 +0x3d fp=0xc8207b93a8 sp=0xc8207b9370
cmd/compile/internal/gc.ordercall(0xc8205f1830, 0xc8207b9730)
	/usr/local/go/src/cmd/compile/internal/gc/order.go:399 +0x21f fp=0xc8207b9400 sp=0xc8207b93a8
cmd/compile/internal/gc.orderstmt(0xc8205f1830, 0xc8207b9730)
	/usr/local/go/src/cmd/compile/internal/gc/order.go:644 +0x1068 fp=0xc8207b9700 sp=0xc8207b9400
cmd/compile/internal/gc.orderstmtlist(0xc8205ee4c0, 0xc8207b9730)
	/usr/local/go/src/cmd/compile/internal/gc/order.go:268 +0x3a fp=0xc8207b9718 sp=0xc8207b9700
cmd/compile/internal/gc.orderblock(0xc8205f13c8)
	/usr/local/go/src/cmd/compile/internal/gc/order.go:277 +0x51 fp=0xc8207b9750 sp=0xc8207b9718
cmd/compile/internal/gc.order(0xc8205f13b0)
	/usr/local/go/src/cmd/compile/internal/gc/order.go:57 +0x123 fp=0xc8207b97d8 sp=0xc8207b9750
cmd/compile/internal/gc.compile(0xc8205f13b0)
	/usr/local/go/src/cmd/compile/internal/gc/pgen.go:403 +0x582 fp=0xc8207b9a48 sp=0xc8207b97d8
cmd/compile/internal/gc.funccompile(0xc8205f13b0)
	/usr/local/go/src/cmd/compile/internal/gc/dcl.go:1450 +0x1c0 fp=0xc8207b9ac0 sp=0xc8207b9a48
cmd/compile/internal/gc.Main()
	/usr/local/go/src/cmd/compile/internal/gc/lex.go:472 +0x2116 fp=0xc8207b9de0 sp=0xc8207b9ac0
cmd/compile/internal/amd64.Main()
	/usr/local/go/src/cmd/compile/internal/amd64/galign.go:127 +0x58d fp=0xc8207b9e48 sp=0xc8207b9de0
main.main()
	/usr/local/go/src/cmd/compile/main.go:32 +0x395 fp=0xc8207b9f20 sp=0xc8207b9e48
runtime.main()
	/usr/local/go/src/runtime/proc.go:188 +0x2b0 fp=0xc8207b9f70 sp=0xc8207b9f20
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1998 +0x1 fp=0xc8207b9f78 sp=0xc8207b9f70
# github.com/openai/go-vncdriver/vendor/github.com/go-gl/gl/v2.1/gl
failed MSpanList_Insert 0xf47708 0x1cad88a558da 0x0 0x0
fatal error: MSpanList_Insert

runtime stack:
runtime.throw(0x46e5d0, 0x10)
	/usr/local/go/src/runtime/panic.go:530 +0x90
runtime.(*mSpanList).insert(0x982d48, 0xf47708)
	/usr/local/go/src/runtime/mheap.go:933 +0x293
runtime.(*mheap).freeSpanLocked(0x982540, 0xf47708, 0x100, 0x0)
	/usr/local/go/src/runtime/mheap.go:809 +0x4be
runtime.(*mheap).grow(0x982540, 0x8, 0x0)
	/usr/local/go/src/runtime/mheap.go:675 +0x2a0
runtime.(*mheap).allocSpanLocked(0x982540, 0x1, 0xf47690)
	/usr/local/go/src/runtime/mheap.go:553 +0x4e3
runtime.(*mheap).alloc_m(0x982540, 0x1, 0xf, 0xf47690)
	/usr/local/go/src/runtime/mheap.go:437 +0x119
runtime.(*mheap).alloc.func1()
	/usr/local/go/src/runtime/mheap.go:502 +0x41
runtime.systemstack(0xc820475e58)
	/usr/local/go/src/runtime/asm_amd64.s:307 +0xab
runtime.(*mheap).alloc(0x982540, 0x1, 0x1000000000f, 0xeb4f)
	/usr/local/go/src/runtime/mheap.go:503 +0x63
runtime.(*mcentral).grow(0x983ea0, 0x0)
	/usr/local/go/src/runtime/mcentral.go:209 +0x93
runtime.(*mcentral).cacheSpan(0x983ea0, 0xf47690)
	/usr/local/go/src/runtime/mcentral.go:89 +0x47d
runtime.(*mcache).refill(0xaf4000, 0xf, 0xf47690)
	/usr/local/go/src/runtime/mcache.go:119 +0xcc
runtime.mallocgc.func2()
	/usr/local/go/src/runtime/malloc.go:642 +0x2b
runtime.systemstack(0xc82002a000)
	/usr/local/go/src/runtime/asm_amd64.s:291 +0x79
runtime.mstart()
	/usr/local/go/src/runtime/proc.go:1048

goroutine 1 [running]:
runtime.systemstack_switch()
	/usr/local/go/src/runtime/asm_amd64.s:245 fp=0xc826405108 sp=0xc826405100
runtime.mallocgc(0xe0, 0x425760, 0x1, 0xc8230afea0)
	/usr/local/go/src/runtime/malloc.go:643 +0x869 fp=0xc8264051e0 sp=0xc826405108
runtime.newobject(0x425760, 0xc8230afea0)
	/usr/local/go/src/runtime/malloc.go:781 +0x42 fp=0xc826405208 sp=0xc8264051e0
cmd/compile/internal/gc.regopt.func1(0x0, 0x0)
	/usr/local/go/src/cmd/compile/internal/gc/reg.go:1064 +0x2f fp=0xc826405220 sp=0xc826405208
cmd/compile/internal/gc.Flowstart(0xc8230a9e60, 0x4c3fd8, 0x20)
	/usr/local/go/src/cmd/compile/internal/gc/popt.go:288 +0x90c fp=0xc826405330 sp=0xc826405220
cmd/compile/internal/gc.regopt(0xc8230a9e60)
	/usr/local/go/src/cmd/compile/internal/gc/reg.go:1064 +0x3d0 fp=0xc8264057e8 sp=0xc826405330
cmd/compile/internal/gc.compile(0xc820b5b320)
	/usr/local/go/src/cmd/compile/internal/gc/pgen.go:521 +0xe6d fp=0xc826405a58 sp=0xc8264057e8
cmd/compile/internal/gc.funccompile(0xc820b5b320)
	/usr/local/go/src/cmd/compile/internal/gc/dcl.go:1450 +0x1c0 fp=0xc826405ad0 sp=0xc826405a58
cmd/compile/internal/gc.Main()
	/usr/local/go/src/cmd/compile/internal/gc/lex.go:472 +0x2116 fp=0xc826405df0 sp=0xc826405ad0
cmd/compile/internal/amd64.Main()
	/usr/local/go/src/cmd/compile/internal/amd64/galign.go:127 +0x58d fp=0xc826405e58 sp=0xc826405df0
main.main()
	/usr/local/go/src/cmd/compile/main.go:32 +0x395 fp=0xc826405f30 sp=0xc826405e58
runtime.main()
	/usr/local/go/src/runtime/proc.go:188 +0x2b0 fp=0xc826405f80 sp=0xc826405f30
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1998 +0x1 fp=0xc826405f88 sp=0xc826405f80

@tlbtlbtlb
Copy link
Contributor

tlbtlbtlb commented Feb 13, 2017 via email

@metaroboman
Copy link

I got an error like:

github.com/openai/go-vncdriver

go-vncdriver/.build/src/github.com/openai/go-vncdriver/main.go:5:10: fatal error: 'Python.h' file not found
#include <Python.h>
^~~~~~~~~~
1 error generated.

if I modify the GOPATH to where the .build folder is, then run:
go build -buildmode=c-shared -o go_vncdriver.so github.com/openai/go-vncdriver

@barracuda156
Copy link

I'm not sure where Macports puts the library, but

Add this to depends_lib-append in portfile:

path:include/turbojpeg.h:libjpeg-turbo

And possibly something like:

configure.args      --with-jpeg-includedir=${prefix}/include \
                    --with-jpeg-libdir=${prefix}/lib

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@tlbtlbtlb @igorsyl @dogayuksel @metaroboman @barracuda156