Training LeNet on MNIST

[hpoit]

Hi! Need assistance...

include("/.../convert.jl")
ERROR: LoadError: AssertionError: ntoh(label_header[1]) == 2049
[inlined code] from /.../convert.jl:17
in anonymous at no file:0
in include at /Applications/Julia-0.4.3.app/Contents/Resources/julia/lib/julia/sys.dylib
in include_from_node1 at /Applications/Julia-0.4.3.app/Contents/Resources/julia/lib/julia/sys.dylib
while loading /.../convert.jl, in expression starting on line 11

[pluskid]

@tenthdimension Are you running on some big endian architectures? Or maybe your downloaded data gets corrupted?

[hpoit]

Hi Chiyuan. No, running on a little endian, x86 processor, and downloading the four files from http://yann.lecun.com/exdb/mnist/

[hpoit]

Chiyuan, where are these files supposed to be in for convert.jl to read them?

datasets = @compat(Dict("train" => ["train-labels-idx1-ubyte","train-images-idx3-ubyte"],
"test" => ["t10k-labels-idx1-ubyte","t10k-images-idx3-ubyte"]))

[pluskid]

@hpoit You need to run get-mnist.sh which will download the MNIST data and then run convert.jl to convert them into HDF5 format.

[hpoit]

I tried doing that. Let me finish a course and I'll get back to this shortly.

[hpoit]

I'm having trouble making convert.jl locate t10k-labels-idx1-ubyte.gz file (which is in my Downloads folder):

julia> include("/Users/Corvus/Google Drive/convert.jl")
ERROR: LoadError: SystemError: opening file t10k-labels-idx1-ubyte: No such file or directory
in open at /Applications/Julia-0.4.3.app/Contents/Resources/julia/lib/julia/sys.dylib
in open at iostream.jl:99
[inlined code] from /Users/Corvus/Google Drive/convert.jl:13
in anonymous at no file:0
in include at /Applications/Julia-0.4.3.app/Contents/Resources/julia/lib/julia/sys.dylib
in include_from_node1 at /Applications/Julia-0.4.3.app/Contents/Resources/julia/lib/julia/sys.dylib
while loading /Users/Corvus/Google Drive/convert.jl, in expression starting on line 11

[hpoit]

I think it worked-- I just changed the directory on convert.jl to my Downloads folder

include("/Users/Corvus/Google Drive/convert.jl")
Exporting 10000 digits of size 28 x 28
1:10000...
Exporting 60000 digits of size 28 x 28
1:60000...

[hpoit]

Now I got this

julia> include("/Users/Corvus/Google Drive/mnist.jl")
WARNING: Method definition info(Any...) in module Base at util.jl:334 overwritten in module Logging at /Users/Corvus/.julia/v0.4/Logging/src/Logging.jl:61.
WARNING: Method definition warn(Any...) in module Base at util.jl:364 overwritten in module Logging at /Users/Corvus/.julia/v0.4/Logging/src/Logging.jl:61.
Configuring Mocha...

• CUDA disabled by default
• Native Ext disabled by default
  Mocha configured, continue loading module...
  DefaultBackend = Mocha.CPUBackend
  19-Apr 18:11:24:INFO:root:Constructing net MNIST-train on Mocha.CPUBackend...
  19-Apr 18:11:24:INFO:root:Topological sorting 8 layers...
  19-Apr 18:11:24:INFO:root:Setup layers...
  HDF5-DIAG: Error detected in HDF5 (1.8.15-patch1) thread 0:
  #000: H5F.c line 439 in H5Fis_hdf5(): unable open file
  major: File accessibilty
  minor: Not an HDF5 file
  Accuracy computation #1: H5Fint.c line 554 in H5F_is_hdf5(): unable to open file
  major: Low-level I/O
  minor: Unable to initialize object
  Problem compiling logistic_loss CUDA implementation #2: H5FD.c line 993 in H5FD_open(): open failed
  major: Virtual File Layer
  minor: Unable to initialize object
  could not load module libcuda: dlopen(libcuda.dylib, 1): image not found #3: H5FDsec2.c line 343 in H5FD_sec2_open(): unable to open file: name = 'data/train.hdf5', errno = 2, error message = 'No such file or directory', flags = 0, o_flags = 0
  major: File accessibilty
  minor: Unable to open file
  ERROR: LoadError: Cannot access file data/train.hdf5
  in h5open at /Users/Corvus/.julia/v0.4/HDF5/src/plain.jl:547
  in h5open at /Users/Corvus/.julia/v0.4/HDF5/src/plain.jl:557
  in setup at /Users/Corvus/.julia/v0.4/Mocha/src/layers/async-hdf5-data.jl:50
  in setup at /Users/Corvus/.julia/v0.4/Mocha/src/layers.jl:112
  in call at /Users/Corvus/.julia/v0.4/Mocha/src/net.jl:227
  in include at /Applications/Julia-0.4.3.app/Contents/Resources/julia/lib/julia/sys.dylib
  in include_from_node1 at /Applications/Julia-0.4.3.app/Contents/Resources/julia/lib/julia/sys.dylib
  while loading /Users/Corvus/Google Drive/mnist.jl, in expression starting on line 18

line 18 being
net = Net("MNIST-train", backend, [data_layer, common_layers..., loss_layer])

[hpoit]

Is "MNIST-train" supposed to be .../train.hdf5? I tried changing it for this file directory but it didn't work, so I guess not.

[hpoit]

On http://mochajl.readthedocs.org/en/latest/tutorial/mnist.html

Then we will define a data layer, which reads the HDF5 file and provides input for the network:
data_layer = HDF5DataLayer(name="train-data", source="data/train.txt", batch_size=64, shuffle=true)

Why read from train.txt instead of train.hdf5? @staticfloat @uschmidt83

@pluskid I know you have a handful so attempting to distribute it a little.

[JobJob]

Hi, in the docs it says "Note the source is a simple text file that contains a list of real data files (in this case data/train.hdf5)"

From my brief reading of the above, you haven't used get-mnist.sh so you may have the hdf5 in directories where Mocha isn't expecting them. I think if you change the path(s) in that txt file to point to your hdf5 file(s), I expect it should work.

[hpoit]

I ran get-mnist.sh and generated train.hdf5 and test.hdf5 with convert.jl. It was all a directory issue.

Network constructed!

Objective completed. Thanks.