Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using songbird standalone #134

Open
coralzhang opened this issue Aug 5, 2020 · 5 comments
Open

Using songbird standalone #134

coralzhang opened this issue Aug 5, 2020 · 5 comments

Comments

@coralzhang
Copy link

coralzhang commented Aug 5, 2020

Hi,

I am using a Mac Catelina
This is what I did when installing

conda create -n songbird_env python=3.6 numpy=1.15.4 scikit-bio=0.5.5 seaborn pandas=0.23.4 -c conda-forge
source activate songbird_env
conda install tensorflow=1.30 tqdm nomkl
conda install biom-format h5py -c conda-forge
conda install jupyter notebook
conda install songbird -c conda-forge

I keep getting the following warnings when I run model as simple as this

songbird multinomial \
	--input-biom ./data/redsea.biom \
	--metadata-file ./data/redsea_metadata.txt \
	--formula "1" \
	--epochs 10000 \
	--differential-prior 0.5 \
	--training-column Testing \
	--summary-interval 1 \
	--summary-dir results-redsea/redsea-results-null

Below are warnings

WARNING:tensorflow:From /usr/local/anaconda3/envs/songbird_env/bin/songbird:145: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.

2020-08-05 13:59:42.889984: I tensorflow/core/platform/cpu_feature_guard.cc:145] This TensorFlow binary is optimized with Intel(R) MKL-DNN to use the following CPU instructions in performance critical operations: SSE4.1 SSE4.2 AVX AVX2 FMA
To enable them in non-MKL-DNN operations, rebuild TensorFlow with the appropriate compiler flags.
2020-08-05 13:59:42.890306: I tensorflow/core/common_runtime/process_util.cc:115] Creating new thread pool with default inter op setting: 8. Tune using inter_op_parallelism_threads for best performance.
WARNING:tensorflow:From /usr/local/anaconda3/envs/songbird_env/lib/python3.6/site-packages/songbird/multinomial.py:75: multinomial (from tensorflow.python.ops.random_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.random.categorical instead.
WARNING:tensorflow:From /usr/local/anaconda3/envs/songbird_env/lib/python3.6/site-packages/songbird/multinomial.py:86: The name tf.random_normal is deprecated. Please use tf.random.normal instead.

WARNING:tensorflow:From /usr/local/anaconda3/envs/songbird_env/lib/python3.6/site-packages/songbird/multinomial.py:91: Normal.init (from tensorflow.python.ops.distributions.normal) is deprecated and will be removed after 2019-01-01.
Instructions for updating:
The TensorFlow Distributions library has moved to TensorFlow Probability (https://github.com/tensorflow/probability). You should update all references to use tfp.distributions instead of tf.distributions.
WARNING:tensorflow:From /usr/local/anaconda3/envs/songbird_env/lib/python3.6/site-packages/tensorflow/python/ops/distributions/normal.py:160: Distribution.init (from tensorflow.python.ops.distributions.distribution) is deprecated and will be removed after 2019-01-01.
Instructions for updating:
The TensorFlow Distributions library has moved to TensorFlow Probability (https://github.com/tensorflow/probability). You should update all references to use tfp.distributions instead of tf.distributions.
WARNING:tensorflow:From /usr/local/anaconda3/envs/songbird_env/lib/python3.6/site-packages/songbird/multinomial.py:100: Multinomial.init (from tensorflow.python.ops.distributions.multinomial) is deprecated and will be removed after 2019-01-01.
Instructions for updating:
The TensorFlow Distributions library has moved to TensorFlow Probability (https://github.com/tensorflow/probability). You should update all references to use tfp.distributions instead of tf.distributions.
WARNING:tensorflow:From /usr/local/anaconda3/envs/songbird_env/lib/python3.6/site-packages/songbird/multinomial.py:115: The name tf.summary.scalar is deprecated. Please use tf.compat.v1.summary.scalar instead.

WARNING:tensorflow:From /usr/local/anaconda3/envs/songbird_env/lib/python3.6/site-packages/songbird/multinomial.py:121: The name tf.train.AdamOptimizer is deprecated. Please use tf.compat.v1.train.AdamOptimizer instead.

WARNING:tensorflow:From /usr/local/anaconda3/envs/songbird_env/lib/python3.6/site-packages/tensorflow/python/ops/clip_ops.py:286: add_dispatch_support..wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
WARNING:tensorflow:From /usr/local/anaconda3/envs/songbird_env/lib/python3.6/site-packages/songbird/multinomial.py:129: The name tf.summary.histogram is deprecated. Please use tf.compat.v1.summary.histogram instead.

WARNING:tensorflow:From /usr/local/anaconda3/envs/songbird_env/lib/python3.6/site-packages/songbird/multinomial.py:130: The name tf.summary.merge_all is deprecated. Please use tf.compat.v1.summary.merge_all instead.

WARNING:tensorflow:From /usr/local/anaconda3/envs/songbird_env/lib/python3.6/site-packages/songbird/multinomial.py:132: The name tf.summary.FileWriter is deprecated. Please use tf.compat.v1.summary.FileWriter instead.

WARNING:tensorflow:From /usr/local/anaconda3/envs/songbird_env/lib/python3.6/site-packages/songbird/multinomial.py:136: The name tf.global_variables_initializer is deprecated. Please use tf.compat.v1.global_variables_initializer instead.

WARNING:tensorflow:From /usr/local/anaconda3/envs/songbird_env/lib/python3.6/site-packages/songbird/multinomial.py:163: The name tf.train.Saver is deprecated. Please use tf.compat.v1.train.Saver instead.

0%| | 0/80000 [00:00<?, ?it/s]WARNING:tensorflow:From /usr/local/anaconda3/envs/songbird_env/lib/python3.6/site-packages/songbird/multinomial.py:172: The name tf.RunOptions is deprecated. Please use tf.compat.v1.RunOptions instead.

WARNING:tensorflow:From /usr/local/anaconda3/envs/songbird_env/lib/python3.6/site-packages/songbird/multinomial.py:174: The name tf.RunMetadata is deprecated. Please use tf.compat.v1.RunMetadata instead.

In addition, when I try to load tensorboard, I keep getting these warnings (which goes longer than this but similar)
tensorboard --logdir .

I0805 14:05:20.793066 123145466290176 _internal.py:113] ::ffff:192.168.0.184 - - [05/Aug/2020 14:05:20] "GET / HTTP/1.1" 200 -
I0805 14:05:21.125150 123145461035008 _internal.py:113] ::ffff:192.168.0.184 - - [05/Aug/2020 14:05:21] "GET /font-roboto/oMMgfZMQthOryQo9n22dcuvvDin1pK8aKteLpeZ5c0A.woff2 HTTP/1.1" 200 -
I0805 14:05:21.136273 123145471545344 _internal.py:113] ::ffff:192.168.0.184 - - [05/Aug/2020 14:05:21] "GET /font-roboto/RxZJdnzeo3R5zSexge8UUZBw1xU1rKptJj_0jans920.woff2 HTTP/1.1" 200 -
I0805 14:05:21.153836 123145476800512 _internal.py:113] ::ffff:192.168.0.184 - - [05/Aug/2020 14:05:21] "GET /font-roboto/d-6IYplOFocCacKzxwXSOJBw1xU1rKptJj_0jans920.woff2 HTTP/1.1" 200 -
I0805 14:05:21.648594 123145482055680 _internal.py:113] ::ffff:192.168.0.184 - - [05/Aug/2020 14:05:21] "GET /data/environment HTTP/1.1" 200 -
I0805 14:05:21.652559 123145487310848 _internal.py:113] ::ffff:192.168.0.184 - - [05/Aug/2020 14:05:21] "GET /data/runs HTTP/1.1" 200 -
I0805 14:05:21.654026 123145492566016 _internal.py:113] ::ffff:192.168.0.184 - - [05/Aug/2020 14:05:21] "GET /data/experiments HTTP/1.1" 200 -
I0805 14:05:21.677382 123145466290176 _internal.py:113] ::ffff:192.168.0.184 - - [05/Aug/2020 14:05:21] "GET /data/plugins_listing HTTP/1.1" 200 -
I0805 14:05:21.824388 123145461035008 _internal.py:113] ::ffff:192.168.0.184 - - [05/Aug/2020 14:05:21] "GET /data/plugin/scalars/tags HTTP/1.1" 200 -
I0805 14:05:22.475150 123145471545344 _internal.py:113] ::ffff:192.168.0.184 - - [05/Aug/2020 14:05:22] "GET /data/plugin/scalars/scalars?tag=loss&run=.&experiment= HTTP/1.1" 200 -
I0805 14:05:22.479615 123145461035008 _internal.py:113] ::ffff:192.168.0.184 - - [05/Aug/2020 14:05:22] "GET /data/plugin/scalars/scalars?tag=accuracy%2Fcv_error&run=.&experiment= HTTP/1.1" 200 -

I cannot load the HPARAMS page, which is just empty... Here are the warnings when I try to load hparams

E0805 14:09:00.260663 123145461035008 hparams_plugin.py:108] HParams error: Can't find an HParams-plugin experiment data in the log directory. Note that it takes some time to scan the log directory; if you just started Tensorboard it could be that we haven't finished scanning it yet. Consider trying again in a few seconds.
I0805 14:09:00.262099 123145461035008 _internal.py:113] ::ffff:192.168.0.184 - - [05/Aug/2020 14:09:00] "POST /data/plugin/hparams/experiment HTTP/1.1" 400 -
I0805 14:09:00.263484 123145466290176 _internal.py:113] ::ffff:192.168.0.184 - - [05/Aug/2020 14:09:00] "GET /data/plugins_listing HTTP/1.1" 200 -
E0805 14:09:00.280741 123145461035008 hparams_plugin.py:108] HParams error: Can't find an HParams-plugin experiment data in the log directory. Note that it takes some time to scan the log directory; if you just started Tensorboard it could be that we haven't finished scanning it yet. Consider trying again in a few seconds.
I0805 14:09:00.280971 123145461035008 _internal.py:113] ::ffff:192.168.0.184 - - [05/Aug/2020 14:09:00] "POST /data/plugin/hparams/experiment HTTP/1.1" 400 -
E0805 14:09:00.289092 123145461035008 hparams_plugin.py:108] HParams error: Can't find an HParams-plugin experiment data in the log directory. Note that it takes some time to scan the log directory; if you just started Tensorboard it could be that we haven't finished scanning it yet. Consider trying again in a few seconds.
I0805 14:09:00.289318 123145461035008 _internal.py:113] ::ffff:192.168.0.184 - - [05/Aug/2020 14:09:00] "POST /data/plugin/hparams/experiment HTTP/1.1" 400 -
E0805 14:09:00.324394 123145461035008 hparams_plugin.py:108] HParams error: Can't find an HParams-plugin experiment data in the log directory. Note that it takes some time to scan the log directory; if you just started Tensorboard it could be that we haven't finished scanning it yet. Consider trying again in a few seconds.
I0805 14:09:00.324654 123145461035008 _internal.py:113] ::ffff:192.168.0.184 - - [05/Aug/2020 14:09:00] "POST /data/plugin/hparams/experiment HTTP/1.1" 400 -

This is what I see
image

@mortonjt
Copy link
Collaborator

mortonjt commented Aug 5, 2020

hmm. We'll need more information in order to debug. How did you run the tensorboard command? Did you run everything in the same folder? Do you have any files generated in your results folder?

@coralzhang
Copy link
Author

coralzhang commented Aug 5, 2020

Thanks for the response! Here are responses.
How did you run the tensorboard command?

Yes and this is what I did tensorboard --logdir .

Did you run everything in the same folder?

Yes. I am staying in the same big folder except go in and out of the result folder...

Do you have any files generated in your results folder?

Yes and saved here https://www.dropbox.com/sh/3sxl94dh9xhirt5/AADLzmVwihtaAvb9IYyjezV4a?dl=0

I am not sure but I think it may have a relationship with the tensorflow version? when i install and use 1.10. Everything goes fine; the only issue is that I cannot see the HPARAMS tab, which was not available in that version.

@mortonjt
Copy link
Collaborator

mortonjt commented Aug 5, 2020 via email

@coralzhang
Copy link
Author

coralzhang commented Aug 5, 2020

That is part of my question.. How do I upgrade tensorboard itself properly? I have never used conda and tensor flow before. As you see, the way I installed tensorborad, I guess, is through installing of tensorflow version=xx, and I tried to simply uninstall tensorboard and upgrade, but end up with messing up stuff with tb-nightly as well. So, I then decided to use a higher version of tensorflow, say 1.15. Then got here.
In addition, since the key results are the differentials.tsv, if I run a multinomial regression somewhere else, will I be able to reproduce the results, at least largely?

@mortonjt
Copy link
Collaborator

mortonjt commented Aug 5, 2020 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants