Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

version 0.2.22 #62

Merged
merged 114 commits into from
Apr 14, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
114 commits
Select commit Hold shift + click to select a range
b82fb71
WIP torch 1.11 compatibility
matthias-k May 30, 2022
410a157
torch 1.11 compatibility
matthias-k May 30, 2022
744fef2
Merge branch 'dev' of github.com:matthias-k/pysaliency into dev
matthias-k May 30, 2022
01f0350
trying to solve h5py issues in github testing
matthias-k May 31, 2022
3a94348
fix numpy deprecation warnings
matthias-k May 31, 2022
03ab7de
compare old and new torch gaussian implementations
matthias-k May 31, 2022
7eb8577
Merge branch 'dev' of github.com:matthias-k/pysaliency into dev
matthias-k Jun 17, 2022
fecfc55
Remove obsolete generics
matthias-k Jun 17, 2022
e9975d7
Merge branch 'dev' of github.com:matthias-k/pysaliency into dev
matthias-k Jun 17, 2022
3c19e84
mark MIT dataset tests as requiring matlab
matthias-k Jun 27, 2022
b247772
Bugfix: convert saliency to float before computing NSS and CC
matthias-k Jun 30, 2022
7c199ce
Bugfix: t_hist got replaced with y_hist in Fixations instances but no…
matthias-k Jun 30, 2022
df0a9c5
Feature: scanpaths_from_fixations
matthias-k Jun 30, 2022
04e0657
scanpath_fixation_attributes, auto-generation of attributes in Fixati…
matthias-k Jul 1, 2022
2d7f74a
FixationTrains.train_lengths
matthias-k Jul 1, 2022
6414b6f
COCO Search18
matthias-k Jul 3, 2022
9412f84
dataset filter "clip_out_of_stimulus_fixations"
matthias-k Jul 3, 2022
883ff65
Disable in-memory tests of external datasets
matthias-k Jul 3, 2022
27228ad
bugfix loading old fixation train files
matthias-k Jul 4, 2022
2052839
FileStimuli.cached to control im memory caching of stimuli
matthias-k Jul 5, 2022
5bda1c5
Make LazyList use LRU caches to allow for limited caching
matthias-k Jul 8, 2022
d3bf244
dataset config filter for removing out of stimulus fixations
matthias-k Jul 16, 2022
bd08e47
slices of noncached file stimuli are also not cached
matthias-k Jul 18, 2022
b571b12
handle deprecation
matthias-k Aug 2, 2022
13cbc08
Fix FileStimuli caching
matthias-k Aug 2, 2022
7cdb456
MIT fixation durations with proper attributes
matthias-k Aug 30, 2022
d8905d9
Option to get MIT1003 with initial fixation conistent with standard v…
matthias-k Aug 31, 2022
8b0c225
scanpaths_from_fixation now builds proper scanpath attributes
matthias-k Aug 31, 2022
c18f512
Bugfix: redundant computation in loglikelihood
matthias-k Sep 1, 2022
91ad0f1
Avoid f-style format strings to keep compatibility with older python …
matthias-k Sep 26, 2022
e397121
Make compatible with older imageio versions
matthias-k Oct 11, 2022
17bb960
More verbose debug output in directory models
matthias-k Oct 14, 2022
e70fb23
bugfix torch dataset with saliency map models from image files
matthias-k Oct 21, 2022
fe322e6
Bugfixes and better testing for torch saliency map conversion
matthias-k Oct 22, 2022
b074eae
Bugfixes
matthias-k Oct 22, 2022
0b2840f
save and load torch saliency map processing models
matthias-k Oct 22, 2022
8cb2194
cache_directory for optimize_for_information_gain
matthias-k Oct 23, 2022
abd1cbb
allow pathlib locations for hdf5
matthias-k Nov 3, 2022
ca87b7d
bugfix: adapt setup.py to moved scripts
matthias-k Mar 5, 2023
942cb15
skip failing theano tests
matthias-k Apr 17, 2023
f8dedcd
skip failing cached optimization tests
matthias-k Apr 17, 2023
edc9bf8
relax assertion for similarity optimization test
matthias-k Apr 17, 2023
d0eb705
extend COCO-Search18: options for treating same images under differen…
matthias-k Apr 17, 2023
ef41380
ScikitLearnImageCrossValidationGenerator: allow to only compare withi…
matthias-k Apr 17, 2023
ec45644
add options to COCO commands
matthias-k Apr 20, 2023
b9786c6
create_subset accepts boolean masks now
matthias-k Apr 20, 2023
aa2fe02
COCO Search18 subject ids are now zero indexed
matthias-k Apr 29, 2023
dc4390f
Disable ssl when downloading VOC in Judd model
matthias-k May 10, 2023
df0378a
Adapt COCO Search18 dataset to website changes
matthias-k Jun 19, 2023
dda8381
Refactoring external models
matthias-k Jun 19, 2023
2578988
COCO Freeview dataset
matthias-k Jun 19, 2023
8e85c6e
COCO Freeview test stimuli
matthias-k Jun 19, 2023
c4e49bc
COCO Freeview: Code for extracting test fixations
matthias-k Jun 20, 2023
4953967
dataset slicing
matthias-k Jun 21, 2023
dbbf303
DeepGaze models
matthias-k Jun 21, 2023
d749722
Add torchvision to test setup
matthias-k Jun 22, 2023
a0fc2ed
Bugfix
matthias-k Jun 22, 2023
81b5925
trying to fix failing test on github
matthias-k Jun 22, 2023
15dae53
plot_scanpath
matthias-k Jun 26, 2023
0cfb5ea
Update COCO Freeview for new split data
matthias-k Jun 29, 2023
6ea3349
cleanup
matthias-k Jun 30, 2023
310deff
option for export_model_to_hdf5
matthias-k Jul 2, 2023
411c1f4
keep API of external_models more consistent
matthias-k Jul 9, 2023
40fb819
Speed up torch SIM optimization by making sampling more efficient
matthias-k Jul 10, 2023
05dcd3c
Some extensions to ShuffledBaselineModel
matthias-k Jul 11, 2023
55a588c
Make KDEGoldModel more memory efficient by not keeping the fixations …
matthias-k Jul 12, 2023
29c2c1d
Extend average_predictions to be able to handle generators of long se…
matthias-k Jul 14, 2023
2e7a409
Made ShuffledBaselineModel much more efficient in terms of memory and…
matthias-k Jul 14, 2023
9603f63
Bugfix plot_scanpath
matthias-k Jul 16, 2023
bf45668
ShuffledAUCScanpathSaliencyMapModel
matthias-k Jul 17, 2023
61c9062
clear cache when setting LazyList.cache to False
matthias-k Jul 19, 2023
71661cb
ShuffledBaselineModel: option to not remove current prediction from a…
matthias-k Jul 28, 2023
91fb8c5
Bugfix constructor Fixations
matthias-k Aug 6, 2023
51c9b62
FixationTrains.set_scanpath_attribute
matthias-k Sep 27, 2023
4bda526
concatenate file stimuli, cleanup attribute handling
matthias-k Sep 27, 2023
374c49b
refactor and extend data concatenation tools
matthias-k Sep 27, 2023
0e94e3b
Feature: DVAAwareScanpathModel
matthias-k Oct 13, 2023
caeab7b
Updated numba functions (#28)
hkhanuja Oct 25, 2023
6818696
add GeneralMixtureKernelDensityEstimator to baseline_utils for testin…
matthias-k Oct 25, 2023
89eef35
add classes to baseline utils for efficiently computing crossvalidati…
matthias-k Oct 25, 2023
c680269
Updated changelog
matthias-k Oct 25, 2023
5121f46
Fix and refactor SALICON tests
matthias-k Oct 26, 2023
c28ec40
Fix tests and GitHub test workflow (#29)
matthias-k Oct 26, 2023
3d22ad7
Update nusef.py (#31)
hkhanuja Nov 2, 2023
602d1a3
run tests on PRs
matthias-k Nov 5, 2023
36304ad
Bugfix: NUSEF fixation locations often not correctly scaled to image …
matthias-k Nov 9, 2023
4a846b8
Update dataset_config.py (#33)
hkhanuja Nov 10, 2023
b7f4b09
Added functionality to call filter functions from config file (#35)
hkhanuja Nov 14, 2023
c4cdbb2
fix rescaling bug in NUSEF (#36)
matthias-k Nov 14, 2023
a66a5c5
Make CrossvalMultipleRegularizations more effective for very large da…
matthias-k Nov 18, 2023
0a48f5d
Speedup and output (#38)
matthias-k Nov 18, 2023
06b718d
specify number of parallel jobs for CrossvalMultipleRegularizations (…
matthias-k Nov 19, 2023
fbe2aff
parallelization (#40)
matthias-k Nov 20, 2023
b3ae536
Remove buggy and uncessary crossval class (#41)
matthias-k Nov 25, 2023
16e212d
Removing stimuli with no fixations (#42)
hkhanuja Dec 16, 2023
90d6a99
allow filtering stimuli by multiple attribute values (#43)
matthias-k Dec 16, 2023
e32e2af
Added remove stimuli with no fixation filter to dataset_config.py (#44)
hkhanuja Dec 16, 2023
332bfb6
Added functionality to use combined centerbias (#45)
hkhanuja Jan 22, 2024
1b2e591
Save memory in gold standard crossvalidation (#46)
matthias-k Jan 26, 2024
1a6d965
Bugfix: ExpSaliencyMapModel doesn't apply to probabilistic models (#47)
matthias-k Jan 27, 2024
a34a69e
Don't compute saliency maps in NSS if there are no fixations on the i…
matthias-k Jan 27, 2024
8a5b829
[Bugfix] stimuli with attributes couldn't be sliced with lists
matthias-k Mar 2, 2024
761fe10
[Bugfix] Precomputed models failed for subsets of FileStimuli which w…
matthias-k Mar 2, 2024
6a685b7
Merge pull request #52 from matthias-k/bugfix-hdf5-filenames-for-subs…
matthias-k Mar 2, 2024
64bcde4
StimulusDependentSaliencyMapModel (#53)
matthias-k Mar 8, 2024
e2029a1
Stimuli can be filtered by boolean masks (#54)
matthias-k Mar 8, 2024
5f93ed1
Warnings for deprecated attribute in FixationTrains was oversensitive…
matthias-k Mar 10, 2024
54ac7fc
Better error message in HDF5 models (#56)
matthias-k Mar 14, 2024
17537fe
[Bugfix] CAT2000 contained unnecessary files (#57)
matthias-k Mar 14, 2024
03b71ea
ENH: check prediction of log densities and saliency maps before compu…
matthias-k Mar 23, 2024
e87a2bd
ENH: make MIT1003 loading less verbose (#59)
matthias-k Mar 25, 2024
6c067cb
Small bugs and enhancements (#60)
matthias-k Mar 26, 2024
245cda9
new tutorial (#61)
matthias-k Mar 26, 2024
607fa71
publish 0.2.22
matthias-k Apr 14, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 21 additions & 22 deletions .github/workflows/test-package-conda.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
name: Tests

on: [push]
on: [push, pull_request]

jobs:
build-linux:
Expand All @@ -9,29 +9,31 @@ jobs:
max-parallel: 5
matrix:
python-version:
- "3.7"
# - "3.7" # conda takes forever to install the dependencies
- "3.8"
- "3.9"

- "3.10"
- "3.11"
steps:
- uses: actions/checkout@v2
- name: Set up Python
uses: actions/setup-python@v2
- uses: conda-incubator/setup-miniconda@v2
with:
python-version: ${{ matrix.python-version }}
- name: Add conda to system path
run: |
# $CONDA is an environment variable pointing to the root of the miniconda directory
echo $CONDA/bin >> $GITHUB_PATH
channels: conda-forge
- name: Conda info
# the shell setting is necessary for loading profile etc which activates the conda environment
shell: bash -el {0}
run: conda info
- name: Install dependencies
shell: bash -el {0}
run: |
# conda env update --file environment.yml --name base
conda config --add channels conda-forge
conda install \
boltons \
cython \
deprecation \
dill \
diskcache \
h5py \
imageio \
natsort \
numba \
Expand All @@ -40,6 +42,7 @@ jobs:
pandas \
piexif \
pillow \
pip \
pkg-config \
pytorch \
requests \
Expand All @@ -48,23 +51,19 @@ jobs:
scipy \
setuptools \
sphinx \
theano \
torchvision \
tqdm
pip install h5py # https://github.com/h5py/h5py/issues/1880
# - name: Lint with flake8
# run: |
# conda install flake8
# # stop the build if there are Python syntax errors or undefined names
# flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
# # exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
# flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
- name: Conda list
shell: bash -el {0}
run: conda list
- name: Test with pytest
shell: bash -el {0}
run: |
conda install pytest hypothesis

python setup.py build_ext --inplace
python -m pytest --nomatlab tests
python -m pytest --nomatlab --notheano --nodownload tests
- name: test build and install
shell: bash -el {0}
run: |
python setup.py sdist
pip install dist/*.tar.gz
Expand Down
28 changes: 27 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,32 @@
# Changelog

* 0.2.22 (dev):
* 0.2.22:
* Enhancement: New [Tutorial](notebooks/Tutorial.ipynb).
* Bugfix: `SaliencyMapModel.AUC` failed if some images didn't have any fixations.
* Feature: `StimulusDependentSaliencyMapModel`
* Bugfix: The NUSEF dataset scaled some fixations not correctly to image coordinates. Also, we now account for some typos in the
dataset source data.
* Feature: CrossvalMultipleRegularizations and GeneralMixtureKernelDensityEstimator in baseline utils (names might change!)
* Feature: DVAAwareScanpathModel
* Feature: ShuffledBaselineModel is now much more efficient and able to handle large numbers of stimuli.
hence, ShuffledSimpleBaselineModel is not necessary anymore and a deprecated alias to ShuffledBaselineModel
* Feature: ShuffledBaselineModel can now compute predictions for very large numbers of stimuli without needing
to have all individual predictions in memory due to a recursive reduce logsumexp implementation.
* Feature: `plotting.plot_scanpath` to visualize scanpaths and saccades. WIP, expect the API to change!
* Feature: DeepGaze I and DeepGazeIIE models
* Feature: COCO Freeview dataset
* Feature: `optimize_for_information_gain(framework='torch', ...) now supports a `cache_directory`,
where intermediate steps are cached. This supports resuming crashed optimization runs.
* Bugfix: fixed some edge cases in `optimize_for_information_gain(framework='torch')`
* Feature: COCO Seach18 dataset
* Feature: `FixationTrains.train_lengths`
* Feature: `FixationTrains.scanpath_fixation_attributes` allows handling of per-fixation attributes on scanpath level,
e.g. fixation durations. According attributes as in a Fixations instance are automatically created,
e.g. for durations there will be an attribute `durations` and an attribute `duration_hist`. Also
for scanpath_attributes (e.g., attributes applying to a whole scanpath, such as task) will also generate
an attribute for each fixation to make this information available in Fixations instance.
* Feature: `scanpaths_from_fixations` reconstructs a FixationTrains object from a Fixations instance
* Bugfix: `t_hist` got replaced with `y_hist` in Fixations instances (but luckily not in FixationTrains instances)
* Bugfix: torch code was broken due to changes in torch 1.11
* Bugfix: SALICON dataset download did not work anymore
* Bugfix: NUSEF datast links changed
Expand Down
67 changes: 35 additions & 32 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,37 +11,6 @@ Pysaliency can evaluate most commonly used saliency metrics, including AUC, sAUC
image-based KL divergence, fixation based KL divergence and SIM for saliency map models and
log likelihoods and information gain for probabilistic models.

Pysaliency provides several important datasets:

* MIT1003
* MIT300
* CAT2000
* Toronto
* Koehler
* iSUN
* SALICON (both the 2015 and the 2017 edition and each with both the original mouse traces and the inferred fixations)
* FIGRIM
* OSIE
* NUSEF (the part with public images)

and some influential models:
* AIM
* SUN
* ContextAwareSaliency
* BMS
* GBVS
* GBVSIttiKoch
* Judd
* IttiKoch
* RARE2012
* CovSal


These models are using the original code which is often matlab.
Therefore, a matlab licence is required to make use of these models, although quite some of them
work with octave, too (see below).


Installation
------------

Expand All @@ -54,7 +23,7 @@ Quickstart
----------

import pysaliency

dataset_location = 'datasets'
model_location = 'models'

Expand All @@ -72,6 +41,40 @@ If you already have saliency maps for some dataset, you can import them into pys
my_model = pysaliency.SaliencyMapModelFromDirectory(mit_stimuli, '/path/to/my/saliency_maps')
auc = my_model.AUC(mit_stimuli, mit_fixations)

Check out the [Tutorial](notebooks/Tutorial.ipynb) for a more detailed introduction!

Included datasets and models
----------------------------

Pysaliency provides several important datasets:

* MIT1003
* MIT300
* CAT2000
* Toronto
* Koehler
* iSUN
* SALICON (both the 2015 and the 2017 edition and each with both the original mouse traces and the inferred fixations)
* FIGRIM
* OSIE
* NUSEF (the part with public images)

and some influential models:
* AIM
* SUN
* ContextAwareSaliency
* BMS
* GBVS
* GBVSIttiKoch
* Judd
* IttiKoch
* RARE2012
* CovSal

These models are using the original code which is often matlab.
Therefore, a matlab licence is required to make use of these models, although quite some of them
work with octave, too (see below).


Using Octave
------------
Expand Down
Loading
Loading