Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for scaling a specific region #2

Open
wants to merge 35 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
92f6e0f
Changes to support the scaling a certain region specified by start/en…
cgoina Oct 2, 2015
6f7700c
fixed the z increment
cgoina Oct 2, 2015
296f0ee
restored the tile format
cgoina Oct 2, 2015
2e817a6
Changed the scaler to work both with and without region boundaries
cgoina Oct 6, 2015
1a54857
do not append the extension in the code - let the user specify that i…
cgoina Oct 6, 2015
7a7a073
removed print statements
cgoina Oct 8, 2015
4d56340
prevent crating an absolute path when no output path is provided
cgoina Oct 28, 2015
96d8fd6
remove commented code
cgoina Oct 28, 2015
c770c82
debug message to show successful loading of a tile
cgoina Oct 28, 2015
3bd2736
commented debug message
cgoina Oct 28, 2015
4ecc826
changed the tiler to ignore empty tiles as well
cgoina Nov 3, 2015
e4730d1
limited the cache size to prevent OOM errors
cgoina Nov 13, 2015
13a3a1a
configurable bg value
cgoina Nov 18, 2015
62566b2
changes to allow me to save an orthoview at the proper offset
cgoina Dec 8, 2015
5518322
use the scale level to determine the z step
cgoina Dec 10, 2015
3bbd8c9
put z in the tile loading debug message
cgoina Dec 10, 2015
ea1b78f
changed the export parameters to be pixel coordinates at level 0
cgoina Dec 11, 2015
44434c9
made tile cache size a configurable parameter
cgoina Dec 11, 2015
93dbc4e
used explicit pixel coordinates for exported max instead of the width
cgoina Dec 11, 2015
2e133d2
don't set the tile cache size by default
cgoina Dec 11, 2015
568c711
put the previous exportMin[R,C], exportMax[R,C] parameters back such …
cgoina Dec 11, 2015
c3890a9
option to set the background for the scaler as well
cgoina Dec 24, 2015
058abc3
http url writer
cgoina May 17, 2016
1953355
removed unused import
cgoina May 17, 2016
3180276
output error messages
cgoina May 17, 2016
5a46b83
change scaler to read from file or url
cgoina May 19, 2016
53fde3d
don't attempt to read image if status is not OK
cgoina May 19, 2016
c5f7b19
moved the tile reading to Utils
cgoina May 22, 2016
70443fb
added scale level to the print statement
cgoina Jun 2, 2016
614bd6d
close the connection on read
cgoina Jun 3, 2016
cc9fd22
print the zoom level
cgoina Jun 3, 2016
112d079
make the orientation not dependent of the order
cgoina Jun 3, 2016
d55c7a5
added guava cache as the previous cache based on linkedhashmap didn't…
cgoina Jun 30, 2016
a7351fe
removed unnecessary imports
cgoina Jun 30, 2016
1e735c6
removed the weak keys
cgoina Jun 30, 2016
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,9 @@
*.jar
*.war
*.ear
/target
target
.classpath
.project
.settings
*.iml
.idea
24 changes: 12 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,20 +102,20 @@ exported in scale level 0 pixels in <em>xyz</em> orientation (long, 0)</dd>
<dd>width of exported image tiles in pixels (int, 256)</dd>
<dt>tileHeight</dt>
<dd>height of exported image tiles in pixels (int, 256)</dd>
<dt>exportMinZ</dt>
<dt>exportedMinZ</dt>
<dd>first <em>z</em>-section index to be exported (long, 0)</dd>
<dt>exportMaxZ</dt>
<dt>exportedMaxZ</dt>
<dd>last <em>z</em>-section index to be exported (long, depth-1)</dd>
<dt>exportMinR</dt>
<dd>first row of tiles to be exported (long, 0)</dd>
<dt>exportMaxR</dt>
<dd>last row of tiles to be exported (long, depth-1)</dd>
<dt>exportMinC</dt>
<dd>first column of tiles to be exported (long, 0)</dd>
<dt>exportMaxC</dt>
<dd>last column of tiles to be exported (long, depth-1)</dd>
<dt>exportedMinX</dt>
<dd>first X in level 0 pixel coordinates to be exported (long, 0)</dd>
<dt>exportedMaxX</dt>
<dd>last X in level 0 pixel coordinates to be exported (long, width-1)</dd>
<dt>exportedMinY</dt>
<dd>first Y in level 0 pixel coordinates to be exported (long, 0)</dd>
<dt>exportedMaxY</dt>
<dd>last Y in level 0 pixel coordinates to be exported (long, depth-1)</dd>
<dt>exportBasePath</dt>
<dd>base path for the stakc to be exported (string, "")</dd>
<dd>base path for the stack to be exported (string, "")</dd>
<dt>tilePattern</dt>
<dd>tilePattern the file name convention for export tile coordinates without
extension and base path, must contain "&lt;s&gt;","&lt;z&gt;", "&lt;r&gt;",
Expand Down Expand Up @@ -155,7 +155,7 @@ contains all parameters in key=value rows (escaped according to Bash's needs).
depth=100
orientation=xy
exportBasePath=/var/www/catmaid/test/xy/
tilePattern="<z>/<r>_<c>_<s>"
tilePattern="%5$d/%8$d_%9$d_%1$d.jpg"
format=jpg
quality=0.85
type=gray
Expand Down
12 changes: 9 additions & 3 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -22,11 +22,17 @@
<artifactId>imglib2</artifactId>
<version>2.0.0-SNAPSHOT</version>
</dependency>
<dependency>
<dependency>
<groupId>net.imglib2</groupId>
<artifactId>imglib2-realtransform</artifactId>
<version>2.0.0-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>19.0</version>
</dependency>

</dependencies>
<build>
<defaultGoal>install</defaultGoal>
Expand All @@ -35,8 +41,8 @@
<artifactId>maven-compiler-plugin</artifactId>
<version>2.5</version>
<configuration>
<source>1.6</source>
<target>1.6</target>
<source>1.7</source>
<target>1.7</target>
<compilerArgument></compilerArgument>
</configuration>
</plugin>
Expand Down
128 changes: 56 additions & 72 deletions src/main/java/org/catmaid/CATMAIDRandomAccessibleInterval.java
Original file line number Diff line number Diff line change
Expand Up @@ -39,13 +39,15 @@

import java.awt.image.BufferedImage;
import java.awt.image.PixelGrabber;
import java.io.IOException;
import java.lang.ref.SoftReference;
import java.net.URL;
import java.util.HashMap;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutionException;

import javax.imageio.ImageIO;

import com.google.common.cache.Cache;
import com.google.common.cache.CacheBuilder;

import net.imglib2.AbstractInterval;
import net.imglib2.AbstractLocalizable;
import net.imglib2.Interval;
Expand Down Expand Up @@ -122,30 +124,17 @@ public int hashCode() {
return ( int )( value ^ ( value >>> 32 ) );
}
}

class Entry
{
final protected Key key;
final protected int[] data;
final int[] data;

public Entry( final Key key, final int[] data )
public Entry( final int[] data )
{
this.key = key;
this.data = data;
}

@Override
public void finalize()
{
synchronized ( cache )
{
// System.out.println( "finalizing..." );
cache.remove( key );
// System.out.println( cache.size() + " tiles chached." );
}
}
}

public class CATMAIDRandomAccess extends AbstractLocalizable implements RandomAccess< ARGBType >
{
protected long r, c;
Expand Down Expand Up @@ -590,22 +579,24 @@ public CATMAIDRandomAccess copyRandomAccess()
return copy();
}
}

final protected HashMap< Key, SoftReference< Entry > > cache = new HashMap< CATMAIDRandomAccessibleInterval.Key, SoftReference< Entry > >();
final protected String urlFormat;
final protected long rows, cols, s;
final protected int tileWidth, tileHeight;
final protected double scale;



private static final int MAX_CACHE_SIZE = 2048;

final private Cache< Key, Entry > tileCache;
final private String urlFormat;
final private long rows, cols, s;
final private int tileWidth, tileHeight;
final private double scale;

public CATMAIDRandomAccessibleInterval(
final String urlFormat,
final long width,
final long height,
final long depth,
final long s,
final int tileWidth,
final int tileHeight )
final int tileHeight,
final int cacheSize )
{
super( 3 );
this.urlFormat = urlFormat;
Expand All @@ -618,8 +609,12 @@ public CATMAIDRandomAccessibleInterval(
max[ 0 ] = ( long )( width * scale ) - 1;
max[ 1 ] = ( long )( height * scale ) - 1;
max[ 2 ] = depth - 1;
tileCache = CacheBuilder.newBuilder()
.maximumSize(cacheSize > 0 ? cacheSize : MAX_CACHE_SIZE)
.weakValues()
.build();
}

@Override
public int numDimensions()
{
Expand All @@ -646,57 +641,46 @@ protected int[] fetchPixels( final long r, final long c, final long z )
}
catch ( final OutOfMemoryError e )
{
System.err.println("Out of memory error while fetching tile (" + c + "," + r + "," + z + "). Trying to recover");
System.gc();
return fetchPixels2( r, c, z );
}
}

protected int[] fetchPixels2( final long r, final long c, final long z )
{
Entry tileEntry = null;
final Key key = new Key( r, c, z );
synchronized ( cache )
{
final SoftReference< Entry > cachedReference = cache.get( key );
if ( cachedReference != null )
{
final Entry cachedEntry = cachedReference.get();
if ( cachedEntry != null )
return cachedEntry.data;
}

final String urlString = String.format( urlFormat, s, scale, c * tileWidth, r * tileHeight, z, tileWidth, tileHeight, r, c );
try {
tileEntry = tileCache.get(key, new Callable<Entry>() {
@Override
public Entry call() {
final String urlString = String.format( urlFormat, s, scale, c * tileWidth, r * tileHeight, z, tileWidth, tileHeight, r, c );
final int[] pixels = new int[ tileWidth * tileHeight ];
try {
System.out.println( "Load s=" + s + " r=" + r + " c=" + c + " z=" + z + " url(" + urlString + ")" );
final URL url = new URL( urlString );
final BufferedImage jpg = ImageIO.read( url );

final int[] pixels = new int[ tileWidth * tileHeight ];
try
{
final URL url = new URL( urlString );
// final Image image = toolkit.createImage( url );
final BufferedImage jpg = ImageIO.read( url );

/* This gymnastic is necessary to get reproducible gray
* values, just opening a JPG or PNG, even when saved by
* ImageIO, and grabbing its pixels results in gray values
* with a non-matching gamma transfer function, I cannot tell
* why... */
final BufferedImage image = new BufferedImage( tileWidth, tileHeight, BufferedImage.TYPE_INT_RGB );
image.createGraphics().drawImage( jpg, 0, 0, null );
final PixelGrabber pg = new PixelGrabber( image, 0, 0, tileWidth, tileHeight, pixels, 0, tileWidth );
pg.grabPixels();

cache.put( key, new SoftReference< Entry >( new Entry( key, pixels ) ) );
// System.out.println( "success loading r=" + r + " c=" + c + " url(" + urlString + ")" );

}
catch (final IOException e)
{
System.out.println( "failed loading r=" + r + " c=" + c + " url(" + urlString + ")" );
cache.put( key, new SoftReference< Entry >( new Entry( key, pixels ) ) );
}
catch (final InterruptedException e)
{
e.printStackTrace();
}
return pixels;
/* This gymnastic is necessary to get reproducible gray
* values, just opening a JPG or PNG, even when saved by
* ImageIO, and grabbing its pixels results in gray values
* with a non-matching gamma transfer function, I cannot tell
* why... */
final BufferedImage image = new BufferedImage( tileWidth, tileHeight, BufferedImage.TYPE_INT_RGB );
image.createGraphics().drawImage( jpg, 0, 0, null );
final PixelGrabber pg = new PixelGrabber( image, 0, 0, tileWidth, tileHeight, pixels, 0, tileWidth );
pg.grabPixels();
System.out.println( "Successfully loaded s=" + s + " r=" + r + " c=" + c + " z=" + z + " url(" + urlString + ")" );
} catch (final Exception e) {
System.out.println( "Failed loading s=" + s + " r=" + r + " c=" + c + " z=" + z + " url(" + urlString + ")" );
}
return new Entry(pixels);
}
});
} catch (ExecutionException ee) {
ee.printStackTrace();
}
return tileEntry != null ? tileEntry.data : null;
}
}
13 changes: 9 additions & 4 deletions src/main/java/org/catmaid/Downsampler.java
Original file line number Diff line number Diff line change
Expand Up @@ -98,8 +98,8 @@ final static public void downsampleBytes( final byte[] aPixels, final byte[] bPi
}
}
}
final static public void downsampleRGB( final int[] aPixels, final int[] bPixels, final int wa, final int ha )

final static public boolean downsampleRGB( final int[] aPixels, final int[] bPixels, final int wa, final int ha, int bgValue )
{
assert aPixels.length == wa * ha && bPixels.length == wa / 2 * ( ha / 2 ) : "Input dimensions do not match.";

Expand All @@ -108,15 +108,20 @@ final static public void downsampleRGB( final int[] aPixels, final int[] bPixels
final int wb = wa / 2;
final int hb = ha / 2;
final int nb = hb * wb;

boolean bresult = false;
for ( int ya = 0, yb = 0; yb < nb; ya += wa2, yb += wb )
{
final int ya1 = ya + wa;
for ( int xa = 0, xb = 0; xb < wb; xa += 2, ++xb )
{
final int xa1 = xa + 1;
bPixels[ yb + xb ] = averageColor( ya + xa, ya + xa1, ya1 + xa, ya1 + xa1, aPixels );
int c = averageColor( ya + xa, ya + xa1, ya1 + xa, ya1 + xa1, aPixels );
bPixels[ yb + xb ] = c;
if ((c & 0xFFFFFF) != bgValue) {
bresult = true; // return true since at least one pixel is non black
}
}
}
return bresult;
}
}
Loading