diff --git a/.gitignore b/.gitignore index 9fc5403e2..6be42962f 100644 --- a/.gitignore +++ b/.gitignore @@ -3,9 +3,12 @@ # Ignore build output /build/* +/target/* +/release.properties # Ignore Javadoc output -/doc/* +cli/doc/* +core/doc/* # Ignore any eventual Eclipse project files, these don't belong in the # repository. @@ -17,4 +20,4 @@ *.swp *.bak *~ -*~ +pom.xml.releaseBackup diff --git a/.travis.yml b/.travis.yml new file mode 100644 index 000000000..dff5f3a5d --- /dev/null +++ b/.travis.yml @@ -0,0 +1 @@ +language: java diff --git a/INSTALL b/INSTALL index d33121eee..a06ba85ff 100644 --- a/INSTALL +++ b/INSTALL @@ -1,20 +1,21 @@ -Howto build and use the BitTorrent library +Howto build and use the ttorrent library ========================================== Dependencies ------------ This Java implementation of the BitTorrent protocol implements a BitTorrent -tracker (an HTTP service), and a BitTorrent client. The only dependencies of -the BitTorrent library are: +tracker (an HTTP service), and a BitTorrent client. All dependencies are managed +by maven. The only dependencies of ttorrent-core are: -* the log4j library * the slf4j logging library * the SimpleHTTPFramework +* the Apache Commons (Codec and IO) -These libraries are provided in the lib/ directory, and are automatically -included in the JAR file created by the build process. +The CLI module also depends on: +* the log4j library +* the jargs library Building the distribution JAR ----------------------------- @@ -23,6 +24,14 @@ Simply execute the following command: $ mvn package -To build the library's JAR file (in the target/ directory). You can then import +To build the library's JAR file (in the core/target/ directory). You can then import this JAR file into your Java project and start using the Java BitTorrent library. + +This will also create a shaded JAR (in the cli/target/ directory). You can then use +this JAR file in conjunction with the three scripts in the bin/ folder. Each script +allows execution of one of the following entry points: + +* ClientMain - for running a torrent client +* TorrentMain - for creating .torrent files +* TrackerMain - for running a tracking server diff --git a/README.md b/README.md index e014586bb..1d6e518a9 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,8 @@ Ttorrent, a Java implementation of the BitTorrent protocol ========================================================== +[![Build Status](https://travis-ci.org/mpetazzoni/ttorrent.png)](https://travis-ci.org/mpetazzoni/ttorrent) + Description ----------- @@ -65,6 +67,22 @@ usage message on the console when invoked with the ``-h`` command-line flag. ### As a library +To use ``ttorrent`` is a library in your project, all you need is to +declare the dependency on the latest version of ``ttorrent``. For +example, if you use Maven, add the following in your POM's dependencies +section: + +```xml + + ... + + com.turn + ttorrent-core + 1.5 + + +``` + *Thanks to Anatoli Vladev for the code examples in #16.* #### Client code @@ -82,6 +100,12 @@ Client client = new Client( new File("/path/to/your.torrent"), new File("/path/to/output/directory"))); +// You can optionally set download/upload rate limits +// in kB/second. Setting a limit to 0.0 disables rate +// limits. +client.setMaxDownloadRate(50.0); +client.setMaxUploadRate(50.0); + // At this point, can you either call download() to download the torrent and // stop immediately after... client.download(); @@ -95,6 +119,9 @@ client.download(); client.waitForCompletion(); // At any time you can call client.stop() to interrupt the download. + +// If you plan on using the downloaded file(s) right away uncomment the line below +//client.getTorrent().finish(); ``` #### Tracker code @@ -124,6 +151,23 @@ tracker.start(); tracker.stop(); ``` +### Track download progress + +You can track the progress of the download and the state of the torrent +by registering an `Observer` on your `Client` instance. The observer is +updated every time a piece of the download completes: + +```java +client.addObserver(new Observer() { + @Override + public void update(Observable observable, Object data) { + Client client = (Client) observable; + float progress = client.getTorrent().getCompletion(); + // Do something with progress. + } +}); +``` + License ------- @@ -134,14 +178,14 @@ License version 2.0. See COPYING file for more details. Authors and contributors ------------------------ -* Maxime Petazzoni <> (Platform Engineer at Turn, Inc) +* Maxime Petazzoni <> (Software Engineer at SignalFuse, Inc) Original author, main developer and maintainer * David Giffin <> Contributed parallel hashing and multi-file torrent support. * Thomas Zink <> Fixed a piece length computation issue when the total torrent size is an exact multiple of the piece size. -* Johan Parent <> +* Johan Parent <> Fixed a bug in unfresh peer collection and issues on download completion on Windows platforms. * Dmitriy Dumanskiy @@ -149,6 +193,7 @@ Authors and contributors * Alexey Ptashniy Fixed an integer overflow in the calculation of a torrent's full size. +And many other helpful contributors on GitHub! Thanks to all of you. Caveats ------- diff --git a/bin/tracker b/bin/tracker deleted file mode 100755 index 95aca596b..000000000 --- a/bin/tracker +++ /dev/null @@ -1,18 +0,0 @@ -#!/bin/bash - -# Copyright (C) 2012 Turn, Inc. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -base=$(dirname $(readlink -f $0)) -exec java -cp $(find ${base}/../build -name "ttorrent-*.jar" | tail -n 1) com.turn.ttorrent.tracker.Tracker $* diff --git a/bin/ttorrent b/bin/ttorrent new file mode 100755 index 000000000..fc21deaf6 --- /dev/null +++ b/bin/ttorrent @@ -0,0 +1,73 @@ +#!/bin/sh + +# Copyright (C) 2012 Turn, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +EXECJAR="ttorrent-*-shaded.jar" + +real_path() { + case $1 in + /*) + SCRIPT="$1" + ;; + *) + PWD=`pwd` + SCRIPT="$PWD/$1" + ;; + esac + CHANGED=true + while [ "X$CHANGED" != "X" ] ; do + # Change spaces to ":" so the tokens can be parsed. + SAFESCRIPT=`echo $SCRIPT | sed -e 's; ;:;g'` + # Get the real path to this script, resolving any symbolic links + TOKENS=`echo $SAFESCRIPT | sed -e 's;/; ;g'` + REALPATH= + for C in $TOKENS; do + # Change any ":" in the token back to a space. + C=`echo $C | sed -e 's;:; ;g'` + REALPATH="$REALPATH/$C" + # If REALPATH is a sym link, resolve it. Loop for nested links. + while [ -h "$REALPATH" ] ; do + LS="`ls -ld "$REALPATH"`" + LINK="`expr "$LS" : '.*-> \(.*\)$'`" + if expr "$LINK" : '/.*' > /dev/null; then + # LINK is absolute. + REALPATH="$LINK" + else + # LINK is relative. + REALPATH="`dirname "$REALPATH"`""/$LINK" + fi + done + done + if [ "$REALPATH" = "$SCRIPT" ] ; then + CHANGED="" + else + SCRIPT="$REALPATH" + fi + done + echo "$REALPATH" +} + +base=$(dirname $(real_path $0)) +CPARG=$(find ${base}/../build -name "$EXECJAR" | tail -n 1) +if [ -z "$CPARG" ] ; then + echo "Unable to find $EXECJAR" + exit 1 +fi +if [ -z "$MAINCLASS" ] ; then + CPARG="-jar $CPARG" +else + CPARG="-cp $CPARG $MAINCLASS" +fi +exec java $CPARG "$@" diff --git a/bin/client b/bin/ttorrent-torrent similarity index 82% rename from bin/client rename to bin/ttorrent-torrent index cd77b761a..4193c4e2c 100755 --- a/bin/client +++ b/bin/ttorrent-torrent @@ -1,4 +1,4 @@ -#!/bin/bash +#!/bin/sh # Copyright (C) 2012 Turn, Inc. # @@ -14,5 +14,5 @@ # See the License for the specific language governing permissions and # limitations under the License. -base=$(dirname $(readlink -f $0)) -exec java -jar $(find ${base}/../build -name "ttorrent-*.jar" | tail -n 1) $* +EXEFILE="${0%-torrent}" +MAINCLASS="com.turn.ttorrent.cli.TorrentMain" "${EXEFILE}" "$@" diff --git a/bin/torrent b/bin/ttorrent-tracker similarity index 78% rename from bin/torrent rename to bin/ttorrent-tracker index 99d46d258..e6bacd5d9 100755 --- a/bin/torrent +++ b/bin/ttorrent-tracker @@ -1,4 +1,4 @@ -#!/bin/bash +#!/bin/sh # Copyright (C) 2012 Turn, Inc. # @@ -14,5 +14,5 @@ # See the License for the specific language governing permissions and # limitations under the License. -base=$(dirname $(readlink -f $0)) -exec java -cp $(find ${base}/../build -name "ttorrent-*.jar" | tail -n 1) com.turn.ttorrent.common.Torrent $* +EXEFILE="${0%-tracker}" +MAINCLASS="com.turn.ttorrent.cli.TrackerMain" "${EXEFILE}" "$@" diff --git a/cli/.gitignore b/cli/.gitignore new file mode 100644 index 000000000..2177fc127 --- /dev/null +++ b/cli/.gitignore @@ -0,0 +1,5 @@ +/target/ + +/.classpath +/.project +/.settings diff --git a/cli/pom.xml b/cli/pom.xml new file mode 100644 index 000000000..0f97f7b7d --- /dev/null +++ b/cli/pom.xml @@ -0,0 +1,77 @@ + + 4.0.0 + + + com.turn + ttorrent + 1.6-SNAPSHOT + + + Java BitTorrent library CLI + ttorrent-cli + jar + + + + com.turn + ttorrent-core + 1.6-SNAPSHOT + + + + org.slf4j + slf4j-log4j12 + 1.6.4 + + + net.sf + jargs + 1.0 + + + + + package + + + + org.apache.maven.plugins + maven-jar-plugin + 2.4 + + + + true + + + + ** + + + + + + maven-shade-plugin + 2.1 + + + package + + shade + + + ${project.build.directory}/${project.artifactId}-${project.version}-shaded.jar + + + + com.turn.ttorrent.cli.ClientMain + + + + + + + + + + diff --git a/cli/src/main/java/com/turn/ttorrent/cli/ClientMain.java b/cli/src/main/java/com/turn/ttorrent/cli/ClientMain.java new file mode 100644 index 000000000..ae56e5f75 --- /dev/null +++ b/cli/src/main/java/com/turn/ttorrent/cli/ClientMain.java @@ -0,0 +1,176 @@ +/** + * Copyright (C) 2011-2013 Turn, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.turn.ttorrent.cli; + +import com.turn.ttorrent.client.Client; +import com.turn.ttorrent.client.SharedTorrent; + +import java.io.File; +import java.io.PrintStream; +import java.net.Inet4Address; +import java.net.InetAddress; +import java.net.NetworkInterface; +import java.net.SocketException; +import java.net.UnknownHostException; +import java.nio.channels.UnsupportedAddressTypeException; +import java.util.Enumeration; + +import jargs.gnu.CmdLineParser; +import org.apache.log4j.BasicConfigurator; +import org.apache.log4j.ConsoleAppender; +import org.apache.log4j.PatternLayout; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** + * Command-line entry-point for starting a {@link Client} + */ +public class ClientMain { + + private static final Logger logger = + LoggerFactory.getLogger(ClientMain.class); + + /** + * Default data output directory. + */ + private static final String DEFAULT_OUTPUT_DIRECTORY = "/tmp"; + + /** + * Returns a usable {@link Inet4Address} for the given interface name. + * + *

+ * If an interface name is given, return the first usable IPv4 address for + * that interface. If no interface name is given or if that interface + * doesn't have an IPv4 address, return's localhost address (if IPv4). + *

+ * + *

+ * It is understood this makes the client IPv4 only, but it is important to + * remember that most BitTorrent extensions (like compact peer lists from + * trackers and UDP tracker support) are IPv4-only anyway. + *

+ * + * @param iface The network interface name. + * @return A usable IPv4 address as a {@link Inet4Address}. + * @throws UnsupportedAddressTypeException If no IPv4 address was available + * to bind on. + */ + private static Inet4Address getIPv4Address(String iface) + throws SocketException, UnsupportedAddressTypeException, + UnknownHostException { + if (iface != null) { + Enumeration addresses = + NetworkInterface.getByName(iface).getInetAddresses(); + while (addresses.hasMoreElements()) { + InetAddress addr = addresses.nextElement(); + if (addr instanceof Inet4Address) { + return (Inet4Address)addr; + } + } + } + + InetAddress localhost = InetAddress.getLocalHost(); + if (localhost instanceof Inet4Address) { + return (Inet4Address)localhost; + } + + throw new UnsupportedAddressTypeException(); + } + + /** + * Display program usage on the given {@link PrintStream}. + */ + private static void usage(PrintStream s) { + s.println("usage: Client [options] "); + s.println(); + s.println("Available options:"); + s.println(" -h,--help Show this help and exit."); + s.println(" -o,--output DIR Read/write data to directory DIR."); + s.println(" -i,--iface IFACE Bind to interface IFACE."); + s.println(" -s,--seed SECONDS Time to seed after downloading (default: infinitely)."); + s.println(" -d,--max-download KB/SEC Max download rate (default: unlimited)."); + s.println(" -u,--max-upload KB/SEC Max upload rate (default: unlimited)."); + s.println(); + } + + /** + * Main client entry point for stand-alone operation. + */ + public static void main(String[] args) { + BasicConfigurator.configure(new ConsoleAppender( + new PatternLayout("%d [%-25t] %-5p: %m%n"))); + + CmdLineParser parser = new CmdLineParser(); + CmdLineParser.Option help = parser.addBooleanOption('h', "help"); + CmdLineParser.Option output = parser.addStringOption('o', "output"); + CmdLineParser.Option iface = parser.addStringOption('i', "iface"); + CmdLineParser.Option seedTime = parser.addIntegerOption('s', "seed"); + CmdLineParser.Option maxUpload = parser.addDoubleOption('u', "max-upload"); + CmdLineParser.Option maxDownload = parser.addDoubleOption('d', "max-download"); + + try { + parser.parse(args); + } catch (CmdLineParser.OptionException oe) { + System.err.println(oe.getMessage()); + usage(System.err); + System.exit(1); + } + + // Display help and exit if requested + if (Boolean.TRUE.equals((Boolean)parser.getOptionValue(help))) { + usage(System.out); + System.exit(0); + } + + String outputValue = (String)parser.getOptionValue(output, + DEFAULT_OUTPUT_DIRECTORY); + String ifaceValue = (String)parser.getOptionValue(iface); + int seedTimeValue = (Integer)parser.getOptionValue(seedTime, -1); + + double maxDownloadRate = (Double)parser.getOptionValue(maxDownload, 0.0); + double maxUploadRate = (Double)parser.getOptionValue(maxUpload, 0.0); + + String[] otherArgs = parser.getRemainingArgs(); + if (otherArgs.length != 1) { + usage(System.err); + System.exit(1); + } + + try { + Client c = new Client( + getIPv4Address(ifaceValue), + SharedTorrent.fromFile( + new File(otherArgs[0]), + new File(outputValue))); + + c.setMaxDownloadRate(maxDownloadRate); + c.setMaxUploadRate(maxUploadRate); + + // Set a shutdown hook that will stop the sharing/seeding and send + // a STOPPED announce request. + Runtime.getRuntime().addShutdownHook( + new Thread(new Client.ClientShutdown(c, null))); + + c.share(seedTimeValue); + if (Client.ClientState.ERROR.equals(c.getState())) { + System.exit(1); + } + } catch (Exception e) { + logger.error("Fatal error: {}", e.getMessage(), e); + System.exit(2); + } + } +} diff --git a/cli/src/main/java/com/turn/ttorrent/cli/TorrentMain.java b/cli/src/main/java/com/turn/ttorrent/cli/TorrentMain.java new file mode 100644 index 000000000..7fb8c1ff5 --- /dev/null +++ b/cli/src/main/java/com/turn/ttorrent/cli/TorrentMain.java @@ -0,0 +1,197 @@ +/** + * Copyright (C) 2011-2013 Turn, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.turn.ttorrent.cli; + +import com.turn.ttorrent.common.Torrent; + +import java.io.File; +import java.io.FileOutputStream; +import java.io.OutputStream; +import java.io.PrintStream; +import java.net.URI; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.Collections; +import java.util.List; +import java.util.Vector; + +import jargs.gnu.CmdLineParser; +import org.apache.commons.io.FileUtils; +import org.apache.commons.io.IOUtils; +import org.apache.commons.io.filefilter.TrueFileFilter; +import org.apache.log4j.BasicConfigurator; +import org.apache.log4j.ConsoleAppender; +import org.apache.log4j.PatternLayout; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** + * Command-line entry-point for reading and writing {@link Torrent} files. + */ +public class TorrentMain { + + private static final Logger logger = + LoggerFactory.getLogger(TorrentMain.class); + + /** + * Display program usage on the given {@link PrintStream}. + */ + private static void usage(PrintStream s) { + usage(s, null); + } + + /** + * Display a message and program usage on the given {@link PrintStream}. + */ + private static void usage(PrintStream s, String msg) { + if (msg != null) { + s.println(msg); + s.println(); + } + + s.println("usage: Torrent [options] [file|directory]"); + s.println(); + s.println("Available options:"); + s.println(" -h,--help Show this help and exit."); + s.println(" -t,--torrent FILE Use FILE to read/write torrent file."); + s.println(); + s.println(" -c,--create Create a new torrent file using " + + "the given announce URL and data."); + s.println(" -l,--length Define the piece length for hashing data"); + s.println(" -a,--announce Tracker URL (can be repeated)."); + s.println(); + } + + /** + * Torrent reader and creator. + * + *

+ * You can use the {@code main()} function of this class to read or create + * torrent files. See usage for details. + *

+ * + */ + public static void main(String[] args) { + BasicConfigurator.configure(new ConsoleAppender( + new PatternLayout("%-5p: %m%n"))); + + CmdLineParser parser = new CmdLineParser(); + CmdLineParser.Option help = parser.addBooleanOption('h', "help"); + CmdLineParser.Option filename = parser.addStringOption('t', "torrent"); + CmdLineParser.Option create = parser.addBooleanOption('c', "create"); + CmdLineParser.Option pieceLength = parser.addIntegerOption('l', "length"); + CmdLineParser.Option announce = parser.addStringOption('a', "announce"); + + try { + parser.parse(args); + } catch (CmdLineParser.OptionException oe) { + System.err.println(oe.getMessage()); + usage(System.err); + System.exit(1); + } + + // Display help and exit if requested + if (Boolean.TRUE.equals((Boolean)parser.getOptionValue(help))) { + usage(System.out); + System.exit(0); + } + + String filenameValue = (String)parser.getOptionValue(filename); + if (filenameValue == null) { + usage(System.err, "Torrent file must be provided!"); + System.exit(1); + } + + Integer pieceLengthVal = (Integer) parser.getOptionValue(pieceLength); + if (pieceLengthVal == null) { + pieceLengthVal = Torrent.DEFAULT_PIECE_LENGTH; + } + else { + pieceLengthVal = pieceLengthVal * 1024; + } + logger.info("Using piece length of {} bytes.", pieceLengthVal); + + Boolean createFlag = (Boolean)parser.getOptionValue(create); + + //For repeated announce urls + @SuppressWarnings("unchecked") + Vector announceURLs = (Vector)parser.getOptionValues(announce); + + + String[] otherArgs = parser.getRemainingArgs(); + + if (Boolean.TRUE.equals(createFlag) && + (otherArgs.length != 1 || announceURLs.isEmpty())) { + usage(System.err, "Announce URL and a file or directory must be " + + "provided to create a torrent file!"); + System.exit(1); + } + + + OutputStream fos = null; + try { + if (Boolean.TRUE.equals(createFlag)) { + if (filenameValue != null) { + fos = new FileOutputStream(filenameValue); + } else { + fos = System.out; + } + + //Process the announce URLs into URIs + List announceURIs = new ArrayList(); + for (String url : announceURLs) { + announceURIs.add(new URI(url)); + } + + //Create the announce-list as a list of lists of URIs + //Assume all the URI's are first tier trackers + List> announceList = new ArrayList>(); + announceList.add(announceURIs); + + File source = new File(otherArgs[0]); + if (!source.exists() || !source.canRead()) { + throw new IllegalArgumentException( + "Cannot access source file or directory " + + source.getName()); + } + + String creator = String.format("%s (ttorrent)", + System.getProperty("user.name")); + + Torrent torrent = null; + if (source.isDirectory()) { + List files = new ArrayList(FileUtils.listFiles(source, TrueFileFilter.TRUE, TrueFileFilter.TRUE)); + Collections.sort(files); + torrent = Torrent.create(source, files, pieceLengthVal, + announceList, creator); + } else { + torrent = Torrent.create(source, pieceLengthVal, announceList, creator); + } + + torrent.save(fos); + } else { + Torrent.load(new File(filenameValue), true); + } + } catch (Exception e) { + logger.error("{}", e.getMessage(), e); + System.exit(2); + } finally { + if (fos != System.out) { + IOUtils.closeQuietly(fos); + } + } + } +} diff --git a/cli/src/main/java/com/turn/ttorrent/cli/TrackerMain.java b/cli/src/main/java/com/turn/ttorrent/cli/TrackerMain.java new file mode 100644 index 000000000..bd3ffa473 --- /dev/null +++ b/cli/src/main/java/com/turn/ttorrent/cli/TrackerMain.java @@ -0,0 +1,118 @@ +/** + * Copyright (C) 2011-2013 Turn, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.turn.ttorrent.cli; + +import com.turn.ttorrent.tracker.TrackedTorrent; +import com.turn.ttorrent.tracker.Tracker; + +import java.io.File; +import java.io.FilenameFilter; +import java.io.PrintStream; +import java.net.InetSocketAddress; + +import jargs.gnu.CmdLineParser; +import org.apache.log4j.BasicConfigurator; +import org.apache.log4j.ConsoleAppender; +import org.apache.log4j.PatternLayout; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** + * Command-line entry-point for starting a {@link Tracker} + */ +public class TrackerMain { + + private static final Logger logger = + LoggerFactory.getLogger(TrackerMain.class); + + /** + * Display program usage on the given {@link PrintStream}. + */ + private static void usage(PrintStream s) { + s.println("usage: Tracker [options] [directory]"); + s.println(); + s.println("Available options:"); + s.println(" -h,--help Show this help and exit."); + s.println(" -p,--port PORT Bind to port PORT."); + s.println(); + } + + /** + * Main function to start a tracker. + */ + public static void main(String[] args) { + BasicConfigurator.configure(new ConsoleAppender( + new PatternLayout("%d [%-25t] %-5p: %m%n"))); + + CmdLineParser parser = new CmdLineParser(); + CmdLineParser.Option help = parser.addBooleanOption('h', "help"); + CmdLineParser.Option port = parser.addIntegerOption('p', "port"); + + try { + parser.parse(args); + } catch (CmdLineParser.OptionException oe) { + System.err.println(oe.getMessage()); + usage(System.err); + System.exit(1); + } + + // Display help and exit if requested + if (Boolean.TRUE.equals((Boolean)parser.getOptionValue(help))) { + usage(System.out); + System.exit(0); + } + + Integer portValue = (Integer)parser.getOptionValue(port, + Integer.valueOf(Tracker.DEFAULT_TRACKER_PORT)); + + String[] otherArgs = parser.getRemainingArgs(); + + if (otherArgs.length > 1) { + usage(System.err); + System.exit(1); + } + + // Get directory from command-line argument or default to current + // directory + String directory = otherArgs.length > 0 + ? otherArgs[0] + : "."; + + FilenameFilter filter = new FilenameFilter() { + @Override + public boolean accept(File dir, String name) { + return name.endsWith(".torrent"); + } + }; + + try { + Tracker t = new Tracker(new InetSocketAddress(portValue.intValue())); + + File parent = new File(directory); + for (File f : parent.listFiles(filter)) { + logger.info("Loading torrent from " + f.getName()); + t.announce(TrackedTorrent.load(f)); + } + + logger.info("Starting tracker with {} announced torrents...", + t.getTrackedTorrents().size()); + t.start(); + } catch (Exception e) { + logger.error("{}", e.getMessage(), e); + System.exit(2); + } + } +} diff --git a/core/.gitignore b/core/.gitignore new file mode 100644 index 000000000..2177fc127 --- /dev/null +++ b/core/.gitignore @@ -0,0 +1,5 @@ +/target/ + +/.classpath +/.project +/.settings diff --git a/core/pom.xml b/core/pom.xml new file mode 100644 index 000000000..a86c7531e --- /dev/null +++ b/core/pom.xml @@ -0,0 +1,37 @@ + + 4.0.0 + + + com.turn + ttorrent + 1.6-SNAPSHOT + + + Java BitTorrent library core + ttorrent-core + jar + + + + commons-io + commons-io + 2.4 + + + org.simpleframework + simple + 4.1.21 + + + org.slf4j + slf4j-api + 1.6.4 + + + org.testng + testng + 6.1.1 + test + + + diff --git a/src/main/java/com/turn/ttorrent/bcodec/BDecoder.java b/core/src/main/java/com/turn/ttorrent/bcodec/BDecoder.java similarity index 99% rename from src/main/java/com/turn/ttorrent/bcodec/BDecoder.java rename to core/src/main/java/com/turn/ttorrent/bcodec/BDecoder.java index 305c56170..29a941dc9 100644 --- a/src/main/java/com/turn/ttorrent/bcodec/BDecoder.java +++ b/core/src/main/java/com/turn/ttorrent/bcodec/BDecoder.java @@ -206,7 +206,7 @@ public BEValue bdecodeNumber() throws IOException { c = this.read(); if (c == '0') throw new InvalidBEncodingException("Negative zero not allowed"); - chars[off] = (char)c; + chars[off] = '-'; off++; } diff --git a/src/main/java/com/turn/ttorrent/bcodec/BEValue.java b/core/src/main/java/com/turn/ttorrent/bcodec/BEValue.java similarity index 100% rename from src/main/java/com/turn/ttorrent/bcodec/BEValue.java rename to core/src/main/java/com/turn/ttorrent/bcodec/BEValue.java diff --git a/src/main/java/com/turn/ttorrent/bcodec/BEncoder.java b/core/src/main/java/com/turn/ttorrent/bcodec/BEncoder.java similarity index 100% rename from src/main/java/com/turn/ttorrent/bcodec/BEncoder.java rename to core/src/main/java/com/turn/ttorrent/bcodec/BEncoder.java diff --git a/src/main/java/com/turn/ttorrent/bcodec/InvalidBEncodingException.java b/core/src/main/java/com/turn/ttorrent/bcodec/InvalidBEncodingException.java similarity index 100% rename from src/main/java/com/turn/ttorrent/bcodec/InvalidBEncodingException.java rename to core/src/main/java/com/turn/ttorrent/bcodec/InvalidBEncodingException.java diff --git a/src/main/java/com/turn/ttorrent/client/Client.java b/core/src/main/java/com/turn/ttorrent/client/Client.java similarity index 86% rename from src/main/java/com/turn/ttorrent/client/Client.java rename to core/src/main/java/com/turn/ttorrent/client/Client.java index 480657dbf..f300bd1b6 100644 --- a/src/main/java/com/turn/ttorrent/client/Client.java +++ b/core/src/main/java/com/turn/ttorrent/client/Client.java @@ -19,26 +19,19 @@ import com.turn.ttorrent.client.announce.AnnounceException; import com.turn.ttorrent.client.announce.AnnounceResponseListener; import com.turn.ttorrent.client.peer.PeerActivityListener; +import com.turn.ttorrent.client.peer.SharingPeer; import com.turn.ttorrent.common.Peer; import com.turn.ttorrent.common.Torrent; import com.turn.ttorrent.common.protocol.PeerMessage; import com.turn.ttorrent.common.protocol.TrackerMessage; -import com.turn.ttorrent.client.peer.SharingPeer; -import java.io.File; import java.io.IOException; -import java.io.PrintStream; -import java.net.Inet4Address; import java.net.InetAddress; -import java.net.NetworkInterface; -import java.net.SocketException; import java.net.UnknownHostException; import java.nio.ByteBuffer; import java.nio.channels.SocketChannel; -import java.nio.channels.UnsupportedAddressTypeException; import java.util.BitSet; import java.util.Comparator; -import java.util.Enumeration; import java.util.HashSet; import java.util.List; import java.util.Observable; @@ -51,11 +44,6 @@ import java.util.concurrent.ConcurrentHashMap; import java.util.concurrent.ConcurrentMap; -import jargs.gnu.CmdLineParser; - -import org.apache.log4j.BasicConfigurator; -import org.apache.log4j.ConsoleAppender; -import org.apache.log4j.PatternLayout; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -99,9 +87,6 @@ public class Client extends Observable implements Runnable, private static final int RATE_COMPUTATION_ITERATIONS = 2; private static final int MAX_DOWNLOADERS_UNCHOKE = 4; - /** Default data output directory. */ - private static final String DEFAULT_OUTPUT_DIRECTORY = "/tmp"; - public enum ClientState { WAITING, VALIDATING, @@ -150,7 +135,7 @@ public Client(InetAddress address, SharedTorrent torrent) this.self = new Peer( this.service.getSocketAddress() .getAddress().getHostAddress(), - (short)this.service.getSocketAddress().getPort(), + this.service.getSocketAddress().getPort(), ByteBuffer.wrap(id.getBytes(Torrent.BYTE_ENCODING))); // Initialize the announce request thread, and register ourselves to it @@ -172,6 +157,26 @@ public Client(InetAddress address, SharedTorrent torrent) this.random = new Random(System.currentTimeMillis()); } + /** + * Set the maximum download rate (in kB/second) for this + * torrent. A setting of <= 0.0 disables rate limiting. + * + * @param rate The maximum download rate + */ + public void setMaxDownloadRate(double rate) { + this.torrent.setMaxDownloadRate(rate); + } + + /** + * Set the maximum upload rate (in kB/second) for this + * torrent. A setting of <= 0.0 disables rate limiting. + * + * @param rate The maximum upload rate + */ + public void setMaxUploadRate(double rate) { + this.torrent.setMaxUploadRate(rate); + } + /** * Get this client's peer specification. */ @@ -637,6 +642,7 @@ private synchronized void unchokePeers(boolean optimistic) { for (SharingPeer peer : choked) { if (optimistic && peer == randomPeer) { logger.debug("Optimistic unchoke of {}.", peer); + peer.unchoke(); continue; } @@ -921,11 +927,11 @@ public void handleIOException(SharingPeer peer, IOException ioe) { *

* When the download is complete, the client switches to seeding mode for * as long as requested in the share() call, if seeding was - * requested. If not, the StopSeedingTask will execute immediately to stop - * the client's main loop. + * requested. If not, the {@link ClientShutdown} will execute + * immediately to stop the client's main loop. *

* - * @see StopSeedingTask + * @see ClientShutdown */ private synchronized void seed() { // Silently ignore if we're already seeding. @@ -965,12 +971,12 @@ private synchronized void seed() { * * @author mpetazzoni */ - private static class ClientShutdown extends TimerTask { + public static class ClientShutdown extends TimerTask { private final Client client; private final Timer timer; - ClientShutdown(Client client, Timer timer) { + public ClientShutdown(Client client, Timer timer) { this.client = client; this.timer = timer; } @@ -983,120 +989,4 @@ public void run() { } } }; - - /** - * Display program usage on the given {@link PrintStream}. - */ - private static void usage(PrintStream s) { - s.println("usage: Client [options] "); - s.println(); - s.println("Available options:"); - s.println(" -h,--help Show this help and exit."); - s.println(" -o,--output DIR Read/write data to directory DIR."); - s.println(" -i,--iface IFACE Bind to interface IFACE."); - s.println(" -s,--seed SECONDS Time to seed after downloading (default: infinitely)."); - s.println(); - } - - /** - * Returns a usable {@link Inet4Address} for the given interface name. - * - *

- * If an interface name is given, return the first usable IPv4 address for - * that interface. If no interface name is given or if that interface - * doesn't have an IPv4 address, return's localhost address (if IPv4). - *

- * - *

- * It is understood this makes the client IPv4 only, but it is important to - * remember that most BitTorrent extensions (like compact peer lists from - * trackers and UDP tracker support) are IPv4-only anyway. - *

- * - * @param iface The network interface name. - * @return A usable IPv4 address as a {@link Inet4Address}. - * @throws UnsupportedAddressTypeException If no IPv4 address was available - * to bind on. - */ - private static Inet4Address getIPv4Address(String iface) - throws SocketException, UnsupportedAddressTypeException, - UnknownHostException { - if (iface != null) { - Enumeration addresses = - NetworkInterface.getByName(iface).getInetAddresses(); - while (addresses.hasMoreElements()) { - InetAddress addr = addresses.nextElement(); - if (addr instanceof Inet4Address) { - return (Inet4Address)addr; - } - } - } - - InetAddress localhost = InetAddress.getLocalHost(); - if (localhost instanceof Inet4Address) { - return (Inet4Address)localhost; - } - - throw new UnsupportedAddressTypeException(); - } - - /** - * Main client entry point for stand-alone operation. - */ - public static void main(String[] args) { - BasicConfigurator.configure(new ConsoleAppender( - new PatternLayout("%d [%-25t] %-5p: %m%n"))); - - CmdLineParser parser = new CmdLineParser(); - CmdLineParser.Option help = parser.addBooleanOption('h', "help"); - CmdLineParser.Option output = parser.addStringOption('o', "output"); - CmdLineParser.Option iface = parser.addStringOption('i', "iface"); - CmdLineParser.Option seedTime = parser.addIntegerOption('s', "seed"); - - try { - parser.parse(args); - } catch (CmdLineParser.OptionException oe) { - System.err.println(oe.getMessage()); - usage(System.err); - System.exit(1); - } - - // Display help and exit if requested - if (Boolean.TRUE.equals((Boolean)parser.getOptionValue(help))) { - usage(System.out); - System.exit(0); - } - - String outputValue = (String)parser.getOptionValue(output, - DEFAULT_OUTPUT_DIRECTORY); - String ifaceValue = (String)parser.getOptionValue(iface); - int seedTimeValue = (Integer)parser.getOptionValue(seedTime, -1); - - String[] otherArgs = parser.getRemainingArgs(); - if (otherArgs.length != 1) { - usage(System.err); - System.exit(1); - } - - try { - Client c = new Client( - getIPv4Address(ifaceValue), - SharedTorrent.fromFile( - new File(otherArgs[0]), - new File(outputValue))); - - // Set a shutdown hook that will stop the sharing/seeding and send - // a STOPPED announce request. - Runtime.getRuntime().addShutdownHook( - new Thread(new ClientShutdown(c, null))); - - c.share(seedTimeValue); - if (ClientState.ERROR.equals(c.getState())) { - System.exit(1); - } - } catch (Exception e) { - logger.error("Fatal error: {}", e.getMessage(), e); - System.exit(2); - } - } } diff --git a/src/main/java/com/turn/ttorrent/client/ConnectionHandler.java b/core/src/main/java/com/turn/ttorrent/client/ConnectionHandler.java similarity index 95% rename from src/main/java/com/turn/ttorrent/client/ConnectionHandler.java rename to core/src/main/java/com/turn/ttorrent/client/ConnectionHandler.java index 8b56336c9..8eca68578 100644 --- a/src/main/java/com/turn/ttorrent/client/ConnectionHandler.java +++ b/core/src/main/java/com/turn/ttorrent/client/ConnectionHandler.java @@ -17,6 +17,7 @@ import com.turn.ttorrent.common.Torrent; import com.turn.ttorrent.client.peer.SharingPeer; +import com.turn.ttorrent.common.Utils; import java.io.IOException; import java.net.InetAddress; @@ -36,6 +37,7 @@ import java.util.concurrent.ThreadPoolExecutor; import java.util.concurrent.TimeUnit; +import org.apache.commons.io.IOUtils; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -66,8 +68,7 @@ * *

* This class does nothing more. All further peer-to-peer communication happens - * in the {@link com.turn.ttorrent.client.peer.PeerExchange PeerExchange} - * class. + * in the PeerExchange class. *

* * @author mpetazzoni @@ -78,8 +79,8 @@ public class ConnectionHandler implements Runnable { private static final Logger logger = LoggerFactory.getLogger(ConnectionHandler.class); - public static final int PORT_RANGE_START = 6881; - public static final int PORT_RANGE_END = 6889; + public static final int PORT_RANGE_START = 49152; + public static final int PORT_RANGE_END = 65534; private static final int OUTBOUND_CONNECTIONS_POOL_SIZE = 20; private static final int OUTBOUND_CONNECTIONS_THREAD_KEEP_ALIVE_SECS = 10; @@ -309,16 +310,12 @@ private void accept(SocketChannel client) } catch (ParseException pe) { logger.info("Invalid handshake from {}: {}", this.socketRepr(client), pe.getMessage()); - try { client.close(); } catch (IOException e) { } + IOUtils.closeQuietly(client); } catch (IOException ioe) { logger.warn("An error occured while reading an incoming " + "handshake: {}", ioe.getMessage()); - try { - if (client.isConnected()) { - client.close(); - } - } catch (IOException e) { - // Ignore + if (client.isConnected()) { + IOUtils.closeQuietly(client); } } } @@ -395,15 +392,15 @@ private Handshake validateHandshake(SocketChannel channel, byte[] peerId) Handshake hs = Handshake.parse(data); if (!Arrays.equals(hs.getInfoHash(), this.torrent.getInfoHash())) { throw new ParseException("Handshake for unknow torrent " + - Torrent.byteArrayToHexString(hs.getInfoHash()) + + Utils.bytesToHex(hs.getInfoHash()) + " from " + this.socketRepr(channel) + ".", pstrlen + 9); } if (peerId != null && !Arrays.equals(hs.getPeerId(), peerId)) { throw new ParseException("Announced peer ID " + - Torrent.byteArrayToHexString(hs.getPeerId()) + + Utils.bytesToHex(hs.getPeerId()) + " did not match expected peer ID " + - Torrent.byteArrayToHexString(peerId) + ".", pstrlen + 29); + Utils.bytesToHex(peerId) + ".", pstrlen + 29); } return hs; @@ -456,7 +453,7 @@ public Thread newThread(Runnable r) { t.setName("bt-connect-" + ++this.number); return t; } - }; + } /** @@ -504,21 +501,17 @@ public void run() { ? this.peer.getPeerId().array() : null)); logger.info("Handshaked with {}, peer ID is {}.", - this.peer, Torrent.byteArrayToHexString(hs.getPeerId())); + this.peer, Utils.bytesToHex(hs.getPeerId())); // Go to non-blocking mode for peer interaction channel.configureBlocking(false); this.handler.fireNewPeerConnection(channel, hs.getPeerId()); } catch (Exception e) { - try { - if (channel != null && channel.isConnected()) { - channel.close(); - } - } catch (IOException ioe) { - // Ignore + if (channel != null && channel.isConnected()) { + IOUtils.closeQuietly(channel); } this.handler.fireFailedConnection(this.peer, e); } } - }; + } } diff --git a/src/main/java/com/turn/ttorrent/client/Handshake.java b/core/src/main/java/com/turn/ttorrent/client/Handshake.java similarity index 100% rename from src/main/java/com/turn/ttorrent/client/Handshake.java rename to core/src/main/java/com/turn/ttorrent/client/Handshake.java diff --git a/src/main/java/com/turn/ttorrent/client/IncomingConnectionListener.java b/core/src/main/java/com/turn/ttorrent/client/IncomingConnectionListener.java similarity index 100% rename from src/main/java/com/turn/ttorrent/client/IncomingConnectionListener.java rename to core/src/main/java/com/turn/ttorrent/client/IncomingConnectionListener.java diff --git a/src/main/java/com/turn/ttorrent/client/Piece.java b/core/src/main/java/com/turn/ttorrent/client/Piece.java similarity index 95% rename from src/main/java/com/turn/ttorrent/client/Piece.java rename to core/src/main/java/com/turn/ttorrent/client/Piece.java index 77d60640b..36154020d 100644 --- a/src/main/java/com/turn/ttorrent/client/Piece.java +++ b/core/src/main/java/com/turn/ttorrent/client/Piece.java @@ -159,15 +159,13 @@ public synchronized boolean validate() throws IOException { logger.trace("Validating {}...", this); this.valid = false; + ByteBuffer buffer = this._read(0, this.length); + byte[] data = new byte[(int)this.length]; + buffer.get(data); try { - // TODO: remove cast to int when large ByteBuffer support is - // implemented in Java. - ByteBuffer buffer = this._read(0, this.length); - byte[] data = new byte[(int)this.length]; - buffer.get(data); this.valid = Arrays.equals(Torrent.hash(data), this.hash); - } catch (NoSuchAlgorithmException nsae) { - logger.error("{}", nsae); + } catch (NoSuchAlgorithmException e) { + this.valid = false; } return this.isValid(); @@ -283,15 +281,11 @@ public String toString() { * @param other The piece to compare with, should not be null. */ public int compareTo(Piece other) { - if (this == other) { - return 0; - } - - if (this.seen < other.seen) { - return -1; - } else { - return 1; + if (this.seen != other.seen) { + return this.seen < other.seen ? -1 : 1; } + return this.index == other.index ? 0 : + (this.index < other.index ? -1 : 1); } /** diff --git a/src/main/java/com/turn/ttorrent/client/SharedTorrent.java b/core/src/main/java/com/turn/ttorrent/client/SharedTorrent.java similarity index 88% rename from src/main/java/com/turn/ttorrent/client/SharedTorrent.java rename to core/src/main/java/com/turn/ttorrent/client/SharedTorrent.java index 85f4ac310..aec96d517 100644 --- a/src/main/java/com/turn/ttorrent/client/SharedTorrent.java +++ b/core/src/main/java/com/turn/ttorrent/client/SharedTorrent.java @@ -22,20 +22,18 @@ import com.turn.ttorrent.client.storage.TorrentByteStorage; import com.turn.ttorrent.client.storage.FileStorage; import com.turn.ttorrent.client.storage.FileCollectionStorage; +import com.turn.ttorrent.client.strategy.RequestStrategy; +import com.turn.ttorrent.client.strategy.RequestStrategyImplRarest; import java.io.File; -import java.io.FileInputStream; import java.io.FileNotFoundException; import java.io.IOException; import java.nio.ByteBuffer; - import java.security.NoSuchAlgorithmException; -import java.util.ArrayList; import java.util.BitSet; import java.util.Collections; import java.util.LinkedList; import java.util.List; -import java.util.Random; import java.util.SortedSet; import java.util.TreeSet; import java.util.concurrent.Callable; @@ -43,6 +41,7 @@ import java.util.concurrent.Executors; import java.util.concurrent.Future; +import org.apache.commons.io.FileUtils; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -55,11 +54,6 @@ * and logic required by the BitTorrent client implementation. *

* - *

- * Note: this implementation currently only supports single-file - * torrents. - *

- * * @author mpetazzoni */ public class SharedTorrent extends Torrent implements PeerActivityListener { @@ -67,10 +61,6 @@ public class SharedTorrent extends Torrent implements PeerActivityListener { private static final Logger logger = LoggerFactory.getLogger(SharedTorrent.class); - /** Randomly select the next piece to download from a peer from the - * RAREST_PIECE_JITTER available from it. */ - private static final int RAREST_PIECE_JITTER = 42; - /** End-game trigger ratio. * *

@@ -82,7 +72,12 @@ public class SharedTorrent extends Torrent implements PeerActivityListener { */ private static final float ENG_GAME_COMPLETION_RATIO = 0.95f; - private Random random; + /** Default Request Strategy. + * + * Use the rarest-first strategy by default. + */ + private static final RequestStrategy DEFAULT_REQUEST_STRATEGY = new RequestStrategyImplRarest(); + private boolean stop; private long uploaded; @@ -99,7 +94,10 @@ public class SharedTorrent extends Torrent implements PeerActivityListener { private SortedSet rarest; private BitSet completedPieces; private BitSet requestedPieces; - + private RequestStrategy requestStrategy; + + private double maxUploadRate = 0.0; + private double maxDownloadRate = 0.0; /** * Create a new shared torrent from a base Torrent object. * @@ -114,7 +112,6 @@ public class SharedTorrent extends Torrent implements PeerActivityListener { * @throws FileNotFoundException If the torrent file location or * destination directory does not exist and can't be created. * @throws IOException If the torrent file cannot be read or decoded. - * @throws NoSuchAlgorithmException */ public SharedTorrent(Torrent torrent, File destDir) throws FileNotFoundException, IOException, NoSuchAlgorithmException { @@ -137,11 +134,34 @@ public SharedTorrent(Torrent torrent, File destDir) * @throws FileNotFoundException If the torrent file location or * destination directory does not exist and can't be created. * @throws IOException If the torrent file cannot be read or decoded. - * @throws NoSuchAlgorithmException */ public SharedTorrent(Torrent torrent, File destDir, boolean seeder) throws FileNotFoundException, IOException, NoSuchAlgorithmException { - this(torrent.getEncoded(), destDir, seeder); + this(torrent.getEncoded(), destDir, seeder, DEFAULT_REQUEST_STRATEGY); + } + + /** + * Create a new shared torrent from a base Torrent object. + * + *

+ * This will recreate a SharedTorrent object from the provided Torrent + * object's encoded meta-info data. + *

+ * + * @param torrent The Torrent object. + * @param destDir The destination directory or location of the torrent + * files. + * @param seeder Whether we're a seeder for this torrent or not (disables + * validation). + * @param requestStrategy The request strategy implementation. + * @throws FileNotFoundException If the torrent file location or + * destination directory does not exist and can't be created. + * @throws IOException If the torrent file cannot be read or decoded. + */ + public SharedTorrent(Torrent torrent, File destDir, boolean seeder, + RequestStrategy requestStrategy) + throws FileNotFoundException, IOException, NoSuchAlgorithmException { + this(torrent.getEncoded(), destDir, seeder, requestStrategy); } /** @@ -169,12 +189,27 @@ public SharedTorrent(byte[] torrent, File destDir) * @throws FileNotFoundException If the torrent file location or * destination directory does not exist and can't be created. * @throws IOException If the torrent file cannot be read or decoded. - * @throws NoSuchAlgorithmException - * @throws URISyntaxException When one of the defined tracker addresses is - * invalid. */ public SharedTorrent(byte[] torrent, File parent, boolean seeder) throws FileNotFoundException, IOException, NoSuchAlgorithmException { + this(torrent, parent, seeder, DEFAULT_REQUEST_STRATEGY); + } + + /** + * Create a new shared torrent from meta-info binary data. + * + * @param torrent The meta-info byte data. + * @param parent The parent directory or location the torrent files. + * @param seeder Whether we're a seeder for this torrent or not (disables + * validation). + * @param requestStrategy The request strategy implementation. + * @throws FileNotFoundException If the torrent file location or + * destination directory does not exist and can't be created. + * @throws IOException If the torrent file cannot be read or decoded. + */ + public SharedTorrent(byte[] torrent, File parent, boolean seeder, + RequestStrategy requestStrategy) + throws FileNotFoundException, IOException, NoSuchAlgorithmException { super(torrent, seeder); if (parent == null || !parent.isDirectory()) { @@ -214,7 +249,6 @@ public SharedTorrent(byte[] torrent, File parent, boolean seeder) } this.bucket = new FileCollectionStorage(files, this.getSize()); - this.random = new Random(System.currentTimeMillis()); this.stop = false; this.uploaded = 0; @@ -226,6 +260,9 @@ public SharedTorrent(byte[] torrent, File parent, boolean seeder) this.rarest = Collections.synchronizedSortedSet(new TreeSet()); this.completedPieces = new BitSet(); this.requestedPieces = new BitSet(); + + //TODO: should switch to guice + this.requestStrategy = requestStrategy; } /** @@ -235,17 +272,41 @@ public SharedTorrent(byte[] torrent, File parent, boolean seeder) * meta-info from. * @param parent The parent directory or location of the torrent files. * @throws IOException When the torrent file cannot be read or decoded. - * @throws NoSuchAlgorithmException */ public static SharedTorrent fromFile(File source, File parent) throws IOException, NoSuchAlgorithmException { - FileInputStream fis = new FileInputStream(source); - byte[] data = new byte[(int)source.length()]; - fis.read(data); - fis.close(); + byte[] data = FileUtils.readFileToByteArray(source); return new SharedTorrent(data, parent); } + public double getMaxUploadRate() { + return this.maxUploadRate; + } + + /** + * Set the maximum upload rate (in kB/second) for this + * torrent. A setting of <= 0.0 disables rate limiting. + * + * @param rate The maximum upload rate + */ + public void setMaxUploadRate(double rate) { + this.maxUploadRate = rate; + } + + public double getMaxDownloadRate() { + return this.maxDownloadRate; + } + + /** + * Set the maximum download rate (in kB/second) for this + * torrent. A setting of <= 0.0 disables rate limiting. + * + * @param rate The maximum download rate + */ + public void setMaxDownloadRate(double rate) { + this.maxDownloadRate = rate; + } + /** * Get the number of bytes uploaded for this torrent. */ @@ -624,24 +685,7 @@ public synchronized void handlePeerReady(SharingPeer peer) { "that was already requested from another peer."); } - // Extract the RAREST_PIECE_JITTER rarest pieces from the interesting - // pieces of this peer. - ArrayList choice = new ArrayList(RAREST_PIECE_JITTER); - synchronized (this.rarest) { - for (Piece piece : this.rarest) { - if (interesting.get(piece.getIndex())) { - choice.add(piece); - if (choice.size() >= RAREST_PIECE_JITTER) { - break; - } - } - } - } - - Piece chosen = choice.get( - this.random.nextInt( - Math.min(choice.size(), - RAREST_PIECE_JITTER))); + Piece chosen = requestStrategy.choosePiece(rarest, interesting, pieces); this.requestedPieces.set(chosen.getIndex()); logger.trace("Requesting {} from {}, we now have {} " + diff --git a/src/main/java/com/turn/ttorrent/client/announce/Announce.java b/core/src/main/java/com/turn/ttorrent/client/announce/Announce.java similarity index 92% rename from src/main/java/com/turn/ttorrent/client/announce/Announce.java rename to core/src/main/java/com/turn/ttorrent/client/announce/Announce.java index 4c0a1b223..0146b011f 100644 --- a/src/main/java/com/turn/ttorrent/client/announce/Announce.java +++ b/core/src/main/java/com/turn/ttorrent/client/announce/Announce.java @@ -75,8 +75,6 @@ public class Announce implements Runnable { * * @param torrent The torrent we're announcing about. * @param peer Our peer specification. - * @param type A string representing the announce type (used in the thread - * name). */ public Announce(SharedTorrent torrent, Peer peer) { this.peer = peer; @@ -226,7 +224,12 @@ public void run() { event = AnnounceRequestMessage.RequestEvent.NONE; } catch (AnnounceException ae) { logger.warn(ae.getMessage()); - this.moveToNextTrackerClient(); + + try { + this.moveToNextTrackerClient(); + } catch (AnnounceException e) { + logger.error("Unable to move to the next tracker client: {}", e.getMessage()); + } } try { @@ -281,8 +284,15 @@ private TrackerClient createTrackerClient(SharedTorrent torrent, Peer peer, /** * Returns the current tracker client used for announces. + * @throws AnnounceException When the current announce tier isn't defined + * in the torrent. */ - public TrackerClient getCurrentTrackerClient() { + public TrackerClient getCurrentTrackerClient() throws AnnounceException { + if ((this.currentTier >= this.clients.size()) || + (this.currentClient >= this.clients.get(this.currentTier).size())) { + throw new AnnounceException("Current tier or client isn't available"); + } + return this.clients .get(this.currentTier) .get(this.currentClient); @@ -300,8 +310,10 @@ public TrackerClient getCurrentTrackerClient() { * The index of the currently used {@link TrackerClient} is reset to 0 to * reflect this change. *

+ * + * @throws AnnounceException */ - private void promoteCurrentTrackerClient() { + private void promoteCurrentTrackerClient() throws AnnounceException { logger.trace("Promoting current tracker client for {} " + "(tier {}, position {} -> 0).", new Object[] { @@ -327,8 +339,10 @@ private void promoteCurrentTrackerClient() { * By design no empty tier can be in the tracker list structure so we don't * need to check for empty tiers here. *

+ * + * @throws AnnounceException */ - private void moveToNextTrackerClient() { + private void moveToNextTrackerClient() throws AnnounceException { int tier = this.currentTier; int client = this.currentClient + 1; diff --git a/src/main/java/com/turn/ttorrent/client/announce/AnnounceException.java b/core/src/main/java/com/turn/ttorrent/client/announce/AnnounceException.java similarity index 100% rename from src/main/java/com/turn/ttorrent/client/announce/AnnounceException.java rename to core/src/main/java/com/turn/ttorrent/client/announce/AnnounceException.java diff --git a/src/main/java/com/turn/ttorrent/client/announce/AnnounceResponseListener.java b/core/src/main/java/com/turn/ttorrent/client/announce/AnnounceResponseListener.java similarity index 100% rename from src/main/java/com/turn/ttorrent/client/announce/AnnounceResponseListener.java rename to core/src/main/java/com/turn/ttorrent/client/announce/AnnounceResponseListener.java diff --git a/src/main/java/com/turn/ttorrent/client/announce/HTTPTrackerClient.java b/core/src/main/java/com/turn/ttorrent/client/announce/HTTPTrackerClient.java similarity index 100% rename from src/main/java/com/turn/ttorrent/client/announce/HTTPTrackerClient.java rename to core/src/main/java/com/turn/ttorrent/client/announce/HTTPTrackerClient.java diff --git a/src/main/java/com/turn/ttorrent/client/announce/TrackerClient.java b/core/src/main/java/com/turn/ttorrent/client/announce/TrackerClient.java similarity index 98% rename from src/main/java/com/turn/ttorrent/client/announce/TrackerClient.java rename to core/src/main/java/com/turn/ttorrent/client/announce/TrackerClient.java index b757e0e17..7cfa0ece4 100644 --- a/src/main/java/com/turn/ttorrent/client/announce/TrackerClient.java +++ b/core/src/main/java/com/turn/ttorrent/client/announce/TrackerClient.java @@ -83,7 +83,7 @@ public abstract void announce(AnnounceRequestMessage.RequestEvent event, * Close any opened announce connection. * *

- * This method is called by {@link #stop()} to make sure all connections + * This method is called by {@link Announce#stop()} to make sure all connections * are correctly closed when the announce thread is asked to stop. *

*/ diff --git a/src/main/java/com/turn/ttorrent/client/announce/UDPTrackerClient.java b/core/src/main/java/com/turn/ttorrent/client/announce/UDPTrackerClient.java similarity index 98% rename from src/main/java/com/turn/ttorrent/client/announce/UDPTrackerClient.java rename to core/src/main/java/com/turn/ttorrent/client/announce/UDPTrackerClient.java index 01b1c0420..5df3229b0 100644 --- a/src/main/java/com/turn/ttorrent/client/announce/UDPTrackerClient.java +++ b/core/src/main/java/com/turn/ttorrent/client/announce/UDPTrackerClient.java @@ -239,7 +239,7 @@ public void announce(AnnounceRequestMessage.RequestEvent event, * *

* Verifies the transaction ID of the message before passing it over to - * {@link Announce#handleTrackerAnnounceResponse()}. + * any registered {@link AnnounceResponseListener}. *

* * @param message The message received from the tracker in response to the @@ -352,7 +352,7 @@ private void send(ByteBuffer data) { * * @param attempt The attempt number, used to calculate the timeout for the * receive operation. - * @retun Returns a {@link ByteBuffer} containing the packet data. + * @return Returns a {@link ByteBuffer} containing the packet data. */ private ByteBuffer recv(int attempt) throws IOException, SocketException, SocketTimeoutException { diff --git a/src/main/java/com/turn/ttorrent/client/peer/MessageListener.java b/core/src/main/java/com/turn/ttorrent/client/peer/MessageListener.java similarity index 100% rename from src/main/java/com/turn/ttorrent/client/peer/MessageListener.java rename to core/src/main/java/com/turn/ttorrent/client/peer/MessageListener.java diff --git a/src/main/java/com/turn/ttorrent/client/peer/PeerActivityListener.java b/core/src/main/java/com/turn/ttorrent/client/peer/PeerActivityListener.java similarity index 100% rename from src/main/java/com/turn/ttorrent/client/peer/PeerActivityListener.java rename to core/src/main/java/com/turn/ttorrent/client/peer/PeerActivityListener.java diff --git a/src/main/java/com/turn/ttorrent/client/peer/PeerExchange.java b/core/src/main/java/com/turn/ttorrent/client/peer/PeerExchange.java similarity index 61% rename from src/main/java/com/turn/ttorrent/client/peer/PeerExchange.java rename to core/src/main/java/com/turn/ttorrent/client/peer/PeerExchange.java index a7448d00c..fab7a5fa5 100644 --- a/src/main/java/com/turn/ttorrent/client/peer/PeerExchange.java +++ b/core/src/main/java/com/turn/ttorrent/client/peer/PeerExchange.java @@ -17,6 +17,7 @@ import com.turn.ttorrent.client.SharedTorrent; import com.turn.ttorrent.common.protocol.PeerMessage; +import com.turn.ttorrent.common.protocol.PeerMessage.Type; import java.io.EOFException; import java.io.IOException; @@ -24,14 +25,18 @@ import java.net.SocketException; import java.nio.ByteBuffer; import java.nio.channels.SocketChannel; +import java.nio.channels.Selector; +import java.nio.channels.SelectionKey; import java.text.ParseException; import java.util.BitSet; import java.util.HashSet; import java.util.Set; +import java.util.Iterator; import java.util.concurrent.BlockingQueue; import java.util.concurrent.LinkedBlockingQueue; import java.util.concurrent.TimeUnit; +import org.apache.commons.io.IOUtils; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -73,6 +78,7 @@ class PeerExchange { LoggerFactory.getLogger(PeerExchange.class); private static final int KEEP_ALIVE_IDLE_MINUTES = 2; + private static final PeerMessage STOP = PeerMessage.KeepAliveMessage.craft(); private SharingPeer peer; private SharedTorrent torrent; @@ -115,10 +121,7 @@ public PeerExchange(SharingPeer peer, SharedTorrent torrent, this.peer.getShortHexPeerId() + ")-send"); this.out.setDaemon(true); - // Automatically start the exchange activity loops this.stop = false; - this.in.start(); - this.out.start(); logger.debug("Started peer exchange with {} for {}.", this.peer, this.torrent); @@ -126,10 +129,11 @@ public PeerExchange(SharingPeer peer, SharedTorrent torrent, // If we have pieces, start by sending a BITFIELD message to the peer. BitSet pieces = this.torrent.getCompletedPieces(); if (pieces.cardinality() > 0) { - this.send(PeerMessage.BitfieldMessage.craft(pieces)); + this.send(PeerMessage.BitfieldMessage.craft(pieces, torrent.getPieceCount())); } } + /** * Register a new message listener to receive messages. * @@ -167,26 +171,102 @@ public void send(PeerMessage message) { } /** - * Close and stop the peer exchange. + * Start the peer exchange. + * + *

+ * Starts both incoming and outgoing thread. + *

+ */ + public void start() { + this.in.start(); + this.out.start(); + } + + /** + * Stop the peer exchange. * *

* Closes the socket channel and stops both incoming and outgoing threads. *

*/ - public void close() { + public void stop() { this.stop = true; + try { + // Wake-up and shutdown out-going thread immediately + this.sendQueue.put(STOP); + } catch (InterruptedException e) { + Thread.currentThread().interrupt(); + } + if (this.channel.isConnected()) { - try { - this.channel.close(); - } catch (IOException ioe) { - // Ignore - } + IOUtils.closeQuietly(this.channel); } logger.debug("Peer exchange with {} closed.", this.peer); } + /** + * Abstract Thread subclass that allows conditional rate limiting + * for PIECE messages. + * + *

+ * To impose rate limits, we only want to throttle when processing PIECE + * messages. All other peer messages should be exchanged as quickly as + * possible. + *

+ * + * @author ptgoetz + */ + private abstract class RateLimitThread extends Thread { + + protected final Rate rate = new Rate(); + protected long sleep = 1000; + + /** + * Dynamically determines an amount of time to sleep, based on the + * average read/write throughput. + * + *

+ * The algorithm is functional, but could certainly be improved upon. + * One obvious drawback is that with large changes in + * maxRate, it will take a while for the sleep time to + * adjust and the throttled rate to "smooth out." + *

+ * + *

+ * Ideally, it would calculate the optimal sleep time necessary to hit + * a desired throughput rather than continuously adjust toward a goal. + *

+ * + * @param maxRate the target rate in kB/second. + * @param messageSize the size, in bytes, of the last message read/written. + * @param message the last PeerMessage read/written. + */ + protected void rateLimit(double maxRate, long messageSize, PeerMessage message) { + if (message.getType() != Type.PIECE || maxRate <= 0) { + return; + } + + try { + this.rate.add(messageSize); + + // Continuously adjust the sleep time to try to hit our target + // rate limit. + if (rate.get() > (maxRate * 1024)) { + Thread.sleep(this.sleep); + this.sleep += 50; + } else { + this.sleep = this.sleep > 50 + ? this.sleep - 50 + : 0; + } + } catch (InterruptedException e) { + // Not critical, eat it. + } + } + } + /** * Incoming messages thread. * @@ -199,64 +279,110 @@ public void close() { * * @author mpetazzoni */ - private class IncomingThread extends Thread { + private class IncomingThread extends RateLimitThread { + + /** + * Read data from the incoming channel of the socket using a {@link + * Selector}. + * + * @param selector The socket selector into which the peer socket has + * been inserted. + * @param buffer A {@link ByteBuffer} to put the read data into. + * @return The number of bytes read. + */ + private long read(Selector selector, ByteBuffer buffer) throws IOException { + if (selector.select() == 0 || !buffer.hasRemaining()) { + return 0; + } + + long size = 0; + Iterator it = selector.selectedKeys().iterator(); + while (it.hasNext()) { + SelectionKey key = (SelectionKey) it.next(); + if (key.isValid() && key.isReadable()) { + int read = ((SocketChannel) key.channel()).read(buffer); + if (read < 0) { + throw new IOException("Unexpected end-of-stream while reading"); + } + size += read; + } + it.remove(); + } + + return size; + } + + private void handleIOE(IOException ioe) { + logger.debug("Could not read message from {}: {}", + peer, + ioe.getMessage() != null + ? ioe.getMessage() + : ioe.getClass().getName()); + peer.unbind(true); + } @Override public void run() { ByteBuffer buffer = ByteBuffer.allocateDirect(1*1024*1024); + Selector selector = null; try { + selector = Selector.open(); + channel.register(selector, SelectionKey.OP_READ); + while (!stop) { buffer.rewind(); buffer.limit(PeerMessage.MESSAGE_LENGTH_FIELD_SIZE); - if (channel.read(buffer) < 0) { - throw new EOFException( - "Reached end-of-stream while reading size header"); - } - // Keep reading bytes until the length field has been read // entirely. - if (buffer.hasRemaining()) { - try { - Thread.sleep(1); - } catch (InterruptedException ie) { - // Ignore and move along. - } - - continue; + while (!stop && buffer.hasRemaining()) { + this.read(selector, buffer); } + // Reset the buffer limit to the expected message size. int pstrlen = buffer.getInt(0); buffer.limit(PeerMessage.MESSAGE_LENGTH_FIELD_SIZE + pstrlen); + long size = 0; while (!stop && buffer.hasRemaining()) { - if (channel.read(buffer) < 0) { - throw new EOFException( - "Reached end-of-stream while reading message"); - } + size += this.read(selector, buffer); } buffer.rewind(); - + + if (stop) { + // The buffer may contain the type from the last message + // if we were stopped before reading the payload and cause + // BufferUnderflowException in parsing. + break; + } + try { PeerMessage message = PeerMessage.parse(buffer, torrent); logger.trace("Received {} from {}", message, peer); - for (MessageListener listener : listeners) { + // Wait if needed to reach configured download rate. + this.rateLimit( + PeerExchange.this.torrent.getMaxDownloadRate(), + size, message); + + for (MessageListener listener : listeners) listener.handleMessage(message); - } } catch (ParseException pe) { logger.warn("{}", pe.getMessage()); } } } catch (IOException ioe) { - logger.debug("Could not read message from {}: {}", - peer, - ioe.getMessage() != null - ? ioe.getMessage() - : ioe.getClass().getName()); - peer.unbind(true); + this.handleIOE(ioe); + } finally { + try { + if (selector != null) { + selector.close(); + } + } catch (IOException ioe) { + this.handleIOE(ioe); + } } } } @@ -277,7 +403,7 @@ public void run() { * * @author mpetazzoni */ - private class OutgoingThread extends Thread { + private class OutgoingThread extends RateLimitThread { @Override public void run() { @@ -291,23 +417,30 @@ public void run() { PeerExchange.KEEP_ALIVE_IDLE_MINUTES, TimeUnit.MINUTES); - if (message == null) { - if (stop) { - return; - } + if (message == STOP) { + return; + } + if (message == null) { message = PeerMessage.KeepAliveMessage.craft(); } logger.trace("Sending {} to {}", message, peer); ByteBuffer data = message.getData(); + long size = 0; while (!stop && data.hasRemaining()) { - if (channel.write(data) < 0) { + int written = channel.write(data); + size += written; + if (written < 0) { throw new EOFException( "Reached end of stream while writing"); } } + + // Wait if needed to reach configured upload rate. + this.rateLimit(PeerExchange.this.torrent.getMaxUploadRate(), + size, message); } catch (InterruptedException ie) { // Ignore and potentially terminate } diff --git a/src/main/java/com/turn/ttorrent/client/peer/Rate.java b/core/src/main/java/com/turn/ttorrent/client/peer/Rate.java similarity index 100% rename from src/main/java/com/turn/ttorrent/client/peer/Rate.java rename to core/src/main/java/com/turn/ttorrent/client/peer/Rate.java diff --git a/src/main/java/com/turn/ttorrent/client/peer/SharingPeer.java b/core/src/main/java/com/turn/ttorrent/client/peer/SharingPeer.java similarity index 99% rename from src/main/java/com/turn/ttorrent/client/peer/SharingPeer.java rename to core/src/main/java/com/turn/ttorrent/client/peer/SharingPeer.java index c24244ec6..e0fb53c01 100644 --- a/src/main/java/com/turn/ttorrent/client/peer/SharingPeer.java +++ b/core/src/main/java/com/turn/ttorrent/client/peer/SharingPeer.java @@ -270,6 +270,7 @@ public synchronized void bind(SocketChannel channel) throws SocketException { this.exchange = new PeerExchange(this, this.torrent, channel); this.exchange.register(this); + this.exchange.start(); this.download = new Rate(); this.download.reset(); @@ -308,7 +309,7 @@ public void unbind(boolean force) { synchronized (this.exchangeLock) { if (this.exchange != null) { - this.exchange.close(); + this.exchange.stop(); this.exchange = null; } } @@ -747,7 +748,7 @@ private void fireIOException(IOException ioe) { *

* * @author mpetazzoni - * @see Rate.RateComparator + * @see Rate#RATE_COMPARATOR */ public static class DLRateComparator implements Comparator, Serializable { @@ -768,7 +769,7 @@ public int compare(SharingPeer a, SharingPeer b) { *

* * @author mpetazzoni - * @see Rate.RateComparator + * @see Rate#RATE_COMPARATOR */ public static class ULRateComparator implements Comparator, Serializable { diff --git a/src/main/java/com/turn/ttorrent/client/storage/FileCollectionStorage.java b/core/src/main/java/com/turn/ttorrent/client/storage/FileCollectionStorage.java similarity index 99% rename from src/main/java/com/turn/ttorrent/client/storage/FileCollectionStorage.java rename to core/src/main/java/com/turn/ttorrent/client/storage/FileCollectionStorage.java index c753132a6..6bb50d3b6 100644 --- a/src/main/java/com/turn/ttorrent/client/storage/FileCollectionStorage.java +++ b/core/src/main/java/com/turn/ttorrent/client/storage/FileCollectionStorage.java @@ -182,7 +182,7 @@ private List select(long offset, long length) { long bytes = 0; for (FileStorage file : this.files) { - if (file.offset() > offset + length) { + if (file.offset() >= offset + length) { break; } diff --git a/src/main/java/com/turn/ttorrent/client/storage/FileStorage.java b/core/src/main/java/com/turn/ttorrent/client/storage/FileStorage.java similarity index 95% rename from src/main/java/com/turn/ttorrent/client/storage/FileStorage.java rename to core/src/main/java/com/turn/ttorrent/client/storage/FileStorage.java index a47f053cb..05e8207bf 100644 --- a/src/main/java/com/turn/ttorrent/client/storage/FileStorage.java +++ b/core/src/main/java/com/turn/ttorrent/client/storage/FileStorage.java @@ -80,9 +80,11 @@ public FileStorage(File file, long offset, long size) this.raf = new RandomAccessFile(this.current, "rw"); - // Set the file length to the appropriate size, eventually truncating - // or extending the file if it already exists with a different size. - this.raf.setLength(this.size); + if (file.length() != this.size) { + // Set the file length to the appropriate size, eventually truncating + // or extending the file if it already exists with a different size. + this.raf.setLength(this.size); + } this.channel = raf.getChannel(); logger.info("Initialized byte storage file at {} " + diff --git a/src/main/java/com/turn/ttorrent/client/storage/TorrentByteStorage.java b/core/src/main/java/com/turn/ttorrent/client/storage/TorrentByteStorage.java similarity index 100% rename from src/main/java/com/turn/ttorrent/client/storage/TorrentByteStorage.java rename to core/src/main/java/com/turn/ttorrent/client/storage/TorrentByteStorage.java diff --git a/core/src/main/java/com/turn/ttorrent/client/strategy/RequestStrategy.java b/core/src/main/java/com/turn/ttorrent/client/strategy/RequestStrategy.java new file mode 100644 index 000000000..f1cb4bf50 --- /dev/null +++ b/core/src/main/java/com/turn/ttorrent/client/strategy/RequestStrategy.java @@ -0,0 +1,29 @@ +package com.turn.ttorrent.client.strategy; + +import java.util.BitSet; +import java.util.SortedSet; + +import com.turn.ttorrent.client.Piece; + +/** + * Interface for a piece request strategy provider. + * + * @author cjmalloy + * + */ +public interface RequestStrategy { + + /** + * Choose a piece from the remaining pieces. + * + * @param rarest + * A set sorted by how rare the piece is + * @param interesting + * A set of the index of all interesting pieces + * @param pieces + * The complete array of pieces + * + * @return The chosen piece, or null if no piece is interesting + */ + Piece choosePiece(SortedSet rarest, BitSet interesting, Piece[] pieces); +} diff --git a/core/src/main/java/com/turn/ttorrent/client/strategy/RequestStrategyImplRarest.java b/core/src/main/java/com/turn/ttorrent/client/strategy/RequestStrategyImplRarest.java new file mode 100644 index 000000000..1bdc85920 --- /dev/null +++ b/core/src/main/java/com/turn/ttorrent/client/strategy/RequestStrategyImplRarest.java @@ -0,0 +1,52 @@ +package com.turn.ttorrent.client.strategy; + +import java.util.ArrayList; +import java.util.BitSet; +import java.util.Random; +import java.util.SortedSet; + +import com.turn.ttorrent.client.Piece; + +/** + * The default request strategy implementation- rarest first. + * + * @author cjmalloy + * + */ +public class RequestStrategyImplRarest implements RequestStrategy { + + /** Randomly select the next piece to download from a peer from the + * RAREST_PIECE_JITTER available from it. */ + private static final int RAREST_PIECE_JITTER = 42; + + private Random random; + + public RequestStrategyImplRarest() { + this.random = new Random(System.currentTimeMillis()); + } + + @Override + public Piece choosePiece(SortedSet rarest, BitSet interesting, Piece[] pieces) { + // Extract the RAREST_PIECE_JITTER rarest pieces from the interesting + // pieces of this peer. + ArrayList choice = new ArrayList(RAREST_PIECE_JITTER); + synchronized (rarest) { + for (Piece piece : rarest) { + if (interesting.get(piece.getIndex())) { + choice.add(piece); + if (choice.size() >= RAREST_PIECE_JITTER) { + break; + } + } + } + } + + if (choice.size() == 0) return null; + + Piece chosen = choice.get( + this.random.nextInt( + Math.min(choice.size(), + RAREST_PIECE_JITTER))); + return chosen; + } +} diff --git a/core/src/main/java/com/turn/ttorrent/client/strategy/RequestStrategyImplSequential.java b/core/src/main/java/com/turn/ttorrent/client/strategy/RequestStrategyImplSequential.java new file mode 100644 index 000000000..92bc69995 --- /dev/null +++ b/core/src/main/java/com/turn/ttorrent/client/strategy/RequestStrategyImplSequential.java @@ -0,0 +1,24 @@ +package com.turn.ttorrent.client.strategy; + +import java.util.BitSet; +import java.util.SortedSet; + +import com.turn.ttorrent.client.Piece; + +/** + * A sequential request strategy implementation. + * + * @author cjmalloy + * + */ +public class RequestStrategyImplSequential implements RequestStrategy { + + @Override + public Piece choosePiece(SortedSet rarest, BitSet interesting, Piece[] pieces) { + + for (Piece p : pieces) { + if (interesting.get(p.getIndex())) return p; + } + return null; + } +} diff --git a/src/main/java/com/turn/ttorrent/common/Peer.java b/core/src/main/java/com/turn/ttorrent/common/Peer.java similarity index 98% rename from src/main/java/com/turn/ttorrent/common/Peer.java rename to core/src/main/java/com/turn/ttorrent/common/Peer.java index 86af9fcf0..38745d897 100644 --- a/src/main/java/com/turn/ttorrent/common/Peer.java +++ b/core/src/main/java/com/turn/ttorrent/common/Peer.java @@ -107,7 +107,7 @@ public ByteBuffer getPeerId() { public void setPeerId(ByteBuffer peerId) { if (peerId != null) { this.peerId = peerId; - this.hexPeerId = Torrent.byteArrayToHexString(peerId.array()); + this.hexPeerId = Utils.bytesToHex(peerId.array()); } else { this.peerId = null; this.hexPeerId = null; diff --git a/src/main/java/com/turn/ttorrent/common/Torrent.java b/core/src/main/java/com/turn/ttorrent/common/Torrent.java similarity index 76% rename from src/main/java/com/turn/ttorrent/common/Torrent.java rename to core/src/main/java/com/turn/ttorrent/common/Torrent.java index e468dc560..2b3af8812 100644 --- a/src/main/java/com/turn/ttorrent/common/Torrent.java +++ b/core/src/main/java/com/turn/ttorrent/common/Torrent.java @@ -23,10 +23,8 @@ import java.io.ByteArrayOutputStream; import java.io.File; import java.io.FileInputStream; -import java.io.FileOutputStream; import java.io.IOException; import java.io.OutputStream; -import java.io.PrintStream; import java.io.UnsupportedEncodingException; import java.math.BigInteger; import java.net.URI; @@ -51,12 +49,7 @@ import java.util.concurrent.Executors; import java.util.concurrent.Future; -import jargs.gnu.CmdLineParser; - -import org.apache.log4j.BasicConfigurator; -import org.apache.log4j.ConsoleAppender; -import org.apache.log4j.PatternLayout; - +import org.apache.commons.io.FileUtils; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -83,7 +76,7 @@ public class Torrent { LoggerFactory.getLogger(Torrent.class); /** Torrent file piece length (in bytes), we use 512 kB. */ - private static final int PIECE_LENGTH = 512 * 1024; + public static final int DEFAULT_PIECE_LENGTH = 512 * 1024; public static final int PIECE_HASH_SIZE = 20; @@ -104,7 +97,7 @@ public TorrentFile(File file, long size) { this.file = file; this.size = size; } - }; + } protected final byte[] encoded; @@ -122,6 +115,8 @@ public TorrentFile(File file, long size) { private final String createdBy; private final String name; private final long size; + private final int pieceLength; + protected final List files; private final boolean seeder; @@ -133,15 +128,11 @@ public TorrentFile(File file, long size) { * BitTorrent specification) and create a Torrent object from it. * * @param torrent The meta-info byte data. - * @param parent The parent directory or location of the torrent files. * @param seeder Whether we'll be seeding for this torrent or not. * @throws IOException When the info dictionary can't be read or * encoded and hashed back to create the torrent's SHA-1 hash. - * @throws NoSuchAlgorithmException If the SHA-1 algorithm is not - * available. */ - public Torrent(byte[] torrent, boolean seeder) - throws IOException, NoSuchAlgorithmException { + public Torrent(byte[] torrent, boolean seeder) throws IOException, NoSuchAlgorithmException { this.encoded = torrent; this.seeder = seeder; @@ -153,7 +144,7 @@ public Torrent(byte[] torrent, boolean seeder) BEncoder.bencode(this.decoded_info, baos); this.encoded_info = baos.toByteArray(); this.info_hash = Torrent.hash(this.encoded_info); - this.hex_info_hash = Torrent.byteArrayToHexString(this.info_hash); + this.hex_info_hash = Utils.bytesToHex(this.info_hash); /** * Parses the announce information from the decoded meta-info @@ -219,6 +210,7 @@ public Torrent(byte[] torrent, boolean seeder) ? this.decoded.get("created by").getString() : null; this.name = this.decoded_info.get("name").getString(); + this.pieceLength = this.decoded_info.get("piece length").getInt(); this.files = new LinkedList(); @@ -413,20 +405,11 @@ public void save(OutputStream output) throws IOException { } public static byte[] hash(byte[] data) throws NoSuchAlgorithmException { - MessageDigest md = MessageDigest.getInstance("SHA-1"); - md.update(data); - return md.digest(); - } - - /** - * Convert a byte string to a string containing an hexadecimal - * representation of the original data. - * - * @param bytes The byte array to convert. - */ - public static String byteArrayToHexString(byte[] bytes) { - BigInteger bi = new BigInteger(1, bytes); - return String.format("%0" + (bytes.length << 1) + "X", bi); + MessageDigest crypt; + crypt = MessageDigest.getInstance("SHA-1"); + crypt.reset(); + crypt.update(data); + return crypt.digest(); } /** @@ -438,7 +421,7 @@ public static String byteArrayToHexString(byte[] bytes) { public static String toHexString(String input) { try { byte[] bytes = input.getBytes(Torrent.BYTE_ENCODING); - return Torrent.byteArrayToHexString(bytes); + return Utils.bytesToHex(bytes); } catch (UnsupportedEncodingException uee) { return null; } @@ -485,10 +468,8 @@ protected static int getHashingThreadsCount() { * @param torrent The abstract {@link File} object representing the * .torrent file to load. * @throws IOException When the torrent file cannot be read. - * @throws NoSuchAlgorithmException */ - public static Torrent load(File torrent) - throws IOException, NoSuchAlgorithmException { + public static Torrent load(File torrent) throws IOException, NoSuchAlgorithmException { return Torrent.load(torrent, false); } @@ -500,21 +481,11 @@ public static Torrent load(File torrent) * @param seeder Whether we are a seeder for this torrent or not (disables * local data validation). * @throws IOException When the torrent file cannot be read. - * @throws NoSuchAlgorithmException */ public static Torrent load(File torrent, boolean seeder) throws IOException, NoSuchAlgorithmException { - FileInputStream fis = null; - try { - fis = new FileInputStream(torrent); - byte[] data = new byte[(int)torrent.length()]; - fis.read(data); - return new Torrent(data, seeder); - } finally { - if (fis != null) { - fis.close(); - } - } + byte[] data = FileUtils.readFileToByteArray(torrent); + return new Torrent(data, seeder); } /** Torrent creation --------------------------------------------------- */ @@ -534,8 +505,9 @@ public static Torrent load(File torrent, boolean seeder) * torrent's creator. */ public static Torrent create(File source, URI announce, String createdBy) - throws NoSuchAlgorithmException, InterruptedException, IOException { - return Torrent.create(source, null, announce, null, createdBy); + throws InterruptedException, IOException, NoSuchAlgorithmException { + return Torrent.create(source, null, DEFAULT_PIECE_LENGTH, + announce, null, createdBy); } /** @@ -556,9 +528,9 @@ public static Torrent create(File source, URI announce, String createdBy) * torrent's creator. */ public static Torrent create(File parent, List files, URI announce, - String createdBy) throws NoSuchAlgorithmException, - InterruptedException, IOException { - return Torrent.create(parent, files, announce, null, createdBy); + String createdBy) throws InterruptedException, IOException, NoSuchAlgorithmException { + return Torrent.create(parent, files, DEFAULT_PIECE_LENGTH, + announce, null, createdBy); } /** @@ -576,10 +548,10 @@ public static Torrent create(File parent, List files, URI announce, * @param createdBy The creator's name, or any string identifying the * torrent's creator. */ - public static Torrent create(File source, List> announceList, - String createdBy) throws NoSuchAlgorithmException, - InterruptedException, IOException { - return Torrent.create(source, null, null, announceList, createdBy); + public static Torrent create(File source, int pieceLength, List> announceList, + String createdBy) throws InterruptedException, IOException, NoSuchAlgorithmException { + return Torrent.create(source, null, pieceLength, + null, announceList, createdBy); } /** @@ -592,7 +564,7 @@ public static Torrent create(File source, List> announceList, * considering we'll be a full initial seeder for it. *

* - * @param parent The parent directory or location of the torrent files, + * @param source The parent directory or location of the torrent files, * also used as the torrent's name. * @param files The files to add into this torrent. * @param announceList The announce URIs organized as tiers that will @@ -600,10 +572,11 @@ public static Torrent create(File source, List> announceList, * @param createdBy The creator's name, or any string identifying the * torrent's creator. */ - public static Torrent create(File source, List files, + public static Torrent create(File source, List files, int pieceLength, List> announceList, String createdBy) - throws NoSuchAlgorithmException, InterruptedException, IOException { - return Torrent.create(source, files, null, announceList, createdBy); + throws InterruptedException, IOException, NoSuchAlgorithmException { + return Torrent.create(source, files, pieceLength, + null, announceList, createdBy); } /** @@ -625,9 +598,9 @@ public static Torrent create(File source, List files, * @param createdBy The creator's name, or any string identifying the * torrent's creator. */ - private static Torrent create(File parent, List files, URI announce, - List> announceList, String createdBy) - throws NoSuchAlgorithmException, InterruptedException, IOException { + private static Torrent create(File parent, List files, int pieceLength, + URI announce, List> announceList, String createdBy) + throws InterruptedException, IOException, NoSuchAlgorithmException { if (files == null || files.isEmpty()) { logger.info("Creating single-file torrent for {}...", parent.getName()); @@ -658,11 +631,11 @@ private static Torrent create(File parent, List files, URI announce, Map info = new TreeMap(); info.put("name", new BEValue(parent.getName())); - info.put("piece length", new BEValue(Torrent.PIECE_LENGTH)); + info.put("piece length", new BEValue(pieceLength)); if (files == null || files.isEmpty()) { info.put("length", new BEValue(parent.length())); - info.put("pieces", new BEValue(Torrent.hashFile(parent), + info.put("pieces", new BEValue(Torrent.hashFile(parent, pieceLength), Torrent.BYTE_ENCODING)); } else { List fileInfo = new LinkedList(); @@ -684,7 +657,7 @@ private static Torrent create(File parent, List files, URI announce, fileInfo.add(new BEValue(fileMap)); } info.put("files", new BEValue(fileInfo)); - info.put("pieces", new BEValue(Torrent.hashFiles(files), + info.put("pieces", new BEValue(Torrent.hashFiles(files, pieceLength), Torrent.BYTE_ENCODING)); } torrent.put("info", new BEValue(info)); @@ -704,8 +677,7 @@ private static class CallableChunkHasher implements Callable { private final MessageDigest md; private final ByteBuffer data; - CallableChunkHasher(ByteBuffer buffer) - throws NoSuchAlgorithmException { + CallableChunkHasher(ByteBuffer buffer) throws NoSuchAlgorithmException { this.md = MessageDigest.getInstance("SHA-1"); this.data = ByteBuffer.allocate(buffer.remaining()); @@ -738,16 +710,16 @@ public String call() throws UnsupportedEncodingException { * * @param file The file to hash. */ - private static String hashFile(File file) - throws NoSuchAlgorithmException, InterruptedException, IOException { - return Torrent.hashFiles(Arrays.asList(new File[] { file })); + private static String hashFile(File file, int pieceLenght) + throws InterruptedException, IOException, NoSuchAlgorithmException { + return Torrent.hashFiles(Arrays.asList(new File[] { file }), pieceLenght); } - private static String hashFiles(List files) - throws NoSuchAlgorithmException, InterruptedException, IOException { + private static String hashFiles(List files, int pieceLenght) + throws InterruptedException, IOException, NoSuchAlgorithmException { int threads = getHashingThreadsCount(); ExecutorService executor = Executors.newFixedThreadPool(threads); - ByteBuffer buffer = ByteBuffer.allocate(Torrent.PIECE_LENGTH); + ByteBuffer buffer = ByteBuffer.allocate(pieceLenght); List> results = new LinkedList>(); StringBuilder hashes = new StringBuilder(); @@ -761,7 +733,7 @@ private static String hashFiles(List files) file.getName(), threads, (int) (Math.ceil( - (double)file.length() / Torrent.PIECE_LENGTH)) + (double)file.length() / pieceLenght)) }); length += file.length(); @@ -810,7 +782,7 @@ private static String hashFiles(List files) long elapsed = System.nanoTime() - start; int expectedPieces = (int) (Math.ceil( - (double)length / Torrent.PIECE_LENGTH)); + (double)length / pieceLenght)); logger.info("Hashed {} file(s) ({} bytes) in {} pieces ({} expected) in {}ms.", new Object[] { files.size(), @@ -843,131 +815,4 @@ private static int accumulateHashes(StringBuilder hashes, throw new IOException("Error while hashing the torrent data!", ee); } } - - /** - * Display program usage on the given {@link PrintStream}. - */ - private static void usage(PrintStream s) { - usage(s, null); - } - - /** - * Display a message and program usage on the given {@link PrintStream}. - */ - private static void usage(PrintStream s, String msg) { - if (msg != null) { - s.println(msg); - s.println(); - } - - s.println("usage: Torrent [options] [file|directory]"); - s.println(); - s.println("Available options:"); - s.println(" -h,--help Show this help and exit."); - s.println(" -t,--torrent FILE Use FILE to read/write torrent file."); - s.println(); - s.println(" -c,--create Create a new torrent file using " + - "the given announce URL and data."); - s.println(" -a,--announce Tracker URL (can be repeated)."); - s.println(); - } - - /** - * Torrent reader and creator. - * - *

- * You can use the {@code main()} function of this {@link Torrent} class to - * read or create torrent files. See usage for details. - *

- * - * TODO: support multiple announce URLs. - */ - public static void main(String[] args) { - BasicConfigurator.configure(new ConsoleAppender( - new PatternLayout("%-5p: %m%n"))); - - CmdLineParser parser = new CmdLineParser(); - CmdLineParser.Option help = parser.addBooleanOption('h', "help"); - CmdLineParser.Option filename = parser.addStringOption('t', "torrent"); - CmdLineParser.Option create = parser.addBooleanOption('c', "create"); - CmdLineParser.Option announce = parser.addStringOption('a', "announce"); - - try { - parser.parse(args); - } catch (CmdLineParser.OptionException oe) { - System.err.println(oe.getMessage()); - usage(System.err); - System.exit(1); - } - - // Display help and exit if requested - if (Boolean.TRUE.equals((Boolean)parser.getOptionValue(help))) { - usage(System.out); - System.exit(0); - } - - String filenameValue = (String)parser.getOptionValue(filename); - if (filenameValue == null) { - usage(System.err, "Torrent file must be provided!"); - System.exit(1); - } - - Boolean createFlag = (Boolean)parser.getOptionValue(create); - String announceURL = (String)parser.getOptionValue(announce); - - String[] otherArgs = parser.getRemainingArgs(); - - if (Boolean.TRUE.equals(createFlag) && - (otherArgs.length != 1 || announceURL == null)) { - usage(System.err, "Announce URL and a file or directory must be " + - "provided to create a torrent file!"); - System.exit(1); - } - - OutputStream fos = null; - try { - if (Boolean.TRUE.equals(createFlag)) { - if (filenameValue != null) { - fos = new FileOutputStream(filenameValue); - } else { - fos = System.out; - } - - URI announceURI = new URI(announceURL); - File source = new File(otherArgs[0]); - if (!source.exists() || !source.canRead()) { - throw new IllegalArgumentException( - "Cannot access source file or directory " + - source.getName()); - } - - String creator = String.format("%s (ttorrent)", - System.getProperty("user.name")); - - Torrent torrent = null; - if (source.isDirectory()) { - File[] files = source.listFiles(); - Arrays.sort(files); - torrent = Torrent.create(source, Arrays.asList(files), - announceURI, creator); - } else { - torrent = Torrent.create(source, announceURI, creator); - } - - torrent.save(fos); - } else { - Torrent.load(new File(filenameValue), true); - } - } catch (Exception e) { - logger.error("{}", e.getMessage(), e); - System.exit(2); - } finally { - if (fos != null && fos != System.out) { - try { - fos.close(); - } catch (IOException ioe) { - } - } - } - } } diff --git a/core/src/main/java/com/turn/ttorrent/common/Utils.java b/core/src/main/java/com/turn/ttorrent/common/Utils.java new file mode 100644 index 000000000..25ac88e6e --- /dev/null +++ b/core/src/main/java/com/turn/ttorrent/common/Utils.java @@ -0,0 +1,41 @@ +/** + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + *

+ * http://www.apache.org/licenses/LICENSE-2.0 + *

+ * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.turn.ttorrent.common; + +public class Utils { + + private final static char[] HEX_SYMBOLS = "0123456789ABCDEF".toCharArray(); + + private Utils() { + } + + /** + * Convert a byte string to a string containing the hexadecimal + * representation of the original data. + * + * @param bytes The byte array to convert. + * @see http://stackoverflow.com/questions/332079 + */ + public static String bytesToHex(byte[] bytes) { + char[] hexChars = new char[bytes.length * 2]; + for (int j = 0; j < bytes.length; j++) { + int v = bytes[j] & 0xFF; + hexChars[j * 2] = HEX_SYMBOLS[v >>> 4]; + hexChars[j * 2 + 1] = HEX_SYMBOLS[v & 0x0F]; + } + return new String(hexChars); + } + +} diff --git a/src/main/java/com/turn/ttorrent/common/protocol/PeerMessage.java b/core/src/main/java/com/turn/ttorrent/common/protocol/PeerMessage.java similarity index 93% rename from src/main/java/com/turn/ttorrent/common/protocol/PeerMessage.java rename to core/src/main/java/com/turn/ttorrent/common/protocol/PeerMessage.java index d9d01d31b..2c7075ca3 100644 --- a/src/main/java/com/turn/ttorrent/common/protocol/PeerMessage.java +++ b/core/src/main/java/com/turn/ttorrent/common/protocol/PeerMessage.java @@ -198,7 +198,7 @@ public MessageValidationException(PeerMessage m) { /** * Keep alive message. * - * + * <len=0000> */ public static class KeepAliveMessage extends PeerMessage { @@ -225,7 +225,7 @@ public static KeepAliveMessage craft() { /** * Choke message. * - * + * <len=0001><id=0> */ public static class ChokeMessage extends PeerMessage { @@ -253,7 +253,7 @@ public static ChokeMessage craft() { /** * Unchoke message. * - * + * <len=0001><id=1> */ public static class UnchokeMessage extends PeerMessage { @@ -281,7 +281,7 @@ public static UnchokeMessage craft() { /** * Interested message. * - * + * <len=0001<>id=2> */ public static class InterestedMessage extends PeerMessage { @@ -309,7 +309,7 @@ public static InterestedMessage craft() { /** * Not interested message. * - * + * <len=0001><id=3> */ public static class NotInterestedMessage extends PeerMessage { @@ -337,7 +337,7 @@ public static NotInterestedMessage craft() { /** * Have message. * - * + * <len=0005><id=4><piece index=xxxx> */ public static class HaveMessage extends PeerMessage { @@ -387,7 +387,7 @@ public String toString() { /** * Bitfield message. * - * + * <len=0001+X><id=5><bitfield> */ public static class BitfieldMessage extends PeerMessage { @@ -427,20 +427,25 @@ public static BitfieldMessage parse(ByteBuffer buffer, .validate(torrent); } - public static BitfieldMessage craft(BitSet availablePieces) { - byte[] bitfield = new byte[ - (int) Math.ceil((double)availablePieces.length()/8)]; - for (int i=availablePieces.nextSetBit(0); i >= 0; - i=availablePieces.nextSetBit(i+1)) { - bitfield[i/8] |= 1 << (7 -(i % 8)); + public static BitfieldMessage craft(BitSet availablePieces, int pieceCount) { + BitSet bitfield = new BitSet(); + int bitfieldBufferSize= (pieceCount + 8 - 1) / 8; + byte[] bitfieldBuffer = new byte[bitfieldBufferSize]; + + for (int i=availablePieces.nextSetBit(0); + 0 <= i && i < pieceCount; + i=availablePieces.nextSetBit(i+1)) { + bitfieldBuffer[i/8] |= 1 << (7 -(i % 8)); + bitfield.set(i); } ByteBuffer buffer = ByteBuffer.allocateDirect( - MESSAGE_LENGTH_FIELD_SIZE + BitfieldMessage.BASE_SIZE + bitfield.length); - buffer.putInt(BitfieldMessage.BASE_SIZE + bitfield.length); + MESSAGE_LENGTH_FIELD_SIZE + BitfieldMessage.BASE_SIZE + bitfieldBufferSize); + buffer.putInt(BitfieldMessage.BASE_SIZE + bitfieldBufferSize); buffer.put(PeerMessage.Type.BITFIELD.getTypeByte()); - buffer.put(ByteBuffer.wrap(bitfield)); - return new BitfieldMessage(buffer, availablePieces); + buffer.put(ByteBuffer.wrap(bitfieldBuffer)); + + return new BitfieldMessage(buffer, bitfield); } public String toString() { @@ -451,7 +456,7 @@ public String toString() { /** * Request message. * - * + * <len=00013><id=6><piece index><block offset><block length> */ public static class RequestMessage extends PeerMessage { @@ -528,7 +533,7 @@ public String toString() { /** * Piece message. * - * + * <len=0009+X><id=7><piece index><block offset><block data> */ public static class PieceMessage extends PeerMessage { @@ -600,7 +605,7 @@ public String toString() { /** * Cancel message. * - * + * <len=00013><id=8><piece index><block offset><block length> */ public static class CancelMessage extends PeerMessage { diff --git a/src/main/java/com/turn/ttorrent/common/protocol/TrackerMessage.java b/core/src/main/java/com/turn/ttorrent/common/protocol/TrackerMessage.java similarity index 100% rename from src/main/java/com/turn/ttorrent/common/protocol/TrackerMessage.java rename to core/src/main/java/com/turn/ttorrent/common/protocol/TrackerMessage.java diff --git a/src/main/java/com/turn/ttorrent/common/protocol/http/HTTPAnnounceRequestMessage.java b/core/src/main/java/com/turn/ttorrent/common/protocol/http/HTTPAnnounceRequestMessage.java similarity index 98% rename from src/main/java/com/turn/ttorrent/common/protocol/http/HTTPAnnounceRequestMessage.java rename to core/src/main/java/com/turn/ttorrent/common/protocol/http/HTTPAnnounceRequestMessage.java index 0362ba72f..5524afd43 100644 --- a/src/main/java/com/turn/ttorrent/common/protocol/http/HTTPAnnounceRequestMessage.java +++ b/core/src/main/java/com/turn/ttorrent/common/protocol/http/HTTPAnnounceRequestMessage.java @@ -21,6 +21,7 @@ import com.turn.ttorrent.bcodec.InvalidBEncodingException; import com.turn.ttorrent.common.Peer; import com.turn.ttorrent.common.Torrent; +import com.turn.ttorrent.common.Utils; import com.turn.ttorrent.common.protocol.TrackerMessage.AnnounceRequestMessage; import java.io.IOException; @@ -81,7 +82,7 @@ public byte[] getInfoHash() { @Override public String getHexInfoHash() { - return Torrent.byteArrayToHexString(this.infoHash); + return Utils.bytesToHex(this.infoHash); } @Override @@ -254,7 +255,7 @@ public static HTTPAnnounceRequestMessage parse(ByteBuffer data) return new HTTPAnnounceRequestMessage(data, infoHash, new Peer(ip, port, ByteBuffer.wrap(peerId)), - downloaded, uploaded, left, compact, noPeerId, + uploaded, downloaded, left, compact, noPeerId, event, numWant); } catch (InvalidBEncodingException ibee) { throw new MessageValidationException( diff --git a/src/main/java/com/turn/ttorrent/common/protocol/http/HTTPAnnounceResponseMessage.java b/core/src/main/java/com/turn/ttorrent/common/protocol/http/HTTPAnnounceResponseMessage.java similarity index 95% rename from src/main/java/com/turn/ttorrent/common/protocol/http/HTTPAnnounceResponseMessage.java rename to core/src/main/java/com/turn/ttorrent/common/protocol/http/HTTPAnnounceResponseMessage.java index efb75fce3..01d27cd7f 100644 --- a/src/main/java/com/turn/ttorrent/common/protocol/http/HTTPAnnounceResponseMessage.java +++ b/core/src/main/java/com/turn/ttorrent/common/protocol/http/HTTPAnnounceResponseMessage.java @@ -87,6 +87,11 @@ public static HTTPAnnounceResponseMessage parse(ByteBuffer data) Map params = decoded.getMap(); + if (params.get("interval") == null) { + throw new MessageValidationException( + "Tracker message missing mandatory field 'interval'!"); + } + try { List peers; @@ -102,8 +107,8 @@ public static HTTPAnnounceResponseMessage parse(ByteBuffer data) return new HTTPAnnounceResponseMessage(data, params.get("interval").getInt(), - params.get("complete").getInt(), - params.get("incomplete").getInt(), + params.get("complete") != null ? params.get("complete").getInt() : 0, + params.get("incomplete") != null ? params.get("incomplete").getInt() : 0, peers); } catch (InvalidBEncodingException ibee) { throw new MessageValidationException("Invalid response " + diff --git a/src/main/java/com/turn/ttorrent/common/protocol/http/HTTPTrackerErrorMessage.java b/core/src/main/java/com/turn/ttorrent/common/protocol/http/HTTPTrackerErrorMessage.java similarity index 100% rename from src/main/java/com/turn/ttorrent/common/protocol/http/HTTPTrackerErrorMessage.java rename to core/src/main/java/com/turn/ttorrent/common/protocol/http/HTTPTrackerErrorMessage.java diff --git a/src/main/java/com/turn/ttorrent/common/protocol/http/HTTPTrackerMessage.java b/core/src/main/java/com/turn/ttorrent/common/protocol/http/HTTPTrackerMessage.java similarity index 100% rename from src/main/java/com/turn/ttorrent/common/protocol/http/HTTPTrackerMessage.java rename to core/src/main/java/com/turn/ttorrent/common/protocol/http/HTTPTrackerMessage.java diff --git a/src/main/java/com/turn/ttorrent/common/protocol/udp/UDPAnnounceRequestMessage.java b/core/src/main/java/com/turn/ttorrent/common/protocol/udp/UDPAnnounceRequestMessage.java similarity index 97% rename from src/main/java/com/turn/ttorrent/common/protocol/udp/UDPAnnounceRequestMessage.java rename to core/src/main/java/com/turn/ttorrent/common/protocol/udp/UDPAnnounceRequestMessage.java index 3150ea890..b6e019a83 100644 --- a/src/main/java/com/turn/ttorrent/common/protocol/udp/UDPAnnounceRequestMessage.java +++ b/core/src/main/java/com/turn/ttorrent/common/protocol/udp/UDPAnnounceRequestMessage.java @@ -16,6 +16,7 @@ package com.turn.ttorrent.common.protocol.udp; import com.turn.ttorrent.common.Torrent; +import com.turn.ttorrent.common.Utils; import com.turn.ttorrent.common.protocol.TrackerMessage; import java.net.InetAddress; @@ -89,7 +90,7 @@ public byte[] getInfoHash() { @Override public String getHexInfoHash() { - return Torrent.byteArrayToHexString(this.infoHash); + return Utils.bytesToHex(this.infoHash); } @Override @@ -99,7 +100,7 @@ public byte[] getPeerId() { @Override public String getHexPeerId() { - return Torrent.byteArrayToHexString(this.peerId); + return Utils.bytesToHex(this.peerId); } @Override @@ -229,8 +230,8 @@ public static UDPAnnounceRequestMessage craft(long connectionId, data.put(infoHash); data.put(peerId); data.putLong(downloaded); - data.putLong(uploaded); data.putLong(left); + data.putLong(uploaded); data.putInt(event.getId()); data.put(ip.getAddress()); data.putInt(key); diff --git a/src/main/java/com/turn/ttorrent/common/protocol/udp/UDPAnnounceResponseMessage.java b/core/src/main/java/com/turn/ttorrent/common/protocol/udp/UDPAnnounceResponseMessage.java similarity index 92% rename from src/main/java/com/turn/ttorrent/common/protocol/udp/UDPAnnounceResponseMessage.java rename to core/src/main/java/com/turn/ttorrent/common/protocol/udp/UDPAnnounceResponseMessage.java index 612012d3b..badf8667a 100644 --- a/src/main/java/com/turn/ttorrent/common/protocol/udp/UDPAnnounceResponseMessage.java +++ b/core/src/main/java/com/turn/ttorrent/common/protocol/udp/UDPAnnounceResponseMessage.java @@ -102,7 +102,11 @@ public static UDPAnnounceResponseMessage parse(ByteBuffer data) int complete = data.getInt(); List peers = new LinkedList(); - for (int i=0; i < data.remaining() / 6; i++) { + // the line below replaces this: for (int i=0; i < data.remaining() / 6; i++) + // That for loop fails when data.remaining() is 6, even if data.remaining() / 6 is + // placed in parentheses. The reason why it fails is not clear. Replacing it + // with while (data.remaining() > 5) works however. + while(data.remaining() > 5) { try { byte[] ipBytes = new byte[4]; data.get(ipBytes); diff --git a/src/main/java/com/turn/ttorrent/common/protocol/udp/UDPConnectRequestMessage.java b/core/src/main/java/com/turn/ttorrent/common/protocol/udp/UDPConnectRequestMessage.java similarity index 100% rename from src/main/java/com/turn/ttorrent/common/protocol/udp/UDPConnectRequestMessage.java rename to core/src/main/java/com/turn/ttorrent/common/protocol/udp/UDPConnectRequestMessage.java diff --git a/src/main/java/com/turn/ttorrent/common/protocol/udp/UDPConnectResponseMessage.java b/core/src/main/java/com/turn/ttorrent/common/protocol/udp/UDPConnectResponseMessage.java similarity index 100% rename from src/main/java/com/turn/ttorrent/common/protocol/udp/UDPConnectResponseMessage.java rename to core/src/main/java/com/turn/ttorrent/common/protocol/udp/UDPConnectResponseMessage.java diff --git a/src/main/java/com/turn/ttorrent/common/protocol/udp/UDPTrackerErrorMessage.java b/core/src/main/java/com/turn/ttorrent/common/protocol/udp/UDPTrackerErrorMessage.java similarity index 100% rename from src/main/java/com/turn/ttorrent/common/protocol/udp/UDPTrackerErrorMessage.java rename to core/src/main/java/com/turn/ttorrent/common/protocol/udp/UDPTrackerErrorMessage.java diff --git a/src/main/java/com/turn/ttorrent/common/protocol/udp/UDPTrackerMessage.java b/core/src/main/java/com/turn/ttorrent/common/protocol/udp/UDPTrackerMessage.java similarity index 100% rename from src/main/java/com/turn/ttorrent/common/protocol/udp/UDPTrackerMessage.java rename to core/src/main/java/com/turn/ttorrent/common/protocol/udp/UDPTrackerMessage.java diff --git a/src/main/java/com/turn/ttorrent/tracker/TrackedPeer.java b/core/src/main/java/com/turn/ttorrent/tracker/TrackedPeer.java similarity index 100% rename from src/main/java/com/turn/ttorrent/tracker/TrackedPeer.java rename to core/src/main/java/com/turn/ttorrent/tracker/TrackedPeer.java diff --git a/src/main/java/com/turn/ttorrent/tracker/TrackedTorrent.java b/core/src/main/java/com/turn/ttorrent/tracker/TrackedTorrent.java similarity index 93% rename from src/main/java/com/turn/ttorrent/tracker/TrackedTorrent.java rename to core/src/main/java/com/turn/ttorrent/tracker/TrackedTorrent.java index 98a2734b5..35125b875 100644 --- a/src/main/java/com/turn/ttorrent/tracker/TrackedTorrent.java +++ b/core/src/main/java/com/turn/ttorrent/tracker/TrackedTorrent.java @@ -20,7 +20,6 @@ import com.turn.ttorrent.common.protocol.TrackerMessage.AnnounceRequestMessage.RequestEvent; import java.io.File; -import java.io.FileInputStream; import java.io.IOException; import java.io.UnsupportedEncodingException; import java.nio.ByteBuffer; @@ -33,6 +32,7 @@ import java.util.concurrent.ConcurrentHashMap; import java.util.concurrent.ConcurrentMap; +import org.apache.commons.io.FileUtils; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -77,11 +77,8 @@ public class TrackedTorrent extends Torrent { * @param torrent The meta-info byte data. * @throws IOException When the info dictionary can't be * encoded and hashed back to create the torrent's SHA-1 hash. - * @throws NoSuchAlgorithmException If the SHA-1 algorithm is not - * available. */ - public TrackedTorrent(byte[] torrent) - throws IOException, NoSuchAlgorithmException { + public TrackedTorrent(byte[] torrent) throws IOException, NoSuchAlgorithmException { super(torrent, false); this.peers = new ConcurrentHashMap(); @@ -89,8 +86,7 @@ public TrackedTorrent(byte[] torrent) this.announceInterval = TrackedTorrent.DEFAULT_ANNOUNCE_INTERVAL_SECONDS; } - public TrackedTorrent(Torrent torrent) - throws IOException, NoSuchAlgorithmException { + public TrackedTorrent(Torrent torrent) throws IOException, NoSuchAlgorithmException { this(torrent.getEncoded()); } @@ -293,20 +289,9 @@ public List getSomePeers(TrackedPeer peer) { * @param torrent The abstract {@link File} object representing the * .torrent file to load. * @throws IOException When the torrent file cannot be read. - * @throws NoSuchAlgorithmException */ - public static TrackedTorrent load(File torrent) throws IOException, - NoSuchAlgorithmException { - FileInputStream fis = null; - try { - fis = new FileInputStream(torrent); - byte[] data = new byte[(int)torrent.length()]; - fis.read(data); - return new TrackedTorrent(data); - } finally { - if (fis != null) { - fis.close(); - } - } + public static TrackedTorrent load(File torrent) throws IOException, NoSuchAlgorithmException { + byte[] data = FileUtils.readFileToByteArray(torrent); + return new TrackedTorrent(data); } } diff --git a/src/main/java/com/turn/ttorrent/tracker/Tracker.java b/core/src/main/java/com/turn/ttorrent/tracker/Tracker.java similarity index 78% rename from src/main/java/com/turn/ttorrent/tracker/Tracker.java rename to core/src/main/java/com/turn/ttorrent/tracker/Tracker.java index 6ba05d8b1..84bf03e04 100644 --- a/src/main/java/com/turn/ttorrent/tracker/Tracker.java +++ b/core/src/main/java/com/turn/ttorrent/tracker/Tracker.java @@ -17,38 +17,29 @@ import com.turn.ttorrent.common.Torrent; -import java.io.File; -import java.io.FilenameFilter; import java.io.IOException; -import java.io.PrintStream; import java.net.InetAddress; import java.net.InetSocketAddress; import java.net.MalformedURLException; import java.net.URL; +import java.util.Collection; import java.util.Timer; import java.util.TimerTask; import java.util.concurrent.ConcurrentHashMap; import java.util.concurrent.ConcurrentMap; -import jargs.gnu.CmdLineParser; - -import org.apache.log4j.BasicConfigurator; -import org.apache.log4j.ConsoleAppender; -import org.apache.log4j.PatternLayout; - -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - import org.simpleframework.transport.connect.Connection; import org.simpleframework.transport.connect.SocketConnection; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; /** * BitTorrent tracker. * *

* The tracker usually listens on port 6969 (the standard BitTorrent tracker - * port). Torrents must be registered directly to this tracker with the - * {@link #announce(TrackedTorrent torrent)} method. + * port). Torrents must be registered directly to this tracker with the {@link + * #announce(TrackedTorrent torrent)} method. *

* * @author mpetazzoni @@ -180,6 +171,13 @@ public void stop() { } } + /** + * Returns the list of tracker's torrents + */ + public Collection getTrackedTorrents() { + return torrents.values(); + } + /** * Announce a new torrent on this tracker. * @@ -318,82 +316,4 @@ public void run() { } } } - - /** - * Display program usage on the given {@link PrintStream}. - */ - private static void usage(PrintStream s) { - s.println("usage: Tracker [options] [directory]"); - s.println(); - s.println("Available options:"); - s.println(" -h,--help Show this help and exit."); - s.println(" -p,--port PORT Bind to port PORT."); - s.println(); - } - - /** - * Main function to start a tracker. - */ - public static void main(String[] args) { - BasicConfigurator.configure(new ConsoleAppender( - new PatternLayout("%d [%-25t] %-5p: %m%n"))); - - CmdLineParser parser = new CmdLineParser(); - CmdLineParser.Option help = parser.addBooleanOption('h', "help"); - CmdLineParser.Option port = parser.addIntegerOption('p', "port"); - - try { - parser.parse(args); - } catch (CmdLineParser.OptionException oe) { - System.err.println(oe.getMessage()); - usage(System.err); - System.exit(1); - } - - // Display help and exit if requested - if (Boolean.TRUE.equals((Boolean)parser.getOptionValue(help))) { - usage(System.out); - System.exit(0); - } - - Integer portValue = (Integer)parser.getOptionValue(port, - Integer.valueOf(DEFAULT_TRACKER_PORT)); - - String[] otherArgs = parser.getRemainingArgs(); - - if (otherArgs.length > 1) { - usage(System.err); - System.exit(1); - } - - // Get directory from command-line argument or default to current - // directory - String directory = otherArgs.length > 0 - ? otherArgs[0] - : "."; - - FilenameFilter filter = new FilenameFilter() { - @Override - public boolean accept(File dir, String name) { - return name.endsWith(".torrent"); - } - }; - - try { - Tracker t = new Tracker(new InetSocketAddress(portValue.intValue())); - - File parent = new File(directory); - for (File f : parent.listFiles(filter)) { - logger.info("Loading torrent from " + f.getName()); - t.announce(TrackedTorrent.load(f)); - } - - logger.info("Starting tracker with {} announced torrents...", - t.torrents.size()); - t.start(); - } catch (Exception e) { - logger.error("{}", e.getMessage(), e); - System.exit(2); - } - } } diff --git a/src/main/java/com/turn/ttorrent/tracker/TrackerService.java b/core/src/main/java/com/turn/ttorrent/tracker/TrackerService.java similarity index 97% rename from src/main/java/com/turn/ttorrent/tracker/TrackerService.java rename to core/src/main/java/com/turn/ttorrent/tracker/TrackerService.java index 014ec5da6..ae24cabd3 100644 --- a/src/main/java/com/turn/ttorrent/tracker/TrackerService.java +++ b/core/src/main/java/com/turn/ttorrent/tracker/TrackerService.java @@ -31,6 +31,7 @@ import java.util.Map; import java.util.concurrent.ConcurrentMap; +import org.apache.commons.io.IOUtils; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -117,13 +118,7 @@ public void handle(Request request, Response response) { } catch (IOException ioe) { logger.warn("Error while writing response: {}!", ioe.getMessage()); } finally { - if (body != null) { - try { - body.close(); - } catch (IOException ioe) { - // Ignore - } - } + IOUtils.closeQuietly(body); } } @@ -252,7 +247,7 @@ private void process(Request request, Response response, * Tracker HTTP protocol. *

* - * @param uri The request's full URI, including query parameters. + * @param request The request's full URI, including query parameters. * @return The {@link AnnounceRequestMessage} representing the client's * announce request. */ @@ -350,7 +345,7 @@ private void serveError(Response response, OutputStream body, * @param response The HTTP response object. * @param body The response output stream to write to. * @param status The HTTP status code to return. - * @param error The failure reason reported by the tracker. + * @param reason The failure reason reported by the tracker. */ private void serveError(Response response, OutputStream body, Status status, ErrorMessage.FailureReason reason) throws IOException { diff --git a/core/src/test/java/com/turn/ttorrent/client/storage/FileCollectionStorageTest.java b/core/src/test/java/com/turn/ttorrent/client/storage/FileCollectionStorageTest.java new file mode 100644 index 000000000..005013f2c --- /dev/null +++ b/core/src/test/java/com/turn/ttorrent/client/storage/FileCollectionStorageTest.java @@ -0,0 +1,61 @@ +package com.turn.ttorrent.client.storage; + +import org.testng.annotations.Test; + +import java.io.File; +import java.io.FileInputStream; +import java.io.IOException; +import java.nio.ByteBuffer; +import java.util.ArrayList; +import java.util.List; + +import static org.testng.Assert.assertEquals; +import static org.testng.Assert.assertTrue; + +/** + * User: loyd + * Date: 11/24/13 + */ +public class FileCollectionStorageTest { + @Test + public void testSelect() throws Exception { + final File file1 = File.createTempFile("testng", "fcst"); + file1.deleteOnExit(); + final File file2 = File.createTempFile("testng", "fcst"); + file2.deleteOnExit(); + + final List files = new ArrayList(); + files.add(new FileStorage(file1, 0, 2)); + files.add(new FileStorage(file2, 2, 2)); + final FileCollectionStorage storage = new FileCollectionStorage(files, 4); + // since all of these files already exist, we are considered finished + assertTrue(storage.isFinished()); + + // write to first file works + write(new byte[]{1, 2}, 0, storage); + check(new byte[]{1, 2}, file1); + + // write to second file works + write(new byte[]{5, 6}, 2, storage); + check(new byte[]{5, 6}, file2); + + // write to two files works + write(new byte[]{8,9,10,11}, 0, storage); + check(new byte[]{8,9}, file1); + check(new byte[]{10,11}, file2); + + // make sure partial write into next file works + write(new byte[]{100,101,102}, 0, storage); + check(new byte[]{102,11}, file2); + } + + private void write(byte[] bytes, int offset, FileCollectionStorage storage) throws IOException { + storage.write(ByteBuffer.wrap(bytes), offset); + storage.finish(); + } + private void check(byte[] bytes, File f) throws IOException { + final byte[] temp = new byte[bytes.length]; + assertEquals(new FileInputStream(f).read(temp), temp.length); + assertEquals(temp, bytes); + } +} diff --git a/core/src/test/java/com/turn/ttorrent/common/UtilsTest.java b/core/src/test/java/com/turn/ttorrent/common/UtilsTest.java new file mode 100644 index 000000000..5916c34b0 --- /dev/null +++ b/core/src/test/java/com/turn/ttorrent/common/UtilsTest.java @@ -0,0 +1,70 @@ +/** + * Copyright (C) 2016 Philipp Henkel + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + + package com.turn.ttorrent.common; + +import org.testng.annotations.Test; + +import static org.testng.AssertJUnit.assertEquals; + + +public class UtilsTest { + + @Test(expectedExceptions = NullPointerException.class) + public void testBytesToHexWithNull() { + Utils.bytesToHex(null); + } + + @Test + public void testBytesToHexWithEmptyByteArray() { + assertEquals("", Utils.bytesToHex(new byte[0])); + } + + @Test + public void testBytesToHexWithSingleByte() { + assertEquals("BC", Utils.bytesToHex(new byte[]{ + (byte) 0xBC + })); + } + + @Test + public void testBytesToHexWithZeroByte() { + assertEquals("00", Utils.bytesToHex(new byte[1])); + } + + @Test + public void testBytesToHexWithLeadingZero() { + assertEquals("0053FF", Utils.bytesToHex(new byte[]{ + (byte) 0x00, (byte) 0x53, (byte) 0xFF + })); + } + + @Test + public void testBytesToHexTrailingZero() { + assertEquals("AA004500", Utils.bytesToHex(new byte[]{ + (byte) 0xAA, (byte) 0x00, (byte) 0x45, (byte) 0x00 + })); + } + + @Test + public void testBytesToHexAllSymbols() { + assertEquals("0123456789ABCDEF", Utils.bytesToHex(new byte[]{ + (byte) 0x01, (byte) 0x23, (byte) 0x45, (byte) 0x67, + (byte) 0x89, (byte) 0xAB, (byte) 0xCD, (byte) 0xEF + })); + } + +} diff --git a/core/src/test/java/com/turn/ttorrent/common/protocol/PeerMessageTest.java b/core/src/test/java/com/turn/ttorrent/common/protocol/PeerMessageTest.java new file mode 100644 index 000000000..3bc911fa6 --- /dev/null +++ b/core/src/test/java/com/turn/ttorrent/common/protocol/PeerMessageTest.java @@ -0,0 +1,146 @@ +package com.turn.ttorrent.common; + +import com.turn.ttorrent.common.protocol.PeerMessage; + +import org.testng.annotations.Test; + +import java.nio.ByteBuffer; +import java.util.BitSet; + +import static org.testng.Assert.assertEquals; +import static org.testng.Assert.assertEqualsNoOrder; +import static org.testng.Assert.assertTrue; + + +public class PeerMessageTest { + + @Test + public void testCraftBitfieldMessage() { + // See https://wiki.theory.org/BitTorrentSpecification#bitfield + + // Create message with 744 (= 93 * 8) pieces + BitSet availablePieces = new BitSet(); + availablePieces.set(0); + availablePieces.set(700); + availablePieces.set(743); // last piece + availablePieces.set(744); // out of range - should be ignored + PeerMessage.BitfieldMessage msg = PeerMessage.BitfieldMessage.craft(availablePieces, 744); + + // Check bitfield + assertEquals(3, msg.getBitfield().cardinality()); + assertEquals(true, msg.getBitfield().get(0)); + assertEquals(true, msg.getBitfield().get(700)); + assertEquals(true, msg.getBitfield().get(743)); + + // Check raw data - bitfield: + ByteBuffer buffer = msg.getData(); + + // total size + assertEquals(4 + 1 + 93, buffer.remaining()); + + // len + assertEquals(0, buffer.get(0)); + assertEquals(0, buffer.get(1)); + assertEquals(0, buffer.get(2)); + assertEquals(1 + 93, (int)buffer.get(3)); + + // id + assertEquals(5, buffer.get(4)); + + // bitfield + buffer.position(5); + ByteBuffer bitfieldBuffer = buffer.slice(); + BitSet bitfield = convertByteBufferToBitfieldBitSet(bitfieldBuffer); + assertEquals(3, bitfield.cardinality()); + assertEquals(true, bitfield.get(00)); + assertEquals(true, bitfield.get(700)); + assertEquals(true, bitfield.get(743)); + } + + @Test + public void testCraftBitfieldMessageEmpty() { + // See https://wiki.theory.org/BitTorrentSpecification#bitfield + + // Create message with 744 (= 93 * 8) pieces + BitSet availablePieces = new BitSet(); + PeerMessage.BitfieldMessage msg = PeerMessage.BitfieldMessage.craft(availablePieces, 744); + + // Check bitfield + assertEquals(0, msg.getBitfield().cardinality()); + + // Check raw data - bitfield: + ByteBuffer buffer = msg.getData(); + + // total size + assertEquals(4 + 1 + 93, buffer.remaining()); + + // len + assertEquals(0, buffer.get(0)); + assertEquals(0, buffer.get(1)); + assertEquals(0, buffer.get(2)); + assertEquals(1 + 93, (int)buffer.get(3)); + + // id + assertEquals(5, buffer.get(4)); + + // bitfield + buffer.position(5); + ByteBuffer bitfieldBuffer = buffer.slice(); + BitSet bitfield = convertByteBufferToBitfieldBitSet(bitfieldBuffer); + assertEquals(0, bitfield.cardinality()); + } + + @Test + public void testCreateBitfieldMessageWithSparseBits() { + // See https://wiki.theory.org/BitTorrentSpecification#bitfield + + // Create message with 745 (= 93 * 8 + 1) pieces + BitSet availablePieces = new BitSet(); + availablePieces.set(10); + availablePieces.set(700); + availablePieces.set(744); + availablePieces.set(745); // out of range - should be ignored + PeerMessage.BitfieldMessage msg = PeerMessage.BitfieldMessage.craft(availablePieces, 745); + + // Check bitfield + assertEquals(3, msg.getBitfield().cardinality()); + assertEquals(true, msg.getBitfield().get(10)); + assertEquals(true, msg.getBitfield().get(700)); + assertEquals(true, msg.getBitfield().get(744)); + + // Check raw data - bitfield: + ByteBuffer buffer = msg.getData(); + + // total size + assertEquals(4 + 1 + 94, buffer.remaining()); + + // len + assertEquals(0, buffer.get(0)); + assertEquals(0, buffer.get(1)); + assertEquals(0, buffer.get(2)); + assertEquals(1 + 94, (int)buffer.get(3)); + + // id + assertEquals(5, buffer.get(4)); + + // bitfield with 7 spare bits + buffer.position(5); + ByteBuffer bitfieldBuffer = buffer.slice(); + BitSet bitfield = convertByteBufferToBitfieldBitSet(bitfieldBuffer); + assertEquals(3, bitfield.cardinality()); + assertEquals(true, bitfield.get(10)); + assertEquals(true, bitfield.get(700)); + assertEquals(true, bitfield.get(744)); + } + + private BitSet convertByteBufferToBitfieldBitSet(ByteBuffer buffer) { + BitSet bitfield = new BitSet(); + for (int i=0; i < buffer.remaining()*8; i++) { + if ((buffer.get(i/8) & (1 << (7 -(i % 8)))) > 0) { + bitfield.set(i); + } + } + return bitfield; + } + +} diff --git a/pom.xml b/pom.xml index 33d6c98ca..606ea9383 100644 --- a/pom.xml +++ b/pom.xml @@ -1,174 +1,170 @@ - - 4.0.0 - - - org.sonatype.oss - oss-parent - 7 - - - Java BitTorrent library - - ttorrent is a pure-Java implementation of the BitTorrent protocol, - including support for several BEPs. It also provides a standalone client, - a tracker and a torrent manipulation utility. - - http://turn.github.com/ttorrent/ - com.turn - ttorrent - 1.2 - jar - - - Turn, Inc. - http://www.turn.com - - - - scm:git:git://github.com/turn/ttorrent.git - http://github.com/turn/ttorrent - - - - - Apache Software License version 2.0 - http://www.apache.org/licenses/LICENSE-2.0.txt - - - - - GitHub - https://github.com/turn/ttorrent/issues - - - - - mpetazzoni - Maxime Petazzoni - mpetazzoni@turn.com - http://www.bulix.org - Turn, Inc - http://www.turn.com - - maintainer - architect - developer - - -8 - - https://secure.gravatar.com/avatar/6f705e0c299bca294444de3a6a3308b3 - - - - - - UTF-8 - - - - - jboss-thirdparty-releases - JBoss Thirdparty Releases - https://repository.jboss.org/nexus/content/repositories/thirdparty-releases/ - - - - - - commons-io - commons-io - 2.1 - - - - org.simpleframework - simple - 4.1.21 - - - - org.slf4j - slf4j-log4j12 - 1.6.4 - - - - org.testng - testng - 6.1.1 - test - - - - net.sf - jargs - 1.0 - - - - - package - ${basedir}/build - - - - org.apache.maven.plugins - maven-compiler-plugin - 2.3.2 - - 1.6 - 1.6 - - - - - org.apache.maven.plugins - maven-jar-plugin - 2.4 - - - ** - - - - - - org.apache.maven.plugins - maven-javadoc-plugin - 2.8.1 - - ${basedir} - doc - - - - - maven-assembly-plugin - - - jar-with-dependencies - - false - - - false - true - com.turn.ttorrent.client.Client - - - - - - make-my-jar-with-dependencies - package - - - assembly - - - - - - + + 4.0.0 + + Java BitTorrent library + + ttorrent is a pure-Java implementation of the BitTorrent protocol, + including support for several BEPs. It also provides a standalone client, + a tracker and a torrent manipulation utility. + + http://mpetazzoni.github.io/ttorrent/ + com.turn + ttorrent + 1.6-SNAPSHOT + pom + + + core + cli + + + + scm:git:git://github.com/mpetazzoni/ttorrent.git + scm:git:ssh://git@github.com/mpetazzoni/ttorrent.git + https://github.com/mpetazzoni/ttorrent + master + + + + + Apache Software License version 2.0 + http://www.apache.org/licenses/LICENSE-2.0.txt + + + + + GitHub + https://github.com/mpetazzoni/ttorrent/issues + + + + + mpetazzoni + Maxime Petazzoni + maxime.petazzoni@bulix.org + http://www.bulix.org + SignalFx, Inc + http://www.signalfx.com + + maintainer + architect + developer + + -8 + + https://secure.gravatar.com/avatar/6f705e0c299bca294444de3a6a3308b3 + + + + + + UTF-8 + + + + + ossrh + https://oss.sonatype.org/content/repositories/snapshots + + + ossrh + https://oss.sonatype.org/service/local/staging/deploy/maven2/ + + + + + + jboss-thirdparty-releases + JBoss Thirdparty Releases + https://repository.jboss.org/nexus/content/repositories/thirdparty-releases/ + + + + + + release + + + + org.apache.maven.plugins + maven-source-plugin + 2.2.1 + + + attach-sources + + jar-no-fork + + + + + + + org.apache.maven.plugins + maven-gpg-plugin + 1.5 + + + sign-artifacts + verify + + sign + + + + + + + org.apache.maven.plugins + maven-release-plugin + 2.4.2 + + + + org.sonatype.plugins + nexus-staging-maven-plugin + 1.6.3 + true + + ossrh + https://oss.sonatype.org/ + true + + + + + + + + + + + org.apache.maven.plugins + maven-compiler-plugin + 2.3.2 + + 1.6 + 1.6 + + + + + org.apache.maven.plugins + maven-javadoc-plugin + 2.8.1 + + ${basedir} + doc + + + + attach-javadocs + + jar + + + + + +