diff --git a/CHANGELOG-ZH.md b/CHANGELOG-ZH.md index 4ebf8dd5..5200849b 100644 --- a/CHANGELOG-ZH.md +++ b/CHANGELOG-ZH.md @@ -2,6 +2,49 @@ > [ Change log in english](https://github.com/lingochamp/FileDownloader/blob/master/CHANGELOG.md) +## Version 1.6.9 + +_2017_12_16_ + +#### 修复 + +- 修复(serial-queue): 修复在`FileDownloadSerialQueue`遇到的死锁。 closes #858 +- 修复: 不再在非单元测试环境使用`j-unit`，避免在一些小米手机上发生`no-static-method-found`的问题。 closes #867 +- 修复: 修复每次重试减少两次重试机会的问题。 closes #838 +- 修复: 修复在`pending`的时候暂停任务，而后获取到该任务都是`pending`的状态的问题。 closes #855 + +#### 性能与提高 + +- 提高实用性: 开放`SqliteDatabaseImpl`、`RemitDatabase`、`NoDatabaseImpl`，便于上层覆盖他们。 +- 提高实用性: 支持从更高的版本降级到该版本。 +- 提高实用性: 当上层没有主动添加`User-Agent`的时候，内部添加默认的`User-Agent`。 closes #848 +- 提高性能: 修改所有的线程池中线程的存活时间(从5s修改为15s)，避免在高频下载中，各池子频繁的释放与创建线程 +- 提高性能: 使用`RemitDatabase`作为默认的数据库，在很多小任务很快的结束下载(2s内)，其数据库操作将会变得十分冗余，而这部分的数据库操作将被取消 + +#### RemitDatabase + +FileDownloader中大多数数据库长尾问题，是由于有很多很小的任务同时执行： + +- 由于很小的任务每次启动、等待、连接、下载进度、结束都会促发入库 +- 一旦任务很小网速很快的时候，一个小任务实际下载耗时可能在1-2s完成 +- 因此整个引擎不得不为该1-2s完成的任务完成一连串的数据库入库、更新到从数据库删除的操作 +- 也就是说单个类似的任务在1-2s内促发了至少5次数据库操作，期间包含入库与最后的删除 +- 一个任务还好，当这样的任务数上升到几百个的层面，这样高频持续的数据库操作，就很容易暴露各种数据库问题（包含文件系统问题） +- 而现有体系在上层推任务大量任务到下载服务的时候， 会高频持续的3个并行对这些任务做入库处理，在这个点上数据库问题也容易发生（包含文件系统问题） + +--- + +而相比之下写入数据库是为了断点续传，这个短期的频繁数据库操作，实质的作用甚微，早期的提供外接接口来控制下载进度间的入库频率显然无法覆盖该问题。 + +--- + +因此，还是为FileDownloader推出新的`RemitDatabase`用于解决该问题，除去期间的多线程安全问题的处理，核心思想如下: +![][RemitDatabase-png] + +- 如果某一个任务的整体数据更新与结束在2s(该值可定制)内，则不再有数据库操作，全程只存内存 +- 如果某一个任务的数据更新与结束操过2s，则分为两部分，2s前只存内存，2s开始同时存内存与数据库 +- 如果某一个任务最终的结束是暂停或错误，则在最后的状态更新中，同时存内存与数据库 + ## Version 1.6.8 _2017-10-13_ diff --git a/CHANGELOG.md b/CHANGELOG.md index e515d16f..e876e743 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,27 @@ > [中文迭代日志](https://github.com/lingochamp/FileDownloader/blob/master/CHANGELOG-ZH.md) +## Version 1.6.9 + +_2017_12_16_ + +#### Fix + +- Fix(serial-queue): fix deadlock on `FileDownloadSerialQueue`. closes #858 +- Fix: do not use j-unit on library non-unit-test env to fix the `no-static-method-found` issue on some mi-phones. closes #867 +- Fix: fix decrease two times of retry-chance each time of retry. closes #838 +- Fix: fix get status is pending when a task has been paused on pending status. closes #855 + +#### Enhancement + +- Improve Practicability: public `SqliteDatabaseImpl`、`RemitDatabase`、`NoDatabaseImpl`, so you can overwrite them +- Improve Practicability: support downgrade version from newer version +- Improve Practicability: add the default `User-Agent` if upper layer does not add. closes #848 +- Improve Performance: change the keepalive second(5s to 15s) for each executor, since when downloading multiple tasks thread release and recreate too frequently +- Improve Performance: using `RemitDatabase` instead of `DefaultFiledownloadDatabase` to avoid some small task start and finished on the very short time but consume too much time on sync status to database what is useless + +![][RemitDatabase-png] + ## Version 1.6.8 _2017-10-13_ @@ -816,5 +837,6 @@ _2015-12-22_ - initial release +[RemitDatabase-png]: https://github.com/lingochamp/FileDownloader/raw/master/art/remit-database.gif [FileDownloadConnection-java-link]: https://github.com/lingochamp/FileDownloader/blob/master/library/src/main/java/com/liulishuo/filedownloader/connection/FileDownloadConnection.java [FileDownloadUrlConnection-java-link]: https://github.com/lingochamp/FileDownloader/blob/master/library/src/main/java/com/liulishuo/filedownloader/connection/FileDownloadUrlConnection.java diff --git a/README-zh.md b/README-zh.md index 4f5599d1..5dfe1a7a 100644 --- a/README-zh.md +++ b/README-zh.md @@ -59,7 +59,7 @@ Android 文件下载引擎，稳定、高效、灵活、简单易用 在项目中引用: ```groovy -compile 'com.liulishuo.filedownloader:library:1.6.8' +compile 'com.liulishuo.filedownloader:library:1.6.9' ``` > 如果是eclipse引入jar包参考: [这里](https://github.com/lingochamp/FileDownloader/issues/212#issuecomment-232240415) @@ -253,7 +253,7 @@ if (parallel) { | 方法名 | 需实现接口 | 已有组件 | 默认组件 | 说明 | --- | --- | --- | --- | --- -| database | FileDownloadDatabase | DefaultDatabaseImpl、NoDatabaseImpl | DefaultDatabaseImpl | 传入定制化数据库组件，用于存储用于断点续传的数据 +| database | FileDownloadDatabase | RemitDatabase、SqliteDatabaseImpl、NoDatabaseImpl | RemitDatabase | 传入定制化数据库组件，用于存储用于断点续传的数据 | connection | FileDownloadConnection | FileDownloadUrlConnection | FileDownloadUrlConnection | 传入定制化的网络连接组件，用于下载时建立网络连接 | outputStreamCreator | FileDownloadOutputStream | FileDownloadRandomAccessFile | FileDownloadRandomAccessFile | 传入输出流组件，用于下载时写文件使用 | maxNetworkThreadCount | - | - | 3 | 传入创建下载引擎时，指定可用的下载线程个数 diff --git a/README.md b/README.md index dbdabac9..40dd155b 100644 --- a/README.md +++ b/README.md @@ -26,7 +26,7 @@ FileDownloader is installed by adding the following dependency to your `build.gr ```groovy dependencies { - compile 'com.liulishuo.filedownloader:library:1.6.8' + compile 'com.liulishuo.filedownloader:library:1.6.9' } ``` @@ -38,7 +38,7 @@ From now on, FileDownloader support following components to be customized by you | --- | --- | --- | Connection | [FileDownloadConnection][FileDownloadConnection-java-link] | [FileDownloadUrlConnection][FileDownloadUrlConnection-java-link] | OutputStream | [FileDownloadOutputStream][FileDownloadOutputStream-java-link] | [FileDownloadRandomAccessFile][FileDownloadRandomAccessFile-java-link] -| Database | [FileDownloadDatabase][FileDownloadDatabase-java-link] | [DefaultDatabaseImpl][DefaultDatabaseImpl-java-link] +| Database | [FileDownloadDatabase][FileDownloadDatabase-java-link] | [RemitDatabase][RemitDatabase-java-link] | ConnectionCountAdapter | [ConnectionCountAdapter][ConnectionCountAdapter-java-link] | [DefaultConnectionCountAdapter][DefaultConnectionCountAdapter-java-link] | IdGenerator | [IdGenerator][IdGenerator-java-link] | [DefaultIdGenerator][DefaultIdGenerator-java-link] @@ -99,8 +99,8 @@ limitations under the License. [build_status_link]: https://travis-ci.org/lingochamp/FileDownloader [FileDownloadConnection-java-link]: https://github.com/lingochamp/FileDownloader/blob/master/library/src/main/java/com/liulishuo/filedownloader/connection/FileDownloadConnection.java [FileDownloadUrlConnection-java-link]: https://github.com/lingochamp/FileDownloader/blob/master/library/src/main/java/com/liulishuo/filedownloader/connection/FileDownloadUrlConnection.java -[FileDownloadDatabase-java-link]: https://github.com/lingochamp/FileDownloader/blob/master/library/src/main/java/com/liulishuo/filedownloader/services/FileDownloadDatabase.java -[DefaultDatabaseImpl-java-link]: https://github.com/lingochamp/FileDownloader/blob/master/library/src/main/java/com/liulishuo/filedownloader/services/DefaultDatabaseImpl.java +[FileDownloadDatabase-java-link]: https://github.com/lingochamp/FileDownloader/blob/master/library/src/main/java/com/liulishuo/filedownloader/database/RemitDatabase.java +[RemitDatabase-java-link]: https://github.com/lingochamp/FileDownloader/blob/master/library/src/main/java/com/liulishuo/filedownloader/database/RemitDatabase.java [FileDownloadOutputStream-java-link]: https://github.com/lingochamp/FileDownloader/blob/master/library/src/main/java/com/liulishuo/filedownloader/stream/FileDownloadOutputStream.java [FileDownloadRandomAccessFile-java-link]: https://github.com/lingochamp/FileDownloader/blob/master/library/src/main/java/com/liulishuo/filedownloader/stream/FileDownloadRandomAccessFile.java [ConnectionCountAdapter-java-link]: https://github.com/lingochamp/FileDownloader/blob/master/library/src/main/java/com/liulishuo/filedownloader/util/FileDownloadHelper.java#L100 diff --git a/art/remit-database.png b/art/remit-database.png new file mode 100644 index 00000000..32c63262 Binary files /dev/null and b/art/remit-database.png differ diff --git a/gradle.properties b/gradle.properties index 85239303..525e2e42 100644 --- a/gradle.properties +++ b/gradle.properties @@ -1,5 +1,5 @@ -VERSION_NAME=1.6.8 -VERSION_CODE=59 +VERSION_NAME=1.6.9 +VERSION_CODE=60 BUILD_TOOLS_VERSION=26.0.2 COMPILE_SDK_VERSION=26