Skip to content

Commit

Permalink
Merge branch 'dev' into alpha
Browse files Browse the repository at this point in the history
  • Loading branch information
BingLingGroup committed Mar 20, 2020
2 parents db0850c + b1ae5a0 commit 2d1ce83
Show file tree
Hide file tree
Showing 21 changed files with 765 additions and 368 deletions.
12 changes: 7 additions & 5 deletions .github/ISSUE_TEMPLATE/bug_report.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,13 +14,14 @@ A clear and concise description of what the bug is.

**To Reproduce**
Steps to reproduce the behavior:

1. Command line arguments you are using.
2. A complete copy of command line output of the autosub. You can use `Ctrl-A` and `Ctrl-C` to copy all of them. Use the following markdown code block syntax is recommended.
2. A complete copy of command line output of the autosub. You can use `Ctrl-A` and `Ctrl-C` to copy all of them. Use the following markdown code block syntax is recommended. Copy them into the place between \`\`\`.

```
```

3. etc.
3. etc.

**Expected behavior**
A clear and concise description of what you expected to happen.
Expand All @@ -29,9 +30,10 @@ A clear and concise description of what you expected to happen.
If applicable, add screenshots to help explain your problem. But it is not recommended using screenshots to demonstrate the commandline output unless you think it really matters.

**Environment (please complete the following information):**
- OS: [e.g. windows]
- Python Version: [e.g. Python 2.7]
- Autosub Version: [e.g. 0.4.0]

- OS: [e.g. windows]
- Python Version: [e.g. Python 2.7]
- Autosub Version: [e.g. 0.4.1-alpha/0.5.4-alpha Nuitka windows release]

**Additional context** (Optional)
Add any other context about the problem here.
10 changes: 6 additions & 4 deletions .github/ISSUE_TEMPLATE/bug_report.zh-Hans.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,9 @@ assignees: ''

**复现问题**
复现问题的步骤:

1. 你使用的命令行参数。
2. 一份完整的autosub命令行输出。你可以使用`Ctrl-A``Ctrl-C`去复制所有输出。推荐使用以下的代码块markdown语法。
2. 一份完整的autosub命令行输出。你可以使用`Ctrl-A``Ctrl-C`去复制所有输出。推荐使用以下的代码块markdown语法,将代码块复制到\`\`\`\`\`\`之间即可

```
```
Expand All @@ -29,9 +30,10 @@ assignees: ''
合适的话可以提供用以描述问题的截图。但是不推荐用截图来展示命令行输出,除非你真的认为这很有必要。

**操作环境(请提供以下完整数据):**
- 操作系统: [譬如 windows]
- Python版本: [譬如 Python 2.7]
- Autosub版本: [譬如 0.4.0]

- 操作系统: [譬如 windows]
- Python版本: [譬如 Python 2.7]
- Autosub版本: [譬如 0.4.1-alpha或0.5.4-alpha Nuitka windows发布版]

**额外信息**(可选)
任何其他的能描述问题的信息。
88 changes: 69 additions & 19 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,31 +9,69 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## TOC

- [Unreleased](#unreleased)
- [Added](#addedunreleased)
- [Changed](#changedunreleased)
- [0.5.6-alpha - 2020-03-20](#056-alpha---2020-03-20)
- [Added](#added056-alpha)
- [Changed](#changed056-alpha)
- [Deprecated](#deprecated056-alpha)
- [Fixed](#fixed056-alpha)
- [0.5.5-alpha - 2020-03-04](#055-alpha---2020-03-04)
- [Added](#added055-alpha)
- [Changed](#changed055-alpha)
- [Fixed](#fixed055-alpha)
- [0.5.4-alpha - 2020-01-31](#054-alpha---2020-01-31)
- [Added](#added054-alpha)
- [Changed](#changed054-alpha)
- [Removed](#removed054-alpha)
- [Fixed](#fixed054-alpha)
- [0.5.3-alpha - 2019-12-30](#053-alpha---2019-12-30)
- [Changed](#changed053-alpha)
- [Fixed](#fixed053-alpha)
- [0.5.2-alpha - 2019-11-05](#052-alpha---2019-11-05)
- [Added](#added052-alpha)
- [Changed](#changed052-alpha)
- [Fixed](#fixed052-alpha)
- [0.5.1-alpha - 2019-08-02](#051-alpha---2019-08-02)
- [Added](#added051-alpha)
- [Changed](#changed051-alpha)
- [Fixed](#fixed051-alpha)
- [0.5.0-alpha - 2019-07-27](#050-alpha---2019-07-27)
- [Added](#added050-alpha)
- [Changed](#changed050-alpha)
- [Fixed](#fixed050-alpha)
- [0.4.1-alpha - 2019-07-11](#041-alpha---2019-07-11)
- [Added](#added041-alpha)
- [Changed](#changed041-alpha)
- [Fixed](#fixed041-alpha)
- [0.4.0-alpha - 2019-02-17](#040-alpha---2019-02-17)
- [Changed](#changed040-alpha)
- [Fixed](#fixed040-alpha)

Click up arrow to go back to TOC.

### [Unreleased]

#### Added(Unreleased)

#### Changed(Unreleased)

### [0.5.6-alpha] - 2020-03-20

#### Added(0.5.6-alpha)

- Add extra environment variables check when finding dependencies. [issue #91](https://github.com/BingLingGroup/autosub/issues/91)
- Add arguments parser input when open it without arguments. [issue #92](https://github.com/BingLingGroup/autosub/issues/92)
- Add subtitles processing when not input `-SRC`.

#### Changed(0.5.6-alpha)

- Change option `-sml` into `-nsml`.
- Change defaults for Auditok.

#### Fixed(0.5.6-alpha)

- Fix Google Speech-to-Text API empty result response bug. [issue #89](https://github.com/BingLingGroup/autosub/issues/89)

#### Deprecated(0.5.6-alpha)

- Deprecate Python 2.7 support.

### [0.5.5-alpha] - 2020-03-04

#### Added(0.5.5-alpha)
Expand All @@ -43,29 +81,34 @@ Click up arrow to go back to TOC.
- Add exception when receiving error from [Google-Speech-v2](https://github.com/gillesdemey/google-speech-v2).
- Add Nuitka compatibility codes to stop program compiled by Nuitka when using Google Cloud service account credentials. [Nuitka pkg_resources.DistributionNotFound error](https://github.com/Nuitka/Nuitka/issues/146)

#### Changed(0.5.5-alpha)
#### Fixed(0.5.5-alpha)

- Fix high memory/RAM usage partially by terminating child processes, lowering default multiprocessing counts and using `gc.collect(0)`. [issue #67](https://github.com/BingLingGroup/autosub/issues/67), [issue #74](https://github.com/BingLingGroup/autosub/issues/74)
- Fix dependency finding issue. [issue #82](https://github.com/BingLingGroup/autosub/issues/82)
- Fix ass override tags translation issue. [issue #79](https://github.com/BingLingGroup/autosub/issues/79)

<escape><a href = "#TOC">&nbsp;&nbsp;</a></escape>

### [0.5.4-alpha] - 2020-01-31

#### Added(0.5.4-alpha)

- Add basic Google Cloud Speech-to-Text support. [issue #10](https://github.com/BingLingGroup/autosub/issues/10)
- Add more bilingual subtitles formats output support. [issue #72](https://github.com/BingLingGroup/autosub/issues/72)

#### Changed(0.5.4-alpha)
#### Removed(0.5.4-alpha)

- Fix output format limits when input is a subtitles file.
- Remove gtransv2 support.

#### Fixed(0.5.4-alpha)

- Fix output format limits when input is a subtitles file.

<escape><a href = "#TOC">&nbsp;&nbsp;</a></escape>

### [0.5.3-alpha] - 2019-12-30

#### Changed(0.5.3-alpha)
#### Fixed(0.5.3-alpha)

- Fix excessive transcoding time issue. [pull request #66](https://github.com/BingLingGroup/autosub/pull/66)
- Fix Auditok option issues. [issue #70](https://github.com/BingLingGroup/autosub/issues/70)
Expand All @@ -79,7 +122,7 @@ Click up arrow to go back to TOC.

- Add issue templates.

#### Changed(0.5.2-alpha)
#### Fixed(0.5.2-alpha)

- Fix last row of empty translation text missing issue. [issue #62](https://github.com/BingLingGroup/autosub/issues/62)
- Fix executable file detection problem in the current directory.
Expand All @@ -92,7 +135,7 @@ Click up arrow to go back to TOC.

- Add translation source lang code auto match.

#### Changed(0.5.1-alpha)
#### Fixed(0.5.1-alpha)

- Fix method list_to_googletrans index error bug. [issue #48](https://github.com/BingLingGroup/autosub/issues/48)
- Fix unix subprocess.check_output compatibility. [issue #47](https://github.com/BingLingGroup/autosub/issues/47)
Expand Down Expand Up @@ -149,14 +192,17 @@ Click up arrow to go back to TOC.
- Refactor internal regions unit to millisecond. [issue #23](https://github.com/BingLingGroup/autosub/issues/23)
- Refactor speech regions detection by using auditok. [issue #27](https://github.com/BingLingGroup/autosub/issues/27)
- Refactor generate_subtitles into 3 parts. [issue #24](https://github.com/BingLingGroup/autosub/issues/24)
- [issue #8](https://github.com/BingLingGroup/autosub/issues/8)
- Fix python3 compatibility issues.
- Fix Nuitka build after updating Nuitka to 0.6.4(Environment Anaconda2 python3.5).
- Refactor api_gen_text to 2 parts. One is speech_to_text. Another is text_translation. [issue #33](https://github.com/BingLingGroup/autosub/issues/33)
- Refactor txt output. Now txt can output regions.
- Fix vtt output replacing all commas to dots issues.
- Refactor list_to_sub_str. [issue #37](https://github.com/BingLingGroup/autosub/issues/37)

#### Fixed(0.5.0-alpha)

- Fix vtt output replacing all commas to dots issues.
- [issue #8](https://github.com/BingLingGroup/autosub/issues/8)
- Fix python3 compatibility issues.
- Fix Nuitka build after updating Nuitka to 0.6.4(Environment Anaconda2 python3.5).

<escape><a href = "#TOC">&nbsp;&nbsp;</a></escape>

### [0.4.1-alpha] - 2019-07-11
Expand All @@ -171,16 +217,19 @@ Click up arrow to go back to TOC.

#### Changed(0.4.1-alpha)

- Fix vague language codes caused wrong recognition result. [agermanidis/autosub pull request #136](https://github.com/agermanidis/autosub/pull/136)
- Change docs.

#### Fixed(0.4.1-alpha)

- Fix vague language codes caused wrong recognition result. [agermanidis/autosub pull request #136](https://github.com/agermanidis/autosub/pull/136)

<escape><a href = "#TOC">&nbsp;&nbsp;</a></escape>

### [0.4.0-alpha] - 2019-02-17

[0.4.0-alpha release](https://github.com/BingLingGroup/autosub/releases/tag/0.4.0-alpha)

#### Changed(0.4.0-alpha)
#### Fixed(0.4.0-alpha)

- Fix several issues. [agermanidis/autosub pull request #128](https://github.com/agermanidis/autosub/pull/128) by [@iWangJiaxiang](https://github.com/iWangJiaxiang)
- Fix "ffmpeg.exe" causes "Dependency not found: ffmpeg" on Windows.
Expand All @@ -190,7 +239,8 @@ Click up arrow to go back to TOC.

<escape><a href = "#TOC">&nbsp;&nbsp;</a></escape>

[Unreleased]: https://github.com/BingLingGroup/autosub/compare/0.5.5-alpha...HEAD
[Unreleased]: https://github.com/BingLingGroup/autosub/compare/0.5.6-alpha...HEAD
[0.5.6-alpha]: https://github.com/BingLingGroup/autosub/compare/0.5.5-alpha...0.5.6-alpha
[0.5.5-alpha]: https://github.com/BingLingGroup/autosub/compare/0.5.4-alpha...0.5.5-alpha
[0.5.4-alpha]: https://github.com/BingLingGroup/autosub/compare/0.5.3-alpha...0.5.4-alpha
[0.5.3-alpha]: https://github.com/BingLingGroup/autosub/compare/0.5.2-alpha...0.5.3-alpha
Expand Down
14 changes: 12 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,7 @@ This repo has a different license from [the original repo](https://github.com/ag
Autosub depends on these third party softwares or Python site-packages. Much appreciation to all of these projects.

- [ffmpeg](https://ffmpeg.org/)
- [ffprobe](https://ffmpeg.org/ffprobe.html)
- [auditok](https://github.com/amsehili/auditok)
- [pysubs2](https://github.com/tkarabela/pysubs2)
- [py-googletrans](https://github.com/ssut/py-googletrans)
Expand All @@ -96,6 +97,13 @@ After autosub-0.4.0, all of the codes is compatible with both Python 2.7 and Pyt

About the dependencies installation. If you install autosub by pip, ffmpeg and ffmpeg-normalize won't be installed together not like the Python site-packages already listed on the `setup.py` or `requirements.txt`. You need to install them separately. But of course they are optional. They aren't necessary if you only use autosub to translate your subtitles.

ffmpeg, ffprobe, ffmpeg-normalize need to be put on one of these places to let the autosub detect and use them. The following codes are in the [constants.py](autosub/constants.py). Priority is determined in order.

1. Set the following environment variables before running the program: `FFMPEG_PATH`, `FFPROBE_PATH` and `FFMPEG_NORMALIZE_PATH`. It will override the ones located at the environment variable `PATH`. This will be helpful if you don't want to use the one in the `PATH`.
2. Add them to the environment variable `PATH`. No need to worry about if using package manager to install such as using pip to install ffmpeg-normalize and using chocolatey to install ffmpeg.
3. Add them to the same directory as the autosub executable.
4. Add them to the current command line working directory.

About the git installation. If you don't want to install git to use pip [VCS](https://pip.pypa.io/en/stable/reference/pip_install/#vcs-support) support to install python package or just confused with git environment variables, you can manually click that clone and download button to download the source code and use pip to install the source code [locally](https://pip.pypa.io/en/stable/reference/pip_install/#description) by input these commands.

```batch
Expand Down Expand Up @@ -157,6 +165,8 @@ You can just go to the [release page](https://github.com/BingLingGroup/autosub/r

Tips: `Shift - Right Click` is the keyboard shortcut for opening a Powershell on current directory. To open an exe at current directory, the format is like `.\autosub`.

Or you can just directly open it and input the args manually though I don't recommend doing this due to its less efficiency.

- The one without pyinstaller suffix is compiled by Nuitka. It's faster than the pyinstaller due to its compiling feature different from pyinstaller which just bundles the application.
- ffmpeg and ffmpeg-normalize are also in the package. The original ffmpeg-normalize doesn't have a standalone version. The standalone version of ffmpeg-normalize is built separately. Codes are [here](https://github.com/BingLingGroup/ffmpeg-normalize).
- If there's anything wrong on the both releases, or the package size and any other things are annoying you, you can just use the traditional pip installation method below.
Expand Down Expand Up @@ -260,13 +270,13 @@ To solve this problem, autosub uses [langcodes](https://github.com/LuminosoInsig

To manually match or see the full list of the lang codes, run the utility with the argument `-lsc`/`--list-speech-codes` and `-ltc`/ `--list-translation-codes`. Or open [constants.py](autosub/constants.py) and check.

To get a subtitles first line language, you can use `-dsl` to detect.
To get the language of the first line of the subtitles file, you can use `-dsl` to detect.

- Currently, autosub allows to send the lang codes not from the `--list-speech-codes`, which means in this case the program won't stop.

- Though you can input the speech lang code whatever you want, need to point out that if not using the codes on the list but somehow the API accept it, [Google-Speech-v2](https://github.com/gillesdemey/google-speech-v2) recognizes your audio in the ways that depend on your IP address which is uncontrollable by yourself. This is a known issue and I ask for a [pull request](https://github.com/agermanidis/autosub/pull/136) in the original repo.

- On the other hand, [py-googletrans](https://github.com/ssut/py-googletrans) is stricter. When it receive a lang code not on its list, it will throw an exception. Of course it can be designed as a throw-catch code block and ask user to input once again but currently I don't add this support so an improper translation lang code input will stop the program running unless you use the best match method mentioned above.
- On the other hand, [py-googletrans](https://github.com/ssut/py-googletrans) is stricter. When it receive a lang code not on its list, it will throw an exception and stop translation.

- Apart from the user input, another notable change is I split the `-S` option into two parts, `-S` and `-SRC`. `-S` option is for speech recognition lang code. `-SRC` is for translation source language. When not offering the arg of `-SRC`, autosub will automatically match the `-S` arg by using [langcodes](https://github.com/LuminosoInsight/langcodes) and get a best-match lang code for translation source language though [py-googletrans](https://github.com/ssut/py-googletrans) can auto-detect source language. Of course you can manually specify one by input `-SRC` option. `-D` is for translation destination language, still the same as before.

Expand Down
54 changes: 37 additions & 17 deletions autosub/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@
from __future__ import absolute_import, print_function, unicode_literals
import os
import gettext
import sys
import shlex

# Import third-party modules
import pysubs2
Expand Down Expand Up @@ -35,7 +37,21 @@ def main(): # pylint: disable=too-many-branches, too-many-statements, too-many-
Run autosub as a command-line program.
"""

args = options.get_cmd_args()
is_pause = False

try:
input_main = raw_input
except NameError:
input_main = input

option_parser = options.get_cmd_parser()
if len(sys.argv) > 1:
args = option_parser.parse_args()
else:
option_parser.print_help()
new_argv = input_main(_("\nInput args(without \"autosub\"): "))
args = option_parser.parse_args(shlex.split(new_argv))
is_pause = True

if args.https_proxy:
os.environ['https_proxy'] = args.https_proxy
Expand All @@ -57,17 +73,14 @@ def main(): # pylint: disable=too-many-branches, too-many-statements, too-many-
raise exceptions.AutosubException(_("\nAll works done."))

if not args.yes:
try:
input_m = raw_input
except NameError:
input_m = input
input_m = input_main
else:
input_m = None

styles_list = []
validate_result = cmdline_utils.validate_io(args, styles_list)
result = cmdline_utils.validate_io(args, styles_list)

if validate_result == 0:
if result:
if not constants.FFMPEG_CMD:
raise exceptions.AutosubException(
_("Error: Dependency ffmpeg"
Expand Down Expand Up @@ -147,23 +160,30 @@ def main(): # pylint: disable=too-many-branches, too-many-statements, too-many-
input_m=input_m,
styles_list=styles_list)

elif validate_result == 1:
cmdline_utils.validate_sp_args(args)
else:
result = cmdline_utils.validate_sp_args(args)
fps = cmdline_utils.get_fps(args=args, input_m=input_m)
cmdline_utils.sub_trans(args,
input_m=input_m,
fps=fps,
styles_list=None)
if result:
cmdline_utils.sub_trans(args,
input_m=input_m,
fps=fps,
styles_list=None)
else:
cmdline_utils.sub_conversion(
args,
input_m=input_m,
fps=fps
)

raise exceptions.AutosubException(_("\nAll works done."))

except KeyboardInterrupt:
print(_("\nKeyboardInterrupt. Works stopped."))
return 1
except pysubs2.exceptions.Pysubs2Error:
print(_("\nError: pysubs2.exceptions. Check your file format."))
return 1
except exceptions.AutosubException as err_msg:
print(err_msg)
return 0

print(_("\nAll works done."))
if is_pause:
input_main(_("Press Enter to exit..."))
return 0
Loading

0 comments on commit 2d1ce83

Please sign in to comment.