On many systems, python refers to Python 3.

Fixes bbolli#87

Fixes bbolli#179

Certain options should never be negative, and a count of zero should
actually download zero posts, not all of them. Also, make sure we can
read cookiefile before we back up any blogs.

Included revisions:
- Add check for confusing -i -n 0

Posts are now read from disk each time they are needed, instead of
keeping them all in memory.

This is especially needed when --save-notes is used with a large blog,
as this often results in many gigabytes worth of HTML that does not fit
into free memory all at once.

qsize() acquires a mutex on the queue, which is shared between threads.
Read it only once to avoid a race condition.

Included revisions:
- Make typing optional

Included revisions:
- Saner non-option arguments

Included revisions:
- Remove log_queue, better status and account logic
- Better tracking and synchronization on ThreadPool.queue.qsize
- Remove remaining_posts
- Remove getting_tup
- Put back the account parameter

Fixes bbolli#169

Fixes #1

Fixes bbolli#202

Behaves similar to cp -u or or rsync when copying local files into the
media directory.

Included revisions:
- Use prev_archive for style.css

Fixes bbolli#205

- posixpath provides functions to split and join paths compatible with
  URLs, so there is no need to manually split on and insert slashes.
- get_media_url would return the saved filename instead of the URL if the
  download fails, unlike get_image_url. Fix it to be consistent.
- get_filename now applies the extension to generated filenames if it is
  known, and uses urlparse to make sure the filename comes only from the
  path part of the URL.

Based on PR bbolli#114 by @aggroskater

The API has apparently changed since aggroskater's fork was written.

This is especially helpful if less posts were backed up than the user
expected.

Reading from and writing to a JSON directory simultaneously is not
supported by --prev-archives. Tell the user to use --reuse-json for
that.

Correctly sort responses to avoid breaking most timestamp/id related
options. Everything was in reverse order and the key was wrong for
likes.

Without this, the combination of --prev-archives and --period does not
work.

Use path_to instead of open_file to avoid creating directories
unnecessarily early.

Uses urllib3 and a temporary file to make downloading media smoother.
Also supports various modes of timestamping, inspired by wget.

Included revisions:
- Fix urllib3 status code handling
- Cleanup fsync logic
- Specific type: ignore comments
- Cleanup file_isfat and merge into util.py
- note_scraper: Smarter handling of cookies and error messages
- Remove enospc handler
- Use os.supports_dir_fd
- No more log_queue
- Global, abstract no_internet
- Support "canceled" WaitOnMainThread status
- Fix "TypeError: 'bytes' object does not support item assignment"
- Fix some incorrectly sorted imports

Fixes bbolli#201

Create WaitOnMainThread.destroy, and use it to prevent blocking in
signal() when we're trying to stop the thread.

…oding

decode_content=True is necessary for chunked responses if we want to
decode them.

Importing requests from pip is tricky. Fix a potential recursive import.

Windows does not support os.open() of directories, os.fchmod(), or
os.fdatasync().

Fixes #4

Also add tumblr_login.py to generate cookies without a browser.
Credit to @johanneszab for the C# implementation in TumblThree.
Credit to @MrEldritch for the initial Python port.

Fixes bbolli#161

Importing requests from pip is tricky. Fix a potential recursive import.

Included revisions:
- The API thread is now a daemon thread that is quit and joined after
  the backup pool is canceled
- Try to shut down more reliably (join-except-cancel, idempotent
  AsyncCallable.quit)

Included revisions:
- Be more paranoid to avoid swallowing exceptions

Included revisions:
- Allow setting options in BACKUP_CHANGING_OPTIONS to match the backup
- Fix incorrectly sorted imports
- Use lists in a few places for consistency with JSON
- Set before to oldest_tstamp even without --period
- Fix argument check to allow --auto 0
- Pass all path parts to open_text in save_post to ensure mkdir

Fixes bbolli#115

Don't bother spinning up a thread pool or calling out to the API.

This allows the script to find media files in a structure other than the
one it is saving them in.

This is a reuse of the option previously called --continue=ignore.
Sometimes it is useful to resume a backup with new
BACKUP_CHANGING_OPTIONS (such as --count).

Incremental and resume modes only back up certain posts, setting a
start or end to the range. Make this clear to the user so they don't
assume that it is a full backup, especially because the "expected" count
appears to indicate that it is.

Posts that failed to save (e.g., if urllib3 throws a fatal error) were
difficult to recover from. Now content is lazy so save_post can save
JSON first and report if it can't save the HTML.

Add rate limit sleeping inspired by gallery-dl, and make the response
status logic more robust.

Included revisions:
- Fix format arguments
- Wait an extra second to be sure we're past the ratelimit reset
- Add missing try_count increment

This changes the way status codes 429, 503, and 504 are handled.

Included revisions:
- Don't assume there is a Server header in HTTP responses

Writing this file later saves us from a confusing, pointless empty blog
directory if the first API request fails.

The post offset is now clearer and --no-post-clobber is turned on when
using --continue.

This condition is like safe mode - there is no no usable notes list. In
this case, there is a notes list, but not in an easily parsable format.

Make sure of this with a lock. And if we do somehow signal a dead PID,
ignoring it is the most reasonable option.

Windows does not support os.open() of directories, os.fchmod(), or
os.fdatasync().

Related to #4

This silences a name-error from pytype.

I apparently overlooked some information in [1], which says:

  However, if the target thread is executing the Python interpreter,
  the Python signal handlers will be executed by the main thread of the
  main interpreter. Therefore, the only point of sending a signal to a
  particular Python thread would be to force a running system call to
  fail with InterruptedError.

Further, [2] notes:

  Changed in version 3.5: Python now retries system calls when a syscall
  is interrupted by a signal, except if the signal handler raises an
  exception (see PEP 475 for the rationale), instead of raising
  InterruptedError.

So after Python 3.5, there is no use for signal.pthread_kill. It cannot
do what it does in C, which is send a signal to a spefic thread - at
best, the thread will wake up with EINTR and then resume the system
call.

Also, the sys.exc_info() check was wrong: There will always be an active
KeyboardInterrupt exception in a signal handler. This means that the
pthread_kill() never had any effect, masking the fact that only the main
thread received the signal.

The signal handlers are being kept because they are still useful if the
program receives SIGTERM or SIGHUP, allowing resources to be cleaned up
via exception handling instead of being left for the OS to deal with.

[1] https://docs.python.org/3/library/signal.html#signal.pthread_kill
[2] https://docs.python.org/3/library/exceptions.html#InterruptedError

There were three "type: ignore" comments that were not specific and
ignored all type errors. Make one of them specific. Remove the other two
since python/typeshed#4544 was merged and they are no longer needed.

This method was meant to be using the "image_names" parameter as of
commit 3cac7d8 ("tumblr_backup: Use .first_run_options of
--prev-archives"), but was still using options.image_names.

Included revisions:
- Don't assume there is a Server header in HTTP responses

This condition is like safe mode - there is no no usable notes list. In
this case, there is a notes list, but not in an easily parsable format.

This silences a name-error from pytype.

Included revisions:
- Don't assume there is a Server header in HTTP responses

This prevents ambiguity for URLs such as SoundCloud embeds that do not
have a unique filename (for SoundCloud embeds it is "stream").

Use post timestamp information to find more plausible timestamps for
media. Tumblr's Last-Modified headers are usually not accurate and often
years off.

This code never worked well with Tumblr media because the Last-Modified
timestamp is hit-or-miss. This only removes the conditional get code;
files are still given mtime based on the Last-Modified header.

Calling os.utime() on a file that may (and likely does) have data left
in a userspace buffer is a bad move, as the next write() call will
clobber the modification time. Flush the buffer early to prevent this.

Consider using SecureTransport if the linked OpenSSL doesn't support
SNI, and print a warning if either SecureTransport or pyOpenSSL cannot
be injected.

Related to #5

Include some information about this fork and its differences from
upstream tumblr-utils.

There is no `WGWrongCodeError.reason`, only `WGWrongCodeError.statmsg`.