Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implemented JustAnotherArchivist's requested changes to Telegram scraper from PR #2

Merged
merged 10 commits into from
May 9, 2022
Merged

Implemented JustAnotherArchivist's requested changes to Telegram scraper from PR #2

merged 10 commits into from
May 9, 2022

Commits on Mar 31, 2022

  1. implemented Media dataclasses for Telegram, and added variable for ex… 

    …tracting a post's view count
    @trislee
    trislee committed Mar 31, 2022
    Configuration menu
    Copy the full SHA
    a7eb54d View commit details
    Browse the repository at this point in the history

  2. added a forwardedUrl attribute to TelegramPost and made forwarded att… 

    …ribute type Channel.
    @trislee
    trislee committed Mar 31, 2022
    Configuration menu
    Copy the full SHA
    4e59638 View commit details
    Browse the repository at this point in the history

Commits on Apr 17, 2022

  1. made Telegram scraper not return full channel info for forwarded_from… 

    … attribute; fixed video edge cases.
    @trislee
    trislee committed Apr 17, 2022
    Configuration menu
    Copy the full SHA
    babcddd View commit details
    Browse the repository at this point in the history

  2. fixed issue where Telegram scraper terminated early because some page… 

    …s didn't have a next page link (added reasonable default)
    @trislee
    trislee committed Apr 17, 2022
    Configuration menu
    Copy the full SHA
    1e4e0c2 View commit details
    Browse the repository at this point in the history

  3. fixed issue where some videos and photos weren't being scraped (becau… 

    …se they weren't in a post containing a 'tgme_widget_message_text' div
    @trislee
    trislee committed Apr 17, 2022
    Configuration menu
    Copy the full SHA
    b276c3c View commit details
    Browse the repository at this point in the history

Commits on Apr 21, 2022

  1. added additional termination criteria to Telegram scraper

    @trislee
    trislee committed Apr 21, 2022
    Configuration menu
    Copy the full SHA
    97d38e5 View commit details
    Browse the repository at this point in the history

  2. added additional attributes for hashtags and user mentions, removed r… 

    …edundant outlinks
    @trislee
    trislee committed Apr 21, 2022
    Configuration menu
    Copy the full SHA
    9b3faec View commit details
    Browse the repository at this point in the history

  3. moved forward finding out of tgme_widget_message_text clause, since i… 

    …t wasn't correctly getting the forwarding information in forwarded posts that contained attachments but no text
    @trislee
    trislee committed Apr 21, 2022
    Configuration menu
    Copy the full SHA
    21f7b62 View commit details
    Browse the repository at this point in the history

Commits on Apr 27, 2022

  1. improved consistency of code formatting and added _STYLE_MEDIA_URL_PA… 

    …TTERN as variable
    @trislee
    trislee committed Apr 27, 2022
    Configuration menu
    Copy the full SHA
    5648e95 View commit details
    Browse the repository at this point in the history

Commits on May 9, 2022

  1. Merge branch 'master' into telegram-media

    @trislee
    trislee committed May 9, 2022
    Configuration menu
    Copy the full SHA
    c18ca0f View commit details
    Browse the repository at this point in the history