Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Javascript not executed on Facebook #329

Open
1453064 opened this issue Jun 6, 2017 · 0 comments
Open

Javascript not executed on Facebook #329

1453064 opened this issue Jun 6, 2017 · 0 comments

Comments

@1453064
Copy link

1453064 commented Jun 6, 2017

Hi, i am currently use ghost.py to write a mini crawler on facebook.

The problem is that the page doesn't execute any of it's script

  • Python 2.7
  • Ghost.py latest

Homepage:
image

Friend:
image

This is my cmd log: I have tried install TimeSlice and BigPipe, no change.
image

The code:

from ghost import Ghost, Session
from bs4 import BeautifulSoup
ghost = Ghost()

with ghost.start():
    session = Session(ghost, user_agent='Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.121 Safari/535.2',
    wait_timeout=8, wait_callback=None, display=True, viewport_size=(800, 600),
    ignore_ssl_errors=True, plugins_enabled=True, java_enabled=True, javascript_enabled=True,
    download_images=True, show_scrollbars=True)
    session.wait_timeout = 999

    # This a a cookie i generated from my login_facebook script
    session.load_cookies('fbookie')

    # Some random facebook id
    page, extra_resources = session.open("https://www.facebook.com/100017894403983")
    
    # Just some test with beautifulsoup
    result, extra = session.evaluate('document.title;')
    
    print('The title is: {}'.format(result))
    if page.http_status == 200:
        soup = BeautifulSoup(session.content, 'lxml')

    soup.find_all("p", "title")

    for a in soup.findAll('a'):
        print a['href']

I am still unsure whether this is Ghost.py's fault or something else.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant