How to exclude disclaimer?

[fedecaccia]

I wonder if there is a way to exclude a disclaimer section in the bottom of the content. For example in this page: https://medium.com/eosio/eosio-stack-exchange-proposal-a4b3787e1562
I'm using:

import requests
from dragnet import extract_content
url = "https://medium.com/eosio/eosio-stack-exchange-proposal-a4b3787e1562"
r = requests.get(url)
extract_content(r.content) 

[pakelley]

@fedecaccia If you have a labelled dataset with examples of those disclaimers you could probably train a model that excludes them. Other than that, I don't know of a good way to explicitly exclude those (except maybe some post-processing that explicitly finds those disclaimers).
in other words, the current model considers those disclaimers content, and you would have to teach it otherwise to have them filtered out on the dragnet level.