Having fun with language processing

Installation

pip install funlp

you may want to checkout the version

funlp version

Haha, funlp is now on your environment, having fun with it, enjoy ...

Usage

TextRank4Sentence

from funlp import TextRank4Sentence
import codecs

rank4sent = TextRank4Sentence(use_w2v=False,tol=0.0001)

text = codecs.open('./datasets/input/当前中国经济实现完全数字化，小微企业的投资机会将是蓝海.txt', 'r', 'utf-8').read()
rank4sent.summarize(text, 10)

TextRank4Words

from funlp import TextRank4Words
import codecs

rank4words = TextRank4Words(tol=0.0001,window=2)

text = codecs.open('../datasets/input/当前中国经济实现完全数字化，小微企业的投资机会将是蓝海.txt', 'r', 'utf-8').read()
rank4words.summarize(text, 10)

Acknowlegements

• google research
• fairseq
• RoBERTa: A Robustly Optimized BERT Pretraining Approach
• LARGE BATCH OPTIMIZATION FOR DEEP LEARNING: TRAINING BERT IN 76 MINUTES
• 预训练小模型也能拿下13项NLP任务，ALBERT三大改造登顶GLUE基准
• Pre-Training with Whole Word Masking for Chinese BERT