refactor: PortalFetch #20
Conversation
crawler_download.py can accept multiple quarterValues now
uniquePaul
requested review from
ak840206,
AmberWow123,
biankaiwen111,
emilylxf,
kis87988 and
a team
as code owners
PortalFetch/crawler_download.py
Outdated
| if school == "De Anza": | ||
| school = "De_Anza" | ||
| fileNameOutput = year + "_" + quarter + "_" + school + "_courseData.json" | ||
| n = 6 |
There was a problem hiding this comment.
avoid naming "n". the naming should be meaningful here. moreover, if a number is a constant, please use upper case. reference:https://www.python.org/dev/peps/pep-0008/#constants
|
A friendly suggestion: |
There was a problem hiding this comment.
Have you tested the code locally? The logic does not seem correct, you use quarter_downlist.select_by_value(value) at line 283 where value is a string of multiple quarter values, webdriver will not be able to find this string and will crash when multiple quarter values are inputted.
example input: 202111_202022
example input: 202111_202022
example input: 202122_202121
PortalFetch/crawler_download.py
Outdated
| school = schoolSwitcher.get(quarterValue[5], "") | ||
| quarter = quarterSwitcher.get(quarterValue[4], "") | ||
| if quarter == "Summer": |
There was a problem hiding this comment.
Please remember to add the comment back...
| list_group_item = None | ||
| while web_driver_counter: | ||
| quartervalue = parser.get('config', 'quarter_value') | ||
| quartervalueList = quartervalue.split('_') |
There was a problem hiding this comment.
It is better to use ',' to represent a list of values, it should be a quick fix since all other functionalities are working, please remember to trim after split(',') since users could add spaces after comma
| login_myportal(driver) | ||
| # Wait for the 'list-group-item' can be found and clicked | ||
| web_driver_counter = 400 | ||
| list_group_item = None |
There was a problem hiding this comment.
Actually you don't need this line of code
There was a problem hiding this comment.
I mean list_group_item = None
crawler_download.py:
can accept multiple quarter values now