fix execution of execute and load_next_page method #587
Conversation
There is a bug while working with Facebook request API, that is not giving all the elements as it is returning 25 elements. So by checking the execute method of the FacebookRequest, I notice one thing that, it is calling only for the first paginated elements and the default returning limit is 25 so It is returning only first page data, and will not check for the next page. So in the execute method of FacebookRequest, there is a method called load_next_page which is the method of the Cursor, which is calling the graph API but it will run only for the first time and it will not check the next page of data. So I updated it like: add a while loop and check the status of the load_next_page and store the result in the status. So if the status will True then It will check the re-execute the load_next_page, which will call the next page graph API as on every execution of load_next_page, URL-PATH which is self._path, will update to next page URL. And this process will continue until there is no Next Page URL. If there is not Next Page URL then self._finished_iteration from Cursor will set to True and when the load_next_page will run again then it first checks if self._finished_iteration then return False. And we will store this value in the status which is in the loop of the execute method of FacebookRequest. So when the status will False break the loop and continue the process as it is already there.
|
Hi @deepprajapati18! Thank you for your pull request and welcome to our community. Action RequiredIn order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you. ProcessIn order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA. Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with If you have received this in error or have any questions, please contact us at [email protected]. Thanks! |
|
Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Facebook open source project. Thanks! |
This was amazing, thank you! |
|
Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Facebook open source project. Thanks! |
2 similar comments
|
Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Facebook open source project. Thanks! |
|
Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Facebook open source project. Thanks! |
You're welcome. @thatalfredh |
|
Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Facebook open source project. Thanks! |
|
was this ever fixed? |
|
This actually isn't an issue as long as you iterate the returned cursor from execute(). The Cursor object will load_next_page() for you if the queue is empty. |
|
Now the real problem is how to make this load_next_page() faster. If I have 10k ad creatives, how do I fetch all the rows quickly without taking over 80 seconds. |
There is a bug while working with Facebook request API, that is not giving all the elements as it is returning 25 elements. So by checking the execute method of the FacebookRequest, I notice one thing that, it is calling only for the first paginated elements and the default returning limit is 25 so It is returning only first page data, and will not check for the next page.
So in the execute method of FacebookRequest, there is a method called load_next_page which is the method of the Cursor, which is calling the graph API but it will run only for the first time and it will not check the next page of data. So I updated it like: add a while loop and check the status of the load_next_page and store the result in the status. So if the status will True then It will check the re-execute the load_next_page, which will call the next page graph API as on every execution of load_next_page, URL-PATH which is self._path, will update to next page URL.
And this process will continue until there is no next Page URL. If there is not Next Page URL then self._finished_iteration from Cursor will set to True and when the load_next_page will run again then it first check if self._finished_iteration then return False.
And we will store this value in the status which is in the loop of the execute method of FacebookRequest. So when the status will False break the loop and continue the process as it is already there.