upload.dataset() can't deal with large data sets

[fboylu]

while using upload.dataset() on a data frame, the following error is returned:

Error: AzureML returns error code:
HTTP status code : 500
Maximum request length exceeded.
Traceback:

1. upload.dataset(labeledfeatures, ws, name = "labeledfeatures")
2. try_fetch(url, handle = h)
3. validate_response(r)
4. stop(msg, call. = FALSE)

I was able to upload a smaller subset of the data frame without issues.

[gilbertw]

We need to switch to a chunked upload to handle the large payloads. See the python sdk change for reference, Azure-Samples/Azure-MachineLearning-ClientLibrary-Python@4204e2b

[stephlocke]

Any ETA available on a resolution to this issue?

[andrie]

Pull requests are welcome. I don't think anybody is actively working on this issue.

[stephlocke]

Ok

There's no contributor guidelines and whatnot on this - before PRs can be accepted, does this need to be once-overed for Microsoft compliance stuff or is it ok as-is?

http://open.microsoft.com/