Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

can't upload large size file #31

Open
y0uzha opened this issue Nov 1, 2019 · 1 comment
Open

can't upload large size file #31

y0uzha opened this issue Nov 1, 2019 · 1 comment

Comments

@y0uzha
Copy link

y0uzha commented Nov 1, 2019

Hi, i've try to upload small size of file and it succeed. but when i try to upload 130Mb file, it always failed. is there any options configurations that i must set first to do this operation?

sometimes i got this error msg.

Error: BP-1312390296-192.168.1.228-1545800077608:blk_1073838522_97940 does not exist or is not under Constructionnull

i've try to increase the buffer size like below this but it still failed :

var localFileStream = fs.createReadStream('/tmp/test.jar', { highWaterMark: 1024 * 1024 });
var remoteFileStream = hdfs.createWriteStream('/user/training/upload/test.jar', { highWaterMark: 1024 * 1024 });
@maziyarpanahi
Copy link

Have you found any workaround for this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants