readline crashes R

[haighr]

I really like your read/readline function; however, when I'm looping through 2000 'readline' commands for files approximately 6,320,000 characters each, R crashes. If I do the same thing using 'read', the operation is successful (and fast!). I can probably use string splits (on '\r\n') to recovery the line breaks, but I thought you should be aware of this issue. Cheers, Rowan
R code
for (i in 1:length(repfiles))
replist[[i]] = Kmisc::readlines(repfiles[i])

OS=MS Windows 10, Processor=Intel(R) Core(TM) i9-9980HK

[kevinushey]

Thanks for the kind words! However, I'm no longer actively developing this package. Do you have any better luck with readr::read_lines(), or readLines() in base R (which should be a fair bit faster in recent R releases)?

[haighr]

Hi Kevin, Thank you for the quick response. I often use readLines but have never had to read in so many big files before. Others have found this function to be the quickest. Your read function seems quick but then I have to break the lines with strsplit inside lapply (~66 seconds), so I'm not sure if I'm further ahead or not. Will try readr:read_lines(). Thanks for the tip! Cheers/salut/ciao, Rowan From: Kevin Ushey ***@***.***> Sent: Thursday, March 16, 2023 1:45 PM To: kevinushey/Kmisc ***@***.***> Cc: Haigh, Rowan ***@***.***>; Author ***@***.***> Subject: Re: [kevinushey/Kmisc] readline crashes R (Issue #14) Thanks for the kind words! However, I'm no longer actively developing this package. Do you have any better luck with readr::read_lines(), or readLines() in base R (which should be a fair bit faster in recent R releases)? — Reply to this email directly, view it on GitHub<https://can01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fkevinushey%2FKmisc%2Fissues%2F14%23issuecomment-1472714957&data=05%7C01%7CRowan.Haigh%40dfo-mpo.gc.ca%7C3fd7ca3716f44faf107c08db265f638f%7C1594fdaea1d94405915d011467234338%7C0%7C0%7C638145963371420664%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=M8j0B0lGdKkMJbn9wkxibqOmToR8Fw%2F6fgXUNzy9z%2B0%3D&reserved=0>, or unsubscribe<https://can01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FABU66WUNDGZE7CIY7QML5DLW4N3VBANCNFSM6AAAAAAV5VCYK4&data=05%7C01%7CRowan.Haigh%40dfo-mpo.gc.ca%7C3fd7ca3716f44faf107c08db265f638f%7C1594fdaea1d94405915d011467234338%7C0%7C0%7C638145963371420664%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=L9atENFYCqs6PhJvfiJXjLQEkCF3ckk3XAVfvxI3z%2Bw%3D&reserved=0>. You are receiving this because you authored the thread.Message ID: ***@***.***>