Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

"ls" can take ages on a large folder. Is there a way to make it more immediate, i.e. streaming output without sorting?


It's something like ls -u from memory.


Looks like it's -U (capital U). But I just tried it and it still took several seconds for the first filename to appear. It was not the spinning up of the disk because I first did ls in the parent folder which was immediate. The second time I did ls on the large folder, though, it was fast (even without -U).


> It was not the spinning up of the disk because I first did ls in the parent folder which was immediate

That doesn't tell you anything; the parent's dentries could have been cached days ago and still present, meaning it didn't actually access the disk or cause it to be spun up (if it wasn't) at all.

When doing any kind of repeatable measurement or experimentation on disks you will want to drop the page cache every time first:

    # echo 3 >/proc/sys/vm/drop_caches


Well, I just booted the computer ;) But you are right, dropping the cache is probably a better way.

I have a folder with 5500 subfolders. Doing "ls -U" in that folder (after dropping the page cache like above) takes 50 seconds (!) And the dir entries appear all at once, i.e. not in a streaming way.

Its parent folder only contains 6 subfolders. Doing a cache drop followed by "ls -U" gives immediate results.

How to investigate this further? (Using an Ubuntu 18.04 system)


strace can tell you what system calls it's doing, what the results are, and how long they're taking, which may help narrow it down.


Thanks. Interestingly, strace speeds up the operation. What took 50s after a cache drop now becomes immediate with "strace -f ls -U".

Dropping the cache and doing "time ls -U" gives:

    real   0m51,204s
    user   0m0,116s
    sys    0m0,718s
Update: never mind, it appears to be something in my shell. Switching to tcsh completely eradicated the problem.


It's probably that ls in the other shell is the builtin instead of the binary, when strace runs ls -U it does run the binary and not the shell builtin. tcsh must also delegate to the binary instead of a builtin, or their builtin is faster.


Yeah, I haven't figured it out yet, but when I do /bin/ls, then indeed the problem does not show. Probably a case of Bash trying to be smarter than it needs to be.


Yes you're right, it's -U on linux (1). On Mac its -f (2). Linux also has -f, which is equivalent to -a -U .

(1) https://man7.org/linux/man-pages/man1/ls.1.html

(2) https://ss64.com/mac/ls.html




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: