refactor `uu_ls` so that crate users can call the `ls` without having to print everything to stdout by fdncred · Pull Request #9851 · uutils/coreutils

fdncred · 2025-12-26T01:36:44Z

I was wanting to use uu_ls in nushell and found out that I couldn't because some of the internals weren't exposed in a way such that crate callers could really use it. So, I was wondering if something like this PR would be something worth considering?

It refactors the code in such a way so that crate callers could call uu_ls and get structured data back instead of everything printing to stdout. It does this without a breaking change by implementing a type of visitor pattern through the LsOutput trait.

I created a TextOutput struct that impelments the LsOutput trait so now the list() function now looks like this:

   pub fn list(locs: Vec<&Path>, config: &Config) -> UResult<()> {
       let mut output = TextOutput::new(config);
       list_with_output(locs, config, &mut output)
   }

All of this allows crate callers like nushell to now do something like this:

use std::ffi::OsString;
use std::path::Path;
use uu_ls::{CollectorOutput, Config, list_with_output, uu_app};

fn main() -> Result<(), String> {
    let matches = uu_app()
        .try_get_matches_from(vec![OsString::from("ls")])
        .map_err(|err| err.to_string())?;

    let config = Config::from(&matches).map_err(|err| err.to_string())?;
    let mut output = CollectorOutput::new();

    list_with_output(vec![Path::new(".")], &config, &mut output).map_err(|err| err.to_string())?;

    for entry in output.entries() {
        // Convert to Nushell Value with full type information
        let name = entry.display_name.to_string_lossy();
        let size = entry.size().unwrap_or(0);
        let is_dir = entry.is_dir();
        // ...
        println!("name: {name}\nsize: {size}\ndir: {is_dir}\nentry: {entry:?}");
    }

    Ok(())
}

One last note, I know I didn't discuss this with anyone so it won't hurt my feelings if this PR isn't accepted. I'm open to whatever. I was just trying to figure out a way to reuse more of the excellent coreutils code. Thanks for such a great repo!!!

github-actions · 2025-12-26T06:21:48Z

GNU testsuite comparison:

Note: The gnu test tests/pr/pr-tests was skipped on 'main' but is now failing.

sylvestre · 2025-12-29T12:04:15Z

sorry, it needs to be rebased

sylvestre · 2026-04-09T15:01:11Z

Sorry, could you please rebase it? Sorry again

github-actions · 2026-04-09T17:15:14Z

GNU testsuite comparison:

GNU test failed: tests/ls/recursive. tests/ls/recursive is passing on 'main'. Maybe you have to rebase?
Skip an intermittent issue tests/tail/inotify-dir-recreate (fails in this run but passes in the 'main' branch)
Skip an intermittent issue tests/tail/tail-n0f (fails in this run but passes in the 'main' branch)
Skip an intermittent issue tests/tty/tty-eof (fails in this run but passes in the 'main' branch)
Note: The gnu test tests/basenc/bounded-memory is now being skipped but was previously passing.
Note: The gnu test tests/printf/printf-surprise is now being skipped but was previously passing.

codspeed-hq · 2026-04-09T17:27:23Z

Merging this PR will degrade performance by 10.19%

⚡ 5 improved benchmarks
❌ 4 regressed benchmarks
✅ 300 untouched benchmarks
⏩ 46 skipped benchmarks¹

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	`BASE`	`HEAD`	Efficiency
❌	Memory	`ls_recursive_deep_tree[(200, 2)]`	143.3 KB	159.5 KB	-10.19%
❌	Memory	`ls_recursive_mixed_tree`	133 KB	141.6 KB	-6.12%
❌	Memory	`ls_recursive_long_all_deep_tree[(100, 4)]`	126.3 KB	133.7 KB	-5.59%
❌	Memory	`ls_recursive_long_all_mixed_tree`	133.6 KB	142.3 KB	-6.11%
⚡	Simulation	`ls_recursive_long_all_deep_tree[(100, 4)]`	2.6 ms	2.4 ms	+4.44%
⚡	Simulation	`ls_recursive_mixed_tree`	1.3 ms	1.2 ms	+5.24%
⚡	Simulation	`ls_recursive_long_all_mixed_tree`	2.3 ms	2.3 ms	+4.16%
⚡	Simulation	`ls_recursive_deep_tree[(200, 2)]`	1.7 ms	1.5 ms	+15.12%
⚡	Simulation	`ls_recursive_wide_tree[(10000, 1000)]`	35.7 ms	32.9 ms	+8.29%

_{Comparing fdncred:refactor_uu_ls_for_others (0f506de) with main (5daf0a5)}

46 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

sylvestre · 2026-04-09T17:28:27Z

some jobs are failing and we see some quite significant regressions

sylvestre · 2026-04-09T17:35:49Z


    for (pos, path_data) in dirs.iter().enumerate() {
-        let needs_blank_line = pos != 0 || !files.is_empty();
+        let _needs_blank_line = pos != 0 || !files.is_empty();


oops. fixed.

fdncred · 2026-04-10T14:23:52Z

some jobs are failing and we see some quite significant regressions

@sylvestre I tried to address these performance concerns. I'm not sure how to validate my changes other than pushing a commit.

I also changed things up a bit because one of the most important things for nushell is streaming and I forgot that when I added the CollectorOutput, which collected, duh. I removed that and replaced it with a StreamingOutput which has a batch mode as the default.

github-actions · 2026-04-10T14:32:46Z

GNU testsuite comparison:

Skip an intermittent issue tests/date/date-locale-hour (fails in this run but passes in the 'main' branch)
Skipping an intermittent issue tests/tty/tty-eof (passes in this run but fails in the 'main' branch)

fdncred · 2026-04-10T17:22:41Z

@sylvestre can you start the CI so we can see if I caught everything?

github-actions · 2026-04-11T06:24:39Z

GNU testsuite comparison:

Skip an intermittent issue tests/date/resolution (fails in this run but passes in the 'main' branch)
Skip an intermittent issue tests/tty/tty-eof (fails in this run but passes in the 'main' branch)
Skipping an intermittent issue tests/tail/tail-n0f (passes in this run but fails in the 'main' branch)
Note: The gnu test tests/csplit/csplit-heap is now being skipped but was previously passing.

fdncred · 2026-04-11T13:54:08Z

@sylvestre hopefully this fixes the last broken test. it passes on my ubuntu vm. if we can start the ci again. i feel for y'all. so many regression tests. 🤯

github-actions · 2026-04-11T14:07:42Z

GNU testsuite comparison:

Skip an intermittent issue tests/date/date-locale-hour (fails in this run but passes in the 'main' branch)
Skipping an intermittent issue tests/tty/tty-eof (passes in this run but fails in the 'main' branch)
Note: The gnu test tests/misc/write-errors was skipped on 'main' but is now failing.

sylvestre · 2026-04-11T14:19:15Z

done!

github-actions · 2026-04-11T14:29:25Z

GNU testsuite comparison:

Skip an intermittent issue tests/tty/tty-eof (fails in this run but passes in the 'main' branch)
Note: The gnu test tests/pr/bounded-memory is now being skipped but was previously passing.
Note: The gnu test tests/tail/pipe-f is now being skipped but was previously passing.

github-actions · 2026-04-11T20:08:09Z

GNU testsuite comparison:

Skip an intermittent issue tests/cut/bounded-memory (fails in this run but passes in the 'main' branch)

github-actions · 2026-04-12T08:30:06Z

GNU testsuite comparison:

Skip an intermittent issue tests/pr/bounded-memory (fails in this run but passes in the 'main' branch)
Note: The gnu test tests/cut/bounded-memory is now being skipped but was previously passing.
Congrats! The gnu test tests/cut/cut-huge-range is now passing!

github-actions · 2026-04-13T05:36:41Z

GNU testsuite comparison:

Skip an intermittent issue tests/tty/tty-eof (fails in this run but passes in the 'main' branch)
Skipping an intermittent issue tests/date/date-locale-hour (passes in this run but fails in the 'main' branch)
Congrats! The gnu test tests/dd/no-allocate is now passing!
Congrats! The gnu test tests/printf/printf-surprise is now passing!

sylvestre · 2026-04-13T05:40:32Z

the memory regressions are acceptable

sylvestre · 2026-04-13T05:40:46Z

please let me known when it is ready to be reviewed

fdncred · 2026-04-13T11:30:13Z

@sylvestre it's ready. just was trying to get that memory regression closer but i've kind of had my fill of messing with it. thanks!

fdncred · 2026-04-15T19:56:28Z

@sylvestre any thoughts? Would like to get this landed if it's acceptible before bit rot occurs again.

github-actions · 2026-04-16T11:41:40Z

GNU testsuite comparison:

Skipping an intermittent issue tests/tail/symlink (passes in this run but fails in the 'main' branch)
Note: The gnu test tests/expand/bounded-memory is now being skipped but was previously passing.
Note: The gnu test tests/tail/tail-n0f is now being skipped but was previously passing.
Congrats! The gnu test tests/rm/many-dir-entries-vs-OOM is now passing!
Congrats! The gnu test tests/seq/seq-epipe is now passing!

fdncred · 2026-04-16T20:02:01Z

I don't think these CI errors are related to this PR

the internal print stdout output

update output handling to add streaming mode.

… output calculations

…rectory_entries and enter_directory functions

cakebaker

Good work, kudos!

Co-authored-by: Daniel Hofstetter <daniel.hofstetter@42dh.com>

cakebaker · 2026-04-17T12:26:34Z

Thanks for your PR!

…11930) * ls: restore WASI ".." metadata fallback in collect_directory_entries The ls refactor in #9851 dropped the WASI guard added in #11633, so `ls -al` at a preopened root once again fails with "Capabilities insufficient" when stat'ing "..". Extract the logic into a small `dotdot_path` helper and add a regression test covering `ls -al`. * Update spell-checker ignore words in test_ls.rs

fdncred mentioned this pull request Jan 28, 2026

Display accurate file mode (include ACL "+" indicator) nushell/nushell#17438

Open

2 tasks

fdncred mentioned this pull request Apr 9, 2026

add uutils/coreutils commands into nushell nushell/nushell#11549

Open

16 tasks

fdncred force-pushed the refactor_uu_ls_for_others branch from 59813c6 to a2e601e Compare April 9, 2026 16:32

sylvestre reviewed Apr 9, 2026

View reviewed changes

cakebaker reviewed Apr 16, 2026

View reviewed changes

Comment thread src/uu/ls/src/ls.rs

fdncred added 2 commits April 17, 2026 08:00

refactor uu_ls so that crate users can call the command without having

a1be742

the internal print stdout output

redo changes

f7ce22c

fdncred added 11 commits April 17, 2026 08:00

clippy

440ca9f

help cspell understand dired

fcf657a

rebase

11f7f6c

try and optimize for memory and speed performance.

77a5546

update output handling to add streaming mode.

work on ci issues, optimize uid and gid cache handling, improve dired…

b4b2de4

… output calculations

test fx: change read_dir parameter to mutable reference in collect_di…

a4aa2a8

…rectory_entries and enter_directory functions

update documentation

7ef4f68

more perf tweaks

a8ed2ea

fmt

6781aa2

tweak perf

a5204c4

Add back (and shorten) comment for 31_556_952

dbed28e

sylvestre force-pushed the refactor_uu_ls_for_others branch from 430a09b to dbed28e Compare April 17, 2026 06:00

cakebaker reviewed Apr 17, 2026

View reviewed changes

Comment thread src/uu/ls/src/ls.rs Outdated

cakebaker reviewed Apr 17, 2026

View reviewed changes

Comment thread src/uu/ls/src/ls.rs Outdated

cakebaker reviewed Apr 17, 2026

View reviewed changes

sylvestre and others added 2 commits April 17, 2026 11:25

Update src/uu/ls/src/ls.rs

a454875

Co-authored-by: Daniel Hofstetter <daniel.hofstetter@42dh.com>

Update src/uu/ls/src/ls.rs

0f506de

Co-authored-by: Daniel Hofstetter <daniel.hofstetter@42dh.com>

cakebaker merged commit ceb09c7 into uutils:main Apr 17, 2026
168 of 169 checks passed

fdncred deleted the refactor_uu_ls_for_others branch April 17, 2026 12:32

dmatos2012 mentioned this pull request Apr 20, 2026

ln command is missing nushell/nushell#16504

Open

sylvestre mentioned this pull request Apr 21, 2026

ls: restore WASI ".." metadata fallback in collect_directory_entries #11930

Merged

Uh oh!

Conversation

fdncred commented Dec 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Dec 26, 2025

Uh oh!

sylvestre commented Dec 29, 2025

Uh oh!

sylvestre commented Apr 9, 2026

Uh oh!

github-actions Bot commented Apr 9, 2026

Uh oh!

codspeed-hq Bot commented Apr 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will degrade performance by 10.19%

Performance Changes

Footnotes

Uh oh!

sylvestre commented Apr 9, 2026

Uh oh!

sylvestre Apr 9, 2026

Choose a reason for hiding this comment

Uh oh!

fdncred Apr 10, 2026

Choose a reason for hiding this comment

Uh oh!

fdncred commented Apr 10, 2026

Uh oh!

github-actions Bot commented Apr 10, 2026

Uh oh!

fdncred commented Apr 10, 2026

Uh oh!

github-actions Bot commented Apr 11, 2026

Uh oh!

fdncred commented Apr 11, 2026

Uh oh!

github-actions Bot commented Apr 11, 2026

Uh oh!

sylvestre commented Apr 11, 2026

Uh oh!

github-actions Bot commented Apr 11, 2026

Uh oh!

github-actions Bot commented Apr 11, 2026

Uh oh!

github-actions Bot commented Apr 12, 2026

Uh oh!

github-actions Bot commented Apr 13, 2026

Uh oh!

sylvestre commented Apr 13, 2026

Uh oh!

sylvestre commented Apr 13, 2026

Uh oh!

fdncred commented Apr 13, 2026

Uh oh!

fdncred commented Apr 15, 2026

Uh oh!

Uh oh!

github-actions Bot commented Apr 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

fdncred commented Apr 16, 2026

Uh oh!

Uh oh!

Uh oh!

cakebaker left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cakebaker commented Apr 17, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

fdncred commented Dec 26, 2025 •

edited

Loading

codspeed-hq Bot commented Apr 9, 2026 •

edited

Loading

github-actions Bot commented Apr 16, 2026 •

edited

Loading