archive: cache parent directory fd during tar extraction - #49
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #49 +/- ##
==========================================
- Coverage 65.81% 63.91% -1.91%
==========================================
Files 42 35 -7
Lines 2039 2203 +164
==========================================
+ Hits 1342 1408 +66
- Misses 519 597 +78
- Partials 178 198 +20 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
7bb6548 to
68a5292
Compare
45dae29 to
8cc48de
Compare
There was a problem hiding this comment.
Pull request overview
This PR introduces a dirCache abstraction to reduce repeated path-walk overhead during tar extraction by caching the most recently used parent-directory handle and performing *at(2) operations relative to it (Unix), while keeping Windows behavior as a thin delegation to os.Root.
Changes:
- Add
dirCacheimplementations for Unix (openat/*at) and Windows (delegation). - Thread
dirCachethroughUnpack,UnpackLayer, andcreateTarFileto reuse parent-directory context across consecutive entries. - Adjust chmod/chtime/lchown/lchtime call sites to go through the cache, and update tests to use the new
createTarFilesignature.
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
dircache_unix.go |
Adds Unix dirCache using cached parent dir fd + *at(2) syscalls for extraction operations. |
dircache_windows.go |
Adds Windows dirCache delegating to os.Root (no fd caching). |
archive.go |
Updates createTarFile and Unpack to use dirCache for extraction operations. |
diff.go |
Updates UnpackLayer to use dirCache, including AUFS temp-root handling. |
archive_unix.go |
Updates handleLChmod to route chmod through dirCache. |
archive_windows.go |
Updates handleLChmod signature for the new dirCache plumbing. |
archive_test.go |
Adjusts tests to construct/close dirCache when calling createTarFile. |
time_nonwindows.go |
Marks lchtimes helper as unused after moving call sites to dirCache. |
time_windows.go |
Removes the Windows stub lchtimes helper. |
Comments suppressed due to low confidence (2)
dircache_unix.go:147
- UtimesNanoAt is called with flags=0 (follows symlinks). If a path is swapped to a symlink concurrently, this can apply timestamps to an unintended target. Using AT_SYMLINK_NOFOLLOW avoids following symlinks for non-symlink entries while keeping the syscall count low.
if err := unix.UtimesNanoAt(int(d.Fd()), filepath.Base(path), utimes[0:], 0); err != nil {
dircache_unix.go:132
- chmod currently uses fchmodat with flags=0, which follows symlinks; if the path is swapped to a symlink concurrently, this can chmod an unintended target (potentially outside the extraction root). Use AT_SYMLINK_NOFOLLOW with a fallback (like the previous chmodNoSymlink implementation) to avoid following symlinks.
if err := unix.Fchmodat(int(d.Fd()), filepath.Base(path), fileModeToPerm(mode), 0); err != nil {
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| // Source is a regular file. Use os.Root.OpenFile so that all | ||
| // path resolution is bounded within root using openat(2) semantics. | ||
| // os.Root.OpenFile only accepts the nine least-significant permission | ||
| // bits; special bits are applied afterward by handleLChmod. | ||
| // We use sequential file access to avoid depleting the standby list |
Each root.*(path) call re-walks every path component via doInRoot, costing 2D syscalls per call at path depth D. A typical file entry triggers ~5 root.* calls, so at depth 4 that is ~40 syscalls in path traversal alone. Add a dirCache that keeps one parent-directory fd open between entries. Consecutive entries in the same directory reuse the cached fd for all *at(2) operations, reducing path-traversal cost to ~5 syscalls per entry regardless of depth. Signed-off-by: Cesar Talledo <cesar.talledo@docker.com> Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
|
Last commit probably won't work on Windows (yet); working on that |
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Each
root.*(path)call re-walks every path component viadoInRoot, costing 2D syscalls per call at path depth D. A typical file entry triggers ~5root.*calls, so at depth 4 that is ~40 syscalls in path traversal alone.This PR adds a
dirCachethat keeps one parent-directory fd open between entries. Consecutive entries in the same directory reuse the cached fd for all*at(2)operations, reducing path-traversal cost to ~5 syscalls per entry regardless of depth.