🐕 GoFetch

A smart file prefetching system for deep learning workloads that predicts and pre-loads data files into memory before they're needed, reducing I/O bottlenecks during model training.

Overview

GoFetch is an intelligent prefetching daemon designed to accelerate deep learning training pipelines. It learns file access patterns during training and proactively loads files into the kernel page cache, minimizing disk I/O wait times.

With our latest updates, GoFetch now features a fully integrated GRU neural network predictor, automated dataset benchmarking, and a zero-code Docker deployment that intercepts DataLoader calls transparently!

Key Features

• Access Pattern Learning — Uses Markov chains and a PyTorch-based GRU neural network to predict which files will be accessed next
• Kernel-Level Prefetching — Leverages Linux posix_fadvise() to hint the OS kernel about upcoming file reads
• Zero-Code Docker Deployment — Automatically instruments PyTorch DataLoaders using sitecustomize.py with no code changes needed
• Real-Time Metrics & Hit Rates — Tracks windowed cache hits, prediction accuracy, and prefetch latency
• Configurable Policies — Choose between strict FIFO or Confidence-driven Heap prefetch scheduling
• Automated Benchmarks — Built-in test suites for real-world datasets like FashionMNIST and Flowers-102

Getting Started

GoFetch is Docker-only. It relies on Linux kernel prefetching, so host installs are not supported.

Docker Containers (Quick Reference)

GoFetch is published on Docker Hub at parthhan/gofetch. No build step needed — just pull and run.

Image Tag	Description
parthhan/gofetch:pytorch-latest	Recommended — Full PyTorch + CUDA + GoFetch daemon (all-in-one)
parthhan/gofetch:daemon-latest	Lightweight daemon-only image for multi-container setups

# Pull the images
docker pull parthhan/gofetch:pytorch-latest
docker pull parthhan/gofetch:daemon-latest

For a full walkthrough, see docker/README.md.

Docker Deployment

Because posix_fadvise relies on the Linux kernel, we highly recommend running GoFetch via Docker. Our pre-built images on Docker Hub auto-instrument your code — no code changes required in your training scripts!

1. Pull the image directly from Docker Hub (no clone needed)

   docker pull parthhan/gofetch:pytorch-latest

2. Run your own training script (zero-code instrumentation)

   docker run --privileged \
     -v $(pwd)/data:/app/data \
     -v $(pwd)/models:/app/models \
     parthhan/gofetch:pytorch-latest \
     python your_train_script.py

   (The --privileged flag gives the container optimal access to the Linux kernel page cache.)

3. Or use it as a base image in your own Dockerfile

   FROM parthhan/gofetch:pytorch-latest
   COPY . /my_project
   WORKDIR /my_project
   RUN pip install -r requirements.txt
   CMD ["python", "train.py"]

Benchmarking

GoFetch includes a built-in suite to download datasets and test performance. Pull the image and run the benchmark directly — no build step needed:

# Run the full benchmark (FashionMNIST + Flowers-102, all modes)
docker run --privileged \
  -v $(pwd)/data:/app/data \
  -v $(pwd)/models:/app/models \
  parthhan/gofetch:pytorch-latest \
  python -m benchmarks.run_all

This downloads FashionMNIST and Flowers-102, runs Standard PyTorch, GoFetch (Markov), and GoFetch (GRU), then prints comparative speedups. Results are archived under benchmarks/results/ including runs.csv, summary.json, per-run metrics/*.csv, and predictor_accuracy.json.

For more reliable numbers across multiple iterations with cache drops:

docker run --privileged \
  -v $(pwd)/data:/app/data \
  -v $(pwd)/models:/app/models \
  parthhan/gofetch:pytorch-latest \
  python -m benchmarks.run_all --runs 3 --warmup-epochs 1 --drop-caches

To benchmark only the GRU predictor on FashionMNIST:

docker run --privileged \
  -v $(pwd)/data:/app/data \
  -v $(pwd)/models:/app/models \
  parthhan/gofetch:pytorch-latest \
  python -m benchmarks.run_all --dataset fashion --predictor gru --trace-epochs 2 --gru-epochs 5 --runs 1 --no-shuffle

For a more aggressive GRU prefetch configuration:

docker run --privileged \
  -v $(pwd)/data:/app/data \
  -v $(pwd)/models:/app/models \
  parthhan/gofetch:pytorch-latest \
  python -m benchmarks.run_all --dataset fashion --predictor gru --trace-epochs 2 --gru-epochs 5 --runs 1 --no-shuffle \
  --prefetch-confidence 0.9 --prefetch-top-k 8 --prefetch-batch-size 8 --prefetch-size-mb 0.25 --prefetch-rate-limit 30

Note: The dataset benchmarks run synthetic training loops with pseudo-labels to measure pipeline throughput. They are not intended to train a useful model or report accuracy.

Configuration

Edit config.yaml to customize behavior:

storage:
  db_path: "traces.db"

prefetcher:
  policy_type: "heap"          # Options: "heap" or "fifo"
  confidence_threshold: 0.6     # Minimum confidence to prefetch
  rate_limit_per_sec: 20        # Max prefetch operations per second
  top_k_predictions: 5          # Number of files to predict
  max_prefetch_size_mb: 16      # Maximum file size to prefetch

predictor:
  baseline:
    enabled: true               # Markov chain predictor
  ml:
    enabled: true               # GRU neural network predictor
    model_path: "models/prefetch_gru.pt"

How It Works

1. Trace Collection — Captures file access events (path, offset, timestamp, worker_id) natively or via Docker auto-instrumentation.
2. Pattern Learning — Our GRU model encodes temporal features to predict complex data loading behaviors.
3. Prediction — Uses learned patterns to predict upcoming file accesses up to top_k steps ahead.
4. Prefetching — Calls posix_fadvise() to hint the kernel to load predicted files into cache before PyTorch asks for them.
5. Cache Hit — When training reads the file, it's already in memory!

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

Learn More

• See ARCHITECTURE.md for detailed system design.
• Check the examples/ directory for sample usage.