PiPic image-compression benchmark — 2026 Q3

Method

• Stratify the input set across PNG, JPEG, WebP and AVIF.
• Use only images whose licenses allow public redistribution, and record each source and license in manifest.csv.
• Record every tool name, exact version, command, run date, machine and config.
• Compare same-format input and output; format conversion is not compression.
• Record input bytes, output bytes, saved bytes, duration, failures and files that did not get smaller.
• Keep outliers and document their cause instead of deleting them.
• Run each case three times on the same machine and report median duration.
• Randomly select at least three outputs per format for decode and visual checks.
• Generate results.csv from the runner; do not hand-edit measurements.
• Publish limitations and cases where a competing tool performs better.

Inputs

Three photographs cover landscape, food and architecture. Each source is CC0 or public domain and is encoded as PNG, JPEG, WebP and AVIF, giving 12 inputs. Every benchmark operation is same-format: PNG in and PNG out, and likewise for the other formats. manifest.csv records the public source page, license, byte size and SHA-256 of every committed input; SOURCES.md records author and the Wikimedia source SHA-1 checked before creating derivatives.

The derivatives are intentionally high-quality inputs, not a claim that they represent all web images. Each subject has the same pixel dimensions across its four formats, and every file stays below PiPic's documented 8 MB limit.

Tools and configurations

• PiPic Agent CLI 1.0.6: the published CLI and its remote compression service, one file at a time with -o, --json and --concurrency 1.
• sharp 0.35.3: local library calls with explicit documented defaults: PNG compression level 6 without palette conversion; JPEG quality 80, 4:2:0 chroma and optimized coding; WebP quality 80 and effort 4; AVIF quality 50, effort 4, 4:4:4 chroma and 8-bit output.

The command column records the exact executable path and arguments used for each run. The sharp options live in scripts/sharp-compress.mjs, which is part of the published reproduction package.

Recorded observations

• The run produced 72 raw rows: 12 inputs × 2 tools × 3 consecutive runs.
• PiPic completed 35 of 36 operations. The second food.webp run returned fetch failed; the row is retained as an error with zero output and zero saved bytes. It was not retried.
• With the explicit non-palette PNG configuration above, sharp produced a larger file for all three PNG inputs in all three runs. Those nine rows are retained as not_smaller, not counted as savings and not called failures.
• All other recorded outputs decoded in their input format. A deterministic sample covering three different subjects per format passed visual review.

These are dataset observations, not a "best compressor" claim. summary.csv reports medians; results.csv remains the source for failures and variation.

Reproduce

Requirements:

• macOS or another environment providing Node.js 20+, npm and ffprobe;
• ffmpeg built with libsvtav1 and cwebp only if regenerating the committed inputs;
• a user-authorized PiPic CLI session with at least 36 remaining monthly operations. The runner does not read or print credentials, but it consumes real CLI allowance because PiPic is a remote service.

From this directory:

# Optional: re-download the three sources, verify their Wikimedia SHA-1,
# and recreate all 12 inputs and manifest rows.
node scripts/prepare-inputs.mjs

# Install exact tool versions into ignored .tools/, check PiPic quota,
# execute three runs, decode outputs and generate the CSV files.
node scripts/run.mjs

# Run after completing the selected visual review.
node scripts/validate.mjs

The recorded machine was macOS 26.5.1 on an Apple M5 with 24 GiB RAM. PiPic durations include the remote request; sharp durations are local execution. They must not be presented as a CPU-speed comparison.

Limitations

• The tools are compared as documented workflows, not at matched perceptual quality. File-size differences can reflect different quality decisions.
• The dataset contains three photographic subjects. It does not represent illustrations, alpha-heavy graphics, screenshots, animation or every camera.
• PiPic is remote while sharp is local. Network, service load and geography affect PiPic duration; machine load affects both.
• Visual review establishes that selected outputs are usable, not a numeric perceptual-quality score. No PSNR, SSIM or SSIMULACRA2 claim is made.
• The single retained PiPic fetch failure means its completion rate in this run was not 100%. A later run must be published separately, not overwrite this result.

Reproduction gate

The dataset passed its local reproduction gate: all inputs have a public license and SHA-256, raw rows contain documented commands, every non-error output decoded, three different outputs per format passed visual review, and the median rows can be recomputed from the raw CSV.

Do not use “fastest”, “smallest”, “best” or equivalent claims from this data.