feat: cog weights import/pull download progress by anish-sahoo · Pull Request #3087 · replicate/cog

anish-sahoo · 2026-06-30T21:50:04Z

Summary

show live Docker-style download progress while cog weights import warms the local weight store
show live Docker-style download progress while cog weights pull populates a cold local cache
add focused tests for import and pull progress events

Testing

mise run fmt:go:fix
go test ./pkg/cli
go test ./pkg/model
go test ./pkg/weights
mise run lint:go

ask-bonk · 2026-06-30T21:52:54Z

LGTM

github run

ask-bonk · 2026-06-30T22:09:44Z

LGTM

github run

ask-bonk

I'm Bonk, and I've done a quick review of your PR.

This PR adds live download progress display during cog weights import by plumbing a progress callback through the weight builder and ingress pipeline, with a focused test verifying progress event generation.

I found one issue with the progress display logic.

ask-bonk · 2026-06-30T22:14:24Z

I'm Bonk, and I've done a quick review of your PR.

This PR adds live download progress display during cog weights import by plumbing a progress callback through the weight builder and ingress pipeline, with a focused test verifying progress event generation.

I found one issue with the progress display logic.

Posted a batch review with a suggestion to fix intermediate files not being marked as "Download complete" in pkg/cli/weights.go:175.

github run

ask-bonk

I'm Bonk, and I've done a quick review of your PR.

This PR adds live download progress display during cog weights import and cog weights pull by plumbing progress callbacks through the weight builder and ingress/pull pipeline, with focused tests verifying progress event generation.

I found two issues with the progress display logic.

ask-bonk · 2026-06-30T22:36:14Z

Posted a batch review on PR #3087 with two suggestions:

pkg/cli/weights.go:175 — Remove prog.Complete >= prog.Total so intermediate files show "Download complete" when they finish.
pkg/weights/pull.go:305-318 — Add throttling to pullProgressReader (matching progressReader's 250ms interval) to prevent console flooding on large files.

github run

ask-bonk · 2026-06-30T23:06:44Z

@anish-sahoo Bonk workflow was cancelled.

View workflow run · To retry, trigger Bonk again.

ask-bonk

I'm Bonk, and I've done a quick review of your PR.

This PR adds live download progress display during cog weights import and cog weights pull by plumbing progress callbacks through the weight builder and ingress/pull pipeline, with focused tests verifying progress event generation.

I found one remaining issue with the progress display logic.

ask-bonk · 2026-06-30T23:17:28Z

I'm Bonk, and I've done a quick review of your PR.

This PR adds live download progress display during cog weights import and cog weights pull by plumbing progress callbacks through the weight builder and ingress/pull pipeline, with focused tests verifying progress event generation.

I found one remaining issue with the progress display logic.

1. pkg/weights/pull.go:305-318 — pullProgressReader emits a progress event on every Read() call without throttling, while progressReader in pkg/model/packer.go uses a 250ms interval to avoid flooding the console. For large files this can mean thousands of progress events per file. Add the same throttling behavior for consistent UX (also add "time" to the imports):

type pullProgressReader struct {
	r            io.Reader
	complete     int64
	lastReported int64
	lastUpdate   time.Time
	interval     time.Duration
	fn           func(int64)
}

func (r *pullProgressReader) Read(p []byte) (int, error) {
	n, err := r.r.Read(p)
	if n > 0 {
		r.complete += int64(n)
		now := time.Now()
		interval := r.interval
		if interval == 0 {
			interval = 250 * time.Millisecond
		}
		if r.lastReported == 0 || now.Sub(r.lastUpdate) >= interval {
			r.lastReported = r.complete
			r.lastUpdate = now
			r.fn(r.complete)
		}
	}
	return n, err
}

Posted a batch review on PR #3087 with this suggestion.

github run

ask-bonk · 2026-07-01T15:50:13Z

I've reviewed the current state of PR #3087. Both previously identified issues have been addressed:

pkg/cli/weights.go:175 — Fixed to mark each file complete when prog.Done is set, without the cumulative progress check.
pkg/weights/pull.go — pullProgressReader now implements proper throttling with a 250ms default interval, matching progressReader in packer.go.

I've also verified the progress event plumbing through weight_builder.go, the pull command in weights_pull.go, and the focused tests in both packer_test.go and pull_test.go. Everything looks consistent and correct.

LGTM

github run

ask-bonk · 2026-07-06T20:55:15Z

LGTM

github run

ask-bonk · 2026-07-15T17:23:01Z

LGTM

github run

ask-bonk · 2026-07-16T18:34:11Z

LGTM

github run

Copilot

Pull request overview

Adds Docker-style live progress reporting for weight downloads during cog weights import (source ingress into the local store) and cog weights pull (registry/cache fetch), including new event types and focused unit/integration tests to validate progress emission.

Changes:

Emit per-file download progress events during weights pull, and render them via Docker’s jsonmessage progress UI.
Add an io.Reader wrapper (NewProgressReader) to throttle byte-progress callbacks and wire it into both pull and import/ingress paths.
Extend tests (unit + integration) to validate progress and completion output.

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
pkg/weights/pull.go	Emits `PullEventFileProgress` while streaming files from layers into the store.
pkg/weights/pull_test.go	Updates event-sequence expectations and asserts monotonic per-file progress.
pkg/util/progress_reader.go	Introduces throttled progress-reporting reader wrapper.
pkg/util/progress_reader_test.go	Verifies throttling behavior of the progress reader.
pkg/model/weight_builder.go	Adds aggregate `WeightBuildProgress` callback plumbing for imports.
pkg/model/packer.go	Adds progress-aware ingress path; de-dupes by digest to avoid duplicate downloads/opens.
pkg/model/packer_test.go	Tests aggregate ingress progress and digest de-dupe behavior.
pkg/docker/progress.go	Generalizes progress rendering and adds `WriteLine` for non-progress messages.
pkg/docker/progress_test.go	Ensures status updates reuse a single progress line.
pkg/cli/weights.go	Renders import download progress and adjusts push-status output to `WriteLine`.
pkg/cli/weights_pull.go	Renders pull progress bars/status via Docker-style progress writer.
integration-tests/tests/weights_pull.txtar	Asserts “Download complete” appears for import and pull flows.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

ask-bonk · 2026-08-03T21:43:30Z

@anish-sahoo Bonk workflow was cancelled.

View workflow run · To retry, trigger Bonk again.

Copilot

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated no new comments.

Suppressed comments (2)

pkg/util/progress_reader.go:26

NewProgressReader will panic if called with a nil report function (it unconditionally calls r.report in Read), but the exported API/comment doesn’t document that report must be non-nil. It’s safer for this util to treat a nil report as a no-op and just return the underlying reader.

// NewProgressReader reports throttled intermediate byte counts. It does not
// report EOF; callers should emit their own completion event.
func NewProgressReader(reader io.Reader, report func(int64)) io.Reader {
	return &progressReader{
		reader:   reader,

pkg/weights/pull.go:273

Per-file progress/status is emitted for every tar entry in a fetched layer, including files that were already present in the local store. This can make user-visible output inconsistent with MissingFiles (e.g. "Pulling … (1 file(s))" but multiple "Download complete" rows). Consider only emitting progress and FileStored for the files in needed (the missing files for this layer), and just draining cached entries to keep the tar stream in sync.

		reader := util.NewProgressReader(tr, reportProgress)
		if err := m.store.PutFile(ctx, file.Digest, file.Size, reader); err != nil {
			return fmt.Errorf("store %s (%s): %w", file.Path, file.Digest, err)
		}
		written[file.Path] = true

ask-bonk · 2026-08-10T15:40:41Z

LGTM

github run

markphelps

lgtm!!!

feat: show weights import download progress

2700249

anish-sahoo requested a review from a team as a code owner June 30, 2026 21:50

anish-sahoo changed the title ~~Show weights import download progress~~ feat: cog weights import progress bar Jun 30, 2026

anish-sahoo changed the title ~~feat: cog weights import progress bar~~ feat: weights import progress bar Jun 30, 2026

fix: keep weights import progress on one line

6e828f4

fix: use docker progress for weight downloads

f536402

ask-bonk Bot reviewed Jun 30, 2026

View reviewed changes

Comment thread pkg/cli/weights.go Outdated

feat: show weights pull download progress

669a993

anish-sahoo changed the title ~~feat: weights import progress bar~~ Show weights download progress Jun 30, 2026

anish-sahoo changed the title ~~Show weights download progress~~ feat: cog weights download progress Jun 30, 2026

fix: mark each imported weight file complete

7f1d0fb

ask-bonk Bot reviewed Jun 30, 2026

View reviewed changes

Comment thread pkg/cli/weights.go

Comment thread pkg/weights/pull.go Outdated

ask-bonk Bot reviewed Jun 30, 2026

View reviewed changes

Comment thread pkg/weights/pull.go Outdated

fix: throttle weights pull progress events

548a143

anish-sahoo changed the title ~~feat: cog weights download progress~~ feat: cog weights download/upload progress Jul 1, 2026

anish-sahoo changed the title ~~feat: cog weights download/upload progress~~ feat: cog weights download progress Jul 1, 2026

anish-sahoo changed the title ~~feat: cog weights download progress~~ feat: cog weights import/pull download progress Jul 1, 2026

Merge branch 'main' into weights-loading

b4a6bf0

Merge branch 'main' into weights-loading

b470a17

Merge branch 'main' into weights-loading

6de19d5

fix: clean up weight download progress

fef2769

Copilot AI review requested due to automatic review settings August 3, 2026 21:13

Copilot started reviewing on behalf of anish-sahoo August 3, 2026 21:13 View session

Copilot AI reviewed Aug 3, 2026

View reviewed changes

Comment thread pkg/util/progress_reader.go Outdated

Comment thread pkg/cli/weights_pull.go

fix: disambiguate weight pull progress rows

dc7ebc1

Copilot AI review requested due to automatic review settings August 3, 2026 21:56

Copilot started reviewing on behalf of anish-sahoo August 3, 2026 21:56 View session

Copilot AI reviewed Aug 3, 2026

View reviewed changes

Merge branch 'main' into weights-loading

88be408

markphelps approved these changes Aug 11, 2026

View reviewed changes

anish-sahoo added this pull request to the merge queue Aug 11, 2026

Merged via the queue into main with commit 01098ba Aug 11, 2026
44 checks passed

anish-sahoo deleted the weights-loading branch August 11, 2026 16:10

Uh oh!

Conversation

anish-sahoo commented Jun 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Testing

Uh oh!

ask-bonk Bot commented Jun 30, 2026

Uh oh!

ask-bonk Bot commented Jun 30, 2026

Uh oh!

ask-bonk Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ask-bonk Bot commented Jun 30, 2026

Uh oh!

ask-bonk Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

ask-bonk Bot commented Jun 30, 2026

Uh oh!

ask-bonk Bot commented Jun 30, 2026

Uh oh!

ask-bonk Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ask-bonk Bot commented Jun 30, 2026

Uh oh!

ask-bonk Bot commented Jul 1, 2026

Uh oh!

ask-bonk Bot commented Jul 6, 2026

Uh oh!

ask-bonk Bot commented Jul 15, 2026

Uh oh!

ask-bonk Bot commented Jul 16, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

ask-bonk Bot commented Aug 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

ask-bonk Bot commented Aug 10, 2026

Uh oh!

markphelps left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

anish-sahoo commented Jun 30, 2026 •

edited

Loading

ask-bonk Bot commented Aug 3, 2026 •

edited

Loading