Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 26 additions & 4 deletions macros/l0.DependencyControl.Toolbox.moon
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,10 @@ logger.usePrefixWindow = false
msgs = {
install: {
scanning: "Scanning %d available feeds...",
scanningTask: "Scanning available feeds..."
loadingTask: "Loading feed data..."
empty: "All available scripts are already installed."
emptyUnfetched: "No scripts are available to install (%d of %d known feeds could not be fetched)."
createScriptUpdateRecordFailed: "Failed to create an update record for %s '%s' from feed %s: %s"
}
uninstall: {
Expand Down Expand Up @@ -218,11 +222,16 @@ promptUntrustedFeed = (url, ft) ->

-- Crawls the feed inventory with the untrusted-feed prompter active (so the `prompt` policy asks), scoped
-- so the prompter never leaks into background fetches. Shared by install discovery and Manage Feeds.
crawlWithPrompt = (inventory) ->
-- The pcall keeps a cancellation raised from onProgress from skipping the prompter reset;
-- returns nil when the crawl was cancelled and re-raises anything else.
crawlWithPrompt = (inventory, onProgress) ->
feedTrust = DepCtrl.updater.feedTrust
feedTrust\setPrompter promptUntrustedFeed
entries = inventory\crawl!
ok, entries = pcall inventory.crawl, inventory, onProgress
feedTrust\setPrompter nil
if not ok and entries != "cancelled"
error entries, 0
return nil unless ok
entries

-- Macros
Expand Down Expand Up @@ -261,8 +270,14 @@ install = ->
-- FeedInventory crawls the known feeds, which are trust-gated and bounded. The shared feed loader then
-- serves each reachable feed's data from the cache the crawl just populated.
macros, modules = {}, {}
entries = crawlWithPrompt buildFeedInventory!

aegisub.progress.task msgs.install.scanningTask

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it would be even better if this one actually displayed a moving progress bar but that would involve adding a progress callback to FeedInventory.crawl(), so if you'd rather keep it simple, leave it is and I'll add it to my TODO list as a low priority item.

entries = crawlWithPrompt buildFeedInventory!, (fetched, known) ->
aegisub.progress.set math.floor fetched * 100 / known
error "cancelled", 0 if aegisub.progress.is_cancelled!
aegisub.progress.set 100
return unless entries

aegisub.progress.task msgs.install.loadingTask

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this one show for any appreciable time for you? It doesn't hurt to have this, but if loading the feeds after fetching is near-instant for you and you haven't disabled the feed cache then it's worth investigating why.

logger\log msgs.install.scanning, #entries
for entry in *entries
continue unless entry.fetched
Expand All @@ -273,6 +288,13 @@ install = ->
moduleList, moduleMap = buildDlgList modules
macroList, macroMap = buildDlgList macros

if #moduleList == 0 and #macroList == 0
unfetched = #[entry for entry in *entries when not entry.fetched]
message = unfetched > 0 and msgs.install.emptyUnfetched\format(unfetched, #entries) or msgs.install.empty
aegisub.dialog.display {{class: "label", x: 0, y: 0, width: 1, height: 1, label: message}},
{buttons.close}, {ok: buttons.close, cancel: buttons.close}
return

btn, res = aegisub.dialog.display getScriptListDlg macroList, moduleList
return unless btn

Expand Down
9 changes: 6 additions & 3 deletions modules/l0/DependencyControl/FeedInventory.moon
Original file line number Diff line number Diff line change
Expand Up @@ -242,20 +242,22 @@ class FeedInventory

---Fetches feeds and discovers transitively-advertised ones by crawling the `knownFeeds` graph out from the
---config-derived feeds; untrusted expansion is bounded, so check `stats.truncated` for incomplete results.
---@param onProgress? fun(fetched: integer, known: integer) Called after each feed is fetched, with the running fetch count and the number of feeds currently known to the crawl (more are discovered as it goes, so `known` may grow).
---@return FeedInventoryEntry[] feeds The known feeds, enriched with what the crawl discovered.
---@return FeedCrawlStats stats What the crawl explored and where it stopped short.
crawl: =>
crawl: (onProgress) =>
inventoryEntriesByUrl = @__collectConfigFeeds!
stats = @__crawlKnownFeeds inventoryEntriesByUrl
stats = @__crawlKnownFeeds inventoryEntriesByUrl, onProgress
return @__finalize(inventoryEntriesByUrl), stats

---Breadth-first crawl of the `knownFeeds` graph, extending inventoryEntriesByUrl in place with the transitively-discovered
---feeds under the untrusted-expansion bounds. Each config-derived feed is its own budget subtree, so a
---malicious subtree can't starve the others.
---@private
---@param inventoryEntriesByUrl table<string, FeedInventoryEntry> The config-derived feeds to start from; extended in place.
---@param onProgress? fun(fetched: integer, known: integer) Called after each feed is fetched (see `crawl`).
---@return FeedCrawlStats stats
__crawlKnownFeeds: (inventoryEntriesByUrl) =>
__crawlKnownFeeds: (inventoryEntriesByUrl, onProgress) =>
c = @config.c
limits = c.feeds.crawlLimits or {}
defaults = @@defaultCrawlLimits
Expand Down Expand Up @@ -300,6 +302,7 @@ class FeedInventory
continue unless knownFeeds
stats.fetched += 1
inventoryEntriesByUrl[feedUrl].fetched = true
onProgress stats.fetched, #queue - head + 1 + stats.fetched if onProgress

perFeedUntrusted = 0
perFeedDrops, perRootDrops = newDrops!, newDrops!
Expand Down