Re-capture starting portfolio value after warm-up and fire OnWarmupFinished before post-warm-up data

[jhonabreul]

Description

Two related warm-up fixes:

1. Starting portfolio value captured with stale currency conversion rates (#9647).
In deployments with a warm-up period, StartingPortfolioValue is captured at setup time with the cash book conversion rates seeded point-in-time at the warm-up start (in live mode additionally mixed with deploy-time holdings prices). For portfolios holding cash or positions in a currency other than the account currency, the further back warm-up reaches, the staler the FX component of the snapshot — producing a permanent phantom net return, contaminated drawdown statistics and an incorrect initial reported equity. It recurs on every live restart. Backtesting is affected by the same mechanism in a milder form: BacktestingSetupHandler captures Portfolio.Cash right after the conversion rates are seeded at the warm-up start, so the reported net return absorbs the FX drift that occurs during the warm-up window, before the algorithm can trade.

2. OnWarmupFinished fired after the first non-warm-up data reached OnData.
AlgorithmManager.Stream could only detect the warm-up transition when pulling the next time slice — one slice after IsWarmingUp flipped inside the engine loop. When data ends exactly at the warm-up boundary (the normal case for intraday subscriptions on markets trading through it), one OnData call with IsWarmingUp already false executed before OnWarmupFinished, and orders placed there (the common "trade as soon as warm-up is done" pattern) could fill before the callback ran.

The fix:

• The warm-up finished transition moves from AlgorithmManager.Stream into the Run loop, where it is handled in the same time slice that flips IsWarmingUp: after the slice's securities and cash book updates — so OnWarmupFinished sees the same market data as before — and before any user code runs for that slice. The transition is also checked at the time-pulse shortcut, since the boundary slice can be a dataless pulse; this keeps OnWarmupFinished firing with the algorithm time at the warm-up end.
• The transition first notifies the result handler through a new IResultHandler.OnWarmupFinished, then triggers the algorithm's OnWarmupFinished callback (still invoked through the Python wrapper) and the running status update.
• BaseResultsHandler.OnWarmupFinished re-captures StartingPortfolioValue, CumulativeMaxPortfolioValue and DailyPortfolioValue from the current portfolio value — generically, for both backtesting and live trading — before any post-warm-up user code can trade. The algorithm manager notifies it exactly once, right when warm-up finishes; the re-capture only reassigns values that actually changed (so unaffected algorithms keep byte-identical statistics) and discards the equity bar accumulated during warm-up so the first equity sample opens at the re-captured value.

Seeding the conversion rates point-in-time at the warm-up start is kept as is: it is correct for epoch-consistent warm-up replay, and chart sampling during warm-up was already skipped by the result handlers. Only the capture timing changes. Algorithms without warm-up are unaffected, and algorithms with warm-up but no foreign-currency balances re-capture a value identical to the setup-time one, keeping their statistics unchanged.

Related Issue

Closes #9647

Motivation and Context

Live algorithms with a long warm-up and multi-currency holdings report a permanent, incorrect net return (observed: -4.55% on an account that had not traded) and skewed drawdown statistics from the moment of deployment. Backtests combining warm-up with foreign-currency cash report returns contaminated by FX drift from a period when the algorithm could not trade. Additionally, algorithms could receive tradable post-warm-up data before OnWarmupFinished, breaking the expectation that the callback marks the start of live data.

Requires Documentation Change

No.

How Has This Been Tested?

• New WarmupStartingPortfolioValueRegressionAlgorithm: reproduces the issue in backtesting — foreign-currency cash with a one-year warm-up, so the conversion rate seeded at the warm-up start (EURUSD ~1.05) drifts ~14% during replay. The expected "Start Equity" statistic (220025.00) pins the re-captured warm-up-end value instead of the stale snapshot (~205290), and "Net Profit" reflects only the trading window. Fails on master, passes with the fix.
• New OnWarmupFinishedOrderingRegressionAlgorithm: minute forex data with the warm-up boundary at mid-week midnight, so there is data ending exactly at the boundary; asserts OnWarmupFinished fires at the warm-up end time and before any OnData call with IsWarmingUp false. Fails on master, passes with the fix.
• New LiveTradingResultHandlerTests.RecapturesStartingPortfolioValueAfterWarmup and BacktestingResultHandlerTests.RecapturesStartingPortfolioValueAfterWarmup: verify StartingPortfolioValue, CumulativeMaxPortfolioValue and DailyPortfolioValue are re-captured from the current portfolio value on the warm-up finished notification.
• Full warm-up regression battery (42 tests, including OnWarmupFinished*, WarmupConversionRates and the option/future/universe warm-up variants): all pass with unchanged expected statistics.
• LimitOrdersAreFilledAfterHoursForFuturesRegressionAlgorithm, which places orders inside OnWarmupFinished: unchanged statistics, confirming the callback still sees the same market data as before.
• Engine.Results, AlgorithmManager, PaperBrokerage and AlgorithmLiveTrading unit test fixtures: 50 passed, 0 failed.
• End-to-end reproductions in local paper trading and backtesting, before and after the fix (see below).

End-to-end reproduction in local paper trading and backtesting

Live mode anchors to the real clock while the repo's EURUSD daily sample data covers 2007-2018, so a warm-up long enough to reach back into that data reproduces the mixed-epoch snapshot: setup seeds the EURUSD rate at the warm-up start (2013 => ~1.35) and warm-up replay carries it to the end of the sample data (2018-12-31 => ~1.147).

Scratch algorithm (EUR cash in a USD account, warm-up back to ~2013-09):

public class Issue9647ReproductionAlgorithm : QCAlgorithm
{
    public override void Initialize()
    {
        SetCash(100000);
        SetCash("EUR", 100000);
        AddForex("EURUSD", Resolution.Daily, Market.Oanda);
        SetBenchmark(time => 0m);
        SetWarmUp(TimeSpan.FromDays(4700));
    }

    public override void OnWarmupFinished()
    {
        Log($"OnWarmupFinished at utc {UtcTime:O}: " +
            $"TotalPortfolioValue={Portfolio.TotalPortfolioValue.ToStringInvariant()}, " +
            $"EUR ConversionRate={Portfolio.CashBook["EUR"].ConversionRate.ToStringInvariant()}");
    }
}

Launcher/config.json: "environment": "live-paper", with the environment's data-queue-handler switched to [ "QuantConnect.Lean.Engine.DataFeeds.Queues.FakeDataQueue" ] (the default LiveDataQueue is a stub that throws on subscribe).

Without the fix — the stale snapshot is captured and never corrected, a permanent ~-8.75% phantom return on an account that never traded:

BaseSetupHandler.SetupCurrencyConversions():
EUR: €      100000.00 @     1.3525 = $135251.00
CashBook Total Value:                $235251.00     <- captured as StartingPortfolioValue

OnWarmupFinished: TotalPortfolioValue=214657.00000, EUR ConversionRate=1.14657

With the fix — same setup snapshot (warm-up replay still gets its epoch-consistent seed), but the starting value is re-captured right when warm-up completes, landing exactly on the algorithm's own portfolio value:

EUR: €      100000.00 @     1.3525 = $135251.00
LiveTradingResultHandler.RecaptureStartingPortfolioValueIfWarmupFinished(): Re-captured starting portfolio value after warm-up: 214657.00000
OnWarmupFinished: TotalPortfolioValue=214657.00000, EUR ConversionRate=1.14657

The same effect reproduces in a backtest with the algorithm above modified with SetStartDate(2018, 1, 1), SetEndDate(2018, 1, 15) and SetWarmUp(TimeSpan.FromDays(1461)) (warm-up back to ~2014-01, EURUSD ~1.38 there vs ~1.20 at the start date), run in the backtesting environment. The algorithm never trades, and EUR actually appreciated during the two test weeks:

Without the fix — a phantom -6.378% net profit from the FX drift during the warm-up window:

EUR: €      100000.00 @     1.3788 = $137879.00
CashBook Total Value:                $237879.00     <- captured as StartingPortfolioValue
OnWarmupFinished at 2018-01-01: TotalPortfolioValue=220025.00000, EUR ConversionRate=1.20025
STATISTICS:: Net Profit -6.378%

With the fix — the reported return reflects only the actual FX movement during the trading window:

EUR: €      100000.00 @     1.3788 = $137879.00
BacktestingResultHandler.RecaptureStartingPortfolioValueIfWarmupFinished(): Re-captured starting portfolio value after warm-up: 220025.00000
OnWarmupFinished at 2018-01-01: TotalPortfolioValue=220025.00000, EUR ConversionRate=1.20025
STATISTICS:: Net Profit 1.219%

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• Refactor (non-breaking change which improves implementation)
• Performance (non-breaking change which improves performance. Please add associated performance test and results)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Non-functional change (xml comments/documentation/etc)

Checklist:

• My code follows the code style of this project.
• I have read the CONTRIBUTING document.
• I have added tests to cover my changes.
• All new and existing tests passed.
• My branch follows the naming convention bug-<issue#>-<description> or feature-<issue#>-<description>