Provision of a "Custom Python script" interface to set Dynamic weights in Simulated Strategies

Feature Requests
1

FEATURE REQUEST

A "Custom Python script" interface to set Dynamic weights in Simulated Strategies

  • What P123 passes: point-in-time data for the strategy's candidates and holdings.
  • What the function returns: target weights.
  • What stays as it is: universe, ranking system, buy and sell rules, slippage and commission, fill prices, position limits, reporting.

This would let users run portfolio optimization with constraints inside a point-in-time simulation, without exporting data or importing weights through $$ factors. Examples of constraints: a tracking-error budget against the benchmark, limits on factor and industry exposures, a beta band, turnover costs.

PROPOSED INTERFACE

Simulations have two schedules, Reconstitution and Rebalance. The function is called with an "event" argument that tells it which one applies. A third optional event "update" is proposed.

event = "reconstitution"

  • When: on the strategy's reconstitution schedule.
  • What P123 has done before the call: evaluated sell rules, then buy rules.
  • The function receives: holdings (positions not exited by a sell rule) and Candidates (see below).
  • The function may decide: which names to hold among holdings + candidates, and their weights. Weight 0 = not held.
    event = "rebalance"
  • When: on rebalance dates that are not reconstitution dates.
  • What P123 has done before the call: nothing; no rules are evaluated.
  • The function receives: holdings only.
  • The function may decide: new weights for the holdings. Weight 0 sells the name; no new names can be added.
    event = "update" (optional, user-enabled)
  • When: every other week of the run.
  • What P123 has done before the call: nothing.
  • The function receives: the data snapshot and its state.
  • The function may decide: nothing tradeable. It only returns an updated state.

Why "update" exists. A factor risk model needs a history of factor returns, which it can build one week at a time from each week's data. With weekly "update" calls and a saved state, the function builds that history itself. P123 then never has to pass years of history on each call.

  • Sell-rule exits are final. The function can't keep a name that a sell rule exited, so P123's rule semantics are preserved.
  • Candidate pool size. At reconstitution, candidates = the top N buy-rule passers by the strategy's ranking system, where N is a new setting ("Candidate pool size", e.g. 200). The pool is NOT truncated to Ideal Positions: an optimizer needs a wider choice than the final number of names.

Script Inputs, all as of the call date and all from P123's data
(1) asof: the date of the data the simulation uses for that rebalance.
(2) event, as above.
(3) holdings: ticker -> current weight, after drift.
(4) candidates: tickers, with each one's rank (reconstitution only).
(5) data: a table, one row per ticker, holding a user-declared list of P123 formulas evaluated on asof.

  • Examples: Rank, ZScore(...) descriptors, Sector, Industry, MktCap, AvgDailyTot(63), any $CustomFormula.
  • Rows: holdings + candidates, plus the estimation universe if one is set (item 6).
    (6) Estimation universe (optional setting): a universe for the risk model that may be larger than the strategy's universe, e.g. Russell 3000 + S&P 1500. The declared formulas are evaluated on it too, flagged by membership.
    (7) benchmark: the benchmark's constituent weights, or the benchmark universe with market caps when constituent weights are not available.
    (8) returns: daily total returns over a user-chosen lookback (e.g. 252 bars), as a dates x tickers matrix.
  • Columns: holdings + candidates + benchmark constituents (+ estimation universe if set).
  • Delisted names are included up to their last trading day.
    (9) state: the object the previous call returned in this simulation run; empty on the first call.

Script Outputs,
(1) weights: ticker -> target weight.

  • Only names allowed by the event.
  • Non-negative (long-only strategies), summing to at most 1; the rest is cash.
  • The strategy's Max positions setting caps the number of names returned. Names over the cap, or outside the allowed set, are an error reported in the run log.
    (2) state: any JSON-serialisable object (or numpy arrays), kept for the next call. A size limit, e.g. 50 MB, is reasonable.
    (3) diagnostics (optional): a flat dictionary, e.g. ex-ante tracking error, solver status, active exposures. It is saved for every call and downloadable with the transactions.
    (4) Sale reason: sales the function causes (weight 0 on a holding) appear in the Transactions tab with the reason "Script", next to the existing sell-rule names.

Execution

  • Trading: P123 trades to the returned weights at the strategy's usual fill price, with its slippage and commission settings.
  • Position weight limits: the Min/Max limits apply after the function. A weight outside them is clipped, and the clipping is logged.
  • Drift: the rebalance tolerance (drift band) applies on "rebalance" events as it does today with Dynamic Weight. Between events, holdings drift with prices.
  • Failures: if the function errors or times out, the simulation stops with the error and the date.

SANDBOX AND POINT-IN-TIME INTEGRITY

  • Where it runs: on P123's servers, with no network or file-system access.
  • Libraries: a fixed set: numpy, scipy, pandas, and a convex solver (CVXPY with the Clarabel solver; optionally OSQP).
  • Limits: a time limit per call (e.g. 10-30 seconds) and a memory limit (e.g. 2-4 GB). A full simulation is about 250 reconstitutions over 20 years at a 4-week interval, or about 1,000 weekly calls with "update".
  • Determinism: the same inputs give the same outputs, so re-runs are reproducible.
  • Point-in-time by construction: every input comes from P123's own data as of asof, so the user's code can't look ahead.

EXAMPLE OF THE FUNCTION

def construct(asof, event, holdings, candidates, data, benchmark, returns, state):
"""
asof : date of the data used for this rebalance
event : "reconstitution" | "rebalance" | "update"
holdings : dict ticker -> current weight (after drift)
candidates : DataFrame [ticker, rank] (empty unless event == "reconstitution")
data : DataFrame, one row per ticker, one column per declared P123 formula
(+ column in_estimation_universe if an estimation universe is set)
benchmark : Series ticker -> benchmark weight
returns : DataFrame dates x tickers, daily total returns over the lookback
state : object returned by the previous call (None on the first call)
returns : (weights: dict ticker -> weight, state, diagnostics: dict)
"""
state = update_factor_history(state, data, returns) # every call, incl. "update"
if event == "update":
return {}, state, {}
allowed = set(holdings) | (set(candidates["ticker"]) if event == "reconstitution" else set())
w, diag = optimize(allowed, data, benchmark, state, holdings) # user's risk model + convex solver
return w, state, diag

I hope this draft proposal make sense, has value for P123 and would be feasible.

Thank you for considering it,
Pierre (+ Claude)

NOTE: Todays The workaround is offline:

  1. download factor data (huge file, cost a lot of credits);
  2. optimize outside P123 for every historical rebalance date;
  3. upload the weights as an imported $$ factor;
  4. re-run a simulation that reads them.
    It works (simulations reproduce the offline portfolio to about 0.5% tracking error). But the point-in-time guarantee then depends on the user's own pipeline, and changing the universe or the ranking system means repeating the whole round trip instead of one click.