From 6ee8c8a571c3fee4c8aacf6333227b3a50fecb45 Mon Sep 17 00:00:00 2001 From: Tyrin Todd Date: Tue, 10 Feb 2026 17:52:15 -0800 Subject: [PATCH] (no commit message) --- README.md | 313 +++++++++++++++++++++++++++++++++++++++++++++- probe.json | 1 + probe.safetensors | Bin 0 -> 20636 bytes 3 files changed, 313 insertions(+), 1 deletion(-) create mode 100644 probe.json create mode 100644 probe.safetensors diff --git a/README.md b/README.md index e3fbd92..77c405b 100644 --- a/README.md +++ b/README.md @@ -1 +1,312 @@ -Sentiment Classification \ No newline at end of file +# Bench + +Modaic internal SDK for benchmarking judges and training confidence probes. + +## Installation + +```bash +cd cli +uv sync +``` + +## CLI Commands + +All commands are run from the `cli` directory via `uv run mo `. + +### `create` + +Create benchmark datasets for training confidence probes. This command runs a judge on examples, extracts embeddings via Modal, and pushes the resulting dataset to HuggingFace Hub. + +**Subcommands:** + +- `create ppe` - Create dataset from PPE (human-preference + correctness) benchmarks +- `create judge_bench` - Create dataset from the JudgeBench benchmark + +**Usage:** + +```bash +# Interactive mode (recommended) - prompts for configuration +uv run mo create ppe +uv run mo create judge_bench + +# With config file +uv run mo create ppe --config config.yaml +uv run mo create judge_bench --config config.yaml +``` + +**Options:** + +| Option | Short | Description | +| ---------- | ----- | -------------------------- | +| `--config` | `-c` | Path to config file (YAML) | + +**Config File Example:** + +```yaml +judge: tyrin/ppe-judge-gepa +output: tytodd/my-probe-dataset +n_train: 500 +n_test: 100 +embedding_layer: -1 # -1 for middle layer +``` + +**What it does:** + +1. Loads examples from the benchmark dataset +2. Runs the specified judge on each example to get predictions +3. Extracts embeddings from the judge's LLM via Modal (GPU) +4. Creates a HuggingFace dataset with columns: `question`, `response_a`, `response_b`, `label`, `predicted`, `messages`, `embeddings` +5. Pushes to HuggingFace Hub + +--- + +### `train` + +Train a confidence probe on an embeddings dataset created with `create`. + +**Usage:** + +```bash +# Interactive mode (recommended) - prompts for all configuration +uv run mo train + +# With config file +uv run mo train --config config.yaml + +# With CLI arguments +uv run mo train --dataset tytodd/my-embeddings --epochs 10 --lr 0.0001 +``` + +**Options:** + +| Option | Short | Description | Default | +| ---------------- | ----- | --------------------------------------------------------------------------------- | ----------------- | +| `--config` | `-c` | Path to config file (YAML) | - | +| `--dataset` | `-d` | Dataset path (HuggingFace Hub or local) (must be a dataset created with `create`) | - | +| `--model-path` | `-m` | Output path for trained model | `{dataset}_probe` | +| `--batch-size` | | Batch size | 4 | +| `--epochs` | | Number of training epochs | 10 | +| `--lr` | | Learning rate | 0.0001 | +| `--weight-decay` | | Weight decay | 0.01 | +| `--test-size` | | Validation split ratio (if no test split) | 0.2 | +| `--seed` | | Random seed | 42 | +| `--project` | | W&B project name | model_path | +| `--hub-path` | | HuggingFace Hub path to push model | - | + +**Config File Example:** + +```yaml +dataset_path: tytodd/my-probe-dataset +model_path: ./best_probe +hub_path: tytodd/my-probe # Optional: push to HF Hub +batch_size: 4 +epochs: 10 +learning_rate: 0.0001 +weight_decay: 0.01 +test_size: 0.2 +seed: 42 +``` + +**What it does:** + +1. Loads an embeddings dataset (from HuggingFace Hub or local) +2. Creates binary labels: 1 if `predicted == label`, 0 otherwise +3. Trains a linear probe using MSE loss (Brier score optimization) +4. Logs metrics to Weights & Biases (Brier, ECE, MCE, Kuiper, AUROC) +5. Saves the best model based on validation Brier score +6. Optionally pushes to HuggingFace Hub + +--- + +### `eval` + +Evaluate a trained confidence probe on a dataset. Computes calibration and discrimination metrics. + +**Usage:** + +```bash +# Interactive mode (recommended) - prompts for probe and dataset +uv run mo eval + +# With CLI arguments +uv run mo eval --probe tytodd/my-probe --dataset tytodd/my-embeddings + +# Evaluate on train split instead of test +uv run mo eval --probe tytodd/my-probe --dataset tytodd/my-embeddings --split train +``` + +**Options:** + +| Option | Short | Description | Default | +| ---------------------------- | ----- | ---------------------------------------- | ------------ | +| `--probe` | `-p` | Probe path (HuggingFace Hub or local) | - | +| `--dataset` | `-d` | Dataset path (HuggingFace Hub or local) | - | +| `--split` | `-s` | Dataset split to evaluate on | test | +| `--batch-size` | `-b` | Batch size for evaluation | 64 | +| `--normalize/--no-normalize` | `-n` | Normalize embeddings with StandardScaler | probe config | + +**Metrics computed:** + +| Metric | Description | +| ----------- | ------------------------------------------------- | +| Brier Score | Mean squared error between predictions and labels | +| Accuracy | Classification accuracy at 0.5 threshold | +| F1 Score | Harmonic mean of precision and recall | +| ECE | Expected Calibration Error (10 bins) | +| MCE | Maximum Calibration Error | +| Kuiper | Kuiper statistic for calibration | +| AUROC | Area Under the ROC Curve (discrimination) | + +**What it does:** + +1. Loads a pretrained probe from HuggingFace Hub or local path +2. Loads a dataset created with `create` +3. Creates binary labels: 1 if `predicted == label`, 0 otherwise +4. Runs inference and computes calibration/discrimination metrics +5. Displays results in a formatted table + +--- + +### `compile` + +Compile (optimize) a judge using GEPA over a dataset. GEPA iteratively improves the judge's prompt based on training examples. + +**Subcommands:** + +- `compile` (base) - Compile with custom dataset and parameter mapping +- `compile ppe` - Compile specifically for PPE datasets (human-preference + correctness) + +**Usage:** + +```bash +# Interactive mode +uv run mo compile +uv run mo compile ppe + +# With config file +uv run mo compile --config config.yaml +uv run mo compile ppe --config config.yaml +``` + +**Options:** + +| Option | Short | Description | +| ---------- | ----- | -------------------------- | +| `--config` | `-c` | Path to config file (YAML) | + +**Config File Example:** + +```yaml +judge: tyrin/ppe-judge +dataset: tytodd/ppe-human-preference +inputs: # selects which input columns of the dataset to use (not necearry if using a compile subcommand like ppe or judge_bench) + - name: question + - name: response_a + column: response_A # Map param name to dataset column + - name: response_b + column: response_B +label_column: label +n_train: 100 +n_val: 50 +base_model: gpt-4o-mini +reflection_model: gpt-4o +output: tyrin/ppe-judge-gepa +seed: 42 +``` + +**What it does:** + +1. Loads a judge from Modaic Hub +2. Loads training/validation examples from a HuggingFace dataset +3. Maps judge parameters to dataset columns +4. Runs GEPA optimization to improve the judge's prompt +5. Pushes the optimized judge to Modaic Hub + +--- + +### `embed` + +Regenerate embeddings for an existing dataset using a different model or layer. Useful for experimenting with different embedding configurations without re-running the judge. + +**Usage:** + +```bash +# Interactive mode +uv run mo embed + +# With CLI arguments +uv run mo embed --dataset tytodd/my-dataset --hf-model Qwen/Qwen3-VL-32B-Instruct --layer -1 +``` + +**Options:** + +| Option | Short | Description | +| ------------ | ----- | ---------------------------------------- | +| `--dataset` | `-d` | Dataset path (HuggingFace Hub or local) | +| `--hf-model` | `-m` | HuggingFace model path for embeddings | +| `--layer` | `-l` | Hidden layer index (-1 for middle layer) | + +**What it does:** + +1. Loads an existing dataset (must have a `messages` column) +2. Regenerates embeddings using the specified model/layer via Modal +3. Replaces the `embeddings` column in the dataset +4. Prompts to push the updated dataset to HuggingFace Hub + +**Example workflow:** + +```bash +# Original dataset was created with layer 32 +# Now try middle layer instead +uv run mo embed \ + --dataset tytodd/my-embeddings \ + --hf-model Qwen/Qwen3-VL-32B-Instruct \ + --layer -1 +``` + +--- + +## Recommended Embedding Layers + +When extracting embeddings, use these recommended layer indices for best probe performance: + +| Model | HuggingFace Path | Recommended Layer | +| ------------- | ----------------------------------- | ----------------- | +| GPT-OSS 20B | `openai/gpt-oss-20b` | 8 | +| Qwen3-VL 32B | `Qwen/Qwen3-VL-32B-Instruct` | 16 | +| Llama 3.3 70B | `meta-llama/Llama-3.3-70B-Instruct` | 32 | + +Use `-1` for the middle layer if experimenting with an unlisted model. + +--- + +## Typical Workflow + +```bash +# 1. Create a probe dataset from a benchmark +uv run mo create ppe + +# 2. Train a confidence probe +uv run mo train --dataset tytodd/ppe-qwen3-embeddings + +# 3. Evaluate the probe on a test set +uv run mo eval --probe tytodd/my-probe --dataset tytodd/ppe-qwen3-embeddings + +# 4. (Optional) Compile/optimize a judge with GEPA +uv run mo compile ppe + +# 5. (Optional) Re-embed with different layer +uv run mo embed --dataset tytodd/my-dataset --layer 32 +``` + +## Environment Variables + +Create a `.env` file with: + +```bash +OPENAI_API_KEY=... +WANDB_API_KEY=... +HF_TOKEN=... +MODAIC_TOKEN=... +TOGETHER_API_KEY=... +``` diff --git a/probe.json b/probe.json new file mode 100644 index 0000000..6de82a1 --- /dev/null +++ b/probe.json @@ -0,0 +1 @@ +{"probe_version":"v1","embedding_dim":5120,"model_path":"Qwen/Qwen3-VL-32B-Instruct","dropout":0.0,"layer_index":16,"num_layers":65,"probe_type":"linear"} \ No newline at end of file diff --git a/probe.safetensors b/probe.safetensors new file mode 100644 index 0000000000000000000000000000000000000000..f1428679912d09cdcd0f9ca45c4f9ab383a23038 GIT binary patch literal 20636 zcmaHSc~p*H^l$T^K^aR@Dk_?2e)fJFC~2TbM3hX)5E4qFNrZ~h9GVQN45jHg`!yw@ zXdsD@d5+9|75Dw!yY63it^54h z_=yzv&zb$RJq+_Fd9V-N6*#iq9HTYX)>S&qo zzqzPp6TxP4R%rOfkF?fG&R#_%-?2`W=HYEa6NQxSVXpy4^mT{K5eqN z$pWUPv$J=~Sd5=OxH(;eFTsl`z@wZ~v55xVx)Yq1;~0}O9u7&Sk^z9zTQe8uG)tZ`%Q7xqSKK4ax%tb%Fy{GJE^B&3RyZy!EjI^ zwS`AnhS_@hrlrq@roQAmUI=0Ipckl(j^m2?KgqauzqaG}1> zTkuB9Y?$kfww;E*`FDA_>YGTu;c0zh~#3K!J+TWS+kb?eLfi@J}OXY zXD`{#nLs`lPEqp=fUk8YQQdv5@Ihw^cBIPF`j4;Ki)W7XxGM(kJ~78`wV~MXZa%#C z?`4`z%g|BkF*o_rE+$v@1)T4!#Z%F4b>hZy;F!A%$_~xO$dFHP<8KUGlcpw=_K_A= zj#-7Cn^Xk01?9jJZ}GLOtc8U^Kn~K!Y2u<rND}Xt)Ho ziNCJvTCKu7POHK2&t2??Z#5|18-{(g?X>sW9$R0N6WFCN7elti;(+cmeo?;?B`3&H ze$RIDwI7CmCY197TpbSgG{wY4_u=2ibl4=B2X_*#&_~hGeOMNjsxf_WxqYp8?uXZrI z!XKX4>7m^|4S{8yAV#(`PsH)`(_qZ)dT2@PkV7|MAaW zC4t)tPnhpK4tI(6bV^DJyY)4NO7yqWsY%ytS6p9*weNm#yZ>w<4TT66FDO8heFb3q z^A(1ynZ)+|{>?rqibOBBil(FH;-}x4Ylav>Ek&O<_MaiLYsG4O!BO5!Jg<7%j+KDZs z`GO-Un?!Q53&bRT2s6nvp+B;VxS*;UP+jdzS7xO0FWzs2`>ArYe_}mznXw4n&2#Zd zv;@8KnJrW(l%vE`UP71Kd-32p15QH5hw>~J(*818d_Dj;-qbO>Frk6N>q{3-{G~v0;_rB&RTsmTszKtNkB>%zs;O&dYk3b$t|tFSMX7TL0=K z470eT;falY*?9QIy}1l%5s`Y5WE#eG;2e1=H{=fT3| zyPz*r61$s~sCcRl?Z5g8M$P1@|~R@Bm_}cx8lnV3E}(2 zZ4|6=irjXOrGSZ@ykckxta_V^ojwn^;oEI+Zc7d zYPvTm9>&#_gO*kUj7U<&$hkjQ)Ko?Gx^n`~yOD;YtnKKC+f)A44Wy=VTWG?`*F@cV zSpF)JxAoB!E(xz?342$9v%L;kdX`YufMa_L@e5lBhibex4z_kU1Gg^=!q$<>vTq&2px4!Hb+i9`VaeGFg=G z5-MJBmc;8)sN1X5wzM^wEX=cMX}~FVu67vCIH!$?zdcxtwFPm{3^8QZTblM>9{hLM z;(_&f%q{YKU31Aay0ZHZsI2wIwU%3pT2W@W*aiyl^Sr6*I@$y#uw9Zf>w|(q|U*s+o0Y#L#!2 z$hxLD7gQ>EkKLnAF#|oMNn0wQWA9W>_&OQg-t6Mu9rCAr_WSv&hW*&(evBIHPQdSP z^I)@BI#actz$c7-%H)EK$kA{bt_n=0u+e__;#nOP_V%-=UyJze!czS8bOrzN%mrrs z=m4qCIE8YHAFxkRuh`1_evl+yN|uxKaY@KHd?uR$&)jyPLd{puOLSz(hsv1IHAV}^ z1>q8-JuJ3*78-_T)+v2+qv2>U;0OMw#VjFF?&MSWoLeip%)WH=QQ zkGJ}~xCg=h_^(j~RumnD`TM7nV^K3d=;(lx(!>QHiiY9C)me1=$uBldasz0q$1;zD z(`mCZ50)O&(K~lK6ofQ$(PIAS*EEFHESLv}XFD)mja**2;V9`3TGNczlfZj*4BlFI zhBuqL726{A@^4r6kfB^PNKapYEtkeXTf-tIR&yE`&ly3%|G6^tY3;Ne?@-6&M*4E~ zA=jhT2@jskBjb0-=Y7A$wg-t*OVtj>zh|`G=oDrJ>>#}t@esuM(T$o@bg$?I8CUth z7GE9IFI^8EmmlN9A`XdD!nd;=;6l>`8o&E5Gz}d^V`BAiFmM_Lu1F=f+LIWOosBPZ z#RTbF;@AueH3~M?K*OuUh4!_Ucw~Ny?byl?_?Y;QUneU`!*aq%CF>XBg>Ot(?Ko|o zVTF=~7f`A*0?Rs8sO|GO;qZofICmtL^y-PVSxl+heo%qBc12@l%^=+F%%lT%k8ufu z6|5~glU}Jr!waA5@ZVqxt+|~=pW|f&vI0*U*>4H@b8ax*kE*mbQBu%npG)55(`mfW z73S=;6&#k-#-YpK^Ua6vP~dV+%2YR}XJQ}u`!N9&F;fDyS23nhqsW8_d2Ct9blX4e zS8>WhdpbrA?7RA3)?Rp$S?2s>ZwCpSytBr!q2101snN3z`g>py*dK zkkqGQf3XW58Qa57Zjcq;^w5AnZ(9~w`5CT6*^*kBD>ve*CHpmEr(pG-S;FE$X*xEr z2W^&TP@q;kEcJ}0!0*zmATyc7SL%^Qdm(xrFeB%oN)+N(K}FA8abs&K3%Z&?E;EFj zk*Xw$j|l-Mzdu}vn+vU6CQ0_2%ekY5CnS{VfVKMKX@2jq*TWRbQ;G4uwzC zu9HtgC$!5HP>g*WZCX46eykg0buka2C&d(vCM4ns{R=pcCJQE<31Qc@;!%3D8|?YE zkVUk3!+M1Sc(Ah+!e}3J<_^I?N7zEMtdvtYU*3R$-f0G`>{b zO)oBXv+B#iGMhXVyPb@e@NU+)o6DLsH&TvO^KPgk?d`r**?mj{0_1z}%M zFg>XM4xZjSNY0^#c?{Eld`Brkz~p$Wzq=1qr}@DO2Pb;E&>u%6D?p*SEc#UV2%oL} z4_*fE=l(W3((XjSgNx!(#{E0aXM<7X+ZuUGhe1JSm5`EC2g)^8+}fJ&d3g^__7pd-5f%< zx*oxyHxW2x!d`Z2#8_e9m`}Vx>>3KcRK@P>y9?np=h1fN0chU4l0Jl=#w$*CqH|VE zznj$A-A!9*VZ$_9dwYmr$rLLbe6s)!YTV(DfEu(uUqq^5dhk0;9godjEle$J1(%iM zY2w!eHuCp9=0B9+zZx03_IoXhj&8(PA2R6C_8_`ZHjy4&JBcSQ)=}$;(d6HnL34AJ zF+e4p{2VK2_X@z~h3)VnE*=wyKO(g<=qeA}31sRf)>E*+Wvhd{ba@c=qI_|kYliU^`Vo5XSvrYMqnC`rv zeJ{y|l#}|x@8=Q(MwwOC6FM~n{H=fN%_nJ6E7=aCg~nugK9H?wyg~A>+(p{MY20&3 znkMZq#LKz%baUifT($W)WV=Sw;!l+@to8wI#u6wk=mIUW7xC~Yv1m$FJ5;Bz@*eUPv8V!of^QUSOeF!EU=Z!it(HSSc?ixHt4H_i^G*2v2PWtI3C1m!>oEYS|bp z_khhY98bdzE#M#ZtAqUWl_@lMJlb;f7|L%D)$*Vmp5Fa|pFDLlXRT3)3BSa^sX7Kzs)q^B*lOdRCS`iL_XjFI z55?KHGnm~c7u21Wh?_dC*p8*u(DLyFlZ;BFr)R9eHz^GFO^CxSy({q%*JyjO*?~Ow ztJ@l^oCo?!&1|{u5S-g5$;S1!fT^viQ1SX_aH)R+w``YMpDFRg$b$t>`iG+j5zU^dH-lgHifbzUPXAtcX^!rhbbKB>|)Co?Jw z@Wn$HN?}l~9440bfb4T!!Ml5jaI{thZ^`02g(+?}#e&(XD{yuOjW{+U48OJgBaE)Rd#3(&XV7RRI%ae?16miAZ+a&PK$%RtD> z2~R`V4_B<%V~ji@axc|S z^ne%N=8~TEW_)ofo$b%5;Q!dU(~5lysK{*)q1}-b9#!+Te$!yvq-S`?^EQTNH?f87 zZ^3t_9eRj#l0yL-vAj2td%pY}L>8N2qUi_Fyr{;ov>v98RD=BF1?-phQ*J|ND}O!O zMA$pYRcNbffU{J4!AW;3sK2zQ=VGN$|E~iI>+Z0O_fa%Ed@iqO6-@=F;~{*q936kZ z0!P_g=dW3fBCl=HSm!+fk1m|Ty4n}9*dMV}Z}^fD9K=vNP(n~sZ$?_d26R4qIQ2#C zLH{F4;Nm?Bl-e_Kj)E`qX*Hr6*@JXsttEbIsz5)hadh#IsJDOpg>&yVflZ7DUD}mX zcTQZ5@*8`r%pR_y#OB4+J~ohIm()fw(n8}lS6bGihlW3+NZu$y zFnYHG9BgZ34=z^G!lWE_sdhXWFR5c(mK+=XH;hh5dy-Gjc)C1Wiu%o-!@Vn2{8f)+ z-r`bvo%EjR_+{NCvdz2BVwbuzxymSrs?uN!cFW<4+2WYykPByJ((%x+4S3>;KFOXn zqy_H{D5-M;d*~a>eh;|@rxOh6#?>4=D$;8O|1F1?uclzi`KiRG&g51Xt`mg6>BQxG zN=Uj`24{_#iy1-HOit9#_S7C{uN;$v`##mM1gl!OVJ*(x$x$W+^_}dx>sS7A+Y}nG zHNYSJo1y69Q|4e*N{+s7!1M8Bgp;ZGrfwqJs>QLq>7!;Q6{ZM3R?I}5-DlWrGQ|m(Z_=!vW0CLJAOKr6 zymD|24pg59RdsLr=&sJ{{aRqv-}792+9INhJMa2wf^Rr3Jp~ z^!3>YE~Gh^Cc6KHxq-K_P}&9yZzr+dl}$|gS3GSr-AWVIPQk$;F4TS^4Nq^)fMn@5 z?zHh2kd0Ci>=geBdF++usj<|5vdB!$96JZfm zvKdze-Ssk5Jv9oiyl|rvye>av^getrTtoO(UW3j)(uDW2Z)=OvWl`ad54&zOA7kA7 zSXakt{I`9Qpl#m>_ISGm^YVC7_c=_0BARC6njjzY-B8I{*O^maF9?@fjz(v)1ml>6 z!kG^jgW)wT;ne2|7}78vr>*6{Z+jLsNWWzVD!p;?l0#(M5e(n+c7kUMrr<`U(6Ktt@{$7?N+r&J= z&4rgLWMJ^HBCg50DB_r9(4?o&R5Sj;Gv_4O`M`v=hMeR4BW|%0xruD7#%BEE8BB^v z=3uV4h|>0Ph^}_5s)C?(bgoF5&xOm5a-?*P$0Jvku}z&4SZPv%-&#spk_ZbeKb44b zyQ6VU)ibs!)sg+x^(8UcYUS@y7obvg*NtI zNMVVi``~=pcRnKcAY1&*kgV-~L-lzl{P(t;3pk;JH{<+isrr6?cEJ>(U;1t|GS#7j z!ymJaR~yk%|1x%^igWl}mx>C!Y)k!);@!O(QBfpyRuh*SJL1r)F|z+3iM&0}FFF$IMZb?5iZ4BkCg;s+Loz(2b$q9{T*t1K58uD=8+G+Oe*WVx z%o;1wI3~|$lD?^z6!uVAsTr~xV?et{m%Wovq=xhQSTrpFC2uCflAWV5 zZaU+;i|VP2djYCBP3(lqVz|9F&GygyQW(2_sPMaREwvp;M6vlzaM}GV?ff>MLKA@* z9V=sjT6!2}T+dx{i^s;#6NIj3iF=vA(RXDjI!t<`GS>=Hjz{AprRDffy@P$&bjtSJ zn)Cdw4Hww@ka~R9cLMM2(FRH7ZDcMJ1;K;;SdpE|%H;&q^);C_h_%6_-LdpvwTq#xx;kq2L4k zX72NOs+bg7+blVs=BQL-w)JFT%uGelJ9!4XuKCiGfrEV0_i}Wbw;K2U+C%oLZ(xnY zNt9j{Alx9=1=rUHQstAa)Uoz8lbKUVyDmA9!sj8F`q_of{FXrb-Vgl!>2-BS90LW_ z68wi?3NhjW@d+O;!=X; zgEdUxQpuv>99jO-z?QooAgH*P8LpoOKQ+?e$+B#waag1Ot-W?;if!8QBNDs5>GzGRo>cUlTps3PV{kw4Fj|^zXJ^X3qX6e9_$?Th_myb zD3l0TMLjCHq-8n=FLXImpJpI8yXy;Eop2et_Kv5Q;U>(+CCoYFL` zdHid!CpAD)aW!O=l&6Onx7BeO2Two_az^Vhwi6KIA_e`D0z&K^mv&f=gzmGJ3Iu zNtPZ#kNU~DRDB%k2(D5}qXXWJz0MMce1^TTUaWo2eom{e3;aLXVDLqxi`}2N%Hjw{ zvmfxsUY?>J<8T!FJ(uNu@220OdbG222&-Srv9=+p%tu-R-+7#6`Li0qbn#kzKe(96 zzmA0otGrn5tB(*-tcpS7MhW%}8;-^mzVJ7`4!18p0H+tIaG_HTxn%#t^g8}7-`_cc zi3lumbM7<8nQKt@@(r4IP!@;TYQceJE7**nCVVAxh=yNt!00U>VTG5r@O0Brs2=QL zU)#ix-)~3XbF1-_&TgDOY5pK9hpDSMV9Qh$9CvUMNDj-Or?KwLf5>q% z$=ZfCuLAMOzbQC&my1V3gyH{3Ic9_;nO7s=v;^@^ zKYK}iel^K_Jq_!1Mo>$@SiBq3$!dSRJ$sz&&zbh-VE*v}*x8>?p28Tm`_OWdyy#DAVcukaAQz4s^VD812i0xwp(r$s zDeOAJwqFW_qeEx00IME8Y@HK%%v*!<20PiMQa46st!(s;FjP5^#U@@{K!-i{kxA-V zma!~>KPRDrnGzpyvd?F(_nH8i?=x`!+(=Eo+iN~9Sk2d$%|fy6GRA*P#?oWac=h{H z*7P6)g|)n|W!@`2Njl4l zOf*WNyCa+5^}HX}ud{<+Z#Sd0>_+sxRSl=AZ^8bcqu{OKfCAgSw6a~34t%VDAeG15 zKxP~X+>WD>h_4lHQ=!W;%D5xQ8HbF><^M#NvYw`G;2rgW&1(?H;CBjiruiX�a`L z>InOpu##r}u4KEX++me9;;_WDn;&guOY{F8!`n-3$#dUth|rW2Dw~;8sK$J7h&f85 z{9I`2Lj{o*y@(>m3h|oNXNa3|g1z~@oCZ~5Y2r9BdcNR*F#3-#4$-*DPUM>6x9$a4 z>6*n}iLYSymQF#5=6;qD*9n==W8kV&15Ih_=lc_*InDbCY~!+}f=%^$?4_4pP5;Xp zHpbxwZ~qR#zupkk7KWg8xg*ZotVeejx#7XMJnH^;lh(;zVH$XVe)1E9@^BU>Hm{?y zJ67~~=0Z-p^AFt5iNVV%*J1zW8FXOZPSl^C1)dKCHe8W_8}3rxODglu_ddeZfh6^)XE@~-JW3wWuAR+N6LlgWGRVOei`=+fg4aPnOdIK6wt-kNl? z=;IHuqr#7*&RvJPb z@1Wm<+vwqYa^4p+fl2ag>{?H##4tnDbg$*PKa_? z!$c;R6N* zk;T=rjTG~3H15$HWPgTOGW98~wo)S#Xuz@)Yk$bm8KDyHn08pR;O05#W#y#%N21ff9}YF&WmuMVkm=26WM{%WLDlb2A3ZF z4sGsgXvQn!WkC|fa}7+4b_m}ctKejm)$vQmG_)F>iIWc-(8IO=;p+D`bg@8Jc)dUz zhAHddx3P2K+ozYL?>Smfx!Ih;R6?2Zn>-kHBj46|J7D{lGx+Py2wJvw9FsDv!|9$n z=%*}!hIf|m7w<&llHN)TzbQr&ZMS1+QU$FZI+NsqXS;`7BmLvDr1Q3px#kaYzgJ1& zw#{`+7SfpsDY1N$LDZMBqJqdgZg$T|l5Gm4^f$BE#S5Yyxz`wt=6+@I!)@8+CpgWknuuyI=@O)xW_<;PT6;{$=AgMj(0RD!1WGUzgbFQGybv6z#=xy zMuP+qnIer|2NS;JV!crZ*KHgN+ny|k6GMjxib89#);$1U&6y+8*O#DD;$!rj{R3~q zfAF6u?Q?#Nx2^C}J6kVvm@JHKsL;xe8lPFQzwcMzGYcImP4y?U#kJt$IuX0~)I<22 z6Bz4r6r%qM*ttPYczNYG!H&g9R=ZxoII0e#0ZkZv9jEF@8B1hg}7p(P)fn@GRBIXHzzZnvbzX+POcGYztEK8(J{_6l@kbJ@Z1Ja?{0!D~3)!`)AJ~@XCppjeKKOZu8r=@lrA&)2 z%qT&VMfatVY>E~3*tfv|vx4@o2JHPf8$sDNS9mMRr#Mf~LFaB$N-^kx)YmEC`C5s& z>_`OP5Yen#`<6Yn{m2}$Kk;FYf3W<&cbHPII_9g678u5JtY_U^+BSL{dVdio_URX! z7`6h=1nh^$V=mILuXEV<{mFF8%AW!?BuRhiJ>Wa#d4+4^vH6}jQ#@dZ{P}1RzsZ2= zUuy;5O*WGD+6eq67XeOFesHEoGgyYI8O0gBrt<0+oTIfny2>ikh@bLMH2w<L#oK>N;Zut_rdVU61qF2HCAQm^i@?c|*7J-LJGA(uz&BfjZ z+v_i4NM8RTJX{h$3g6CRLasXnoZ5*uhm1xahhLmsInOOS<|KSMQh}T6dlsL{n&7ne zubAV5U3k>M5x*!tV_hDDth+p(xRMI=?fSyWJ@Vx4{(WO>kK17BvSBz*E&`X1`$+-c zY{`401Sj}=i7Di0(fB>(+jsBA*sO@|UVN%s8d03N%v0d@4xlg_>ks5oIbeZRDw zxdnX#wfz}5uX{4R{dANm*Da%$bCqCi$voUsG6=U421wJknWj~mQMAHhaV zf$b|5Og}vqjC!0%hlSG66n&Ce^^nzEuHfT+s$stV3^cX2qKjAO3nEWRv#^_?yn2u! zUVSl~guAXl`Qeew{dN(a)k|Xc1&+8%-wQ|Xnhbw}b_j2+J_e^p+L3Fw1-s@Yi_4XGO7Scft8*eR`VrnW4B{1Ilj-`n4_v~-W9+4!E*o<159Q{c1@8_; zw5Uj@#3%b{k^T@o_E(!+-`;`owqu}m!4@d?QpI!^QJ&+=ZRVbxOl+J!y4;tiz>3G5 zb(kBOT+$?y{yc76%mLg`QwWDwmT;Y-+Nu9D7VX1b(LDYUOq#4AT%UFdW|}GpM!7^` z)(vl**(yePL7MoFXQ*x+NUzHf4nI7HUmKFKzI~-&qg5$*-5f#d^5?=_EhV<-Ne`5N z{tp(+t)_yFY9JXnj5>ZUVRyXM@Mvp0E0}!%yB-H}yS&9{&%0LEoCEka!ied$2Gd=` zGk8bh9)8^zLodgsG3h8b`Z4bqeRe#FA&08q>7NyNzVRG*_{TBBgIes$v1|<4UW6&Z z?&xCwh~h5)gvevDAme31?T0?Hxv_3AEXt1DigVZ|<3Duqw>~)hm&x8|Y{lREQpw=? zK6>I}f?rgY2;?{TG0UI(>9vyriqG7@(tD2JqTM^0-U>gao@Pql!spY}9$7L!ASP&b zY-1Y~U9fZEDG1ygMO8QUlB^qtUu_?=>p550$1&z)tyjm67rCM^ECz#8-!i{qDUz3O z;Vx~F6lm`VW!IEr!S3=oL-|F;&})p9>m!>&95@dyq%g_usO2!eW+m zOp($*eul=MY54AM0zLh=n~C>57VY(Gwr_eSU$XKgjqDAw)o_cVgU7_t$a4uR+Lh0K z-MwG;e*Ptha6z*W*jz z(H|oc7A?lCTVG&ESRq)i*n@K>=#p*KH}LZK3K5chbUytb3;ufy9(S~HaU#F(Socuy z(_Tt{_dCLd*K3*e@8kH7>w!{Pbt)Jk&h`ecxAh%*5)va_@N|+c8BCO-l?yMkr-D^< zV^T66nQMi5@$pb}d^f3jh1>ii30$oGibg&XdAr;-vXIOp$f#uhjGf_+qB&YjEN6+Q zhSEU3gur)L9P^Tq$0ydGA>@Do+{pBWjv?pCDANKhwp?P9b47PkZp)B3XM&n1Iw)+M zCZ<2w0P+!vcs2Ghi~2>anT67UqX8CEq4xw(*n>lOFKKk2^%()#C?nHa><9X-}p@ zFW=!IrLnehL6LMYUXd!&hNAhyK3*|E9!)lGV~aNX2qeC9wC3U{tTm0tY5yX*khAS5 zuDXo1?-;{BGg(ZgQl)4YO`I1^5YIuef?K9a*hKD-=w3#c_Y%!{Q~WMDPTEHK&Qgy;M?+G{5;Np?~V#& z)o;GARqhAbnfjGLGfLsX8C4whBa$_nW|33u3Rdzk7rh@z(4puKNXRgwxwa5tv4OpoX3fc*g$>0kNC*G7<7C42vWg`x)c`~%Pg*Fk^k0Vl%~4`*Q*Vu41aNEp|F)r3jW8|X^tbip+V59 zoyhLGcR+Q@MB6Ld+SvWaoqWcNSa^Cwg^ooPRt{EghB&!OZ$B^dFp6K49w5~p&CRlF3(AuqpzZdf$@Za9mF?(StF_vYdalajj3 z-XeM_n?w%Y;({+Km*`TNt>Et|4|;UAoy2!ZP*PPKHJiJkj`=l~zGefv_(UC_jeLPV zE~2wuT|iqB^D&^klYg)K5_UaW%|$-4LbuAN>_+V-^7Puxn;lUTs_kFG*Gnp4ZK4F$ z{nNulK^}hXtcR%B8K@h+oTff@qH7{g)vrbsRZ}IY^Xoq1Gwirf301)ryJ#wD)1b*4 zL_XQoF!I!W!IuRX;KRsSxV$HW4mu0?KS^b*U+)&ai_K$KTqG!^HID@icZaqJHQdto zlkH8>LDraqLQ(GI)3O`Tc(IRH8~cDOcf4scf1NxIng4>_<>IMyUN)@WD#Ko#cgA(& zmw>HFI2I4zOIf1)=(NsBLQ%MmwrrmtLc(2@8P#lSNhS z(<%Q}FI&7fm=ykM(cAKTrl@1b8s@~aiTnV}XdD2KkDajX^lzB^WxVjCd25~Ivr<+Q zryxwpnGOnP0F|0Qh_ZMbhC&oZ&FyC&imp;(>%z!pFKIZ#}?k0lGGjtpkZ*8x&uvDX>@-})LG{{OlhT+?sZ~V4IT`JvXOXl59s*)534@sMX#LAnJ{cPrjCilJPRYz zO1X`zQ@dGCcQW16m?~_W-VVV%C!s3-9O~*k=9QbASkU_Mv~bRNaymW0ZQd8jrkOj@ zK-fVz6*`<|{?kT@zcP45u>=lVPFXTE+wPmoj-$DF5s13H(9E*7z@H193gD~M4{_&9cz`v;@G)L@O=J1!|G z9xaE}k;B+6tU@k}R%UG_Z3kIF#O$GLYl8+WJuks7=u~j8W+|hO_FgO~Rip8dm)VK$ zLujSUIaYIUAKSG_T3~Vi7In0(6U~+N*ivG^v=a^Z$HOnzxs3S_-2L#Ziuuzr~|ZFcBqBhILka;gR?V;eW)^;o8NMT({;A3?v` zDy~vil9QfM4I2K+G~G;(1r_u_`rSn?R9BY{{W*PFOrxnkLrik;LDR8 zOnzY<@>MPT;&mcj;@~>={>)WYVPD83q93WY&9c^F_9{+vPj*fadl+W|Hj*nSbox-ZwWSE^sv^Oj7stT0 zt8whL5g>S{CCed?9WTmC28QuU33HJkDN1}{QiR3owl3t5`*A!!OSc*l=Px3+^v6j~iX3VEX%oCPI)){f$g(@~C0HG|j8)b(Fpt?U zL3>&jzDwH3;x_AuaEcv1J^7jc_qdGBADha4HO;`&DHXUl6o_-sxi8+EV;?ymAjeNpbMf%NdeB`bWi?*wc#a=>51^hkL63*Mdg zjNAKb7szXd!Ma78V9kFo$aK3oTnNfV!3sCr@-~Ky>Wz3j7R`n%wx{h1qlB`f73hV= z9^B?^Mm;WOIA_x!JE0#3bpl&j_Q@Xkp$BneTNVy28{pS(4M9Vt2s$4v4tE37xU6M+ z*?%8G=uJ`=8GhUf*OeE6iphWM`|S*3(+HYi_3Z7eF z0r$#p!ER+M-hW0S`iRLc`suAyt`w@xLlK^M9_^uMZ0mN`_O>q%ZqRm`~&xo_iL^7 zzOHv!F{6wye0t~-eQDXi9+>YT^Vg#=*(s187TqL-&k3PMb!)1)5rt0%W`T#320ksA z3GTbZJ7%XeKh8KD=GKm9xpVVK<+Uw*^W6xOwl?6PhdDcLt_}_|%PRsE&cGil6v$XR zmGyU&<9`Omlz4sxecqBr*@0h}lY%K9XZKW?IZut*rkc~s(aWH;aW}laGYqm9PNkq? zRrsk?i+;hqoPh$M|2PXW?;9q0AD>3!`P(eKN|jn-OqfkyFHPU=EoPf5g}SGc!7=w0 zlMeMF=VOKB@#`97&A-P~FKA#B8VR2t`r~ISC3v?ri#KU`1kdcRg3cH(7HX=4Uw!3i z)uBH8*_Z_mYb)@`J2%!fB8s`$7t!|cDt79+CUtp^W;5^aGyA$&5k9PGqRCDg0yjAt z2S3D;%%9oJ>zEav;@!Yn1`n{pt^xK-;T^VR9%3B}cY?pAEtT=2k9OXieNjpjnmapj ziSBsn3r&ZY3d-VKx*xpKPt(UlMSe}LJAKP^;LRtfqi07EUJ>2kongn|mrfeZuvLPl zQ6l?W*ErKO0y*wczNEK7#vIMh-=HkWv67b_E00Y#P%J7_M;H^isQnXdH5PUid?MP*-gk~t1{lxWsOMKe$k(P zksH>I&<9InUHWU+B${(=I&_rm6X)DO(I*e4wc~U}c2)&0J(`Lxq0@zpHPU4B>?aG0 z)8KyAmt(rg4!YRp3PnFgQv2!ALP1t0?m4<24n8tr2``K(sqGrt*+&zso(ZQ;3-i6)S#dKS&UWj zp_cEfnXQ;(s=6m)W3~b4pNs^nxDak@&P0HJ2WjNrgIt|*F!pP&2kW2-6$;1oS=lQa z3@K<~&Vl1-Pq6}(9>{^&Qv#WJ&KuT$Lq(DyGlhhQ3AZ>cr|vli zSi$jE>}psVmg^3o0~^ni&Xi&rER3eNnWIsvGeK|<5wjBA1ISCIVDJ|oTr=5B=uI<1 zr;PO&nqkGhL?{4%#g|ud^8dGp=OY|eVRq122O z_9<|x*q9C2I>WBgGt5{!j0K&^W##qHh5zjmX8^}`ro*O@TyY9JA07&dr#LW*oW}oi zE0UkQPG7Qgyc`957!7aVWpSNXPC~t<3O#w{1*abskktGxtUezo%-Cp1xkn9X!H#)U z{--iO$6`2Lxw?{7*~Vj>R~4PAibIVPP9{B#m#D!j8C>NNhd10|Me_UTdsPW?^&@D( zMv+CHSq`-iS3*OiRE6EcXz1#WWkOmHcsyPk-zHz(1N0!`98;SQtZ)g%>=2@WZGILw4+N9QKtiq04`$iGvOTBA-! zdlo@eZW^21>`rT)t4OZ!JI>!X9D?h`zLU}yc%rI>0-NSgQsh*wZfOszj`@S0i@A4B z)O^X2^iceCMwwsC+OT}Nk|3GT${jXIBzK3gEZu(=`Aq-9e%$LYbKiCt_*hT)SjV7r zb3eQs{f&8ColZyLDAm&!SiI zSZHfBqAs`hT=bYc7GSaq&JKx(@GI%m;}C$3ALFR6mx-W_7Ts#y1C={Zu?4EfNFJp5 z$KBomakBZ@kg+G+u&!Z)-Quw@8n{F20;xzgs|Ia_;ltqWQGmUfb zxL+;V+HNQ2vkGJGH8a2BE!>O0Kf#VK&(VE;4V=%$=FHo6V ziuytFkSO%BR!Tv~?V->zKZCSvJB4^{Gurj(BUU6Cvs9n8AQNImru&>ZjkDI&8u6Je zjE#nrL`#|%8cSo_#ea3jIPg9yFU)B5;pTsJC-Xu(7I>i&i;d;U+&V$-7R4>!C@E^P=8WaF@&5C8_~GmVGItg-=_250uLcfUErX{++^}TSJkX0*e|eP(bKj4Exnt9X#;M&HHBS!u{nFSO zPKu5{ONaij87xe;28$G2s9XIUqp8E_%K9oY*mMP+zYD?Ywp`)2yFG*t%2L0`K6~x9 z0E_+JkhiIf(&N8YZqAeR=lMya0FM)QT&Dl z*`h~R1i`+#)aW0LMp^FgtEZo}8F)}$y)pc zL_s-Z4}P8aS;$|K%qHpo<{Tn+3rAa*K)Z`8+4t|D*N;xnm57tgdMP9wE{XN*}%HGNpP|4JRFeGWdojz z=^sM}rgzT-Me2!;l*GYat#?#9S4pz2G66mRX~0Yw4&r}kV28{&XghTiofbJ$n;pp$V3cd6+m77n~%fkUbw_<96NnIzLH zkps!zyc8!UkKv7mUuSoq8Gk%jL;JJMc!Mv!G+WynGXGgd69=W?+KXwt>(4uaSKSc) zXRRgOvKdWEL)9d8haPgi&B{2K6oKlMmE5NqD=5C1M(^+VV(*AP7_o0DY!EwATszWO z-}TEl`biFLQF{Tg%N${=!#3u-cs*J+myzdX8IU#;GntFAK>ej`qkl7-J7ETmZGIun z*2iG)!X9qkW@i>DGHi?e%2=U`GrP3Sl8f9_1Wg{(sWz`um^2QtFi{D%^nPa{JAHB4 zdle!1`~ha2Gt~6K-}Nxm*a$X>{Yj1AltJgJ0jM1`!zJ_1VL<;LG}#bFIU#3h)~Lyn z_<<5QAJBlOV`JD*l}db^?aFq^&cQxD23Gae!4LFyO(`s~Re+12N zw}HDhfD0`(&^@o7j^$it_mqxthvTGBQS%EfJQDogaI*OE_CB z?)~{w&AOi3P*q^6p!xd)J9Riikg2u9>CUg%ZI@gcYf;atmw7_x<9c{`=WvCbz9E$? z(O~BqKasjo5)H%|fY#@DiW@PT?APj%+2_3wC3+fxLoK)*!wj_RDre6&ip;;bP5m>ZqEjC4Dp3)`|*vCTJ*vjih8$|+e( zjd+p;d2fRTwRo5``!sj{wkbbzxj8O9CeUEIJSub!!IbO_yjVR29$LFz3*BS@_m}NL z@AVdD``&uPnLqW|%4x2kVYm<+zwf7cw=c5Tf-*LOU%^^>rC`v-hiv2$*t9*z@kopr zF0;*Ihkv?*R?B!=(EU~zAv*={R`PUSc@NXloK6$AF2_a5As8H91afgFSfIrT$W;?C zZQx(_{&@iJl&2;-sYa0GG>5)SB(s=v`&qMPGiyJ!0_3xo(IVL;G&1o7%1TEv`+JXR zjgv0DT(3v%3-lo-%$-Tq-DhiZvW2;`|E1~H7dg|v;)O{rhTtO`0I}By=amNtKYjIJ zW@M0fZ+U~O6}Ci1HH8pQZ*H{zL85dQ+69xTWz$Yvo%s{3(;Z1; zAPH1t&cggY8@OmFU|rc*)Hcz8=U?aXT`v^qU>CA(8PSi=@1xBVA26ler}54+31-!6 zz=p$n*w+FX*z!FJ*s!z$Gq{bN9Gfaj_a~RJ@8b&C TeKT`d=4u9+KPu2T)rkK;F!(E= literal 0 HcmV?d00001