From bd12c3607edab55c13f49a0ff13c4f8a0b457d60 Mon Sep 17 00:00:00 2001 From: adrianlizarraga Date: Mon, 5 Feb 2024 16:42:52 -0800 Subject: [PATCH 01/14] Start adding more documentation for quantizing/running models on QNN EP --- .../QNN-ExecutionProvider.md | 111 ++++++++++++++++++ images/qnn_ep_quant_workflow.png | Bin 0 -> 115115 bytes 2 files changed, 111 insertions(+) create mode 100644 images/qnn_ep_quant_workflow.png diff --git a/docs/execution-providers/QNN-ExecutionProvider.md b/docs/execution-providers/QNN-ExecutionProvider.md index 8adb87d9cbc51..9e2b7c3bfb498 100644 --- a/docs/execution-providers/QNN-ExecutionProvider.md +++ b/docs/execution-providers/QNN-ExecutionProvider.md @@ -78,6 +78,117 @@ The QNN Execution Provider supports a number of configuration options. The `prov |'2'|longer preparation time, more optimal graph.| |'3'|longest preparation time, most likely even more optimal graph.| +## Running a model with QNN EP's HTP backend (Python) +The QNN HTP backend, which offloads compute to the NPU, only supports quantized models. Models with 32-bit floating-point activations and weights must first be quantized to use a lower integer precision (e.g., 8-bit or 16-bit integers). +This section provides instructions for quantizing a model and then running the quantized model on QNN EP's HTP backend using Python APIs. Please refer to the [quantization page](../performance/model-optimizations/quantization.md) for a broader overview of quantization concepts. + +

Offline workflow for quantizing an ONNX model for use on QNN EP

+ +### Quantizing a model +The ONNX Runtime python package provides utilities for quantizing ONNX models via the `onnxruntime.quantization` import. Note that the quantization utilities are currently only supported on x86_64. +Therefore, it is recommend to either use an x64 machine to quantize models or, alternatively, use a separate x64 python installation on Windows ARM64 machines. + +Install the ONNX Runtime x64 python package. We currently recommend installing the nightly version of ONNX Runtime to get the latest updates to the quantization utilities. +```shell +## Install nightly ORT built from main branch +python -m pip install -i https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/ORT-Nightly/pypi/simple/ ort-nightly +``` + +Model quantization for QNN EP requires the use of calibration input data to compute quantization parameters for all activations and weights in the model. Using a calibration dataset that is representative of typical model inputs is crucial in generating an accurate quantized model. +The following snippet defines a sample `CalibrationDataReader` class that provides calibration data to the quantization utilies. Note, however, that the following example uses random inputs as the calibration dataset only for simplicity. Using random input data results in inaccurate models in practice. + +```Python3 +# data_reader.py + +import numpy as np +import onnxruntime +import os +from onnxruntime.quantization import CalibrationDataReader + + +NP_TYPE = { + "tensor(float)": np.float32, + "tensor(uint8)": np.uint8, + "tensor(int8)": np.int8, + "tensor(uint16)": np.uint16, + "tensor(int16)": np.int16, +} + +def get_np_type(onnx_type): + if onnx_type in NP_TYPE: + return NP_TYPE[onnx_type] + else: + raise Exception("Unhandled onnx_type in np_type_from_onnx_type") + +class DataReader(CalibrationDataReader): + def __init__(self, model_path: str): + self.enum_data = None + + # Use inference session to get input shape. + session = onnxruntime.InferenceSession(model_path, providers=['CPUExecutionProvider']) + + inputs = session.get_inputs() + + self.data_list = [] + + # Generate 10 random inputs + # TODO: Load valid calibration input data + for _ in range(10): + input_data = {inp.name : np.random.random(inp.shape).astype(get_np_type(inp.type)) for inp in inputs} + self.data_list.append(input_data) + + self.datasize = len(self.data_list) + + def get_next(self): + if self.enum_data is None: + self.enum_data = iter( + self.data_list + ) + return next(self.enum_data, None) + + def rewind(self): + self.enum_data = None + +``` + +The following snippet pre-processes the original model and then quantizes the pre-processed model using the above `CalibrationDataReader` class. + +```Python3 +# quantize_model.py + +import data_reader +import numpy as np +import onnx +import onnxruntime +from onnxruntime.quantization import QuantFormat, QuantType, quantize +from onnxruntime.quantization.execution_providers.qnn import get_qnn_qdq_config, qnn_preprocess_model + +if __name__ == "__main__": + input_model_path = "model.onnx" # TODO: Replace with your actual model + output_model_path = "model.qdq.onnx" # Name of final quantized model + my_data_reader = data_reader.DataReader(input_model_path) + + # Pre-process the original float32 model. + preproc_model_path = "model.preproc.onnx" + model_changed = qnn_preprocess_model(input_model_path, prepoc_model_path) + model_to_quantize = preproc_model_path if model_changed else input_model_path + + # Generate a suitable quantization configuration for this model. + # Note that we're choosing to use uint16 activations and uint8 weights. + qnn_config = get_qnn_qdq_config(model_to_quantize, + my_data_reader, + activation_type=QuantType.QUInt16, # uint16 activations + weight_type=QuantType.QUInt8) # uint8 weights + + # Quantize the model. + quantize(model_to_quantize, output_model_path, qnn_config) +``` + +Running `python quantize_model.py` will generate a quantized model (`model.qdq.onnx`) that can be run on Windows ARM64 devices via ONNX Runtime's QNN EP. +Refer to [quantization/execution_providers/qnn/preprocess.py](https://github.com/microsoft/onnxruntime/blob/23996bbbbe0406a5c8edbf6b7dbd71e5780d3f4b/onnxruntime/python/tools/quantization/execution_providers/qnn/preprocess.py#L16) and +[quantization/execution_providers/qnn/quant_config.py](https://github.com/microsoft/onnxruntime/blob/23996bbbbe0406a5c8edbf6b7dbd71e5780d3f4b/onnxruntime/python/tools/quantization/execution_providers/qnn/quant_config.py#L20-L27) +for more information on available function parameters. + ## QNN context binary cache feature There's a QNN context which contains QNN graphs after converting, compiling, filnalizing the model. QNN can serialize the context into binary file, so that user can use it for futher inference direclty (without the QDQ model) to improve the model loading cost. diff --git a/images/qnn_ep_quant_workflow.png b/images/qnn_ep_quant_workflow.png new file mode 100644 index 0000000000000000000000000000000000000000..fbffe29310c71f7869cfc08b17b9201f98861dd9 GIT binary patch literal 115115 zcmeFZ2UL^U);`V@$C*LI3P`mpN)Z*15^%1AiZn4I2n3W+!~_sS4*|y+K?y|>1Zf#j zf`9}QBuEP?C5S*oN(cd>AcPi*Kq#U7PZDHgE;IML_pWcP-&+5VHKXRudC%G9*?T{y zaqG0{39%oge-sfB5i>mb^BED5wFV+0-yHe=JMg3`=i6-X*H@@BC;lNqsFUpjAHH?@ z$>b*y5n`0+?8Q~!^AA@}o=1s@9DKq5|Es-@)=UwRg*3yTe>&@DH`p%rR(TrscG|*$ zc($*w>Ws~1U+=tUPrXE8JHlRN-kJP)`wyqFUnf5P`KRwcnAkZulaRN6-}^q=#L@Ss zu4at0d&f!V8e`|(ccqdXd+egO#{RH{WKNH}kd4SL63yFn+K(K3Y_e^^D=6K*u1|07 zsy;5$V^_hX|BM%|f!237cwvDyCk|x%zn}lJ!2c^2c)wk*sGphDVds4^Oh2kr{{$Ug zjx!mRS|;BM(~5soz(u!3ZMN4xuH;oiZm`(RRp+%5dEYE+hpZtQgdCH;Jb3*Le$YOA z(Astj_3LFV2H$FlZhyE9yUq-@Zi~~xOjjdzdoE|%a?e*^Hft%dijl+)rV=Wyjry95 z<{8b2hB7D-86KrY5(m~Q+9b8))e>oNI+;Y}=Y^%8YbFSZe>jT{R5mL`7yq`6Cu3<9VRUIs%SA%)z6k;vo8=cf+zK8kefBS7k`7zZWu_b%(9zPOh!`lbKq6r&QcRDcwY_IWOg3_w@@T z`i{Y!WqmwVBR*HL@v_v&xB5LE;@+SoDwWGIkT6(=o)?wlb4C$y)`4*lA4Uer?Vr96 zDSuUndQ};?c4Z zlzvCl!@T+`>=}B8E3Q9-JLu;QYvq4ZfOkAk`<51D*As z#qiLDVVIKN{E~>gX0Hs)w_j@Zd$jWSWo;*7TfvM(x?Dq^-$@Lq$d^K07f=!f$)U|OVt#Ll0=z$!VOPOp+u{bJ&Ry8iy^l^Zp+`+ zIgrcw8N;jhL6*))$jLL=EgiR4VcER_Di8f;w{=RHLQKC@R>U+$x#zNU(Mk7H7%aWA zn!H{N_ANaf5ln;0YIc*m_`SjHhbi>O?Qc1g${MmoxCjC~%9PSx!#?J=!Z9K+ro1|wRup=W;>^5>511=Tp`&oCm%X+R zgKfRXAEs5J!W$BYQduio#359m;kX8^sDe9gI?`i>-W+xdd??yYKjD+wvX|X$VAzUO z;h+!BFAd7p?Kw{7T&fw6uupW`k#L4QcPSIK=F2zVlbiD%D_dKW%}n5uBTmelN@2xK zEo`qkxVs}Wva;BnF?j;2dak&!)9B0Bt})NhQ^}WH8G7mwFt0eb9i+XV&LM0!`6C}Z z_)|K`--6w)9HzwVO0A-FMU>+9VVCDw=f=+h5WjZ|N6LU4ksn6g0tXV*6zu&Jz0CDj z@RU_>qkd*S5j#Y3`&q)E<8l;sdl7f^&&>#{uGXIwWt!IKd%%TS#TwihZe-E8*tcmr1vUJQ%CcU*_A3Al7L8mO28vA1Os^ehMtymqz>O?AhfOklO+}H zB@Otc2XK~|?fLJ>msSA3SD{thhkcE6ugRMQqSH7)^ry>f$hCAW@`zI5_$wm809l`b z<$&Z4mBSg_F&l1!yUXfmQu(0JT-N_nH5iteO2l^8)#n{Oq-{aY=0{cpCK&**vJJaE zz13{V?EHVI?6o=2e7Nb=OFft^VIfoCUa;KPUbGQ(e36{)O;C@gsDQd zx)%*h$nk7xKo2B+W4GB)2?lmvrvJ=meFE8btAD zbW1-keuz~5RCpGud=BXtwccdn2ufvL(p6?YFSUx$MGK?e>Ox8e2O(M2orq;Fe}2vC zZLIQmLQ#f9V=Re!%M@lyZ=SGCM(YBt5a#b&Rt2KuQC%&s%20Os1onmg%0DTTv`yl+ z*tFy*ey_YsW`&mo3<3Dgeop`(S@&wBJOn+cc99tI=j&|d~mg!VzwkxMi71#CaXI1aB?7g_(d>bL?u-Gn<$qCJ(CNtzO*0Zw*@EG0 zV1{Ab^($K(^wV{HmQ2F`$L>a@-Pno?CJYDSnYyg1qdIoj$y{;7%$2(?T(Rq%tQA#fuG}?c#ja z$Ji9SjJ=CIea?c#Du*D$N|!TdtG3)MQpIhP(aGh8#qz>3k!p_6S;(^IFFGjz5NlTG zo({aWEwOWWxi=*yk6MtW0F33Ab#uW}A-h`Qm_`L_If9h>>F4Hn!hz>iM1$q=dUKFj5?HpLHuF6nuW~-=zzV+sL zPYd4gKYuNg1-`bo-`uIs43@NC?xHK6!8w^FBcnQ*uYAwtEvQhI+w6U(9*}lw%HQI{ zSXS#lzim#-ysR29Tajt@_{ULP>Di|Fgekq{qGSD?EYL5zFcxW3EKbg?;pN7E9xra0 zC#g$$l(}i4Xii-px{UFQTv<=cxvc6Fo^swIY#|u~3)ElM`HR60Sd(4v6gks))}Mz^ z$X;^ZHRd~1=1|7WjZE&hI)>3Z!GAD3`|CHlO_slp(5w4LUS%SW zS_)bk#tkT(^?p6?WVrYyc@Y*DehPlNPv%9eumdMvi3KdY>!jeF?1vOjVIFh-XfJhJ z(88J`X(=}PDszyxOVE*5L5GDvzZYMp*u;vpof5q1Iv@0)KPA(}GGa6-bBZ!Gk1P^O zkx&}%E3np!8ztG9kPEy^O8^|n3QEm{P_C)$(C;?2itIZFlzvZzUa-NPuK zl@YX1`MAr1^GmC^DtP7n z*Ln~7<1$;n$)ja14U(eL)AWqpI_qFjQzmG2!5L@YB>!m6YDv=D*lBf+mL|?dsCW$} zHxqJ{H@Orq*KBKYTxkW=1_q3M>E_&zF4PXdH@YyAxd13uEI0c&ar-c$4+nsBYkDnt7 z`vXG#Zc~%%N-McFeMBH(&Bx($d8?gEDFT7`k9sA>kgX^W2R&h=ADRy3kD&T^$xMZq zix%i1lNzefM!WSb`6I|Jnfhox>wHrRdgGXBO>1zpj;N^$ z%E7f;b4UyHQLgVFm@F#~OytTY#m5O@Up_ya$-Oq1es;;(e`@fXUPdNFif6QB$rne| zhVc-|Jw3vbec;Xl2$OX@)jw?D>Cj<{o(@4mxNMvz zWO9!VX8pB@d_UjBod3hlrZq*bw2E8Pi+=tXngv8u{iAaIlZXVqh^c=qVj{l-d+Wax z(O+OB|1Zo8!u(4k!2#l1XixsqNV;}Fg*nI<9sJ2pe5ep{=l)WH=}Q*qt=#-Spwd-d zoXE9JiWkzs7eCLR7c}?-Dt~SO3K_7X^IuzW0l>LD=hFKkG^QI0MPRVTHh0|cGSg+R9^n;Zhc{oKN|YSzMRZ04_m`q_#sJ+lxz zJGC(55wS3AAN5tBiADGifmb2Mf9UvBkDg<{;AUxQshSahhG?A;)Xw&f3aOVy4mH2a zf5Aqq>e_u-bG#f_kq^76sVTzY)kDEBXNqLhLe{~5n0UH`Uy&D1wlEkbpb1t{UKurz zf*#V+oqGRksBg&tEcArD|Hms`l@VpD^+tXvTdhC!vTUQ7uICg3H+No8=bzPgE-kMvC(_=5$#&?OKEDgK<+JWI*DQn+T!Wc%dD zomEtYVBO>cih};!qa2W1E5CPmvQ_5T&3?5Im*%oBNM*zGVG2)gw7ehJC>t|OD0FAx}wtWN(z}(onob|!{7s1?Bf8N zaP!8pr5F_vQ5Xg*LBDLY!Plg`%6H}AFh9YZ^r&+ zVC(fN-%?(C-%bOe{#s$gn^%#k{blnNe{5sE?*b?($-aNZ54f)^LPqo5>s!LkQ4;VC zY8v(7(DHZ#NZ?Za5*hh;y-Go4x3u*-p=%|SD|c5D!!i{Jp?_=;Tw^kTCp2U4u3^m$ z`KI4pFH}C%F1HL-W62{=zZX|Ayr`McN;3K477aACyE2CzTswic+%okh^P0HaS$J` zTdPjiram4dn&pMUwSCsY99X&3NPTRnO|GANB*X(=)JSw@zi#P_Pjo<|a68!A+%eBn zo;8c0cyX`ImxaOkd3#UfBpBY9>_B^HPoEZ;#r3sAP3o4?o5XObhR30r7tHY1;jx=V zZ}wO{+RUtx&uLE*jVhElk?bH&ZS-A(;-Fbw1aD${n?4$~C4i7(t*eR%sO{11IMUqm zIj*1j05GxIjrRt2$*}zVW=E+Fw?c(JFp_{vsQJ`~!^`83LyEsbzs(bI@5kG{KkUq6 zTiZ6}iKn-`QJYX6PlSJEMTb|ivTNh%Gi^P76xD(SBbD3XY{GD0N~U^B`l4b-y8x2z z(+MzMxAVadXcn)->)ivPF{Q(ZSZCRX@(q5@keaV-d@^q6IYCuswiN9}tc5CVRpNBV zgALvQXg|7GOJY?Dd_W(4aj-z!z;jJppmJyu74?if_8GI^gAvf)hwhAIdteaSj77_t zR7f2s9{AKWabPvF{){Ll0dJiIcF~#}v9Dge!)~Vs8e^yjyINdztz_!YBt;o5n2neukt{(T12KcWqe_+P zup2)Sr}|pHPM#XObgLpTU8y=D;DI?^FUi4!8sqj5`6GtLur42n;joC!v$>Kp@|yB4 zCmEA#6)CZO)X#K%l$zHES^+kpQsp}=-tMHpVNA7&WK?>JOB%As1d0s)5zS+W z!oW&AbscM?7$gLLWJoc>f2|p2kiaIiH&O*04Y)7dd$QekVcwzh{fD{X;9%{Si&l-O zusW_qUJsX>c)A+-@^*zDRO#N2$0UDACi!KmIbSqqXhkua`9SFcWs@F*AbA5PwtTvro`+5sfJ6H znhsAM%5$&f$&jX;GjHfcJP=JZ#k3I&&cmW~y0<^fkX$!38b2eEkX$&7VJ6Q6>RHeS znh#G&q(>aW%GU$2no#u!(XEyTq|wj zqN=qO@y8EQgsMJ)k>BExb*mg1U9ri9Zy$S%+Hu(?Q00UD%upiN&-ei%v0jmFPj@KL zlE_Kk$1El73g*^nxfI$pWYkq0rz6_Eh7cXTk#+v26A+)znO*6_%)~%xcKiXqsgY3q zBvU0}PPay>;mspG%i~W#X1_u-<^O)Ef0eP@vH4YGs6>dj^3ESIRLBhqMhX!{ZLA_& zY88(I;L?iGi!>O>ypQfH`LQsQX?K4VW=Qhs zaJR!q?Yl-H=uz;`5v##Ta*!zWK%j$VbUmN>EuilzGDtw;_QAC+X!5gU@9=yx3 zZ)oO^tI;*w^UhZZj$>-2B%&O7Z@v1)KJVbUd+^hb=^?`rKza=KY+M{NL#k`9gn?c< z7*YK^(*Z|Ww2Y`Etp?DkT;V5qLINl5$Z%3kZ3 zCJj1D%~>G?grXD4EzW61f}8!?Hm6pLgi(~q8qmE}nh2%CPaIPy9Zo77?`%5IMEqQK z*??sh4PDy(17!dweI$Di46nl0Z*S@ouT#PEDP7wEg%APpz{SyP$)L1lW&Q%)4N*jZ#ewF=v zy&6@FOi2tj@kFBf3SVChgUCWzYBhPon5d%i+UHuqgw_ZMg+QN`P1D=4i~R;69DINq&%F)Dlx1%HUy27ESQ!Nl5tCBVP!2O= zGMJR(A)*3*z9?SD`+q5G9OQ&Kui__9ttO*~KYpPfmAK2fadq3LTUN zL@M8C_UOxyJlrLx#x)oJ^@X)@!IbM|*rZ$KAYZDPp*@jX5V%hHv#!8&=>wyMVZ$~L znIBwKz1#-QfbgVwvr3B6h6}*?nvX$S1=boo$uhwfB=&#T+>#iCYCAH}8VViki=T;f zs+Kv;ZYxiOlJ!xWX}-8YUgTz)&3UdmReMrPf#=8mHtVeMI^#X-7bBQ>wMLKwd!^&# z56L!d;A6LmGEmE)_1l9sPt*DN9RhU#pLPfUt?vt|PDuhy7-HB$bqz z>AM#Xw_Mm$sd6)1W~=9po|f9{Q)7<@j+-{O$QaxwMr|>{G(1Eom=g2h)ns#x8x%Iu zkTGA=w2ms7FP+YcyXb08Mv5|XoiE+#^MKxa=i-cR-V>)4Io=Kk>8Vwrd% z3o67<7nED!R3~=0uh|2S8uViVXS|OY`d~7%juBcTALV+pC4X=!E4WMk5GNyVao^RO~fhBkJWfRcvPLjyx)WeN$(OZv2 z!4(;LniZ&`M+CZt^h2SMF}*rw_&}AO+u(#$T8h0eOpYIOk2LXyKk-nrZP3!(Hsel_ zMO4cuz<5`}K&oZ=`_^55MsPFpR>AFH=HPe5L@Cf{TC};_xKU_6D+tEcK9SITh47*w zW<|4hRXRaCe$_Hyl{|bYbXWn1l5y70rT{op{WXFJ#crM=&yK+q69N>%oBvHW3aqQfk-AF2q655 z6w&>ZNjKmAvFS~WAa&&jTA}y=_rki@y^V?CwOW4tllc=Imv6Jqn69hYz3xwNy%e=2 z&P4qjCX>L^sOQ@>)1$bAwuyJTwM$J*{bR?8P4@ZD|3i!JW{6!QX-VF8in8?SuYFEP zRNclK!)Le$jdRMZK$&=(SKAvimNVcyZ0$bc%xiuo$snp{cYF8R9F@twxL#Q|ZDpSy zQc3Kef0I!-L=4{&itC&=WBH+3^TrtQ%fRd9w|VkT2+Hc{MT7>pQ9HgF;A3`0M8jY-HFzb#h?)lz%>KSOjB?bvMu1oMGmk^*dL{+p7r)4iNc+Tl9c%(SU=Z?@;0EhI0>|Z2 zqh`1d5DO3a#{nJ9cM9N#JWH>Q=#AF=ibPcf^rpuMJ|uJa824g0h!fYaF(N_+?s2qE znMuW$^lIBTj|Ys;wUx8VF0U9Fg+mY}NRVz5=8od-|{+wNlX zN#2b`5fVL_MsL)v!tvT$6qyas4&8mV)4FTgLmvQ7y0Hg&2_E6jq&Z|OR?*{onrr{0 zLcE36PX`i@E^@DmYBd4vN*(T~5b%OS(dK7%`+^nnzlD0#qso%hmyy zg5&F?nzxB1h-i*WgPkL_c@z7c^?0&AqSf;0Uf}{JhxEC$6>4q}@FX?Sww~t;hq!i4 zjl8E+xVCdmkT=Hd+3x%@Fm2JUn<^?W8P#=R(Y1GgyPvW*+#yUx zC4%6VHvt*Spi$#7QVvnn7%it?TBT`hJqAUC^Al^}2sdOTX(q_7`B}~d^GAY}jaH|{ zDnM)=9ik99YCwjD$>O|WmT`$VUllt1XE%7%MMHrYtsT4Uq7)<32kbb}Sn=FBSL=oX z;YwvVA3e(6-(0Acv4LrN9E53JPW=;;b$y+P#E|CPT7T85D=qL!K30E5vuT^fN(r>L zD0nlx#;8|1Q~@3ANZ{$ixt{$P;mkl&PDitdyWjc^=8k}cl*q>&sT}l74ABdsqc$i% z7@qZ95F%RR@$hlFkOMZP1Ez{N&P8ybhV$2th)4sD0ZRj8jQnJ{r?1s6jxr}VM)^k4d<9FX_Z8}IfVDRA4zcHOfMW!J@vZrU#1k+f15GLvrvfTp zINl!4L2yVI_ts3p*h5!k&3YLOub4cy3cFp9vEQler|R}!TxWGRPkC!%e~>kNh^No+ zN$ez+2@t%`#B;49%ZB`&U}D8KM5)w17%q_GwV-r0)2*5Q!XbWod=ya-dsq%TDd#7y zpA&o{IWA5YH!~C^lt`w@d*l0P%^d*4Mk>U;2OU`r;Nn6R=p&>`J}XO&n)?XJN^Ji9 z{Gi_x!sH#F(L99hx-HZ@9FGR+$t_A@LV8uSNCw@S58~T2^@76)@WfN}AS%m55Ld^^ zC3wFYwVAqf^|t+e0rC%(7r^AMONJM)w#ip19b6R22HzXq9NUJT#TTSNh56=denm_3 z_}Z&|{8mVRt0@jFs|3I~NkZv*i#(f#&S>h<%TbSL^0mVSn3}lK`0ScF>I)sv2~CAg zxbZfyq1@+ADBOO5XfML~qG`E539t7(<_bvfntJat=ib_cMIL#UPKgtPM5ps;@6%Q4?4P~e7{x?hg;K)oQA z_0mK42PZ($fGng_@YFS0_B8*JP+dz@iW!EgIWZ>OU$C|E`Q~@mKAEpDn(e@7b?Y#i z$PisK_RujVpyQ9XX}(R>@<=sNW-e$+e<~!4VJM-8*(SZv2{{(^2qB9dxwo%w(`>$%265(y!;BXhq0GCnM$nKW|&o+I`@?{@N9&7Vi{|ig{zVsxIN` zk4#*<$fbn-%IXXN{-2L|ySY53$n3%3TSDIn#UiFo6=w78)v2yyK+Z;m(VZh@lHg(m zeWF?CR>%c;{e_liG@9b4ZliayI}{2&@-sqUHGf{yuQOZ+Va<>gf=i!0ppD%N0*geb z;Gr?Sj})thP1Fq`dOc|h9+_i8$*fNMY~&lv8m5VaotU zPs4jrQc*fRQbwLn;2l^&D!nFG<^JtqWz4bP2flk9pQr{X*~8)Er{XfW4*-RicId5r zLQ<~cqWddnYm!n6h|&@M$vqu8@ENXW#@u!D>L0E%zWs-AjavC!q<>@C)~C*DdL8oc zq5YNb5)F**JJ(=p{Lo%VG~9M(-i+SmlQ1 zxYXa8t~j}k4{R@po#iM75y$auI9Cx-^+59obynAG8&{bbg*O@&PkOQv&7x}#1>SN{ zz+dU(>2E_nfhRkt0wGgOFlJAN^=N0=ZCcGExq$Z0r9p>7P3zlo*l&3uahJr{DuT$p zf+r-38>#LdXw6sCjKvq%GkkaSw}{0oisTs8;MK48#DGjQv5}-2pJ5O`5k#`1+GVaG%)|Vu>x+wiJlBT1)IDMVj&d7LQc;S$yaBh&|wrz ziDP<)N8v}ca=5amy^=htmsZ)9Jtr?*4u1P!xUyQvAAZdqkTR>EOmgD+yav;5>Bzo# z$_-=$CC8={-*K8TufZ{34QzB`GtP$S&L_~;pIj-|A{>efhCAM9)fU`lgOqU)BbsIk z83K!4bRScUT}ZWjF4CmaC1+SmKd;3d8C(TP?~UQvp(q|dd#)LkLOkE2!6s^e$ujDJ zKTH?^5q(9Br;h|=BEDC<62R){tx`l{fP#?Y@XsmLL7ZiAr*i`xD#oNxqC*sJ~yfNXGSNl7?0+0hNB#cP^_+-9G57$imsgc zMtwAg{d_>nRg*h$-=P$ad)RXBu#bX@B3G5Gt`kYDAh(9N>nxs*Y`IO5(J*S?3gkU> zoK7}v0u4CZyMiGxyadfxixqO#em#`mo-J(Mqpgzv3<1aB)u3>HrFrmW#D zS5lzC+)i|1Im?Av73Riui;4-RR>Jb|@=>nQKt-6r>Y?7>X#f71pP zZ2DZ=Y{UGTL`XD9Z}W;rjoMX_nVX{Ige22747HWlK~jIs&2jZ8tV_HUHkn8m+oRo$ z`Sx7K3VBMzrrvkB~f;ypALe%3^YSXCZ?tWf2i_e+$F{?7()nbe^1) z6!`Z-x5MpH$P1SHkjx<%xQh+-IAZJY)kPM?OhE45=0A3OLKzEC)+CZJh}OH?h7YB; zRwKmjt^jC}7tIwVPkQDvdFKWsa-b+yz>t=3BlpF(GZK$#{ZYl!Dw2 zmKuE5maE}X2MWTPuk6?lmeD|TE1O~>cy-BItfC9%E7?Z?f01jAw{L)}#wyaEo~Nta z`w=Ee#2@~6gd|w-vcBd&tb*zDLy2?3(Dy6^Q78a!*`UhtK_n4W0GMMDOo!rEjKMZ?E9NRZP zd-hmxGgWWln1r>;biylxm7;c0Zf&vi$VL?jy^2XWo8LAC168^9a!C&6PtKrmU@OC@AD!H#6_(T2Fk1{#jdnLXa8OIj8(OUSlJPf+l*~xn zhNWgm$ayQLBp7k6sM??&W(@1fkVHJfDXZd*l)c_?)B2(-RSt)DxJ$qR;G|2qLqHM%>itQx&;1;?s#wQP=h-=eM^Jp$FAmycAWkmu&ik-lLD zD3}9}0Y5Q16?YFJNmPgnfE3B1v#Aoy`=7ia+(TC~dh||QyuDHm6x!j%P6zYTjYUzMNIp91T`zm|F-3C>?d+9d;dSztIyUI5wWrS=c+46CW0p9m6qn0 zUPi)Jpf^k3wiN!gJ|ZGdz5m)#A|e+C{sy=pH~setyXPfpddSNMO?@b1vgWyEkjXZb zAOB$3bKX!9PGj4d=tod2;*_mTr}Rd8!t>#j>`E-LcjUch+$ z37l=b!|21Et{I)uUJG-R`u-->1dBopy7ShawVLj2M~b`ThW$Rt7J$=9I9n_9+|0zd zE$9>3t?G6YVPcEwY~4uzeX!L_=d7OoN`puQ(9j(yKXB$~>e=ha=$*80}% zX7dA5;J6Cw?m~d$tmqbT{n?QZu;Bo3ZNWv79`l@BuD+X~?pb>aVdi5JhOn^BB_Zx} znLDC%ulkkAblgta2*j~i6o+f&Ke&?o*4P_+yD29|bL?m5N*5-@#UbOT3$GbOKPo_v zyj+bdmT+3wcfq!Zgh=2vJ&e)|@Ec2H&3fhppPhdfvRdBn{cD=ia3EW!0c9Bdh*+V% zi=gZo(Z@>@)1U9spZ}=7zvvz(<5T>=#dS*xFCytx-ozpOu|sC2LrRl8uqz*M^{X(0 zRnd>sy}_A2J^zS(Ip;mau!!?M5e;0vj018N4x37!{S^Jo(Nio8VNO1Ja9*Vd&q@1a zwNmfu6HxsLsM$P1qFax;06-Pma0@4J&qETz(zbzwNG3Bv^~N9UkxY!qL5-WF+ z8uv%%dBbLVd{0F`Ywm2duzSx8dM=g{j|gS=uL~&o$91&|Y&sVlKiA7RSddx^-tIIESg3 zT%~CE?pzq+yuTY30rL@SEX1zL?jE@G)1#fPS)A!OjTowh_^+SEL{Fr;U|!jqNqhj^ z#@x7LEZlywE#w=_jKhjrn2Ji@cx0DhtF@;Ig5cqq{f6+1wMUfROfNLYe$ztcIj*)* z^t0IN;JG7V{NdWHyVaV&%zR6bf5y&i?a~*U1Vc6V-f$qFrK%eHDEp&iJXl&7Y(6kGrP9!cIws`@@ zKkQvPQ_xk39f;bVv_qx_m?7s2Gc?UjvKAVH&R^MT*ox?W^=p3V9H&p+e4r09>`#N{ z&F+eiN_}gPS+E&ns;p;U&JLd{uAg7C0cX@7wKqH# z$in!C9JtHoH2m6pjx6RP+%b*o?Ws*P* z-aI(LmoYEc^_cB9xri=_%aJ=W!P$h3B;xzMGCKFbZMi`VP_7Upg_ft5v&p`5RA0MN zPqTt`A2xeTy)Hw@`^}D2o)|M&1Mm&qjMMByod}%Ys~=U)zB`q4CfM=fW8?w->hBhI z8IHx3>^U*P)JR&4_0&*>ps5T@Av*>A&M+Z*CnxQBvpT)x`FGCriaI4ASzxY? z{a=W;?Jvc9Y*{>CIb1Y0Q%ilb>U;fhGEUDnn0BJUKop3E+KkihM4g;2_!wE_5x7?` zr%?eWwOytriu5vg1-o%Pf_YG$7ynS<#=`*jMxNOS0@_ooh7e&*0eMxi}PA zVYD6H?7NxNIqaPsj=xTuYfPycCX_Bm(}vi)s&LPf8b3j!AC;V)KX`4?K7z;?_!y^g zGMa10hm&+XOeX&{YRzXjonW*OclI508h&FgMAdig{&SU(dIaH6E>Bn6w~o9$CZ3j)w$l);Nbs)p z_ILx#RB|i0N}8QERs5;(RaHOAcO+1t!rsTN@l$M$Iahk=bK5_<)%fexKdgkw@?^S4@$>U% zN5V7#B4*Hf2`YNv!m72=sQ)g#eBHbQq#0dk@Uv}yiHNg9Pz^OcEG_JKB*epeIe`%* z`svS~yCn8SJ@_k&?-+U>G=|d<{n3ucxA0z8xC(2}QeJJ@a^VL-x>DGOar?NkR)Gd=2*o}xO0jK5{ zKj)wJUjrB&@*f%!vQBVn^rJU1C8{STZ1}%KLFNjzU317?VnZ?arFfd?T9C$@tpkHx z_1?6Bn1oH7kZt{24?8b;^xyLE3prob)U@-qmwC9UlZ7ou!OSMRf*seBYaB?((C>VE z@2_J0zwm7L(DV5gRHDJREMCuKx~LdmRlC4Q|H~I0C#Lc9kB=$-RnmRjmX375gP6@K z<~o5AU_UMGlmqsMgr6&w&&nTLtd#$h+Nwcn>tP`!))mjpi{r6t76*0JYba+Y&OBV3 z_P^lM{l3%*8DDkL?}Nra;0z_V@PoEwO12>oe~YF$aYF9Y-zANW%aR6tFIsFViMT)b zB(=z!j#O7h?lO#1MB9U;!>3Q@h5)lKV5aRg-OSN0S8$ZhR2) zzv9v2ps^jbUtiA(p1R|8=NC}9TCp^>T9<^WK3VrO~<|*44=DLQjRkadq02n6pf{x<`Z2<_)BvQA0@*= zV+Ier?+y0opI-H~EgcmFg40VUANFnk2Fy|OBysQr+TqbJk=VCk?uB?T`>v?8{mgY% zY)WGKCT5shB{`QG(|p!v?J~;pZ>4~D8>Uy`3+3K)#61*}&{%|o_|J^5`GRDh4FVz9 zf4cg+v4_YJ`-H6Fgp#fMfAlb-Q5CV&o76CB4cEZBaIiSyPi5zSj-WK7 zZprw5A61>u&tmMC6nDgmf*>KEnJ)MuCSCjOra$Fq2ewPj_KeFG|2*0P&cIO&jbf9t zzGRjEUZ5U!3Hl|&D@&k(0K)`Z-P_74AoI4K>SibW5uIrn0Zo%ZoDVX%1=qfU{GU}Z zFVi6fW(>-yI2D}}55Jfen1iR~#)i?{bN*m10vOpR^=dnCjcD<+&12n=#R)(pi3oR% zoc$Ww*Y&2ZX|ddQS66()==}Phy`Y_6I{GUsUSa{+bP4_fkDs7O{C&_D6N(BFx# zfAOF!@ZjI;59QhDUuA#Lv74p7LuKdW%sg56IMPjuPUTJ&O(jgVPNhtBAQLealuA7p>h@ZgkLmF*DfhpJ-%S9& zXM!08TZAn3iEV-3RvTM>(uu96seh5<(#J8Z$q7M#t3zm!6S2YL`Z~G`t7uw*AF!ok zBh*q(uwGWnaU}ouyEYRYaBx;GJi13oN8vwrYhO|w5RWyR1RTaxUC1o6S%|Zgch?=C zDbru50x4!7YVK|sZGpKkHu9!)YB)t{9zA=078Ym{=oV-e=s~Wf09+`a=UYfnr1x8@ z$X7{7f;{e)_6bk{^UyOI@d{4AF1%}@@SKA2+PIWaPqt+-XF_KQ1oerh(S}h-8 zWzzSO<)YR{#d6B1li_=!w&`fN5sw+DY z)PJX}z(IzX-Qo=P1wM%y=bdFtOZ_Xazyqd@CT3N32BiEnu``kO83lFUeuysEh$y2eC}y{CaO zO2DtOxt6{6uQJc2;OQH<`ze-Y1BX&Pr=nGPon^VD-Fbbf6-Hnyk++T)Jp?zGh59^s z*XYEy|IZ6Vgj=ub0ZboAaAi7#(n343pVnynXbVFxk1yzJ66%0(vTIdvb@V9tIiE7^ zkHsPSqqJid8QRRuOM3C_$UDhE>j2YO(AC0d89T7d-biwvius_?Ae5Zru|SmcQL64OJAIg+o!XT zx>#WTDzj(g#LZ@~?x&dL2ULD2u%GMny4WclhQVTLCg&`%@R~?iLgrvG_CK#N+r;Kp z;e**ZAb+!Fvx3j5n%LSc8N)N2K{SHKCZiW;qSdk3)uf75fb`pB%-VjzAsU`%0p|TI)W`! z4Qz43G@l5v0mzpf2LCD>5bfsQ>EBN|VPZe(U&hiywKL2}^x<6}JGCqho7DW9J%wXB-I`Y)L)B^;=6~AJ@(s zwEZtI37?qzxM2p zbp^}Mm{Jv}bu@z+S;IXs$u9pnD~M^D6g6I`o1DE7;Qzs=Ld}sX@jsP~G(1yCGo&T= zX6X+=*W!3$tT>eeZ4RDFp2iH~@ljhx+8wV`3@wR2FBAOFCy@%4G>{~l?EQR7o?PK| zT9M%4AUI55J}LhyILgXz&wzZNrpp@3#eOYFrij9aE* zf1G?fmI;3osTIcFi6%TEIEBXSdQf61-n^4gZ%Xcj#l+wS7 zj8y(qz?62s_e~1#e4oQB$b=JZZbu*S#Aur%?fzl>U|ZOKo~KC!`wKH8GIlnqJKAu* z=cxDOQ2dC^B#rI=mvVVbSG%OKjmQ0i`-&QF{D$5asiwo-o02%Fb+2TN_i5oS_WI8V^!FQR1H`cly(BB@k4eulPDe!0S@5evu@$skF6^oxy+1_|FI&0XCl_CP**m1h%SDJ91sgE0!h@x)96|yeSQfD`@V+cB5{pCuy-c z^JS%@BZD46-i24L&??lZL>HQ&t|bGtS4>eS{Xb_d86e|P`tp1m->mXb@T_5=)`M!H zoj#ZZQo)y%%snn-9{U><9D#YJhTZ-UDZ?g@r&bk6pq-;Oi(!hlErhvnG3m{deki3^ zG0pthzc$PDjc^Sj-}nDx?L8c+?)(4o#?`tgDMG@P%7~80$gV_n+1WX#Q#i!2IYt~R zMRu}xD6)>M!!Zt(5ywdO-t!oj&9T0(W7U0K*XQcKuix(<;Qe~;@pwE>xAlNft;EJM zJOFaKfXPF*Hp<@)lQFDx`mT+_-x<}~#&{vc9GjZ`P`*wvg);7DshzvKJ4IZSK-9Z8 zzr2#&FL?Oy%iF)GLN4+1^V3m>d&<}UntJ#bm)~U0`O|wd=*slp^e)q@t|-y%6Lu$h z^_SR*xx3f(>s>NX&_5BWbpNy# zuP)+n)UNCcenctIU*M>RC4__&p0>D_@*Te6|4N(Ul<5Thn~?+Y?FwT2%UzP|!!Oj# z(QgHlCm`$v#WZa_WdWM2*5{rdzj65N^W&#+vG&EG6nlmgm_FZ&wnlgv8+nZPBcDoz zvr4nLN^1B%()pA!77gi#Tv=MEo7$tx+SUA4mw9mT)a(oGRk!)U$f?dqsP)piO^Pvo z?jVvLdZ@uB_~hX4!dAFn-hcAuOkZl1c^}w!@OsWu^TRrFrn+ok+t=~ZDmt9;p`8v3 zP6D7ELl`Y4t*rWKUYclfyZpv~1eg7wx5;O($NdMS-W5}5%P-A}M)`yzJeOYx=JH-R zCD}>-a`NZPU>-4>5AAy*T;FhMZ14?^*(pAC#r()wpo#!3^##8vxhze8G?Jt8q@ zayvK-Ai;aoa;MLql59VmeYfMc5y9ifu6tFORPBEDIZDOxbIopGRB3gt25o8@u`m6o zC*t^XwzD~>xI5Z-Q-lyam)9!kHq0MVLnRacSqs0Pl+sg~)pi}@STTaXO${|;P|<5X z_=z`!Rk~SUQN!(#2qdYjBcq?Yud;e1*6~UDN$>)T?=P@idvUA9!m`G&Tr=+f#}`QR zTrMQNjn-`0->gyjlyDSi0%6gAhubNSdee$-9QKCRiS<++*kkq1M=PDH{sC;iK4bm8 z(6i}Y5N}HaZ*D)ar!vWf3#kQN)(P!U^UfD^t;xv97cAcA6=(* zZMFAhv29#N2IH$o+b=7Kb@tH*3m*XU)%f+co&7$4Lww0i1{NLU3|G2r5X0g z9uA??IT!o6id)wjzW${#BpO_rF*6Pg@8GF9$vJe$cQr+?mH0wfs*{k6@|79E5?s2>`(*_=vfv0 z&|E7{8c@FVIb_@{R#p$dB|&2@o2W2ueaI~^U)#|PRE*eKE2khnTf)-kGy05JqIy*mynnlERg@yURLx@=$5~>9lB%8`k^XGlos+wZ4nd64iS(1-ST+(foIJuTa@c% z2{`2ON!LqQW{py16C%&4n*h~OkX3hTS9L6gblBJ$m)1+=%)rIslP5DaHTGqAxWaeE zZ{UFB{74P6ooIs@MzJ&fGSiGM=eY^^zkOqA(*64v$u&Z1(Yoh*xD3Xy%L73l*=h&B>mT^~*A& zHcI_BfrOqcLRO$Q2aL+xzWOPr;cyDs-|j+YmU}^H@LS;d%i97N_U7~D2)wJh z$e^bz* zgE#lwuzuR@(1dSfLadhWi)Aa9&;vk&)YI3 z^rNC`YzL1?*XMl2UZ*-1y%BKBn0`GVp}*^oD0J0RB07dUUE(*B#I(H}Q57-&)vej(E!#D0CWXw`XBE;^=mPdT&B`npRO>sg?8W^WM_tmwQ^l{({I z#D8_Y8pLYo%(`9sg+1BsgRX^YDdt+Kzwl&_b05W|cu(5ZWj{3!rB5_K@q*I8D5(gApRUx9-Llz2xGM_LcC}$eB4lywc*~sDR5Q#FyLan zDSj+Kuia46q`mnWc%>*$enU(B1 z!GCrg;mi8xv^3H$A#Yok8}L_G)64nAav!UkLwaB+>#{jBD!*#I;5zxB^Cc)3QBSuU zDBsZbF6+rMm($HbYF4fB>?S4sed8_?-8YgCIcr3btlMj}h`7IKJn;?_nye^@ZU{8}wp~)g@M#%t36Now;p~$_r0#%-6Il$} z(rwQFQVb!uSs#Lba;-=e7IAp)ooMNR&O{gq|7W96)Qc0GAOn-h%9GJXq8|1QIb$#hvv%~zm?i;A%iDLDj7M|7rMXK{(L zOC)nWW+NS9LDy~z4Odib%~VvA_v@C5wr3GLSx5NQvH2S$K5XN=%7WnA75+4 zZ^kF3s6ix6=;XJzAN)qE<()OBUo~0P)+L=CMOMx&i=d5qT7de>x$^e9n@rWplyzrdp-cuU=Cevn#_DHNg7P8 zfIF>X2h5zNOe(rJh%2w1mp(}`jua8Q=Y*KzbWuJvD1>^9CT%8?kdZPr_%0TdqeT`| zM>n%K)MtpauRa|esg|BH27*y>w(?uF+g~!%s1nax9$nD+fe!Y}Eo~{C)@FX<>BH@1 zJs>EHtf?a)DM&W!VGoIN1uf^BASM@gZFWo7x4WhC;!7IKJTt;wPNX=0UlV)dnMQ3Y z?7?#EduU+VW9Ztk(y2DJn+w-aYE*q4`*U+9TJ_s!^5`am&p3DP2aT@+$DvBH$pzOM zz#8a2&8uke+xN5$5mU^>Na+=k;Zd&&kh3bUsUoK9D{j>~sn3WGp9yjt>li*CEj(q< zxRJP#>fGFoituT+#}r&7kwl0|<|Zc$qyMc!9@a*2`4bfEV!iWUG9^e}TAmMw2d%bu zclfQ=Z!F{0-%|#@+f?0Rw(8`~_LLm{dM)`u^ap0q*we=L%}Hvf&Dp!% z<~+52Pxmb$J@ptu=0WcO=$5KELjkoU-3_0t<)J zImp@;3lnNX=YK2A2hH+n3jMP@9#vL^(AwJ)q@YayqSYk`(sX=ZV5*2}u;)YgLi-~Z zWVkLR4-YMQ6gKk5n*WsZ6bl`;c)rnjZC%o2;M3qE2+w7cSya;myBIHS>RX&Y)ubI_ zCNw`y0Y8^)%eDc=vYK6{&w`7O}?%izn8il&32Kb@^$pTZZ1(XZS@C7IHxFlU!$Tcm7Bea*kt zN$=rND12>nY9pw+8VWKQ_5!}4 zC#imL?@diPV_Ohcc};qeAjFOlKp)`CtK#DoLmdfrV-??|pH5WBVindT% zHLA9fNo}4i6Fc&l`3Ivn86iGQHt&Bk*{s!VH>c|l7wu0-^-BlrsWH^_0OyJbk>0`t z6o~)Q&ZR6rBjLut4y;R@D$KfUWM-(~c3;9yX*?ZMr}QW-rB@Dbz;YIBjYD<@rN9WP z&w#;JJ`l5AVOr+o)ro}HR1;T)mcL0B)rT?WUqd7xYaG;RapPYA7Q#e)Ei3GTD9@Z?|fu8?~bO3kNO ze+YwO&hW6s+uB;z_2Msu^AvszJJ7s0yL0d+?rsef(*AYfsUvNx-B%4cB5=2MPnOK- zK^cR4P}(H8MeY*oG%-cDyp-pg#EpN~f4KE@9WuRxjYfxf^Uic3eHHP`t@$a+7Kdp( zbu82M#g;83Gn%iwjvon?U+7T4jm>UlLyhRUlN87v((l6NxKI*>9MLIqKB?bXY&kgp zzJa_KI#xt2$CFdBu34+x1L<;m<;b>tzy3l~y}t&!=klR%G52r7%dmL?3`g89OaH1N zBcmk;D|-bhR{51wZb9XRY-!tqN2jHjlKQw~eKq&Q-!4eIlXi4_KzLX8s9$4X?&~`N>2~9 ze^~x-Ykc?h22&@hZrlD(thr8i+hGdvtNZHdHabokhqF?d zit{deb~`2rTs-zvM_CTi^b#+Y%=0y`!{-QH{_T}Re=*EFKIVn%TE_BnwlKyMFkuF1 zVWP-EV3q{>6Tt$sdAg;1yaCDv5qU)jO7Wn*W!K1NNwH&=Y8^J8g$3ii+-fC@Pg9lq z^ws2Cn0R({u4g`bmH;e`?gdaV7!?<#L! z847h{gqUV?2Tl%)wVysw{;W?WV!7oUTKw?TuXLHWR}RE9c`*LcdGOQOm%KRd(A_QG z^ImuK010r6xA#jbzhNJyQ$G88exT{u3vJQ+7jlewoDQ;17WCjMawKhY1kUwQk>l~xQS3=7dHka9hW*2nP?p4MvROjm50ZHJoJbSxDZCbx9N45ZnK6ni zNRjTR(o5g)*Z)$9UAz(@YM_>D#}^hdS=lz?lNNi?0D{Hn9>;Lkofo3kP=8w?OepJl zb+*3IPkHew{?rko0$GShc7#Yfj_bE%R4hwVXgD4kv!v@VQ+6f0d+^;6gjo!1;ViOj z;hE4^{3Ix7yP1M3+kbrZ+`Pkg`@n_?g%5N3*BnXzpU6JzWEN9W#wCp?^ zy%A&h!oK?M$A2_!t+Df=FNN{Nb8Y@n5v!vPW|{gWyfO5&NH1|;{LM|zsA8R<`%6Fn zo^J`aJ=FP_N+qQe$wMa-|G)!UraIzt5*Ef9K9D!AN$q<;Q|LdYqG+qB{W<4@O&-7Y zzSS{1b4xp}w?Ro1y7Ig&O)Ph{p|tRYphI%1DE^4UWWlGhOEpf*?oHRO-fB<}-XC)Y z*O49L>6Q|@tc)<^E}2%EM>XXuI=vQP_F*cWX!B3MVq#l0t7)F@k;U@whQ4I$2AmL={f98x4)SIXXuW7IuEFO%9JU-=bllson07Wia2c&&}$FD@1LB^lW z2Dc>%=0oMC55(8nwfUMkGG&0i%*?)VyM?`%OKD2qi{X`wPo{ z%cdZ<_%D33adDt!T#RshFK|ZT0O;TX>q?uo^2?ogRp;KO&wp`sp}lC8#}-W<`%5GDH-SmCc(?7ov`ioQDN7 znR&aZTuaXvL|9Lwnk2TgyFIVpt>_=5uw}5p52eIbX%73O$@aRj%4!%F2qB=&;|F)Jd%KYZi4uIo!8r^fYbDPKq zi|+B3yoxebZ1hrpg^=W|4e~NsoNWVb7s;b1kr8|Owrt6_z6K~VKw_x5sNloV8V#d3 zWPV5+gRAj69@9&RdOvqoi@YGZ7?V9JG#{(3j<~(n?th802+zj%%7q3*{ z`onuFt6o|xP`hx+W=&*fXiAGUt--RquV?X={H5zH8eqfYp`-18W&t| znd8BJCem*^aFWsNXJ7TU63?W+Ll;T~)VC|`nkCU|B*qk+dF6X~OzI7sk^&Sn_GVbb zaB*QF1v7pio~n}+_5x*&e_R{xS(I`$5!k3>@+XBvpVz6`&NvSG+Msn~qF*R}Pi*@>Ym(~c9XIP$Pq-EZJ zjU7H`u$Ce~uRxo9Yr=AZ)}*MW?E*yNr}&}glc4?|y@W(w_WIaRH{VlcV#-6q(qUpIsZ3Dv4uOXaF0jazN>eSCp7 zcOtE2nP94QjCd|z0)1B_yV%*~F}JDAgP%ELv1DP#RyS*{6qj=R4HOA?S#ZBrIL|L4 zBz{ndo66`P8$WkYBj^$GHVtChWBlr(P@SMFr0j{PP+!g7t(ob;otY^mf+*LaKYpmjg^@PTKD8B4Qt4kh*FKK$9#KZyU-Q)cwBY$KveY;DZNC1dF zXr7fjNbU#`qhMqj+5R9F(?3UPSl`l+xqYaNH{@an>%+rjtOI!s_3A#?VR@NNY(qbu zV`Z$zrQIh(|C8`^ap0|1tS25X(sDozc zjPxqBCGrqh9sv*@8D#Xf`PUo%ew(1XT%OANu}kyL+3lx~3_ouhX@iu_-H^ToI+HZv z!BH?A?Sx+%e_kKl7AuIudL$MWXN}87v*fSoATLJjXHje)YU3u6z76x`E|v5pFz(dS zUud_QJoY57N6DmndaZQ(UWg`mWMFaq!7*b5*UgLz3tby?yxy>u|LCL(7f+laPOaZi zt^f1`N{~COSL3#KylC@ALNtQd{dmBsqS=<&SpK<{)P$-7j5&ZMI4AvkEn#C(Gg^Mn zzc)aM5!(K|4Fhm+UZJtTN9R7Ry;h8vHAb>UP!#!Z+9eC2Sqo;YGo9(J#Pv@*&WVv( z;w$L~tS%UHS84AwNOg`Q=Ly%pjc_jFw)YC{(R#8Y|(ZI_!e>3MvQ{-74HGwPpW41iG=mVax zRwVI<4f1@%Npi;hzoo{g=9(pOR1YN=I$0nx_wcs0>R{So4Rt2429j0Li*&N!J}IRJ zBVdzLHl(}t;ZM?!14m8YK_uE%r#45K67fSJYpE9IlkYnrEM$p=tL<^UQllizvIov>bel}L%Qd)O1-)2=3BlksfR9$cwmk^!EPc4L1%ZiQ2%(C*A`L` zW;L}YYTViPSyVA+=%a8PsFDv4Rt}nTtC9hPIqbP9?do!Oka+u6YZ(?<5Bn8_6hOAU8Q6H|yHL@YkDAmgL^0 zcbeVU`J3%QUYz*itR-TKuN;2CZzc7qxzGDl(7Tt?<7>wH{vMS{O1dwj0dG#$%|#+# zY6o)O<<|Ypdawc1&Uu2;5Zwji(^8GONYSpO$|(_pJ-XyMV*d_QdFnM7hKHwWIIg$t zPc_^!#T(0(_GcKS;b&K)$Ab&TKV53_x_Xc%TxTwQ;|Z_h9dj)Y;J<^OEGFJAR<|Z> zmj7!3Ht=%UAUKkd#REu(58S3=N96o2qrT_X;N9Gc!NnIx%oBy8jpSX3PqwE(KHg-P z?e&V5U!Re6F8ILYqv4^Po+qfcf_xvqn_{I`H+L@Uq!u>w}z&~;cj|JrLEI+<)fsc2FBCHbSzFd1O7J>hTo(t^5nP#_FV$Q)(=XDB`!YPqq-X`u`$on-2Ph~Kn zUlZ4x&4Zn2@%+3fTO&3+M}`b%$&{+S$)k1AN#{KXim0YUC6ML1ug)5zY}>8Q7f`s^ z{k(7)^Y3~V^lg}q5r+07K=P}i4k@`Z%I;G+FBkc}ez~w)zsTBjKaNRVOzNk?>XSjs z<~U_AH7}RxBqxy6V0Pz?)S~;E)Na9Bv%j`@!91*QBq$8<8*Nv>w=pUGqafoeuVH`9 zyL{6_Eu8g5a`ItYCvpC}B`Daduz5r%#LI@<@Jn#A?ozv)My=XrXG3juJPd>`MR`PXnda7OkjnQzra( zJmjx{VZZAsqrpu1Z-_h#M<03^>-=Ut+`Jj{PRKyfK6$(W-f0r}4$bm$V$J?d^coX! z$Lcf+OlkwMHL^ZSbpU2jpk)qIS_wQ9%raO#XjQdZ{8D3$R%UukH`=dYHPGM!I}BeR zM!&n%30^xWP>#9-77yKdqephER>~=|vGm+e`4CFJa|@ZT)EK_#6b8kq6_g z|J(GiXRO|&E_ScZ(EcdJki?aJ^%Fd3qnTE!?xGKz7=wX2pa^Vr|BY-&wl z3(qO22xAO?!>1ZlX^o<_*n~e#w((1Y6hiuAwDtq=sQ<+Vjgs&uaFsm#c_e^KMO}|q zidnxnc4f`Mrb82-l6W;E7gnHEHQ9k3AQF@!C+*(Hr$}aYEVL&4v41zs_%Q8GV#sD7 zs68z0HWj>@z$5l(lUVTvntO)R7LuSFUqK;fw)SE}V0me!E6lM!8Gf(jRDN)!I#Uv% z^aIbV-BUxd4WL@5$-3!?9{ML?D}15UK(HaDSl|b$I6T{#x-Y1eeG#0n8Fa_XqfuUd zB4v?`oCNvk{_D9SxksfL99oHP{mx8N9xsoZyeEegvf)!{wH0-AT)iWYRV{P2A#$LrU zyQL@7_eLAj0@xPVnqV0P@#ZB!O=0Gu+yRbDfxvX z0eMnKTQ!<2V?#^|8kU_}RW;<3c7xZ}^L7(!| z#`)b2pyk?518Q4}8W!(vT6S10Y&!kjawAU^6kE^JfO;#b|5!S(7Bbo9G#J`LPW=tL z0<7NTUf97ia~rQKZ&G!Dtr^$HU~43cs8!zXN+EM4h~jlB4?4+YioDMKSSrX)4zXW3 z6*IT(kzxBk{DgP-+_JI?imG~mRzg@>0U@ZXYYe4eU30zg)o2FHqzFxEu(v-(-WuiY zh#tO#gfr+0fq|YGHM-n|H$!}P-d%oRw|?Tnh&2`!%*nFk)%_?1Rv-lwSqf{-5&GMf z$G;tO{xYQiQV98V8n17W>=L_$%!f^rbg42>PJ=@(DLO^Bqo@#<8!iW5?Tnr@YK@2%M_;JwSJt!76e_kwB-Z2xUNYp(@bh#*{4tn~~hZrth zb&dEh8J4Uf3KNz6KBy5*4QEYX2K)S zo(4s_tGC8nSJXHG*AQy>-ZO^$U@?R2q~PH7M)s*6Nj(XaJeMb;K}G4bni_o1rAH(d zd$=dxdRh3V3AhjNkh?^hk=%VPab*sM7jOne!?`o!7I&{Lup%+WInCO zVQT9Wm*UN5fwKzx_TsDpO<HzK-lif^5+m zE`Jy3DH>caTudRi5(l0U?Yx+}gDI9mgDQ>n)L*EwKxR)8!HDKmjtEEPCcvhf79%k& zl;Z&2KkW^3Ygs)UTq|tVEhXwNYHQYdnIAvh7w11=z!C&aUrww*0CZccWmUhwCw%Vx z(Yfwc6e z9HH#}td5p_*udpDqk@4%?OR&wC4A}nxidGLT?;T}*BC2xMm}{U76~x)T)BEZNwT8F)Kmv=S+cu#1|BTr{Vh=8YuDCSyn8y}T z*jBpkb5q*vyDvvQ@HB}$#(9qWR5u@9I3~PPwMdzmnu84RGma#U3gU_7aEb>;hwI7= z3i0KlXCo8*rv7g#_OKFl#BHnLfVvWzR{r&jGr!s@klUpS`Xnh0-3MBgfV!`p&&3JDKI~8O}WVx+gurv=9joT2({+^ z?jNxugYPz`eOWeXCY?nzM>_9XSFcsGcNuKQUBJuS!Tel`+hhs-okIkhh%_`U4+}9r zl7B{Y1tnr!0S@Rm=8$WxbLBZfLDy2=`BAIoyKgy2cL%m|J$*t%7eV0(U9r53oBq72ZfDD{X?3Djq{?x#r5U)|{>oFZz@jZ~GkHSE zMDcxQMAksgMh`QY>WK%zz{JOuWW*#Fe$}-P+!X0nbS?(z$Ir}pta+hQrDQP640=Xq z`oGCE7NwkuIjHqIfe_XgJFgB%2P{m*D9f)-k{$G?${VK0^MByo5`IHxhc4)DC6_W^ z!5f$UiPlBq-(`7&25-M!@Dg~&)+2q zhc3xXl+*Y(u;|zd{2XSG=sm|m&qmLW8<>hgy-RrK+tciV1dd=BE2XMr%eM5KwQ>9( ziG494b3vJ}5b}>;%7|jC;j6Cv4o7TPFLGX~7MSQ)8~7G@DM+A(Y|=6XF=bDZdE)Lw z=gwZ~m?kI*I(1AdDeU^O#dkmU;8L9D;aU6UW}3C_uUm}es}s@Ug7!20P6KX>7hCN} zZDxf8vgXs!1M7vDY_F|?n^Cf3TO7}2(*94A7lm}|5gohj4{81=4*N}lv4p1g$W!DQ z6&A7C(5?8!%fqmhV7^KGGgs@m&NuCv8)2u1^5UzmUFZ<@QKMhJEUn{lMu#$>vGuK4 z{+V-#6rI(Sli$_7_q)0=qEpM*n(q9m?47hmW}iQa8v~5i^&ZhJ7G_^|5btI+#X2GR zr!6+nd$B5@D0yl#(QF;8nDT%LfmKPt*vQ&WfdFc2#V3^FP4^zdGH;yqF7J^ zIXx)n{PB6$qC~)Abv55CdsEpjoA2$I-e34R;C(7*yPCO8QAz=u*At4ky0F8)YY$%_ zW0AdKT^!^=W2FnhcV=kjh|*kqtC+pY0E}ItVl#N}`tQx3@vW5P9uu^_d}Vvb7Y9ni z_KTG85Z0C2!3;JTjUY{I0js+BL8v%`{E80k+Qqyxo=-SW#GD=61jYu@q9BhSy+AY4JA;-z?0>5kO4~<^AYb@bH;X4e{pF2bvFL!|h+%SI}>^Lw*Zq z{~4LT9ii_ea`iXAl9A~3fuG8j4rpwC9*-8&gM`o7mjqr2x|H@(&HE{zZjAY`YF(q9 zkyL&9;U9`E2&fF7V!NDY2^v41ANkOg1!!h!FLh!guns+;RII&bDdLlwPFxoHLKjtV7PV-G-N`$>%EyV3xRIin?vzRi_? z=7_E1Ng8K(eDHqLofJ5Njde4c9$)pKG21YO6Ok^! zt8~dh>lV12cz+pZUzf zP`T5nQm}dZpWmkkvP|-)QvbS=YXWcm6m$EN-;V`j9vR(aC4Z=(f5^N$JEoQPyrQ`n##F7X9Y+AyVAbG>CN~K{jo3hxYXcU? zG=~4sa>2{9U>ny*{omMibI0R}QS2nrr?orm;IL2I#7vABZuR48-RH(RzKQ42;^PV9 zODBpfhH5oGZ>@ZHtotWu(U_2nvyt1NVfy#+`X1a|$;ivnB(Rtw@b++U6&XqnSs*a4 zvD0Z>-wpbEk%hzv595GVik4M(uQ^o?9&d8P-+z!d(c+S-Ot$42d+Pr|(e~v$_>WDO zB<8|cGRr>VV!asb- z*U;QvS!GMkv73R5CpM{*AxW{k$-OAWH$1G)wd&n^Clma|R{Vx}@4cnPVpDfa!_-xj zx$OFG!8Pr=+c^DspW&8f|kSKIp=K;YgkLU!0Me~slV zn}h~Z4l@ouGaq?Kdw*|-l4#@o;6sDugGYyCoAtj?7JkHnh8~=iulvsh>d*M$pv%zb zs%zkHrbQggjA#3B*4yzD6a2ZTWQNz?V=4o9Kx80r#U}Ft;-tQ{pY1=%0Kn=W5>K&o z+2;uyt6|7)9lXJz0dczwC)qt6t&bXkzg}5K)_!pPh`MHCE6lJ7E$~}7_oveo;PHQI zo)>n_vCmgx|EgyIt4u}8*L1>vzs6rDQ7Uu zIRDM*C$*;KxAq!v??F0GlP_Kmgf_cdS`0d^n!M7J+M88M9{qkF5bE+I2xB8j@EC{h zn-ZbjO$m#!B(VNf0hY=K1G zY9be#S-T^#wb|MnJP`P0IJRhXzOWFjj!5eE*|V(|o+I8zz+VE_rwx3z)~NHNv%602}+mJqvA=fZWg|FHcVX^yvbl zsRo=t?HpJBa7R$yOTO%g%K+V_J;{rD)BDrISqYY%sPW_!ioD%xb)$AtJCppyQrIFC zse7CXY9By`jUUQkLbO9RU(N%LNN0_MVX5Oh=MKq%*A>i`b8oi!?sS;jBzR!*f1Wb3 zn-%3RJte;5Y2#V$V?35s_9-!T;XMpfcw^bz{+`gJ!y^OB;mJ!C;c1dr>;dMQyy@V^ z>CCE-iOxD1gCXVYbVVb||G_L*W5A53*Uw8eYudCnYb_c-js-Zt1KGfsyavIRLGY(u zqN<|raRTNPi1Fgn4JFsK?>N6IBL=L82RI4N?ev?GW29|dL(ryk8lmc)h_Pg{nAgkg ze`xnTb_ndoj_47bho_VyDT*RB)cly^MU!x3CK;6dKzw8tK<;=b);A${tUHi9PZyZ+ ze`PP>j1ZDNqi3#R{dxtwd+4aEiqwRX15}heq{)PI%*vXzUFJ8HTxk`QB)rK`jjyyp z`GjJa^M(kQ9C}Fj;I+2`Gzv`FglrA3Q6LwI|3WT^Pbbjy>8uD>(exDe)D#mFZwu7S zPz|9zq4fi|jGW(k1cJjuJGY;}!IhjWHq4ovIePVJVn0PH8=fsQMHco=|1n z()6n7U8{*g{B6)m1!aik_j_^>ETWrxl;%JEu>D2uj`i@Z=_r41X=BOha(^?Yp$Pz? zmToweU6HLuKD$TU$8)(e92A^n*obeZQ#!Epz1h{-THiIg<%2s^#=mCXv}fYBP}Ba3 zrN&vR4>F$<-z>rM`9Sj=jLX>Bsa2P5@6=?z$=Yy6aNZ37YRQW`c3WaLY51PY4oQa> z3*4Sw8k~*!$GdAvz9MWz5>?8mv59*6T;p)4)-kaE$T*W_=%AG^U7tblveFfh4jL?G zD&mGUsGu%c2llfD;)^R-$oy)?=I2>b$c z*ez$O>49+ML2Z@_cHu0N=Kcb-3JhfNd4V%G5O5dd!cS|HG>=H32wmMuJGC8Osg)zA?>l3=lPkici@$|}d$I}S-+NP8%QeNv_Mu&Oe94$;(O{M(3z zy02(gdXlCmM>5>Hv7Et3r5SOF-Xj$nfW{+0r$C)8ZDMIctGfam4_x7J|Ec3850qUh zr8-}VHV`iZ{7jEN)Px;iuBK1_Ha+urnH`)G+ZXQ1`=*5^&J%sRjh39>dfy^M;eJHSptK z2C6ioLL*3R1b_b=ys)rEdt^oY#3EZce6i=xzyuEU((yLFUo0YS&gF$1sF3sy54LYzp4yYTQ(v`o&( zf1|(>Sn3y(5#|-i9gN}bt$AwnJ(V8VNu|>lab$ov#!D1pr!X^7+iBwMrh7BJ%l;$2 zmMPN!MN$O)h>qzWV3qes_a%*V?N}1P0Q5A;H|bryz|67CI<)(~pLJs!Nq(FYGbaf} zGd7*+E9@I9s6+Rp`o3~NN{9Pu7+J4Gfpq{|d-vSqaNn&{a9{&kHx%=77Cw_Tkdmn= zFKA}3V%fpygrnU>;~TTK3b8`_a#fsMUBwf!~3)1@P0H7ZR(Vp=r;abhTf1@AS1 z?-lUw7~pSM;#=VI<)!9PaJQfuuv*0D87lJ4aKA0LIM}kom#WOzx|hbH*}IpAn*E)Z zCY^7uw=Wju{Q1%e5fIFi0A`e!(VyCC{ws$~bG%nNIb2=@;qs?5qqDihNhH93tR4CB zic69oU ze;y=j9t-UV3yCA$w8#YtLpkuBd$V4@1{=`jh)an;D2GZ9R^|@%aohmdv714jSekj| za9*WTwoGFKiN?mb{@L7FQgP$N2%X(PV}rFB(+F^M+o__+=?^}n z%$fZj^1MU+_Y;DHJrJNnt>=mkc3g@y`yEd*a1H>!T&1LSvA$8QLBJb{;(J-pz zOkl}r{`^5qx-2KroS+tN(mo4ansqyGu=*)E)jeHNHDSz#caWUASIWF8dVQ+RsbHNJ zJ|dtN8~ajzo<(BTAoc9_fe(ZCXx+z|A3l@$@&DQX579izZ-p{Gzmj!5FZpN>n4c<; zL&^53-WVp6qX^iyQnZGKej2@O>lz{@{}`+^!4kx%zGU46&j9Zi5Rwc{Hhg0eaB#y5 zFmarBvnsS;VLsQFqQ48CYPUc;o4fc_`!zs9=<9lsQ!A*_79ij@!@km3fW*>)fh$X4 zV8Gu5UxNGl7EPsA2;oWZUPPulBpL%~4wP~wLt?8q$z~T9Uanv4(g)^sgZH9(vnwSa zIX&Rc4I!6Q+u7s)?4%_5)u^8FeZQ=9Y3H)i^jXC^mvqWPKf$2~7d!rFqie@bm4c%j z22oazEmu)g_qGmTTu*PI-w*hDNH{8BY>ZsI%*`(Q>)D|CQVins%vA^+wvQjQVyhS@vs~iMO0)Xpgm2!;e|a zi+1eKpD=s8=DN%~;d3K?rYC!n(FMLkkB}+-6FnloM4TI@e`aI};gyX7r{pF|eeW=f z%o{en&Aq5v)Mq{|z1fJ^8s{!aiyZvVm9wx-rJ$wl3&Ev4@DtS!!ZuBH>-_l*+b<;z zjcWl{q(`0eb%6t@i4-NaCZ{N9s59pZMNDuk%ObyUKUuy9ns_CNrB3?|EXs+4OFoa$S4lz~mSDu98xvwVOyEJ7=rmDuuAIs8!Qn zI_=`lH6+Xe57Mw@rtu%lQt09+4p0~ZiFx+OC^RGlx*1xV_%Lf*}vCoEa7zC=X3l0&Ohhes@uuy zx~}K*dS1`v{-7DlZ}w1p zOBcwoxk?oq_`X?OQ+7}5iI2ATwnS6a*$L{zTelG&;y=AQ4-(N*n@r{HIX)46NqfNx z0XkjYjRd3>&tuZKKWq3K=-%APigcHkk=`M?zL~6n#)vtvtGS%Y>GzXs4-bIuOEj$ zz{RoizdgPZgb=OD7j>)1H@cZ0JyG53hYI~fpgh$R9*wx#Dto79Xn@S(MEP28c~GQQ zdQh?oLT~fD6OPw446YbyeHI6%7+)G$dQ|$LOKJ;yPWfvdKTqYWA{p9-iJfm!>`!M8 z?jPcVcp4_=f!m<;7CJu?^C<*YwByE0$W z%USaGhaWy_4yTGJ`OSd<6!d^tpYJgTsJ@S!LL;6%!GK@{1|OmI_RY#|`nE6OlWnW! zDb7|~tI6l?ZpmW84)@n48W?Vk6!e$WO8sJ~e-)gW|9t40^yN!LMNHy$&cCFk2c6$THZDQB+JVs$e-!XqBey-My&&~X;l693PFH&jusary6!Oi^t^)fZ-~2)*g8s5_@T_ zlsK|YLF)MSyRq<`qFIBm@?;|y-Mv9tli-?bp89JH!7Z%#(`%`XEamhp+eUn0HdfR# zX}Zwj*^*EHrqP@u|4^WJ9t{3|pjD=HaB}s)jlWv@_^8q5?$r^zI_;#11E7H0+arMs zunSUipZyR#vBZU;Ue(gO=9A-ci{4d+OJ83FT7%JmEop0zF46+m4KTIKz|{UTTaf&Y z<5j^)2?IMdyOs&f7o)@b0JId$Mg7J`_k!9fAlxYt=yUj z#tXXEQ@6zg@sjs=bx2-8Dg*~`{_e^tRGc&l=(~nmMhYN%{eJ(T^=85FA{h}oo z6Gg!I1Axd{CJPK=XrYj?dmKkTg|z{A-aU(S71GaJGrA-V!z2~0yo<2_?ntK=xcSR@ z6=w~?xOUv&plKCBvn2BizUw!0%wfk4EXHmMo_w-RW8qd?2~3M4O%+$D+bYru9${Y0 zVV!gHJ8a7O_*$MPo(xE>xPo1zrYn&;U-eS)0r)_yMcgYCH4 zJT~jc#ZGOi5}nslJxxPn?QiB#KRpOk^^>`gFgm}9n_NGKZ-;A5{|-wSc@uR!tq`jK zkchKI!%4mIWwTx8?WR$7RWs{kjyPmHc12wr0aqJ6VQ~lxUcDax1KJ+>+ZISLc4YdN z7o&(QXq9M}1hKx=W;fGW`Ez(j&&1(C%xjwLT{y?%t~B)q>% zoibcJ>?}xE9SZ2P%Lmu7oM#V)IB{bdA6&2Ny+6z*B-`r8W;-v0yrKRG``HHqyIhjA zCGsbv2ql|6t`-f=%02jqa$kkCYx~jY0<#a3ip)4rzr9TxDEfh5mCDMB$c9@z2tAB@ zYdiGC>3#3wbCyQRo(YJ{i4jL0^782WIl0(PcpuN(k`D>~LN@pX;}d`?Sv44n;kS{u z8s|Av0c6Iw=a-~gpnQ~edZ?N2LRMrw?F<*FfQge;2fLB#zcF%N-gH-m%DowgNLadK zL%YJe5EzOdC1v4;CIg{8>`;Hv$Y}uYI0F9Nk7-k^_jFfec-@h$jUEE<&Oa9kukt{j~T>qenhn5f+8SrnY2bh>Y4Zh%d~#(ntNKZJEYI;*UE7-63W z!hlH{1boTtO8JBRY+n|qiO?4Q2_$Lm5dpCj^!YAc*26jp0D@e(1zi&)GT-LT2Bo}F$9zHxDDa{K6S5BAY0KuiO1d*OVfvkPy zjlm~Wsf5Xn>`SEP8dasxiNeCB+rdLQ&5|6DQvHq-lg*JT?*HW&({do2TD?Pu0O9%biK7tZD# zgfk##+npY92z8Hy#@<}VUVz>RRN_Ep?%ltK{5)sg`U(X29oqh05|P(LB*lr;KU?mr zt?H>|n(K*RNsKu5EJRF>n%dSU;$>`$xy|Ot-19mZFVS$LpTjV?t}j2R|q(gYZlMOsuZO z`B3JjH*|0f7;Z7!grer#K*jUB4t(tyzg>r zTb9A6UHw83X6Nspv~PN|A-FiWx4lX3B_)>KM&EE?lNjQhrCtfY-Z4RWCM4(o18dS` zW-#zRPUv)AIVi33EKyYye618r&Kd<1;AHHDzVux>=~hYUfeCwkom`y zMr{SorG`f9QU0})Ap#bH`M(w-5*Aap7vYcy*oUD-16oy%rBgMJ&^F6g^&3Gyi@}Y3 z7~7sd5>0;I7NdO#ve(UI^a{D(wkEgg{D=0qr17Xnp=&C-?c5hu#@Z9OUkDGr+ zjMYj+VMydh%E!al-A9*J`(NjTax-NNlEo4uvNijD*f;mD-86w)}p9HKmdCX`BF%lA&+hY0!5eFtgF6%mA>mf{6?SI;mu3{wAGL%&l_L90KX$YFf91vY701F zD(y9kq=pXKi_vQF#^axL<)D?5{=RZ5ukysk;=06sv5vVgoS~huhL;FRcUe}blh^5# z+J8tQNkK|yyIq-9(EM>q54ko)-%}b96dp8*Ht$Ou8K1EB(+!cQ5~n7EsJfM>skH(L zDb@V-VYLEYulQ@}9j%z~?82riPjXyL2g1^dywLN)2^w>AE(#4EImmsEG2DHhv;)Ag`@A} z1IL$o+zoVpX;!AefZy&{i#xD-Cbk7X5zh{$UuvIFtUQM>OQZCV?-}Vm=Q~XIJLI7X zcCA79JSVhG3-vv8$E3SnNfQCCepI)`W$-}LS;9f6gh>vzJBqE;H;;ose?`&uV;2bK z&z<7Xc?2XUuk`PK=E7v-a>MKAdvJ&|&~jYpP&vd&7%Q2%B&}h9?Y#*27OV6Q zy%Ek(lSBSHezlVM(c-ZBh6w*oO%-t=V%f)#q@2D)m0;oOXl>6NyT}SuaI=bb863Xw zNIHui;bFqG)STo8?@vjD!zYU0E39{Dm3!z`*n^LS6jvca8^v+ zpv1vpk|@pKu~YjbT}A{HFE?$(%Ldb5KfaWf<`r0(-|zpTr#GtJFMFfOJ7;m?DO1Gf zia-?-{WgcK5l+PW?)~E$Yh^^xc&#G=8~O3k{q)B{@ z$##N3cy&NT0*(fAvVScjIB*pgTV`6D?&#_`bb!@V40I`o%gAb~;w%#Ra)Zp-i7{6i{ZpDjruj?D6; z&ISKcO$V4}aEJmG_srGic8Kg^=Xv6AL^re;2mY}bb;+46Ddec8cQ5*D z-`6J=aTT7mhJPU+RVND#7x?7QogOYt;*mrs#gr^)+G}|Z@Joy>`rs?{QRLQMKkGkQ z!f>R1+x{o``Ilbt-y#k6V^2V>31A&SO5;4%bd$>LoRcj-U=PkltmouSSU`K;R(Agu zk<-Djf3cCs-?*a%B;pjc!9g#ebEaaILu)PiAn&9~yak#(a+e%Baf+dInHwa}O z6VDvbLJDODKzDSdHRN`tH6REe_PgS&51+;rQHvqCp#q1(nHyR#%+GK zuHeK~cK7P35f+;M&Sjf*9Y^isIhP=rRQr*?!f%%`hGQznTi97QDc_Mant5_WdbAT$ z1FC`#+4c)Q5Rfk8D5XbBI+CPy&?#7|`ME)PN4Iy;xuDO*2Va9F=cX$+5FJ`$x=w+| z6h#*~7q7uFCX=0p45n6c$T^&(4rF=S^~I-u0W-Pk8rl0QbE42K<71xs0Y zUFFiXIUp?bhMGy!CN&cv6!%tykbUx_ZGGgj+3bDNpi^D9`+9Bob{uVx5DE*^esy%w zHg&N_?^g>r9)1Mqx=AlZE_ZrfwvAQ`(DR!b9pNu^4J{SaL&EZf-|ZfTSQn(<*3eE2 zJ;qEPmGlxy8_G^V9}SwZwS);PTw;RM4*fXKX^*14st0Zb!Z&GrvP+}F;!MSlpn0+? zMf%PmG?WlYr^~5iX4J7PvYmm11S&|&6Yt{;EDSy!Yt#QSE`wo+MVn$b>^B|=FYhKZ z<0h%KlmWYH=DPgSevdvGwSXRF*K>^vKvVc$(z_MwvE|YQph&GUE_!I_np~0Q$Q$;- zrau+kI#?>I_+uJV{h+}%QdJ|b9NK8V0!TI;f>e=hOh8UA8H+Znw8ZNzimt!ZkFGK( z*L{!TE&)XT(;bv(?T0?sQ9;^DS{%1MrAz3|WdsD01rY~>OMRR&Z(UGNVe|Jk3U{-N zFmZG_q&}5yvLNWL7v#@lt4Oxn`{5S|FoxZD?C-yj;oOuPOMK5@!CCn^l=HhBthb9B z^7x}+)q5&#g6PDa-8I*mKxj9?lfFrzQqm#KM;3%-SaTiGnDcYmh_n+b zR#~?@LwbEd;Kv66rJyS9hffxz);wkPmRlDv-+mOy^mjyV6R}>PFI51m7w%NeVdxNauhR^_8?s}{!WKeeCpo!!6zn3!Hs%~(P88YGbWU8hYW@UPt0Q3L1W&m zVR_wIBtZYqyszgS*9#A8`~x;b><(ee91-vc#H zpgX_Hu+&+5GEZ*)L+`qq+puhRVA*!Fer)zH%=*{Pr)SI@B(zG4z5BUVKGA^7|bTFb9q*yak9WhcPTuCzO6)eA_pf#&qU$d~|b*EdtzOh`k z2;eoCJ4#a}_7)Uk;fgymA?2Oev!~`-o>~Cw8HwwgsIk@A+GZz*pXDo^KqL)(d z!zp~XAEXQUcQ<6?eTcMdKmJ%aUGkr5-d~TWo(g9bptSi%MYaS} zcJ4@gAvDQ-h{{FrrbODf?)dz;S??r=^39~%re4*$b_rWuA-XRER;dQZkMR(46hn9A z6ndn~rdssz4;R2{bm2T_9sSCp1qw&x5N;mQCHu)0YI<@(Glm_)ADG~-Mf^xh5S|@E zEiEgM=VAU8q_hPreDxVqy)soNG4Gsksw(n?sPXsA6Xm_P_6a~ujRn?AbiPbEYysUW zz0JNx5z})(ZE{19SI^+A!X^D~+|kPX$v?C&s=d5$r^p zS{U`v$oekP_Ac*~K+hbRe8q`EDeiO;#ip9<1^q+WV)F|S1iy4@RSBDFI%Ond`sk}^vu=#NU|jfZK9CqF^VQjeYQT;F zK-{@F3(;rk@)_i71^h$DQpod}7DvL;xyv}f1^H&3qJEpXuW+K9ImK;J6tIudPFSww18$3%uz#|tcD)@CW2&*&p5bL z6AW*Dsnh%-sb}(9IThsDhR6x|1Sy3M5!xR(14ISNg9D=VuepWrBJ!6zqL17X^-Cus zEXRsuyy67inj@I4za&jRLeD}o7Rv6CdCzn*Rx#L zS2d#l;v0Oy|Bk~jl>dPq>VZyT^>kzK<@9R$K0l$dV6BHkne`rNMf)5LICKIGwVHOB zpB4natPm}X;4M`&Nlc&YaDrb8ZabG1KXN^#Ugw?^2o>!uT67qfD8KktzbMLs)nSFn3QK64qH?e4 zL33QpjxNkMyhc3Gq>GWCUJ*2|DYdYiPM~DV{k)pK$<5%)!Ly2Re+?Lx-%uRZxc|vx zI_6MZZ3C$2Yvn*U22iu{nBjJgNjBGL*)Q31evWgc#~nz$50tLV0@1o6C@H84mTkTR zc2ytT81d(Sd}CzE8&+fyft1Rz>iM&;pGZamQ58b%x0?Esa%iWcl-@$ic9`^W1wA6l zZ?CQ)bXnlnFAElC%mF{juaIJRm(c-$j{)rqtpXCP`hFKGzjFKzh0)W;6hz_4NtHf- zZZlnQ%j>X&<~RsVt2~UYOyKFg>%ouww#~qbf5DCdgEwA zFCVZtx*?0Y63 zcBPcA@QVop(wFbB6nF8c&^wM;&3?(3LfT#~{=%_t+;O3uphD~@Glz}K~S}FcwE@k+}+C{4t%$Jk- z4_n<)-r35yclc4;L!F1xrTeYqOwExDe&F_4wU?)yyqgq_{XYYJczVcEq(*A!=j11` zAEZ7{f+sQf=fh#OfDfl^eeu>25yg{1;kkt(qnTvJmWp4)zmRKeNQi8&MK6$LZ|nOM z%z8^ShGO^OoE5@jz}V+Wj_xaV<`*$HyfU8B2qU0e#>sGv!^$VhVcB98z=HInaay{d zmD!_x(!QqjZWkt@tv9TkyS<`1Ta-yY$XnC?7SHvj&*DF$mzvNSv?Jf!87;IClOR!m zh~^`VazxEc=<-o;@1om&AzQatgSKQh>tR|52Z(D)mH6s^80~2X0M~}UJjmYs2y$h% zUzS&H#S+%DW2~Gv6^!VzU0oLz&M>Yx)%Lrn=E|tXM^3k4&|h#H z)o-GaR0*>~hWNwCj#M*34+mOUyavsRcd0{jtgXcdW~&L#3KD5aS|))JOJ%YA6`CK0NHD|*Gv{Gx@!bICRnPhjf$=!)k!F+~S4 zS=zg*nWPI7Jpwpi6=6P)O}w59HB{tGoG2`^Vn;VW3%ic*fts67^H_V0w9Q!b98f2EbVHL;MRu|zRTjTtY{}xw@pb`GS^I<^jAQ|Qe50<%%rDxek0$revqE2A zm1-VCc<*nH7*Gv*($J-@yRIm)%Y!8r+b?Q5A6XJ7Y-Q&IsjQhK z3)Jg(5xyyeD#fSvUJ5F2eNIeIk0MMdhAtt@?hYrcw1(}a)G~oVR$E~(ZkbF@BdNq> z&p5NK)F?PlUJv=W%X~{Ng=8Lh1-0kLpQ&AIOCDz}Q z`$M@wwv_UeQ~9m|NmQD)4EJu`=Q~{ZF0%!9Q5&nzp<~oq#@K>#QoP}{xC7-Q=7=9? z&G$cn)zkudqn^K59`+^6;V_bnD{eq;M4e8Lm zqyX`^dsy3U-1*p5QG2QY&s4zVDemDQQZQz>T81C=jC>|zkO&iFt+ z4-`a0-uRxHC?nP{#_W6@(KCkI!ENV-{WOQ&M&!J(iE-7Ugp0YeF(kVrd5*Z3>147&BC0~;!r9b?F_cextW)37cAh3Vv`kg42Q%JcA+_R#k zYNP!*=p=?f1sry7ZsLdN$oxyuOB- zOA*}^2~_T(+K`UcHT3H20(Lgo`qC3DUtz6vOuk7bzyG{5;s=W+arbP5!!7f7Iaz=; zBdt*@Im!Z{y1YRoml6Lu3)@iGvppOz5CXK zZ3Yl*U&rPS+fttCldu+V-ez)?-2wXc%+psJGWxtUIM|BN;zz zCp2T^hh$vWX4&>^J{11E-N~s?M9J8zJu+ueD3C!dz_}jhHU*lHkf&hc_X?qqT4p1p zwklk_Zo2qK;wRVPM-EX%eU%R~XmApwRZwo2{JlcL#L15qM{2Htw}@EH)9~(H7Ypv& z>@lVp9iONf?iQDYH|-=1ecbbj7w#cdS8)T5er5rcEaHcwjDc=}0+DjwZ9P(}80J}~Z;3_P&A@itu5 z+hc^1t6(q_^31fOFP>R=P8s%{V7KL!d`FoBtHaDA0#-#kWEMRMS>D>w$tcx%;(g#R z81x#}Tz?Jw#KKCBhPzktcg}ToQ~~<8HHqA(Y7Mc-A!vsdJ}B4a_>Li{7@wtiPWuen z%krj2HRF~fH7N0otouV*nL+(nVI?m$H*F+|m`a|> z$u6`@uefNK%@{X7)F(i3hO?!eeq}{AuCl9FJUBU{)IMQyidN!T6&wBQ;9Qsb(!u~Hz|ijhI6s9B3a?bqF4I2_*;n2zhBe2H1&KYe5F=Np^C#?*)9RY_zeYJ4Mvg#m z>^_W;It0e33Qk@&wVS)_3A)-YoG12H?i6puuQaSQl@5eo6IA~S`) z3r_7$aDp9|y2%-@T#ZurOexvQLgxyHaD|kKXvJ?KK&b;sz=}*d-D>4!RdnSZ>NQ%e z+^^2j0G#VNUbWHLeHGqphVi4tyE23}MkKR#Ka|oo(KapHIJL2`n3muR=>kCU)pPun zt-BQ5{vik;1Wi9~dzeUPtvxnDYAC%{wnGCYWSN**Us6(D{mGo*7nZ!Kv1hjlT zAxtTBoKLHeZ#*wZ&a6#8;vOw)_3XVc?SQYvFIOHdWFFRDDE@Qe>bcUUaQz+ZTUQ;U z-TQ3#&MMbe(q@fNe7VzQDZC1Fc}hVvu_~s>*a)NpXvsc9l945TQ}l1$A_V}%Zx}9z z8a}b7nll1jqP8cVJY5T|7C@X*EkG4MRG%I`WuXxAam*nMkwgDtJk>T=|O}Kq;DQT=7d@| zr+w{37_8l(uzvw&h^;Yw9tindnSGPyS2v-4{H@Lf)0$7%ey03UTDv3Ae50v+fp*6Z zWcZYr1S39^t5^pFtD$^3y#F+%VsBwq9Jn~gYvyq3>Zs2{Lh%`0FP{|o6Hx3pTj~?Z zX+uQh*9D)vTR!T(0hp92X)vGdB}L?W)!F)SR_p^aEs;mBNvl<$30(qpjZwRwYMmqb z4X${;w%_PWzkBAlhVXshTx71rt;z2I#V4}erID{Hzuv7iKTGwR=g`$?SV+l-+&sud zjO7+W{SuS$%BN_f9u9BjRB5`;O+XuN*rq1)aI;l_L3(sbH8}n2-EZ4Y|L_$vo+FfF zSDNMh7~#Uq7hmWNx}KPqQHFma?4s%NOS9wDf1SFhX!__MRC z-IHZUpjY+phtQ?Ou{$ihL&_aOUOAF#?i7~p&e(}vs14@yv=q3FM@_s&?r^-PWeH_| zapoEi;gbgGS(Sgg%egdGN&M z+sSs|T*t0=Fy~!4$8a9I1L{D9Kb9_mp?67tsZIR}x}h=PO-htN^-C~-Mh&%q;8hyA ze}J?D>Y1f187n;wahR{s%GO$|DZ32axRw?T5y6S|DZnb zHV5H;iFmW>r3kj2c+Bd`{^RjLuExn*`oP|n^Xj1aKYEPj9H3I-MIu>pMac7KLJ;|N zMF&yT=HN3rjqd2V>mIUMzIfwN{sL4#EaBF{Ha}`*PIEN470$@Vn%rdCG=BFJqt>*3 zbDv|n_d)=Dd3Lq+9i`Bz%b_V4SSL>k236CBdZ*2dC=HuO?oI!$AAq}e#e6=FFDE%P zV)cPWs$Pe(0R|OWyE^|*x{YeR{N>~^S4fWx}TOU!A0yj#tkJaf+VV&j7b$> z2TSvpbeJ2pnn>vBfM)Rt1;$9SEw16YIyw`Wg*Ld3 znf|o%$dF({jrE-_A2AZ80v?(}?BHOy@||CALANoIrPOIVKJ$E6ahw)3q1O3OO6Wx6 z6k*X@rNh=KWK|d5)5^8xt<-tL;{KYfJPb3|{r+Gf5-Z=+%Jb18A5q0Z{Fe3OjqUmw zVWy-#6?s~8^o({)S#b0hsK|2yVpB7^MsmFx5xrLe$kBrY#kDY)^cMevf#a$QHrG&v zwfV`Oujj0qV`M0@x@Rtth$0WAOAfA2-xQ28CL`^N*O5f0nP=PG18W-Db^9XPCZ3Ua z8KNeGdV8|Y3(hS~#T1;AGNh)+AT(KVS;Xuwty~{1j&2>uJZt#!zW&YJJ9WDjMyye6 zab$7-(8j>>;aW;-{cOc-_ET#7Y0h~65(}>(*l>ZM(wExt2*H@NFp+t$^vmq7<-ZOK zd0!+|j~04CIyq}&Ej9zQ;$J~`%=0bC{~(dKTr*jXx{^Zmj8Wia}&@Og0>Oj;o$ z%cBf$w1Rc}|1@})Jt8J?boKE6xUp9Kv}g;6$5~MkgY0PPe(+zbfkWcJTSmPma}@Fj zohwbUYCW>^zpoDzKzndSOVi2_3FA|5dckqa1AZ7SA_(ya^BfY^ZI*4PhGbZnC`H0e zu4nAR?YN}#t#UAo%*SVXs-Coc)x|;}D@ZRn4(OfK+~;X-&m?gK$=DCoKkIU(0??I- z{xv)qaRKk9+#)8SgOpad+p*PnkNm|zrr%|GeD}SQ&7(>-goS-|eVxWkpmHWKJZM43dG;zmk%#v<4?p2(+-WBz%25U8uNbvFCPdN}CaSBCnY7 zA5jNMb84R08+bW?yAvqzfND$knnO;2j`MN%!v|(3%?(4%E6D*$~jLo;NZM+7gtqg#FU~E`%2F7XT4!GF; zgXa+*HlW271Rl*%lheTD#orgv0uLv`}MnJIe&-iyp^X z`b>T-5kA@KJu#(5E2@ar5#KDN*&0yQn75ar^78#1PyOu3H8dVPj$IAHP z#0mL(^H7qN^Vvso2lu2n*I=9{OvFpt2h_aNFU<%XWHvncst@T*^6)lHG@;+>HtAde znV!yrCSkQUyZ)P|q2UndFd=_NctEn+VU%h|`S|?8J5%Rsp4i2SGk0(fb>YrxjeZgAr^_MFj>0T#ir(oy9*=^;#+3=OoJCVS#hp>rY8;F_&I z-#;CyoBM_F3N#0O+j;qf6N0BHE(KS^0qFDIymBoOjGmV!PQ#e3E1E)=-zhC1U(vW8 zJ^Q!s#}%`)H+s&~O>Yhuen0N0d7`xw5!8_z6wb_nMpFle-BPM$KFwaxMfJq{>5GS) zz#-9hRNVI!>M1mt)m)w(2JF;# zaau^P_vOWNFYJJd%I@^^zL3%676-GUp~U`^wB|7e(qrQbLcT3|r{pnB5A2%VjlBQ~ z1qqJVTv%_J`ErIB&q3ert1hrH`aI21`%%vQFslGRfvr<}KTTh3@t7%*=-o-T*^w#O zkAEKHQ)pZ8%5e@s*I40voJBqnv27wjKt#@;3sruFbgu>HN11QrV0ijHdjF)nKEuH9 zrkJ7q8JGz^%VyGB+o)#kx*0&q^@4Emvo^tborQ9VRQQfeNDCHZi%j2zFzWw=`sd-i^b=s`r=1T-j>5Y z3VZyx`(H>tR7vRz`EdhZ7Fngwh8pyR{3~El?nCG(A(RMfF}@2cyg6*>He2q{V_`?J z<2Q<-yeMHkFA72YWST2NY!~{JPZ<*DezQaW-^7_m8w_ zE!4XCJgduS6Y>d;sdYak^1fj{cPPTSUf5q%*0wemaIXYU%a#vzzrP}lS|I)Dt(R!B zZ@bm}(~EsfX3fm7T&e+MC{jFc#Hq}}5BYL5^cT+F51BzJg-XIKcxMZ)cy3sW$#-k{ z8Bx5bCtgDD$}wgrcPE?%$LTA%F}s)?R9bW{VVL>G5K+res4)ZVK@3G=HDRCwu^~z_ z-;(xT3e1vY+@%39QiKh)SIaH( zUCfMymL5=fa8)%)ibAni3gX{dUNbdFH zjgep0`GO!-3Q&Te5m+xn-N!nh5VYi_3rJkm*_~9$`PnZk-l_|o;zpI$)GF<(%Z44? zz6IUj4Vf*BmT_6$(R2quCml=9+&z+6!jigDB8n-Xkb>4}YWbSdQ$^j=J`tGcjot-ewZXFcXF?R8!g$%p2S6d#hNk(&&H#~94 zZwu2x?VjxEw!i$qbwAIusl@VI!g$sc*cy8ydjU~C&}+e_g8_W-Sb9f;&}&qJs;mz&(1~a864Hqhxz;RS!93I2QnzzxYJU%;^vs6e&*5?HFZs|lU z6@+_aCh_D-C1vuM&V-hS`^W7=cw89)?XsX};#}+@guxMHct8#z^4KV-SbDS126y@7 z2#@$>$?H|49)O01-#`+sXWv>9&hQuKF+zJq;6(ZnF$J^JZq0c5ye=yGHjNecb#x0W zE}pDzkjtIil%oqyYaGwPe4(JFuKTm zK{aCn5F@{4Z~$u_QcMDR?^Qu5Kq7j!bY`-5ymY1+aF_nRx+X;5Q1PiRwY2CZ@UG23 zq@!moUNI@|LuPP|I!PI;JL-^%EdoG6bTIb$==Wr>T{j z{Ht2I{q)>tF+;1i-FR?ZFeDC3Ux@N;GkV0Ns1F&%HHzJSHSb`n*l|MzI;d;J!yO{+ z#R?!I=R?m5qF9fjT3Aty0omZSLB`IQ+Rr_z;3v;(*$4OMh&y(Ko|XII5x+ya(0nSC zeMXlp@XmN1NiX+H&=t0vua$dJg_O)z^l1>`B&BGOJGOtfT&3K#)Yn_;y5hXwqI>8$ zq*!k&tOOWBQstqcz)BgcI!pmP_Y%eibL78PU8IP37Zls!FNKwckmgoclGmaw9E3`r z)B}w5OiS?erWDItD|k1fc494S5;yEudCG(9iY za4?UQrw!?LYnwK!XKA!u``>Ek;;o-sbrhQ!3{XxOPC^I@+suLceG*bril9&SXpQ?y zaPZ1TVIh{Z_8-R#je3~?$J96ohTIf3-UK`<#4TRi=NN?>H%ot zk@S*&tgxwlxH+Ok_xWDNBwBkj4Vh)@zYZ=Ki(ZHNOgt_qnePWAXD>+i9O^By^y5+L zU=Fi#AJh(f@Ahr^k-9DJYIIs#d5|)AwN{Hhr>%cM=_C;VaREiVSxLCUq80$D-ITXK zuFF}IO`C=&->zLVWIGZ`e;d6^R5?7gCOA(`)~mJkQ5-BG<)EV_=rX^r*nrX3@{8eKvMK$w#QnrYqqLwyrik(1kAP0t%|wJ<}U!{Vaih; zwHsw3ds6BdzbmNg{i}j{oX{SAfQBu!pZN@TckK5<({?13y~KSoAlI!4K$pG1W0CmP zOBn13Algf8-w`@c_@`g(8(&LA>Xsvb_#DhXo!ey3gByBnkEG36$wR>1o6@3Nn%kM$ za62U}T9aZ0qrA&^CTrdob@p{cH;X}g_= zbX+M89iq=GViHYBUp`RQehxf!M~+p^2g*Kz9|Ef^{CH!Pas8#QD^AEd&Glqt+YGIX zDMawkUYPEKLZH@s9osirnkoi!IoyGwhE8U7Ll#1s8lCffz3(uNJdycbk?igr54OXp z;&hpEEhr){UH`t0&VVrF4mHK&Xt`UykF;rfXj0gRbm1Dc_=iUV2Eg)clL63WdoAo$ z3!rG3=`zW;j|a~@A4ro!2*ZqGezf4S#D0{-DdylJ-M`2c2#7c{1aZ1IYDKC zeOm+ydj7VlZ7p14$VJk3ck~~>ao`d5OWaFWd`J8!VjrmzD4Ulw_(+`dQWq}#)Rr^y zpc&s~DnP~)FGR-N1b_D8D0cLy39%0e#2o&yCilf3zEYnt>4UbkzdJqTrv6s|NU+(^*8%YJT_6PU|6vZEO(`xw4i!&ZwPOQ{AeO&>v7_VUqxXwulfGHYPLt8rZWrb}$Co{6&U*WH*HQPY zk-klS%&{0VDKNBz{=Yr6JZl(OyI7S4w*Owc?g;FX{zW0#Mdd(u_o<7ZP{CaCEFXGT zA`FlveXpS^4fNb=8#_Um&!AOl9Ml7MXQ8cQYU+E{Un(}YD+;v>&AkJnCuTtOM4G0B zvz(2@LLwfb+e~|%F}P>Vu2rnF_S=$#%J}K6u;+lDfI0!Ib5RdSO+eEz^l_kx&FA~W z=lFV>01QxcU^9I$$`D3rOe9G11oP^O6|y#|x)2~_)r5r3yz1*DQcr=;;T?N5eJX#{ zSP+;lP^DP!U%UD5suVV)F;&b`_ZP(b1%UuohFHVJJQbktgn>bwG^PNqd%LY<-?ai` zo9vm(9lNvOBnA77bOFs4F~`}Cr>< ziD&QAP6Sw*v?LXTTgB6g*E!3KV^L@uv^R{zQ~f>FYqDt((7T9PE9o6gD(86{l$ssl z+6hrL_i?WG2U!dZt~=B$8FJovwN69wp+;b{pS;Xkm(W%JeKju#tZ6H^tds$NcP`g* zN#7rA>O2tR1{yvo(YLcG4<>si#U-Yj-bZFhooaDt)ohcoe*#e4CU@#@M(@s2Pnmk1 zHxTh^hI6&V)FGZDtC(a`CT7dq~B=X6YT6Zs7{l(0nIA87}=1u z76|rdQXo5Xb>?;6NQ5(_d<9w!R+X;|u`r&1ufqS^0)Fo@EB7fG)(gHYWCAk%aJCDP zFVa3CTUf{L(0$htFWSgfF5G~?MKgwWAciZDb1V>p#GN1wqbq*MvAzl^aQYvIh8{J< z`m2?E4Oqy$Z{wGec{`U<&H_9_sLBHCe|m&heN7fXO4txvy$onKjty3!3_$PAngdWk zqpPA{6$)>hSM@cQK^6aLu>tA~2(y&zo2V)my4$dY{0a9EaJB?dK;FoGi#jn! zDi8$(Jx$7^4s$H?5m@0TFWvvQ+T#v#^+Qy|*!(#~9oFf>v#35jN9{x#VYEF{TUtrw z-YKpP{f9dn2`a~$;@R;RIB$k=nT}2PqB3HRhR|6Qm3A}?Uwsj@I36Vm*5%RKPZImz zhGAi9fW@s0&F$7*)4B`d4V4SyL|wRpLwmjSXF%fJ%1MK8b% zwh8l~?7Ez-a%a5W8#@0bcA%CcuZ=2y+7RY{R%rUKbH#%C3A+pCa)6C>ou4uXc%QM9 z4j89#B?wcwKb9z_`4Teg z#1gV>cTUls{82w;9~j)cr%=AGX=zv@_O|+i;d`xPcJW9e``e;{1?qEf0WD-*D2v$D zyaP0WnGS%T;^fEW?Ug9-joTzFVJ)CKgDGzFZ|OR=3n;F3Q5EOxoE|#Wl|N|mS&Y2t zjVB5iEZzh{ijYig=!q;zW`C6hrfrENIy&s;*x1KpA*|hpG(MDB?eEs_CY} zUmX{7?2W7fTdn}^v`B)GJV1;-;X7RMoq@f{OF!!%&xP-YMY7|92-%|sb<4F4>M;kAWGi>4d1;0WuqjCB|dce zus~7Wk0&~!9Eu>F+1X@C8zSpu1a{)_OKZsOymWguVH>+p{ELVZ*xX}r|Ehdmgn1vF zU#sf(t*)2tRS|9Pf_H~T&=gp+WC)A&xEi9a zLTmDW_@lv09GK@ur;2>TRqLddFm@$k?Sn=&0S~~&6kdf-#pe~`ZElBB>(TG@^7h-$y z+yH(OVApTAuO5901kHp#ywupQ7Kf3_LHk1!;y}pYt+K#);^BwsnbcqAob)Rs9!8y2 zjbbXrtelB)#4YK@f1ToDsUZ_}Wi< zecUk!2$+ZQ2!Mq98?o7C3ys`eZ*U4k8_8qqr|8lib;E_~#vT9tonG28R>O-476)Ih zeTr5mQWj7Ymik&`A{m_D$(js3Fqf^TEy9UdHsJ=~z7PGE3gl0T1x)~d7W#a@cA}aM zzxioyQ&ak+$`jZtOp&yd#~a4Iia!hs2oTX<%6+_~`TE){BcCHI`K-E!o6{hv>u##p z9@=KtT>&wGV&W3e2)jAoYK}knTkk`@!zUi{q|mTHU_-Y@#7vOu%t-Bd(SLQw|4&#(ImG8ruR9tg{bUTT06A<|F=WSC#pDPJr*UL3tm7rRCD_9 zg4MUrzxthSvi1^1m8R-^@jJ9B?6}Dtr@{U%*L2#a0veHV3Wx=`XwpRITt`JUcmHbl zmJxjTk$=5WyqYDF~@gN10Mt+nk40Oq!Q^u6=nM)*K}IgD}!z}#_X<0BcLLyWdTrdd4=Ym2VE>)QMO z%#WL0|Ksj1kh~IX?Pbm{fv*B{dWg3k8V!RYPC8%trYdfaEFi=${yKOe@$F$BtbyT= zzitF?Ghd(a+dn48D#0UW&loUSgMO&OxyfB_CZFASqS#>~R*UcaHt)&IeO_4u`1UAO z6?f0CQ|Y9DfL|U{^}a8DpDzVCwL?F=xBU2_bPu!I^u4h$D!(^R116%!y9Ms|BkOJ1 z&#Mi|LJGHZF{->?nYZ3c2n;&=NVpJz&iP3g^FT1gdsKOaZ=bBTRR$5~)Ias&dv7RK zpt7`%`JI}|J%K#iHn{P(8=%u@V@>3qI6am7L>i%;=6rXxo4H$wuSd*U@vPeLlcIPtdckwHWfNQ*Da+sVhAb#|3^ip&~T>~(s#<}sb z*i(DoV@p{66*B;IF$0HW{a5TKTX!Nf<8I>n)tV3Fo!IlYec`beF`LBwN~ULkY+phF zn^diOXnq#bIiE#Z{XaGp$hwqoz3i4CNRi=Si&pU^HylV` zvKJ&dOP?nJO(I7t`#dD}lf2^i*+RV6Kvjc6)5<=t{GyJN{ z`z*G1*NnVF?CK$@&>fF9b**!>{vlOl_XEvwr|`{jM{Dp}`fvaCTvbijiFZEMLcaN7 z;cBDQU4t!yn|`|q4)Y|JM3csKFq@DjtSzR!pbN??uIn%l{CIXgtjRpjiDHJYa$2ky z_xNjKm)UOullH6`<^M->C2i~7e{4&cx9xdn%bW%O z_eIAt5Af?sXin}@5|ajW!p!gwpzMc_mb+tMH!;H}qr88LaTViX=leF|lGriYTOOxA z-gE8L7HrIE^xkHTx$$mi=+A|Dy3EnOYi3;K=dci_9+dN__W!3?$#QGNkRn zk7}g~_E4`)$o9^%QSc2RNK(>XS5%Bua1V{&Tlb*mjaYoxx z!MXSVZ2)$@*xl_@r%Z85+BV~MGvqlDnZE15#vEMb>Kg?AeTm0C10Q{)IH?>ZwswR! zer=i&1xwZc4}Og|d9gx~$_l#}@!}x=?J!lsi2iOA5x&q+e>AJ# z7c={?-!fA>cv=34&$YFGPzgApo7mg<@KKLO&F17~JGbol?VF^FH-CI`>T{h%D===02gT3w*UWu!00zU&! zXXnfGsdiMjmvaV{epy^I^)po{2@txGRbeKl=BZ_ z&hboRn-_n~ce(2ezx9@KU&fzh~!ct9y zFQDtG!fW~H0Lx9aCpxy~5DK&{U}pH&;CT*O*Fq(q&cBTMs(D?vNPR1 zJ1*$F-CC5n-gz*xjZ5iK{>eY+y=_a2_?SA2MJ+gTNS3Mvgxw=WcW?l2V`RhWSeV2#X zFFc`@v&w^jqIr9Y|C>_cXUpWT|AQWUN=Or9<2~Yu1SO@!FQ~n2#P;~@`MRiqYT$eB z<3C!*n=L4ZLIA#(R~q?IchG3mOolp|CDb4i$s<)-h+bAujeD3Aqg<;v*7ojb{ap1b z)gw1nrmBV3T0lkhS{cUEiilX&JdZ=(r@F>{1&-@w>0RXgwABfNI}}Q-6G!8(H(r)~ zz>eIW2xA5nSb6Z@e6h?j^YA0fFBCEM^Pl%uuSVFj)Ej@|=oq0gkKXqfWn8sy2&4@> zvu`M|w^3epYY%+lTXW5C%rLiZj7D1V3$LVXda7mn7EO1bd>Z{I$h6-C5xXen@mSmK zF4nMyZGt@%*&~bU@!&8yp1c}$ zTMtf}VFmj{2=u6IZ;+(I zhFvDRvxU{%uuGO(Yrk}0wxzW@4aAm~8=9fA8G9ZEWQ(rd1T z<~Vo15_VJGqGzG8*RQa=dHS(O!bc)C%L_&n8TTFT z$*pMn;vuZFLJj^@*=(Z6S0d@eOhr)A(67E(3_&8CQ;pvqTaCi-ONeNGG@es-gBS7B07tT=JNvn z7&q$=^&^dJrrttQ+7xvo*^Vd&`Dlm>>yIFsfvsyx?=gPQM)FJQv%cAKw5jU)>`WS! z?Y}+Rwu)s_(=wB!`yZ!KQjIO~mTN^Af%G76}qZU59;{6XF_gdH3LCC4JLn+9L zj@6r29sXg%tBN?@?qt)XkaKzfo`VmphtaiZ&We#y25bL3nBDIVrVn&BgVhj} zs!w{kLpwmCzI?(SpQ6^S**GwHHI3^H60;3t;(_@Ml=&Hv1N3^F? z*K!awXPj-A)o9}YdX8Q)XBw^XAdmq(Oxxz)988HcbIke|mKJ(M5|X~8K2s73;^ReH zK2fbzSut7>^S@uPo}ZxJM4Da=8S_5k%^(_rC^6S;vQPEv>`X26{rt!Csri?vxuMA) z4cW!Xm-NuIxV#A?_}+jrYs`pMkZT9G&{v^M!z`pI`*xww4>?kpx59k z%9V&L5tfS#w^TGbjw_+DbhjM*!E2<&3VJ4Dt1L4J?Khq2A7S<9e_edE-{qF=hmWn- z-xju~l^BUID0%W2CJ{%gssl2dDFjRzHs-lGcxxYm#YzfR5lA#uizf+#jSI( z*-y57AV$vKED^0Dtc|asV6%`jhfNGrH)O{srSnS!=>lj!rhy|L)fj`-jTF>#6@|#J zBym2Kk<=X#2LijIa2AnXH~KYJVdHA*V{C@>e?qGR=Xd>RfnyLnU#QTlOpImusGQ8> zoKN7mnZ{r=IjjOBgL{7#cj%?!PWU!gYj!AXeu9!6Bu@J_EY1PiwlztrqhmlvP9Qkw z8;$;?{nK0dVp5*=9_v>i%fE|Op*nBZk4$RGt#eL!j<9UNY@_c`VAkxN&I((-taJH4 z#C0fXD%bitd=FGy&JxjgqA&K5+qtC;DzWq=5y{aca=Pd^%6){Z<^;#-s`u?@M|mdLVUeEgu@-q2bV#{qFVunT%yqb(r!ksIDy643+xQ-Mr_itnF4 zL3pY2IF3Kq$$AQxS`mHnt_5qjIHwUth{<_!L?tawCDk0fFJxn`mEwuUmC9xqTLQ9V*gfPq=JKq^ zf9APu1Q~SSd=7cj#{cB|@%Hj6jVfiFp*(?)Co`0Wgh5kh|A<63jrfpWXO~4IpVeQG zK|M{p0G%H2@#Le~78e}dL`_|yY+iM92boS~Bkc2W9o|9gW+F(-q9=}X z&(cmE3c*1O$b)LVK?O(;{zo$0z;kn>OtRl`+wj1VjBV8##10+>()njaY`RntBWFt; zjSf`YjvBQc+YCIkk#5X$CvZJtt+tn`an@MIb32Xsg}b8yER~IRLySRTOlkQKY%gno$DM_NIzvz|hGe0C>&Qm=8yI45FM+qx0px+9`;nNiK zRCH=Fx&%w(7Fs43!`Pj6EFG!{Y~PI$Gv|l<&*{7}nCS>-+F+Y)t64U}cQq*%_~~%% zpr*L=Z1e36tLPrGW(ly6k%^Ks5>Nd{rH=+F3ggeYHnAW9^e*&F%dInq4&sz(*mi1Z z!;V<$CgiJ05~?r%aU-@Eie{1z3JXiZXGT7=j`+^fujglbMAq*u$=#Pcxq(EP0)N&^ z;1xxY$hnbT@l4RU%dgstP4k}{*Ot&VEY&WMQZ3raB z{TcgZmMxVS`6V+CSkDv>PdvU%4(+Xcl=D!{6={y#=cC~*(f120pzm<#q|60lQC-*)462q%*)+`>7HYhl8e{geZG|97;F8^KoZ9Ua8t=F4{ypN6x$`ec z+br;hf3O)KwK&^BDVC$dkYtFm1OA~?qzY*6t}i-q{wo&hg={{es0&dy-{6$3taWEJ z^4Z+B>pK>jTM}~62WuMndNXzl!rGb1Q=u-%c(CDdN`#Pj3^km&{G1yxU3FxRvjfRo0T#t$R;^5gZCh?G0kisvu&_uGy9e5(JLfufd}_hsMLt?CoBA z*=zYLZC^%`-UWkwvcuXymn^>+OtD|}=u>8Jv9Q(1niDsQ1h`Qc(v@y+ zMwuSl@E%4mp?iyoe_@_tIcPSqvztVJicQ$M5LuNTrjlmN2*!2Nc4_0<{Bz}|JX|G! z3_QF|SlZA{Lc}U=Kx$XL;`p!90Gf6$>`P7$?V7iRFAT<09nGk+6k zt6DzVRs|-+%y(nSos^JbpFDE5y0cEQ8YJj_jVitK&KmLD7Si z@&((14U?cIUsEog!wbNbn&?^2mcF2VpXz&$>rbbPHml~6eJf!QFaA6a6vAoWEac+w?JfTFH>zd{ zJi1C7G<(W%H?{t_=^$fFw-VzSei~pALv)jC3s^+-#7b)TQD0kbyp^tYX}9r6^K$OI z=LM{A8+vGhjAqp`y^QkCiyqhEN@|5!0)r`>(u7Lc-fV%f_UZAh4z7h(1;xd*N-nzh zAc1Qe+GWh2&=THDyA;NDsIf&ySo1*Jv#*AtcX#vX7Z2|J^mc0n++6N}8OBIoTJsu_ zM&)x2lx2rYrpJGEC_P`@(PWn`FU2;b_!M{fQVA3NR_H!n)YZ@BttMCv%9u`CoWs0I zDl*9-CDZuZujuO81|?ucwp$aIQQb*MCsLNZ>zDhP6vE8JHQTg;kSnb$l1qZ^_U;)r<@rRl{aihmm=RCJC2W9~ra+z7ez)^`WT48!^jO#Y+%7>f(GLpQt zG3JcW{Q!APTY2SV^QJk|9FW$-E_dS;=Vi}<-*g$Z5~W!`@ZgHV(TC&e+ohdICpA7i zwad{+P?(k$_8$jEVJvYJ?G0)Xq|_yY$z=JBGS)P2fA(>I(4y6=rFFZyqb@FYZT%PQ zC-CX452779){lowIqsBJfYOduTK_QDEY&|UgNVA9+61dQ|0+QYtn}5UqKZnS7vMb= zk@B_EF*wvu!|^D7;sx;_LVP4OBP^qPuzY)+AJcfv_HK8!pgpn> z)Tuu2nzW>!gcq9366~#$t$8Ee;54UTc7C=Dvy=MB{CuXh=e-TLm8-3ox|N`?_Jf%P zd5BCDC|+qp_YNu?$xf7n%M8k|QaC{T+@$|&>_$e1&{}CGpiBl$*-cS?WJ~CF9JL%R zk#T0(URtFzcUMRuq+O=YHbgd{rcEaLuqMI@{|G_E=bpxYQU;p^ifQI~kOFO~AB-u7O%0CT)| zHi$C2$*saNTLFQWU_<i$>ca3ZmJT4@jw z4XssC#&d>7(Ystr1Xt5?!uEjvFs_}{!KPkOoKQhT?Agt^Ql_h!>BOT&`wb&9XOJaf z9V6p>!PQR61WqZCndW;)MM;}`gzf5#EYUszE3?p_{ViC|v5u83IEFquEm$Syeek`w zzPDKcQP$7e=L1dj?|1wgpyu}2{Ks45%gpaq`}8E&KVnq(u{{ekh&TSM%b-hDgvqs% zsL@GJM`$>|v!O!?H6*$~7G=_4dxD(g4Ii;V2Y0q4>#s<6wnadqW{+fb8rTk_9)M;c(|mffEswpBB#{NoZf z5uX`QFrPKsi(Ais53r##B@3WOK3v7+S$=?rO05-L7du#Ome7n(5B{nWp|LF_6M}I%IrWBFjIgxDk!3Zy&9IhrvbQ}t(8ToL^}jSak(3v~XV;(EurU6jOc}(_ z-sx=Vpr9T=F8I&2M7PYp?4VPQK0e~a!t2?&+Q_+bc|^2m9X4L0RMd0c6-R=g!o7$i zYb?=IHKYOyty`M3==RD9H}d`9+{iISM7oIMUl%xCqc3){v@?kz=~pnevpMe^WN-_*I-zzC0H`K?l#VO_RJLztts~F)`cFhEO|Z1Y zgAMWSL6vDws*3IXX;79K%i2L`OJ*;p*_&80hOW>nvLR6~1wFf=UkjZ~1ny{$P;ov< zccF9zm2-I!;RNiSh0OR_pK4MEk%gbdSd->b<^1O}$g&%^RhF}h1stQ{xkiAUs$~jJ zx3IqK%=ouy%~HQ2nMY}U^||F`={#zt6E6sV&_ z$X44s*?hp&)@JCLb-3K? zO}j~n1#%B-a{00}6NF2o9RFE?$}vp~`JSOpkCB~ff!I->giWiLZOoA(I!17oZB<2q zVm8T3v)Kx>GdxTteUq8{c`&$~75%)}+ZD__WFaBGlW9!eZ}FLR!vW}k=?5%Spyeeu znjwi@AZQJ5C<@J=zlWcA;M0ZPT2WZg7s9!Sd@N()KiwJGb8xyW>Ulp_A#qKeZ=gMmudM1|C+dD9@; zbjqfg^nZtjO+!&ECPMA!wtRt&r0pnc${yeXYW*eIU-K0%siVf3s7f|T4C#3U4v^i;HkeX)9;h5-GAk!CO z&+U@gnXXJlp{%E=aLW3YsJjx?I>I#KCVG-*FV{Sq?_VRH7({oV=~NEi9^XU{C?I(_ z9LCiZD8*Qs-8|wbdKprLw<2Gkc2z=fJLwZU!~>_b3*O2Xk#bE^52|_N)Em!deSw3_ECBN&uyy@D;`xzYC|^cXTsB{WawnGr z+LPiec>98Yos%q)>a58XzE=M-R(RJoff?Qr*)Kj*{Z2Z=i&B?ht?9<&7N!R)A`_^{ zCr~VeuK?8+zLdktCCbqWOSp$aCT_IAx(*j*;)oY)gZLLBdJV5>f~yf^dIi~nZ4NdtH`hT@$51yey$bcu}guzU{wnT4aDOetgn{c-6fsY+;0mxjjZ+JG$r^L_PhmpG15xezlKdBj5bF#ESDJAn#dY?rWG+Wf)tN5mZ1@C zh?aRKSCHL>o}G1kCdnPpiupQ}0~x1dUlCUTv(0KQs&L4(g<^0~KUA_k_rgNpf`#+c zH_>0hY5kHK&u|SlJq*^pfy$`AV)Qh?9xYxN0{PA zGS5)_A*C8$QKYB{<>&!MCbpD3!&Ks!F~U2~BzZFY^JSLU!^IC`qt9rZCiJK2%LY1CdGU077GZX=L4bnSM^W%AZ>`iC=z2uYg^)~X!CL<4L) zoe5nmVX>vm*-_|WDWoL~yvT9Y!WkE2WmYu-rY^YQAQ?t6Ib&hMNx;eqAto;~p<5;B zF&oB?bcQ81_@?3`jEi7~R2_*?Dgd| zhg6NP*RU91*kgJ%Fpq+aH!P5M9~^Qf>jH7aN%GJJ5Z zNA3FJu*K(QYv*Vy1hr96Fs}jLY|@^)E?bL6Y3G+VsK$0_LUMh^>CJze_t=O{LX} zsvDn6ymbw;v#}lk{ZF}CUo8Or`2#UfFxF=%qOn~YxqLMOx~7y}GTAD8V*r@mY#r%r`OW*&y-+U|LSDEV-OA)5bb#0wMA^@)xkcS}_u#() zO-s{lNz)G5x*~1G!W@57)Qd@_gOL-d=nErlJBk2eWW%TkNv8H*!`9^JG};54L>@#* zG_TH@+xi6+D}fg#b^`jd0W*lA%PYS{m6e9SeSSupaLPTS2h#oR8!$Wd&AxCu_~Md6 zXcsfv!wgsa{#m1%|CV752~KmfUmaB7)|GO&B6+_9+-kHK;QrPs0Y%*}D)r0D{rwF? z<)T^a;>p~J{DIE7X=Eo$o*z_ku}`2kHM+;vnBf$XH<^+j2=`}K_eXk*Mj2V%s6zPA zIZ2@+e{LZ&n%a|Dk_5reBn3|oXbF7?B^(!?9a1Zcp4o!#`+J zl${fYdV%SIJa{xt$9hLVAXIRpY-FVZti?&JC!qzRXb`ADt-KkIz`|jBhRt?QkmT?g-hiQ-b7O zy0h5Oc&fa4ny*=XvdesWTMDxy$xb)LQn!P2tV6p1A5t8;mZK<_sE!rrUr1)!>)02P zd(JD$@;n^j`}DV!^uZIKvpd1{9Je!Xj%v2lJh7)Jm!C}}-6Y0nt(b1{L^Np!>od}Xril3aW-5E5w~Pd{b-GagfMHKDWa+IXS{!nUjeZ_>n_C=U%@wa7E$)hF3r^ zhgHS$Ua*L?O~)JujZ-9&h{3|BHY(n+@}0;7|{!&yPh zF~KA;Fc~x=(<}4*QN0Ht)h{347TjvMQ~Lx=3F7!Yx2<5OT0p9oH1~?!MsvIzjHdj+ zeuF?=+dug{$pdbl6XB9tUW_7Zcm=$vE;3Akt-gsis z%iL%oH^!3XTdaM8l3Y`@t{Dq^<9vzf_p}|fZ(*&KT0oWi!lNkTDcz+pytu%!w8`(G`s!GFL^5uE zsskObaJXaM$_b$%oYe9@;={yF)d&ohyWyAFY8!TRg)3{_OFVY3xmA@hnI${u0hUW|J>Nowh>VIRteA-oJdIh}`mBgu}v zHMMA0T%Y|=_)|Oou9PnO?^&vR1sNECB(ZxJ^Fny{82bu+%IaRpJQ$BEM#k4P_zeDJ z9d>eQi>lp;mbR7>pukV~%Gn`nILb&tDyYn-WXiPsVa<)ZQdJuFfgJ!jC=U+Fc=`Bu zuz>VPHZ4$2`n~I<3e@DjFwb*ssBVnPbh?ga+%8`1iED>vqkUofJ&7UP$KnNnUbhyMdle!JPUStTw{YY{-H# z+HHIBxI|4xMSx2_7KCyY3jH_QB0a=Hh$*_lYcJnZqNCkQIueY%zCusZ<=^p9R47UG zuVU?%Y>O@~tiC#gXBSQakZkN$evV^y%5Vf|Dpq|;Q}MJdR2KUpImU#tP0}BTC#a!n zmu2G@ysQasNw+h=no9Ab?%LyyW;ZmZ)3*}g-=8NmPnVlrtvPu^BQ$xdGwelT6NcW?X)k% z_2h_QqgI2dM0Oa@PseY#wvKa)JcN9M?>_B-9Ik>mje@hSAQ$89YIq@?)8bETl;`K^ z6`d=@6*2euQ1V85r%*%l5@0Im>O#YAYR!NkJ6#+g@LnovO7;K>LJ>gu zY+)U7uhj8^SKi59rkkuiF!kYsUK#lOK$GZCN_R#`#JF?sovTF2X`Na%SFUON_O%V{ z*M=Ms*eDp4EbD4hIFLbH`K7V89MljG!+i$X8F7+C7hsW z9H1S%$9C6ivA>Y)!>z2+(D83gsG4N zgQ{>Vo<<-oG(OhT-!P6+26`*1%wmmT1VPMx=i`q($~V?Rde{Cy8lU9q)G*83X8oCCMo3@SX4V;+)Jqu6YRdTMu^s|x49 zS)R!e0uwmKGF4vgcdHid(7NJCWUN8xXfb2VgFdb|8Tlmg{F~}g_l^z-7`n#-m>>$Z ze_{B_@~V!%`lL^Aiu*HFL>b>6K4r|^lDIdDq)hK+mE*^1RxAjXv(R498w(1!Qwo`pM$IC0^HzxSLsqTqkFLnSl|(&C&c z+l7Xdyr8?WU`qlq)XW(<-j!-A>@mOd+i4 z!TXXYXIK2|v}kGNUb6Fhd(D+)iQh^8-H8CSJ$#^H$!%B^VImvtL^CR+XZ7CaO#1SxAyo%w+f9NxE?)cxJH6^FX2Q; zQs&KJry85tV|o3N)E%cac(|VPPbv6HTgw|ggLQgCJH`_FtcHpaU12$}FnLMe({pB_ z@1eE@VU_Mp&&OH|uZG1&4elcEQ2o;VsEjrXih zX_}4t{8K<8py|OfEW9X4LsCw-&)(8h`A~e555y%~l)g8t_-_Obj|I;TWnEy-1vojiuu^eAP@~JvO9IIYrV7 zJkwz;wKnY-OA3GNb%rGm#C)ST$rYdCkv zWY8ACd+RV?b#-SqSGJsNllXwa%1`ov;qYIyw2G-nw#A{gkck8S;YU(8{32lxk?tpD zrr~n~O|X&^@#E^}(hWdGCL!lGhyjG^=`7uQxJM-r-dQH>MJAgI@w99m2a&QEW&`n1 z$I9b=Z_raXa1fz=vJOUPfN3G{1w>b0pd;PJkkm7vXr}1Bj+pg*!Rrm3+a>+qG90GrhJQVv?K&Gg(+wWbTl)o8m4#_-P@gU}6>@ibK$%3;BKl!W1$-Cr~Yb_-T|GwU_#3GY$!>adxxqr2k z8wSgc<87l7fY@^)rf^hh+|f>n#IVSHScnZ9_>&50j#_BUCJN0AH(5rFNyELI;u-#9Kziz z=r)|W?Vo?r@$~|4GtiqRGNPI)OGOi{6J#+|OqL2Ps*(uo6Nv=^D|{d@OB0j1C7nHD zHRyPd<~U|}+E2q7P8-OvJz?YO=_WbEu)}(&g1P4e0)ywqZOh2iwiQR%B4ODNc@^v+ zm@J=OofpKzN3;a{5^+9(zP|i}nr=)<64PHwV&?=ik*K**`eOyL&4ADMfiEB56Imp9 z2&TIP&kx6H8QiN&abeNF{-lC(L9IudroqyPm?fS?S0{Zt*NB~KRA|UksB~!j)$Q@L zi+2|Eh(zjlfs2Y%JsqoJT%2(T6yeD=CzBY*gw{LEiY+P1ZG`tl9ujn=9$ZDk6GBUT z45?}Pm+FcQ^`-tU8wW|m#aSCx^w1_feSHxI(LF66N)0%PrUvf29F0UmNwA)}kzY4* zvfoz8lIpJwo{RCSyj_@Y4L@`E-*%S%!KvNdgj)L*=5wEz&XMPTvsispn7Y#fdZr2P zTaw!`8v=N{fVsPDq3IaEWljCwb~Z%b;(52~PkVrsLC8WBiEyeY1$7`Z>y4G<3 z6fC~A7>phRugZfzd*yVv@?cibQ%MB79*JwQGOW-~X8sJL4;1z%#_dD!K*7R*wCt#q z7Qky43*g!`li_)+Vo8F~bg<@KisC((I%3N83ji$CXu6I6$CljxsYH1k@&deddLBKW z1)N#}Eh6*B!K(~=+7~nl=JDVp7ql(pCJIDZZ{Go0Z)Q^oXepC6rBiM5CK`i%)zl_h z;>snA%D8ZFfMuz1!M^_LG<4_ zuABbDFlzF3^&7Bqz&D*^NKa<{I|&#P5-#}LeR84MmZxm^L;XQMiduJa&wAIVMOXJVG@PXo!TxdH zM9nE1WrT+WY>`{)YqkHja{j+D;m|(+H~%hCAK_8vk!rI**TVb{{Lk77Zss8LMmeVU z!zl&4Q*=h>By35DOTI;p3Rp=nWl=jj#{&Oa7d-~-<9A)sLPaF$DO=MpFwjMQlo0q2 z`!lswntxmON_T64m2Mqch$XKSEV2!AGL<*vE?6l}z@Bl zB+W;6c?PTf^at38830}K^Xo$}aJ~z|@Zyi74Cq)3?Ib}c{ zV7_iDNnB+*)Tn)0!o>fzA+JFrR)zR9*!bgwyT9MMNt2~Z8aJ?&5;A=pJ;SWNcR!|C zEIKY0O@MB30z}D2pfQked%)kN*9!$+bcpk1`fZ5*gMGBqlBA%Y^}BwuwW7Kie8h-z z(_boRrPZ`t%}InYy;)w4siRO<$cVG$+y-M3qBAc-j>9*kj}&S~j^)BIAcZ*62b>Q7 zw?SVEMvkMut39bWp3J{~{e7Oc2f&g(*=IW?KFjEv%+oyh_Vt8CWvxM6tk(X)Jn47# zN(B9JWBx+BS_Er6AvCYvc9>P&c(fpHsgNy*ngB5d#&X{MKtloQr&9q7%5IYBwco(d zl;<>8OjeB^DpzDj$Yo~QmsAo^B|*>dY^O{*k<>OmXc%(Sprhr#GhmgJlg=5C8=L*V zG%rJ9-d$QzQK{R*0tx`$xrFXWNXg*uo!_#Y9$KZxb^Q!$bn!5}kT5(W5q1!CnFiF` zn06ZxfMIt_WnHLfCDr>8)n9Ta`G9v{6%)WV23ccT-ud9|07rgjVO*9!O)pjQC2&PK z#u4{PdC5L3^i(+Oz}vw_*eQW`LB7pn6Rj|z?7a79l$s|+D)X6B(zz7D(Y&AG>0-*239)IJZkB|T0{-)yG) zSSi%70>Pw%VjT@~eq7Dg{5}F_v>mkU;l22ChR3;vu*5hmm1FK2AGBsMqX!hn4aQx< zXX=$UZTv#(GL5gRdeFh^t44bOqpu*{I zKb)KMLY>fH5sDeA zyJJ-fk^!8eX~EuGX%_6Fm&Z{}$u(@UY6zBXL^TU4pLLcyxKh~WtTE8WggA>^wPA4( z|7dd^HtSTismsIO;8#-(2@j!P%lE1BO4Vv*68Z#S*{iGAWdhXO?6ib6YUq8XFYa3ERf0PRJ!pKwb#{6*S6Ic`WpbyJ=p zW_SkF$`Ourq!fdW=nyKQDQrX_;8;;W*JJC~3NB&@D+8|Im(>zFV)cSzND?SlqQO1bpKMRsI-&4Dk>Z^J2DB zX{-A6W5ZP*4ob2rn_^?Fd<;X=aUH!bu)7ye7Run1BjJYm1^B?kt0ja`Fs8v{4mrj> z3nKcOv})IZC5h6jIxvWS?p`2fiV!*4#8v z@+FbV26`2l73nn}WUup5lC=AQ!Cj=+N{^2|44U{F8c@CaP zyAFfx3bM#mi4FFRQYG7S!=TJuBUSNV6f&!;G4=d9I07lH+Qq|ZfwV`3>hw$$wkP;z zgP^);R^Q7f*$VFb*bYiG%YwmAf{AL&)+Bq9zQ`oKg;8T(3dzDX(Ri({;<^z-r}l>S zIFAkkTgVaNWL{8yge#@xG}4Nq5%glKMk!oeO#v(8at_)wM+Z5Z*$@Re2E6-OuXKxI zDJ+!oC0h+)y-9$}8h=B=gFf<(2c5vlz?Xb-NUW2r^$C1Pw=EQmpnaD{Vdt0<%mR{z zFP2`&^k+D{MAozzGN+6%+T;2>%RQ!pL(R z)yai5+e$4-ttE#{=-O6KD0189<>|Q}arEbQhv}Dpm!NI}X)VjVYR_xkYL3v!W(6kl5X{83z%p|L~Az zLvc&pSkyjtWTB_*WX8~$bX>N6IZ;p<6Uz6~X(+Dovvv+yM=gut6okAmuyIzTygzD8 zJVNGfre+w=yJOdv+@APahUgEN?idLtrc~e#TGVSj?ZL&LtcBV{3s4ZLFoVgOxt9cX zP#Yzw_?h^!zli@U=MXvP4$SOG^4+8jv4Va!!UOD3%4U_xTihw^-=cDb2k#9jr6-ND z5HHe3gW{U!dxq!o_dCM++bScWu(G|BhN?<#bmSFb1!z>Dk&7U7Vk0-=0qpg`cHuA4 z;L;1dTbF6m?fHF@eHgGURPOMpnNR0W2GQOH29YH5DdqVTk0^f7f({tAXcb%pi*FN@ zdmHZAxLRCyRW`7a_~E4dVnG&|KqU+I_J_0!JspikJ+#N)ZYr)o_0~Qs(5jOx zbPiPHallu9IO z%GJut%LUV-k{Z?NEKtyQ@BP$@gm~jazOvF1U%3Q8CL}95x&=^)_*w2D^@k8|jokd1<1!eA7MC2|dxn+ciE}y>U3eW`mJm;>#-Mo|l)I z&i`&YVgv32km|$??pq9~IhYzf2>O5ZUp!|Gmd`7r^ucDM(+25GzW*~4+egtP@_wTG z`oyh>vor{}?JQUCtKterB^%}wWIF!w``UiBR=?*`WTqc+Fr~^sa^)nUqZ-wVJGj3% z)`+Cv7uG*IIn~D}>9?o=7x9V68^kAild85T^0kA!fQb|*4WxL(Sy~Yh) z2X>KXN~nsc!7H9JwQL6LeL|9VP`NShIygWPADDtP`^=wIXT=StM!x>S3d~j>yKn>{ zB^~#rH{Ee}b5Z=7em8KqG&@`n>$tm-N=<(<64QDf2_~rib8|I4!`m&~C#~mGyR1wn zddtK!FQDAkEePvHRrYk%06;>qCu*OdXq}`x&--3Fi}}a1kSQpnLk|n5abt2NhDA}`atdz&)l?fop1*IXD3Yl!T z3+3hAw*=phF@mj9-oJ0nzj`>2!I+9e)b8?o4nnb8Q8^yU_g-rW?y2K4`lc%wtl=zf zl;z{RJ+=}L01aJAXQ!%h;meS4bZ z4#E$;4_>6TTvV}^{Qu|PC0sE`8WbH!;{ST+#+p)VXUxFPd~Iayblj^pB^19I#2ghF zc=Zn-Tz^#A6WBZoFC?Ex(7Pycy9)FU;(|AlSjOCO)c2U${T1BZuMb6v>e`!aAdJg5 zzBt`pGFKF4Jk|=xr?{5%?(Y>D!M*Q4n}&bYt80w^ue~o1Yx+v}@3fsdR)t!0ToAam z0V}eI3;QxtSCpm7Zdil}h!8*^1c)JRt)!wr1rtC7Mnse?>EE0PF7P$3BrNJO?m z2tgnT5JK(==m1)K`}^Jd*L^O3@bKu_-u*q_^PbP)h{lmrv)~s1l5dp4_Yl^i$%O#i z^s4y=tH~QlQC44Bd;TpjA#iztr&cb|5^!xG)0TbVy^v*dH4gVhbi8#OGw%YYl+46B zWDvtEt4oc_m!ylJq>&&+cIW3=^j;KL7BU+>^@&xu__4mMqsvJ$`?#y0GSROGm3vwh@|@DO zq4bV2r(*>f=1%SVYgVR%vkMZcTpY>t%8L@Bc&bK$0Iu1WLp65=z`-ifrVEes=N}2C zL<-et-3?&xNUZAECEh!f36HJx9axaaPUd6~`rZw)?*akaC-+cC(pU1$0NtL!jX-hc z4{CuAMzcf`Ss+FbSoM{EpRAmJa?Em{^tFckqHq_asiI1@=SI~K6M&FD&hM1mTN!>} z;GjHuqix&Z|HlI0^6l*+wwG1iARhWE4+M<+e#HS*L=nHIL;iXDkx4XQj$mr2#4QEV z)^w)5o9g2c=y_({{d2t3^5vpPEpSD7+FkFROzu5EXh@SRBdckp+v9)&l+$pD@Hq13S3qg z5x7n~QUmnrjcW`SB#_XR_Ww~mO+_Ej+6DlrvuZ`^?bP-T=`tX3Hlzl#+9Vuag8)Qn z@H={S4ItRW#`r;%ZgKSOM1dExc43ZB&i)0m%fHy8Do(_AD~1O=%_u-q|3DPOw^T!< zHI_fT^qL<#=98>|2hWEJ4P<7vWZp6sqII&=$RcHB#Mo>#eFxn5eYGq0m6Gaqo|s5# zw9N!0+^X`c3hab(C-ICx=$R>FwLqmnJx)#Wx#Wlqzd)id5SwI!CY0<-8Gku51Np6i zMe1_$MW}hr;!8jMQKM#%_ZadCKxT9xabcGDP0@VOA4#XlOIqbs=kV&EnkTqn4Bd;D zcUdRXiL3|Sr!g{QaX>(E|MWc5OU5)KXU`AVFAR*4h3Jrat>ek&#HyprGfaX<6`pwu z&#K+c5tsde@`!01PwuWS>W2i>jVbGoCqwLeW0FRA0i&{IIPv$rNWL4U>eZs`Xf%Ea zmFtxN*mV;Z^c@DB&w3E~wBTTi*U#owfp%C7oFAteJmK=Sbutj8ZPn4LKxf&QJI}s5 zc17HPG&Q7UF_tp{cF)WYCBLcs&2t{_b=CSaU|siv$@FRA0a0u_dGueJi^yQRtjBrN zHhL?z1&-)86gM3EG{HCf(gXoX-V?k4oH~yp?G+|Aeda6ED-JN%2u#FIEdVF`dh*~Y z>Phx`pbtb0Q1XCkyG*Inn!{d^ovya^wV_WpkQV-#eK@qY4Rb(iyF$$iLOc~%)_x=n8{tj(xx(!A&MK(ZksCK$u{G@PGnf$1S!kOp8q(JJW$KJ zbvp25u|XB9<5tR-8=B7)UU+&Ua9dj&2pzJD?3t_CKeN9D$j+@u%2e3S6Edgg5eYXS zHH50Xj_%WeE#;CFQvINAD!Zr>pP2&BUXj`)4|v-w`)jR-RYu$xboTLyS|s`#E!8)q zynr0xJ|n++Q{?@=+&9f%@tV2PmBF3PGYg6YE~Asx?HK5IrY>rMB18)_WHU+fsCq-7 zh4?B%=5`#55MC)jeb8@w8nuu>vM8{JL_N>IM2qu&2~v2kcoym;nK~`pC-Q9ngSFf& zqK3T&z6%}M0Jc-!{XoO8<*a|r@o#MD)7QUPqcQvBnBFd049Hm2+s|CSpj&V-pOshX zmsuzyx3Jk6n$bU;&>nUwjrsXag>=Z&cpTJSs$w%j*s{Xn2_mwZSfE&qy zS3tq0F3hLd)~NRYYAS=Lv+V~+&Hg~!{HlAbv@?Mhp5|BWDz-l7X8jF`8^a&70K2Ba z$D0)4)&td_VC3lu1;K&j;knpyI7jnaBC)@H#jXLO42#iAeUrHC-AEw6W);r;^y8NK z8U3fh^H+G+fhPsV;zKU!K%JB;ISBXn7H;@*5iirqI!vDDAGYEs4YX9mt;&Sx5A);a zz>sQDsxs?R#E(Y4Nvn;4ODQp|tXKqMhjkR_HEmvp6SOxH02sM2yq0rNJ@6pc{4<@{)>0d3fW?&N>roMcc z-N1TsZ{9HA;8Hc9%0Rbj2zdl{^)o8O0Faw?9dql8*^{JQhk*i1KELd{gai3r3bM!< zYrE%$L(e}b*?)S@(YzLN14HTsmo-irR>Q^J$c3XeHUQ<{{HDnA1tr@(yrEbl=GUZ! zmR-n_y{`WqItKq5QZEWGQG*P)Kq}Js75hXgVxJ@m)th2I~lc?XD#u~fYOYeW#t%@1JA#Dx=uH_iXC&v)N zxo2aw+4A!?K8l$-VyF)!*JF5FDVIA?){^)i5!Sz6N*);bBg@!pJ$XRzCgt1TH_PTl z4ptfU%rTI5Hei-lr1|EtgCX*SuB4vH5I^}Vf2)0UAQg%bFXm(efR$ZP;W`obJE5{j z{ghBe#+_2P?>!yD!{A^7Im__7<**{M9Iyn=RvRdIkiNm^52&B`yEpdS9xdqjy`| zJ)(mzHi{Kjf9D?FzlQP(mtRHD4ns75{FaFM!! zTrw+tyoQp4YJ6z<;9N$?^qoD{Svl0NzxuK+cM)yx9TR$nKAP&&08*>Tsf97g8=@35 zgP+jji#6Qu81#Bd{Y{Ad8LzXSFSSzqerT|Snz+RMH*RtHzZHM&Mf}cStDD$lF5+^n z*3{Ka9PYnQ#Of9gJN|De3h(%TT>0OsE77Os`$!8x^Cs4wj%3weG}>jQRtdUI=x(~1 zVu!a{@OgHN&|Q$W-jJvfQHB029uwI$Cvl8}=qG!;cqoWD_VNg8X@EnnZFuf%#iFNN z9_02Xi_kT{nEPi-;xdBme;zB&zE=*3K6RU_-qYlmSpTh8|MxQrXEdN^W z-TK3lNoQ3EysnOk*gNgpZl!QFV)<>feFH~cgD>?G!8lAX{^e)mxsp$_a>lW;fFSFv zYa;iNOe_E8aTI?&G{5jzCrI; zq9?pO2^rC5xO$LKr9~XfZzp@;Dqm|J&dLHfKuKNjA!;vLQ=1uGFut&ZyD=*M4NZ2e zihIa~`o$Xz%i|3HV~M9tFmUoUpVsTQK;ET&!{OPVm^Rrw@6Cj%4-U5?tj6qNCns-@ z^{e$(ES3r`Px6-Ukfi+FeS{a2xl_GG%~R-O*bsF%(bd@pxi_~(HqsH-QRU<)Qx>8N$|`RV`^`u zK9Y706@+qU?%y#rCo}E0nKWjhyUnP`)q-7CFaymt&Gxb;aoWvkwwVs+Rxpu;Rjw}fZ*;FLe8&GF?-RL%*YyscTv;? zZOP0MZ;L}%`t>T9v0eUxDMh!vn}jN|sd>;_cymN8an_aE9aguM10G7Fch@zG9K~oi zzwL!$Dypz2CtN=xsi2}c7lus~$-W@$qcCZG#=6^u#1_30P8g1eXS8y<)!CAKncL*; zacmgVG=)JkRZDRnv^-8P=HHXoe(c2IveILz1p0}1_k+9d zBHg}h+rsaK)9a$S8e1JMm7>?6QuuSHqiya0Ov$~vqjUkqm6OIS{2sD_0EK=p~;GV_e~kzT?;dovv-lM1RD{sfqxn>RHoS&1R?79IhNy}2TT_~G@S5_Fld_7@5jFfC2Q-=o z<-iKKj(GIvUC9<*I5&!HFa`aGMN;vwFVvr3>TX4V!Sx+Hq>-I*VV$hl zdS^8u(Z(wF+K$#Enff?>@#G=?jsSCp@34Nmfy|0CwVr{?r$mm$te9Y1;ye@cBVupTo?|iS8Y(2pUP34ISQ&KRIkacwC)VYlIs9BW8TG$c(+GN{Yk5 zJ17nN4!P{MDnh!2c|(adzGtxLJ@nSH3pKkTN0d^o;5CyfeHo5d*$Q?SaWP<0EQTJJ5EXJqIae)4rZjoUI=zG!!w!P_&pK}B0sUOAov9_ zEU0LSaH0i)B$H{&)LH4xV+?V<@2EY$bUx>GFk@`8)`Qu%Loe_Ul(?fVF%D4@_OzIj zW!swz=Ns2MBfFBkdka0+IBezg9i$W`z`MXJ6WGB>bZ@eGJ)%V1Ds)<{0LxLf7*c0A zqKC64($}kqx=l&8@j*#m1KQ}V_wv=|km08W9@aDETd9oIlCIV0IbBa;8l@I|ZQy&?!ioAl=C;P%*LY{fW} zJ4;@%2U2k0-|0?WA!L&l2NriV2A+vM9`8PwCYB7A=RQGj%1Q@v(I@gv4asgL* zUo_qWMTWr+7FAdTcl9Xo;TY1BzS%g$IKvz2vaQZ3DL^4swKKk`!mM(+kl4C-SE9?i zcR18aXMcjfOb6%Tr{Hi z=o4gaw00(484dR>4TxlkOknG=Jr^3RMu&k(?ZNSR4azyO(t z@0FEpXLYHQf`YN4;8&%s=$dgfR4K{r02iL_;Yk954==b+(nW3*Vhw%$C-VAx8i5#sXgk<+`c28C-)Nd~?Kwx-|H=YBDM^KX*vAAiZ7#Y1Z7a7d-meU=L0=abPL(A#OcmXFaA1Y=AfWVjv@cuLXssF^ z>MRT;oKEKwd&*<5YR;1n?-rg%EITfFz^{s~c;+5FwcH(6lNaF|SNH8crL0Ts-yc?j3xoXLh5dh$AlOrYgNWK;_QR6-NE z`*0?C4TrCl<^CMJU##XnJ+H>98#A%6`NHiDwc@n$SYoTO=36@}A|QqegBWUIK}bO< z{8}K)AZMTS_{`Qo`UH{>dAL5w*4G)~%Fc=1z`!la)X^UZ6-uNRh4+{$afqYl32&(F zzfFsBmE?rgg?6$KZTo{j!hC^&zeYnD+g%$4$>kQ&9o>qe%zUmRfuu_+8t+Tf-^1gm zSF~Q_8wYU;B!`50CB5DrFwuIB2cccuS9af+C@Ay20L#1^YpG+V!{*@KSi{~@iNR3_ ziS&mN@G`EH^u+t)v)T?yd_`awsRHODo4VLe@*{EfJC|tm(C9)uqL)5xm~5D9 zJ|ZZ}CV>aF9n`MbOqHFd#PxTlo)jrD?(HKtSnBTKWpsNI;lg6mEeW<}kxj&LO4c2a zN8c7)gs0C*#7MzKCycbt5vdI_IRU4GlX$2guq7h#2>gU|m*X)`D+PY7`Cg2}?DiB2 zK`dz~8)a=acpX4#1E!w~kp^JG;D?U|P-T?G#13W3PI9u6*)`>@3VG)&Z)dk9q!U|m% z1R%hv&2-Qj)U-$Mqq=-(x4KueqnPiSiTjzoK(PIy=rDO<7*}Kz>Fx&fr~yW^5p95T zi1BbK6f1CVgePxi2S~L0OO+kMXlb~j8(`O=6|RG_Q*NU^xCHe}U8_-uSL{P-ywiR4 z_>5hD5q(n=o*S}tn;P6C7liYke_b67{{7A4*PF&90t}&) zGSU4h#RH^~$y;mecAn)>4Cu*Ql`}WF(Fxt-`^V3M^3M#|omod2wP<0#23H1u|+BYa8bdA_y0f1g@-=C4o#NFtU>;uJ-oo z?oZ|u(QfEN?VWjH4EKAPPJXV01v93L68++Nj9LN?N^0fC?r0Y__ZgF@&`7j0r?lu& zpjyUI(2e{=L`nX+TO>GNEipi-Efdf~-_p)<;#8kEhjJzvwDTE!{6Jq3stOrf@}b@B z*%P;*>@rjA+NMxUMBh*cImy<>xgx@dwMrwHvrtE8B zzsJBQE%%h%az^S)2x~o;FjXpp zf7}|n{tPFkS&U?KSB!Qhx*>AKB!pf8G71g%F|*5WjkLqc;7O`PP#4o;%5$}%%~|WCIoC3?UXkrKV~pr}d+L-phs>TJI@usc!k>;jCJDDIpS zn#Km{K(TbN6L1<{G@U=nT$>td0lxph>q@(@Y^4;Mv-L;NMzjZM92*coIqn%D9X{)3 z@WpD43;k^A17b23ydZAvsUuMol0pT6}bN z^bPYA$4PI`!B;m)GhL!_@xUo8o@kKt!7tzc&84{4MvU0cZTL}96Y)yx?OC9kL82x1 zRN3#$+KOO%NV4Z_f26)vrEWC(^38c$Q~bs0*NO3HP%f~VlD!B$kJ<<*)q27OBXa3K z`X^^##iqOjHLu(M%mjqG@mjVh<5M2^+TNJ()0;!Oa-SWxrw90;#9;M;(oET27}-IY z*00&K>DgCR*S?X|xklr;sAG+mzt_P7*{cFb7RIpaUj11LUIGe0t;eR76dbU=p*;S+ zI=CZy5lFt#TD->m%QgLeZ&~D8mSD*GJt4IIa57boe3$pl3`IiL5B_F};@dw`C)QD) zX&s^5U58ms^Y`I>FYcYR=Nql#UYdG(@zC2N&!Wzw8p993!TiGN_8$#DN-*NixIOjz zwF_%C-&dUv>LY-pCvO&~9aZiB-5!tcolFfD7_EqS44+`dL2;d7x!jO=Nk8Rr_p*aomb3O_jqU$4rn0f@%}NxB2f+oA6V*OuDK`f zIeou2@%a^#cRJLBkG#0oRy|(JU#R9CpSebzxpuMQjW~9@mR-~Os@!LL_WS*gyac4^ zj~S^@Mr=cFp?^E15uc&7#N(uqHC8jfGRm%5CY~}#(0lhZNAGhZ-lZuzFv1a}biXPdk0LRPQ!`)dl3s&@xz-zI}<@AwSmH}mz?@nqwMUzJ}v zzlF8B%ST&o0L`?x+v`3vWhI}wr~)qa0G14VC9BDn));OpzjSm<>$_djK^NIS>Y654 z(2#_TjZ2d#dpBxzyEgV{{Pt+;I~`J0zQ)w@&PM^xTVG#d_^~pdFE?lW*1u_U#cz+c zywe`jOr6;xe^sLJbPZ4A8oJtKJ#CFouP$|!milaj2l{fq9rGGhyruWwPWC>=ykT-Lar0qS!AK?9AYKA&v<+I7O^D}7GmR?EBV-Z9CK z#;^)~YN%mLGPb4|^+vRhc>upX<{vxcuNU%Ty@&<%r_L2pO{EFEnS`!mmXkNCy_ZB| z*D!~X&SB=~FtRz!!W>2({})G{fBAg8+xeNR!4+Wri>Q+;c%Y;xFPxxyDjb<7SUSrO z21*STt9%CC6Zc)&PkKcwS}XnxH4m<|}+{hJ4EEseE=<@6gg9Cbsfi>Eh~1EV&eE@cpiq o;6#($!_3G(!7-jrH(kFeP%O&&h}~hrw`YI+PnPNOC;$Ke literal 0 HcmV?d00001 From 57be3c4221d52e5de039d5bb2061493299dc9fb9 Mon Sep 17 00:00:00 2001 From: adrianlizarraga Date: Mon, 5 Feb 2024 17:01:24 -0800 Subject: [PATCH 02/14] Update image size. Update python language tags --- docs/execution-providers/QNN-ExecutionProvider.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/docs/execution-providers/QNN-ExecutionProvider.md b/docs/execution-providers/QNN-ExecutionProvider.md index 9e2b7c3bfb498..0266c5e6c37e4 100644 --- a/docs/execution-providers/QNN-ExecutionProvider.md +++ b/docs/execution-providers/QNN-ExecutionProvider.md @@ -80,9 +80,10 @@ The QNN Execution Provider supports a number of configuration options. The `prov ## Running a model with QNN EP's HTP backend (Python) The QNN HTP backend, which offloads compute to the NPU, only supports quantized models. Models with 32-bit floating-point activations and weights must first be quantized to use a lower integer precision (e.g., 8-bit or 16-bit integers). + This section provides instructions for quantizing a model and then running the quantized model on QNN EP's HTP backend using Python APIs. Please refer to the [quantization page](../performance/model-optimizations/quantization.md) for a broader overview of quantization concepts. -

Offline workflow for quantizing an ONNX model for use on QNN EP

+

Offline workflow for quantizing an ONNX model for use on QNN EP

### Quantizing a model The ONNX Runtime python package provides utilities for quantizing ONNX models via the `onnxruntime.quantization` import. Note that the quantization utilities are currently only supported on x86_64. @@ -97,7 +98,7 @@ python -m pip install -i https://aiinfra.pkgs.visualstudio.com/PublicPackages/_p Model quantization for QNN EP requires the use of calibration input data to compute quantization parameters for all activations and weights in the model. Using a calibration dataset that is representative of typical model inputs is crucial in generating an accurate quantized model. The following snippet defines a sample `CalibrationDataReader` class that provides calibration data to the quantization utilies. Note, however, that the following example uses random inputs as the calibration dataset only for simplicity. Using random input data results in inaccurate models in practice. -```Python3 +```python # data_reader.py import numpy as np @@ -153,7 +154,7 @@ class DataReader(CalibrationDataReader): The following snippet pre-processes the original model and then quantizes the pre-processed model using the above `CalibrationDataReader` class. -```Python3 +```python # quantize_model.py import data_reader From 9a297b421a453a1347d41621b6964a522daedf5f Mon Sep 17 00:00:00 2001 From: adrianlizarraga Date: Mon, 5 Feb 2024 17:05:02 -0800 Subject: [PATCH 03/14] Add list --- docs/execution-providers/QNN-ExecutionProvider.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/docs/execution-providers/QNN-ExecutionProvider.md b/docs/execution-providers/QNN-ExecutionProvider.md index 0266c5e6c37e4..efd0bfe2e82e9 100644 --- a/docs/execution-providers/QNN-ExecutionProvider.md +++ b/docs/execution-providers/QNN-ExecutionProvider.md @@ -186,9 +186,10 @@ if __name__ == "__main__": ``` Running `python quantize_model.py` will generate a quantized model (`model.qdq.onnx`) that can be run on Windows ARM64 devices via ONNX Runtime's QNN EP. -Refer to [quantization/execution_providers/qnn/preprocess.py](https://github.com/microsoft/onnxruntime/blob/23996bbbbe0406a5c8edbf6b7dbd71e5780d3f4b/onnxruntime/python/tools/quantization/execution_providers/qnn/preprocess.py#L16) and -[quantization/execution_providers/qnn/quant_config.py](https://github.com/microsoft/onnxruntime/blob/23996bbbbe0406a5c8edbf6b7dbd71e5780d3f4b/onnxruntime/python/tools/quantization/execution_providers/qnn/quant_config.py#L20-L27) -for more information on available function parameters. + +Refer to the following pages for more information on API usage: +- [quantization/execution_providers/qnn/preprocess.py](https://github.com/microsoft/onnxruntime/blob/23996bbbbe0406a5c8edbf6b7dbd71e5780d3f4b/onnxruntime/python/tools/quantization/execution_providers/qnn/preprocess.py#L16) +- [quantization/execution_providers/qnn/quant_config.py](https://github.com/microsoft/onnxruntime/blob/23996bbbbe0406a5c8edbf6b7dbd71e5780d3f4b/onnxruntime/python/tools/quantization/execution_providers/qnn/quant_config.py#L20-L27) ## QNN context binary cache feature From a5f6569d39ee457133f2249c0c4fd2f1b919e83d Mon Sep 17 00:00:00 2001 From: adrianlizarraga Date: Mon, 5 Feb 2024 17:33:32 -0800 Subject: [PATCH 04/14] Simplify imports --- .../QNN-ExecutionProvider.md | 54 +++++++++---------- 1 file changed, 25 insertions(+), 29 deletions(-) diff --git a/docs/execution-providers/QNN-ExecutionProvider.md b/docs/execution-providers/QNN-ExecutionProvider.md index efd0bfe2e82e9..e2e71c691af20 100644 --- a/docs/execution-providers/QNN-ExecutionProvider.md +++ b/docs/execution-providers/QNN-ExecutionProvider.md @@ -83,44 +83,29 @@ The QNN HTP backend, which offloads compute to the NPU, only supports quantized This section provides instructions for quantizing a model and then running the quantized model on QNN EP's HTP backend using Python APIs. Please refer to the [quantization page](../performance/model-optimizations/quantization.md) for a broader overview of quantization concepts. -

Offline workflow for quantizing an ONNX model for use on QNN EP

+

Offline workflow for quantizing an ONNX model for use on QNN EP

-### Quantizing a model -The ONNX Runtime python package provides utilities for quantizing ONNX models via the `onnxruntime.quantization` import. Note that the quantization utilities are currently only supported on x86_64. +### Generating a quantized model (x64) +The ONNX Runtime python package provides utilities for quantizing ONNX models via the `onnxruntime.quantization` import. The quantization utilities are currently only supported on x86_64. Therefore, it is recommend to either use an x64 machine to quantize models or, alternatively, use a separate x64 python installation on Windows ARM64 machines. -Install the ONNX Runtime x64 python package. We currently recommend installing the nightly version of ONNX Runtime to get the latest updates to the quantization utilities. +Install the nightly ONNX Runtime x64 python package. ```shell -## Install nightly ORT built from main branch python -m pip install -i https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/ORT-Nightly/pypi/simple/ ort-nightly ``` -Model quantization for QNN EP requires the use of calibration input data to compute quantization parameters for all activations and weights in the model. Using a calibration dataset that is representative of typical model inputs is crucial in generating an accurate quantized model. -The following snippet defines a sample `CalibrationDataReader` class that provides calibration data to the quantization utilies. Note, however, that the following example uses random inputs as the calibration dataset only for simplicity. Using random input data results in inaccurate models in practice. +Model quantization for QNN EP requires the use of calibration input data. Using a calibration dataset that is representative of typical model inputs is crucial in generating an accurate quantized model. + +The following snippet defines a sample `CalibrationDataReader` class that generates random float32 input data. Note, that using random input data will most likely produce an inaccurate quantized model. ```python # data_reader.py import numpy as np import onnxruntime -import os from onnxruntime.quantization import CalibrationDataReader -NP_TYPE = { - "tensor(float)": np.float32, - "tensor(uint8)": np.uint8, - "tensor(int8)": np.int8, - "tensor(uint16)": np.uint16, - "tensor(int16)": np.int16, -} - -def get_np_type(onnx_type): - if onnx_type in NP_TYPE: - return NP_TYPE[onnx_type] - else: - raise Exception("Unhandled onnx_type in np_type_from_onnx_type") - class DataReader(CalibrationDataReader): def __init__(self, model_path: str): self.enum_data = None @@ -132,10 +117,10 @@ class DataReader(CalibrationDataReader): self.data_list = [] - # Generate 10 random inputs - # TODO: Load valid calibration input data + # Generate 10 random float32 inputs + # TODO: Load valid calibration input data for your model for _ in range(10): - input_data = {inp.name : np.random.random(inp.shape).astype(get_np_type(inp.type)) for inp in inputs} + input_data = {inp.name : np.random.random(inp.shape).astype(np.float32) for inp in inputs} self.data_list.append(input_data) self.datasize = len(self.data_list) @@ -152,7 +137,8 @@ class DataReader(CalibrationDataReader): ``` -The following snippet pre-processes the original model and then quantizes the pre-processed model using the above `CalibrationDataReader` class. +The following snippet pre-processes the original model and then quantizes the pre-processed model to use `uint16` activations and `uint8` weights. +QNN EP currently supports the `uint8` and `uint16` quantization data types. ```python # quantize_model.py @@ -160,8 +146,7 @@ The following snippet pre-processes the original model and then quantizes the pr import data_reader import numpy as np import onnx -import onnxruntime -from onnxruntime.quantization import QuantFormat, QuantType, quantize +from onnxruntime.quantization import QuantType, quantize from onnxruntime.quantization.execution_providers.qnn import get_qnn_qdq_config, qnn_preprocess_model if __name__ == "__main__": @@ -185,12 +170,23 @@ if __name__ == "__main__": quantize(model_to_quantize, output_model_path, qnn_config) ``` -Running `python quantize_model.py` will generate a quantized model (`model.qdq.onnx`) that can be run on Windows ARM64 devices via ONNX Runtime's QNN EP. +Running `python quantize_model.py` will generate a quantized model called `model.qdq.onnx` that can be run on Windows ARM64 devices via ONNX Runtime's QNN EP. Refer to the following pages for more information on API usage: - [quantization/execution_providers/qnn/preprocess.py](https://github.com/microsoft/onnxruntime/blob/23996bbbbe0406a5c8edbf6b7dbd71e5780d3f4b/onnxruntime/python/tools/quantization/execution_providers/qnn/preprocess.py#L16) - [quantization/execution_providers/qnn/quant_config.py](https://github.com/microsoft/onnxruntime/blob/23996bbbbe0406a5c8edbf6b7dbd71e5780d3f4b/onnxruntime/python/tools/quantization/execution_providers/qnn/quant_config.py#L20-L27) +### Running a quantized model on Windows ARM64 +The QNN HTP backend can execute quantized models on Windows ARM64 devices with Qualcomm chipsets. + +Install the nightly ONNX Runtime ARM64 python package with QNN EP. +```shell +python -m pip install -i https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/ORT-Nightly/pypi/simple/ ort-nightly-qnn +``` + +Download the Qualcomm AI Engine Direct SDK (QNN SDK) from https://qpm.qualcomm.com/main/tools/details/qualcomm_ai_engine_direct. ONNX Runtime's QNN EP is currently compatible with QNN SDK version 2.18. + + ## QNN context binary cache feature There's a QNN context which contains QNN graphs after converting, compiling, filnalizing the model. QNN can serialize the context into binary file, so that user can use it for futher inference direclty (without the QDQ model) to improve the model loading cost. From 5b9195e7dd1ff14c29216621d6cc45ef044b8172 Mon Sep 17 00:00:00 2001 From: adrianlizarraga Date: Tue, 6 Feb 2024 00:35:57 -0800 Subject: [PATCH 05/14] Update provider options. Add snippet for running QDQ model on HTP backend. --- .../QNN-ExecutionProvider.md | 102 +++++++++++++++--- 1 file changed, 90 insertions(+), 12 deletions(-) diff --git a/docs/execution-providers/QNN-ExecutionProvider.md b/docs/execution-providers/QNN-ExecutionProvider.md index e2e71c691af20..d008743d12b15 100644 --- a/docs/execution-providers/QNN-ExecutionProvider.md +++ b/docs/execution-providers/QNN-ExecutionProvider.md @@ -33,24 +33,28 @@ For build instructions, please see the [BUILD page](../build/eps.md#qnn). [prebuilt NuGet package](https://www.nuget.org/packages/Microsoft.ML.OnnxRuntime.QNN) ## Configuration Options -The QNN Execution Provider supports a number of configuration options. The `provider_option_keys`, `provider_options_values` enable different options for the application. Each `provider_options_keys` accepts values as shown below: +The QNN Execution Provider supports a number of configuration options. These provider options are specified as key-value string pairs. -|`provider_options_values` for `provider_options_keys = "backend_path"`|Description| +|`"backend_path"`|Description| |---|-----| |'libQnnCpu.so' or 'QnnCpu.dll'|Enable CPU backend. Useful for integration testing. CPU backend is a reference implementation of QNN operators| -|'libQnnHtp.so' or 'QnnHtp.dll'|Enable Htp backend. Offloads compute to NPU.| +|'libQnnHtp.so' or 'QnnHtp.dll'|Enable HTP backend. Offloads compute to NPU.| -|`provider_options_values` for `provider_options_keys = "profiling_level"`|Description| +|`"profiling_level"`|Description| |---|---| |'off'|| |'basic'|| |'detailed'|| -|`provider_options_values` for `provider_options_keys = "rpc_control_latency"`|Description| +|`"rpc_control_latency"`|Description| |---|---| |microseconds (string)|allows client to set up RPC control latency in microseconds| -|`provider_options_values` for `provider_options_keys = "htp_performance_mode"`|Description| +|`"vtcm_mb"`|Description| +|---|---| +|size in MB (string)|QNN VTCM size in MB, defaults to 0 (not set)| + +|`"htp_performance_mode"`|Description| |---|---| |'burst'|| |'balanced'|| @@ -62,8 +66,12 @@ The QNN Execution Provider supports a number of configuration options. The `prov |'power_saver'|| |'sustained_high_performance'|| +|`"qnn_saver_path"`|Description| +|---|---| +|filpath to 'QnnSaver.dll' or 'libQnnSaver.so'|File path to the QNN Saver backend library. Dumps QNN API calls to disk for replay/debugging.| + -|`provider_options_values` for `provider_options_keys = "qnn_context_priority"`|[Description](https://docs.qualcomm.com/bundle/publicresource/topics/80-63442-50/htp_yielding.html)| +|`"qnn_context_priority"`|[Description](https://docs.qualcomm.com/bundle/publicresource/topics/80-63442-50/htp_yielding.html)| |---|---| |'low'|| |'normal'|default.| @@ -71,13 +79,29 @@ The QNN Execution Provider supports a number of configuration options. The `prov |'high'|| -|`provider_options_values` for `provider_options_keys = "htp_graph_finalization_optimization_mode"`|Description| +|`"htp_graph_finalization_optimization_mode"`|Description| |---|---| |'0'|default.| |'1'|faster preparation time, less optimal graph.| |'2'|longer preparation time, more optimal graph.| |'3'|longest preparation time, most likely even more optimal graph.| +|`"soc_model"`|Description| +|---|---| +|Model number (string)|The SoC model number. Refer to the QNN SDK documentation for valid values. Defaults to "0" (unknown).| + +|`"htp_arch"`|Description| +|---|---| +|"0"|Default (none)| +|"68"|| +|"69"|| +|"73"|| +|"75"|| + +|`"device_id"`|Description| +|---|---| +|Device ID (string)|The ID of the device to use when setting `htp_arch`. Defaults to "0" (for single device).| + ## Running a model with QNN EP's HTP backend (Python) The QNN HTP backend, which offloads compute to the NPU, only supports quantized models. Models with 32-bit floating-point activations and weights must first be quantized to use a lower integer precision (e.g., 8-bit or 16-bit integers). @@ -96,7 +120,7 @@ python -m pip install -i https://aiinfra.pkgs.visualstudio.com/PublicPackages/_p Model quantization for QNN EP requires the use of calibration input data. Using a calibration dataset that is representative of typical model inputs is crucial in generating an accurate quantized model. -The following snippet defines a sample `CalibrationDataReader` class that generates random float32 input data. Note, that using random input data will most likely produce an inaccurate quantized model. +The following snippet defines a sample `CalibrationDataReader` class that generates random float32 input data. Note that using random input data will most likely produce an inaccurate quantized model. ```python # data_reader.py @@ -177,16 +201,70 @@ Refer to the following pages for more information on API usage: - [quantization/execution_providers/qnn/quant_config.py](https://github.com/microsoft/onnxruntime/blob/23996bbbbe0406a5c8edbf6b7dbd71e5780d3f4b/onnxruntime/python/tools/quantization/execution_providers/qnn/quant_config.py#L20-L27) ### Running a quantized model on Windows ARM64 -The QNN HTP backend can execute quantized models on Windows ARM64 devices with Qualcomm chipsets. +The following assumes that the [Qualcomm AI Engine SDK (QNN SDK)](https://qpm.qualcomm.com/main/tools/details/qualcomm_ai_engine_direct) has already been downloaded and installed to a location such as `C:\Qualcomm\AIStack\QNN\2.18.0.240101`. + +First, determine the HTP architecture version for your device by referring to the QNN SDK documentation: +- /docs/QNN/general/htp/htp_backend.html#qnn-htp-backend-api +- /docs/QNN/general/overview.html#supported-snapdragon-devices + +For example, Snapdragon 8cx Gen 3 (SC8280X) devices have an HTP architecture value of 68, and Snapdragon 8cx Gen 4 (SC8380XP) have an HTP architecture value of 73. In the following, replace `` with your device's HTP architecture value. + +Copy the `.so` file `\lib\hexagon-v\unsigned\libQnnHtpVSkel.so` to the folder `\lib\aarch64-windows-msvc\`. For example, the following terminal command copies the `libQnnHtpV73Skel.so` file: +``` +cp C:\Qualcomm\AIStack\QNN\2.18.0.240101\lib\hexagon-v73\unsigned\libQnnHtpV73Skel.so C:\Qualcomm\AIStack\QNN\2.18.0.240101\lib\aarch64-windows-msvc\ +``` -Install the nightly ONNX Runtime ARM64 python package with QNN EP. +Add the `\lib\aarch64-windows-msvc\` directory your Windows PATH environment variable: +``` +- Open the `Edit the system environment variables` Control Panel. +- Click on `Environment variables`. +- Highlight the `Path` entry under `User variables for ..` and click `Edit`. +- Add a new entry that points to `\lib\aarch64-windows-msvc\` +``` + +Install the nightly ONNX Runtime ARM64 python package for QNN EP: ```shell python -m pip install -i https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/ORT-Nightly/pypi/simple/ ort-nightly-qnn ``` -Download the Qualcomm AI Engine Direct SDK (QNN SDK) from https://qpm.qualcomm.com/main/tools/details/qualcomm_ai_engine_direct. ONNX Runtime's QNN EP is currently compatible with QNN SDK version 2.18. +The following Python snippet creates an ONNX Runtime session with QNN EP and runs the quantized model `model.qdq.onnx` on the HTP backend. + +```python +# run_qdq_model.py + +import onnxruntime +import numpy as np + +options = onnxruntime.SessionOptions() + +# (Optional) Enable configuration that raises an exception if the model can't be +# run entirely on the QNN HTP backend. +options.add_session_config_entry("session.disable_cpu_ep_fallback", "1") + +# Create an ONNX Runtime session. +# TODO: Provide the path to your ONNX model +session = onnxruntime.InferenceSession("model.qdq.onnx", + sess_options=options, + providers=["QNNExecutionProvider"], + provider_options=[{"backend_path": "QnnHtp.dll"}]) # Provide path to Htp dll in QNN SDK + +# Run the model with your input. +# TODO: Use numpy to load your actual input from a file or generate random input. +input0 = np.ones((1,2,1,4), dtype=np.float32) +result = session.run(None, {"input0": input0}) + +# Print output. +print(result) +``` + +Running `python run_qdq_model.py` will execute the quantized model on the QNN HTP backend. +Notice that the session has been optionally configured to raise an exception if the entire model cannot be executed on the QNN HTP backend. This is useful to check that the quantized model is fully supported by QNN EP. +Available session configurations include: +- [session.disable_cpu_ep_fallback](https://github.com/microsoft/onnxruntime/blob/a4cfdc1c28ac95ec6fd0667e856b6a6b8dd1020c/include/onnxruntime/core/session/onnxruntime_session_options_config_keys.h#L229): Disables fallback of unsupported operators to the CPU EP. +- [ep.context_enable](https://github.com/microsoft/onnxruntime/blob/a4cfdc1c28ac95ec6fd0667e856b6a6b8dd1020c/include/onnxruntime/core/session/onnxruntime_session_options_config_keys.h#L243): Enable EP context feature to dump a cached version of the model in order to decrease session creation time. +Also, the above snippet only specifies the `backend_path` provider option, which denotes the path to the QNN SDK HTP dll. Refer to the [Configuration options section](./#configuration-options) for an list of all QNN EP provider options. ## QNN context binary cache feature There's a QNN context which contains QNN graphs after converting, compiling, filnalizing the model. QNN can serialize the context into binary file, so that user can use it for futher inference direclty (without the QDQ model) to improve the model loading cost. From 40b3a5533ce124a84e0d3da78f0d7add37c25d8f Mon Sep 17 00:00:00 2001 From: adrianlizarraga Date: Tue, 6 Feb 2024 00:59:08 -0800 Subject: [PATCH 06/14] More clean up --- .../QNN-ExecutionProvider.md | 35 ++++++++++--------- 1 file changed, 18 insertions(+), 17 deletions(-) diff --git a/docs/execution-providers/QNN-ExecutionProvider.md b/docs/execution-providers/QNN-ExecutionProvider.md index d008743d12b15..31535dd9ca0d1 100644 --- a/docs/execution-providers/QNN-ExecutionProvider.md +++ b/docs/execution-providers/QNN-ExecutionProvider.md @@ -103,24 +103,24 @@ The QNN Execution Provider supports a number of configuration options. These pro |Device ID (string)|The ID of the device to use when setting `htp_arch`. Defaults to "0" (for single device).| ## Running a model with QNN EP's HTP backend (Python) -The QNN HTP backend, which offloads compute to the NPU, only supports quantized models. Models with 32-bit floating-point activations and weights must first be quantized to use a lower integer precision (e.g., 8-bit or 16-bit integers). +

Offline workflow for quantizing an ONNX model for use on QNN EP

-This section provides instructions for quantizing a model and then running the quantized model on QNN EP's HTP backend using Python APIs. Please refer to the [quantization page](../performance/model-optimizations/quantization.md) for a broader overview of quantization concepts. +The QNN HTP backend only supports quantized models. Models with 32-bit floating-point activations and weights must first be quantized to use a lower integer precision (e.g., 8-bit or 16-bit integers). -

Offline workflow for quantizing an ONNX model for use on QNN EP

+This section provides instructions for quantizing a model and then running the quantized model on QNN EP's HTP backend using Python APIs. Please refer to the [quantization page](../performance/model-optimizations/quantization.md) for a broader overview of quantization concepts. ### Generating a quantized model (x64) The ONNX Runtime python package provides utilities for quantizing ONNX models via the `onnxruntime.quantization` import. The quantization utilities are currently only supported on x86_64. -Therefore, it is recommend to either use an x64 machine to quantize models or, alternatively, use a separate x64 python installation on Windows ARM64 machines. +Therefore, it is recommended to either use an x64 machine to quantize models or, alternatively, use a separate x64 python installation on Windows ARM64 machines. Install the nightly ONNX Runtime x64 python package. ```shell python -m pip install -i https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/ORT-Nightly/pypi/simple/ ort-nightly ``` -Model quantization for QNN EP requires the use of calibration input data. Using a calibration dataset that is representative of typical model inputs is crucial in generating an accurate quantized model. +Quantization for QNN EP requires the use of calibration input data. Using a calibration dataset that is representative of typical model inputs is crucial in generating an accurate quantized model. -The following snippet defines a sample `CalibrationDataReader` class that generates random float32 input data. Note that using random input data will most likely produce an inaccurate quantized model. +The following snippet defines a sample `DataReader` class that generates random float32 input data. Note that using random input data will most likely produce an inaccurate quantized model. ```python # data_reader.py @@ -162,7 +162,7 @@ class DataReader(CalibrationDataReader): ``` The following snippet pre-processes the original model and then quantizes the pre-processed model to use `uint16` activations and `uint8` weights. -QNN EP currently supports the `uint8` and `uint16` quantization data types. +QNN EP typically supports the `uint8`, and `uint16` quantization data types. Refer to the [QNN SDK operator documentation](https://docs.qualcomm.com/bundle/publicresource/topics/80-63442-50/HtpOpDefSupplement.html) for the data type requirements for each QNN operator. ```python # quantize_model.py @@ -196,30 +196,31 @@ if __name__ == "__main__": Running `python quantize_model.py` will generate a quantized model called `model.qdq.onnx` that can be run on Windows ARM64 devices via ONNX Runtime's QNN EP. -Refer to the following pages for more information on API usage: +Refer to the following pages for more information on quantization utilities usage: +- [Quantization example for mobilenet on CPU EP](https://github.com/microsoft/onnxruntime-inference-examples/tree/main/quantization/image_classification/cpu) - [quantization/execution_providers/qnn/preprocess.py](https://github.com/microsoft/onnxruntime/blob/23996bbbbe0406a5c8edbf6b7dbd71e5780d3f4b/onnxruntime/python/tools/quantization/execution_providers/qnn/preprocess.py#L16) - [quantization/execution_providers/qnn/quant_config.py](https://github.com/microsoft/onnxruntime/blob/23996bbbbe0406a5c8edbf6b7dbd71e5780d3f4b/onnxruntime/python/tools/quantization/execution_providers/qnn/quant_config.py#L20-L27) ### Running a quantized model on Windows ARM64 -The following assumes that the [Qualcomm AI Engine SDK (QNN SDK)](https://qpm.qualcomm.com/main/tools/details/qualcomm_ai_engine_direct) has already been downloaded and installed to a location such as `C:\Qualcomm\AIStack\QNN\2.18.0.240101`. +The following assumes that the [Qualcomm AI Engine SDK (QNN SDK)](https://qpm.qualcomm.com/main/tools/details/qualcomm_ai_engine_direct) has already been downloaded and installed to a location such as `C:\Qualcomm\AIStack\QNN\2.18.0.240101`, hereafter referred to as `QNN_SDK`. First, determine the HTP architecture version for your device by referring to the QNN SDK documentation: -- /docs/QNN/general/htp/htp_backend.html#qnn-htp-backend-api -- /docs/QNN/general/overview.html#supported-snapdragon-devices +- QNN_SDK\docs\QNN\general\htp\htp_backend.html#qnn-htp-backend-api +- QNN_SDK\docs\QNN\general\overview.html#supported-snapdragon-devices For example, Snapdragon 8cx Gen 3 (SC8280X) devices have an HTP architecture value of 68, and Snapdragon 8cx Gen 4 (SC8380XP) have an HTP architecture value of 73. In the following, replace `` with your device's HTP architecture value. -Copy the `.so` file `\lib\hexagon-v\unsigned\libQnnHtpVSkel.so` to the folder `\lib\aarch64-windows-msvc\`. For example, the following terminal command copies the `libQnnHtpV73Skel.so` file: +Copy the `.so` file `QNN_SDK\lib\hexagon-v\unsigned\libQnnHtpVSkel.so` to the folder `QNN_SDK\lib\aarch64-windows-msvc\`. For example, the following terminal command copies the `libQnnHtpV73Skel.so` file: ``` -cp C:\Qualcomm\AIStack\QNN\2.18.0.240101\lib\hexagon-v73\unsigned\libQnnHtpV73Skel.so C:\Qualcomm\AIStack\QNN\2.18.0.240101\lib\aarch64-windows-msvc\ +cp QNN_SDK\lib\hexagon-v73\unsigned\libQnnHtpV73Skel.so QNN_SDK\lib\aarch64-windows-msvc\ ``` -Add the `\lib\aarch64-windows-msvc\` directory your Windows PATH environment variable: +Add the `QNN_SDK\lib\aarch64-windows-msvc\` directory your Windows PATH environment variable: ``` - Open the `Edit the system environment variables` Control Panel. - Click on `Environment variables`. - Highlight the `Path` entry under `User variables for ..` and click `Edit`. -- Add a new entry that points to `\lib\aarch64-windows-msvc\` +- Add a new entry that points to `QNN_SDK\lib\aarch64-windows-msvc\` ``` Install the nightly ONNX Runtime ARM64 python package for QNN EP: @@ -257,14 +258,14 @@ result = session.run(None, {"input0": input0}) print(result) ``` -Running `python run_qdq_model.py` will execute the quantized model on the QNN HTP backend. +Running `python run_qdq_model.py` will execute the quantized `model.qdq.onnx` model on the QNN HTP backend. Notice that the session has been optionally configured to raise an exception if the entire model cannot be executed on the QNN HTP backend. This is useful to check that the quantized model is fully supported by QNN EP. Available session configurations include: - [session.disable_cpu_ep_fallback](https://github.com/microsoft/onnxruntime/blob/a4cfdc1c28ac95ec6fd0667e856b6a6b8dd1020c/include/onnxruntime/core/session/onnxruntime_session_options_config_keys.h#L229): Disables fallback of unsupported operators to the CPU EP. - [ep.context_enable](https://github.com/microsoft/onnxruntime/blob/a4cfdc1c28ac95ec6fd0667e856b6a6b8dd1020c/include/onnxruntime/core/session/onnxruntime_session_options_config_keys.h#L243): Enable EP context feature to dump a cached version of the model in order to decrease session creation time. -Also, the above snippet only specifies the `backend_path` provider option, which denotes the path to the QNN SDK HTP dll. Refer to the [Configuration options section](./#configuration-options) for an list of all QNN EP provider options. +Also, the above snippet only specifies the `backend_path` provider option. Refer to the [Configuration options section](./QNN-ExecutionProvider.md#configuration-options) for a list of all available QNN EP provider options. ## QNN context binary cache feature There's a QNN context which contains QNN graphs after converting, compiling, filnalizing the model. QNN can serialize the context into binary file, so that user can use it for futher inference direclty (without the QDQ model) to improve the model loading cost. From d155202207aaf939ccdb72d75954a9f41bfd85c6 Mon Sep 17 00:00:00 2001 From: adrianlizarraga Date: Tue, 6 Feb 2024 01:24:21 -0800 Subject: [PATCH 07/14] Add python snippets for QNN context cache docs --- .../QNN-ExecutionProvider.md | 30 ++++++++++++++++--- 1 file changed, 26 insertions(+), 4 deletions(-) diff --git a/docs/execution-providers/QNN-ExecutionProvider.md b/docs/execution-providers/QNN-ExecutionProvider.md index 31535dd9ca0d1..3ba9e7ed4e948 100644 --- a/docs/execution-providers/QNN-ExecutionProvider.md +++ b/docs/execution-providers/QNN-ExecutionProvider.md @@ -30,7 +30,10 @@ ONNX Runtime QNN Execution Provider has been built and tested with QNN 2.18.x an ## Build For build instructions, please see the [BUILD page](../build/eps.md#qnn). -[prebuilt NuGet package](https://www.nuget.org/packages/Microsoft.ML.OnnxRuntime.QNN) + +Alternatively, ONNX Runtime with QNN EP can be installed from: +- [NuGet package](https://www.nuget.org/packages/Microsoft.ML.OnnxRuntime.QNN) +- Nightly Python package (Windows ARM64): `python -m pip install -i https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/ORT-Nightly/pypi/simple/ ort-nightly-qnn` ## Configuration Options The QNN Execution Provider supports a number of configuration options. These provider options are specified as key-value string pairs. @@ -162,7 +165,8 @@ class DataReader(CalibrationDataReader): ``` The following snippet pre-processes the original model and then quantizes the pre-processed model to use `uint16` activations and `uint8` weights. -QNN EP typically supports the `uint8`, and `uint16` quantization data types. Refer to the [QNN SDK operator documentation](https://docs.qualcomm.com/bundle/publicresource/topics/80-63442-50/HtpOpDefSupplement.html) for the data type requirements for each QNN operator. +Although the quantization utilities expose the `uint8`, `int8`, `uint16`, and `int16` quantization data types, QNN operators typically support the `uint8` and `uint16` data types. +Refer to the [QNN SDK operator documentation](https://docs.qualcomm.com/bundle/publicresource/topics/80-63442-50/HtpOpDefSupplement.html) for the data type requirements of each QNN operator. ```python # quantize_model.py @@ -215,7 +219,7 @@ Copy the `.so` file `QNN_SDK\lib\hexagon-v\unsigned\libQnnHtpVCreateSessionOptions(&session_options)); g_ort->AddSessionConfigEntry(session_options, kOrtSessionOptionEpContextEnable, "1"); ``` +```python +# Python +import onnxruntime + +options = onnxruntime.SessionOptions() +options.add_session_config_entry("ep.context_enable", "1") +``` + ### Configure the context binary file path The generated Onnx model with QNN context binary is default to [input_QDQ_model_path]_ctx.onnx in case user does not specify the path. User can to set the path in the session option with the key "ep.context_file_path". Example code below: @@ -305,6 +317,11 @@ so.AddConfigEntry(kOrtSessionOptionEpContextFilePath, "./model_a_ctx.onnx"); g_ort->AddSessionConfigEntry(session_options, kOrtSessionOptionEpContextFilePath, "./model_a_ctx.onnx"); ``` +```python +# Python +options.add_session_config_entry("ep.context_file_path", "./model_a_ctx.onnx") +``` + ### Disable the embed mode The QNN context binary content is embeded in the generated Onnx model by default. User can to disable it by setting "ep.context_embed_mode" to "0". In that case, a bin file will be generated separately. The file name looks like [ctx.onnx]_QNNExecutionProvider_QNN_[hash_id]_x_x.bin. The name is provided by Ort and tracked in the generated Onnx model. It will cause problems if any changes to the bin file. This bin file needs to sit together with the generated Onnx file. @@ -316,6 +333,11 @@ so.AddConfigEntry(kOrtSessionOptionEpContextEmbedMode, "0"); g_ort->AddSessionConfigEntry(session_options, kOrtSessionOptionEpContextEmbedMode, "0"); ``` +```python +# Python +options.add_session_config_entry("ep.context_embed_mode", "0") +``` + ## Usage ### C++ C API details are [here](../get-started/with-c.md). From d0dbddac160b3e0a938586a684eb1f86ac838d8c Mon Sep 17 00:00:00 2001 From: adrianlizarraga Date: Tue, 6 Feb 2024 01:33:49 -0800 Subject: [PATCH 08/14] More cleanup --- docs/execution-providers/QNN-ExecutionProvider.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/docs/execution-providers/QNN-ExecutionProvider.md b/docs/execution-providers/QNN-ExecutionProvider.md index 3ba9e7ed4e948..f39a26a6b752a 100644 --- a/docs/execution-providers/QNN-ExecutionProvider.md +++ b/docs/execution-providers/QNN-ExecutionProvider.md @@ -124,6 +124,7 @@ python -m pip install -i https://aiinfra.pkgs.visualstudio.com/PublicPackages/_p Quantization for QNN EP requires the use of calibration input data. Using a calibration dataset that is representative of typical model inputs is crucial in generating an accurate quantized model. The following snippet defines a sample `DataReader` class that generates random float32 input data. Note that using random input data will most likely produce an inaccurate quantized model. +Refer to the [implementation of a Resnet data reader](https://github.com/microsoft/onnxruntime-inference-examples/blob/main/quantization/image_classification/cpu/resnet50_data_reader.py) for one example of how to create a `CalibrationDataReader` that provides input from image files on disk. ```python # data_reader.py @@ -200,7 +201,7 @@ if __name__ == "__main__": Running `python quantize_model.py` will generate a quantized model called `model.qdq.onnx` that can be run on Windows ARM64 devices via ONNX Runtime's QNN EP. -Refer to the following pages for more information on quantization utilities usage: +Refer to the following pages for more information on usage of the quantization utilities: - [Quantization example for mobilenet on CPU EP](https://github.com/microsoft/onnxruntime-inference-examples/tree/main/quantization/image_classification/cpu) - [quantization/execution_providers/qnn/preprocess.py](https://github.com/microsoft/onnxruntime/blob/23996bbbbe0406a5c8edbf6b7dbd71e5780d3f4b/onnxruntime/python/tools/quantization/execution_providers/qnn/preprocess.py#L16) - [quantization/execution_providers/qnn/quant_config.py](https://github.com/microsoft/onnxruntime/blob/23996bbbbe0406a5c8edbf6b7dbd71e5780d3f4b/onnxruntime/python/tools/quantization/execution_providers/qnn/quant_config.py#L20-L27) @@ -264,12 +265,12 @@ print(result) Running `python run_qdq_model.py` will execute the quantized `model.qdq.onnx` model on the QNN HTP backend. -Notice that the session has been optionally configured to raise an exception if the entire model cannot be executed on the QNN HTP backend. This is useful to check that the quantized model is fully supported by QNN EP. +Notice that the session has been optionally configured to raise an exception if the entire model cannot be executed on the QNN HTP backend. This is useful for verifying that the quantized model is fully supported by QNN EP. Available session configurations include: - [session.disable_cpu_ep_fallback](https://github.com/microsoft/onnxruntime/blob/a4cfdc1c28ac95ec6fd0667e856b6a6b8dd1020c/include/onnxruntime/core/session/onnxruntime_session_options_config_keys.h#L229): Disables fallback of unsupported operators to the CPU EP. - [ep.context_enable](https://github.com/microsoft/onnxruntime/blob/a4cfdc1c28ac95ec6fd0667e856b6a6b8dd1020c/include/onnxruntime/core/session/onnxruntime_session_options_config_keys.h#L243): [Enable QNN context cache](./QNN-ExecutionProvider.md#qnn-context-binary-cache-feature) feature to dump a cached version of the model in order to decrease session creation time. -Also, the above snippet only specifies the `backend_path` provider option. Refer to the [Configuration options section](./QNN-ExecutionProvider.md#configuration-options) for a list of all available QNN EP provider options. +The above snippet only specifies the `backend_path` provider option. Refer to the [Configuration options section](./QNN-ExecutionProvider.md#configuration-options) for a list of all available QNN EP provider options. ## QNN context binary cache feature There's a QNN context which contains QNN graphs after converting, compiling, filnalizing the model. QNN can serialize the context into binary file, so that user can use it for futher inference direclty (without the QDQ model) to improve the model loading cost. From b7318e9b1dac2120a31bfd455f871f4b76e8a092 Mon Sep 17 00:00:00 2001 From: adrianlizarraga Date: Tue, 6 Feb 2024 10:16:44 -0800 Subject: [PATCH 09/14] Add requirements for python arm64 package --- docs/execution-providers/QNN-ExecutionProvider.md | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/docs/execution-providers/QNN-ExecutionProvider.md b/docs/execution-providers/QNN-ExecutionProvider.md index f39a26a6b752a..e5e27deb8ad50 100644 --- a/docs/execution-providers/QNN-ExecutionProvider.md +++ b/docs/execution-providers/QNN-ExecutionProvider.md @@ -31,9 +31,15 @@ ONNX Runtime QNN Execution Provider has been built and tested with QNN 2.18.x an ## Build For build instructions, please see the [BUILD page](../build/eps.md#qnn). +## Pre-built Packages Alternatively, ONNX Runtime with QNN EP can be installed from: - [NuGet package](https://www.nuget.org/packages/Microsoft.ML.OnnxRuntime.QNN) -- Nightly Python package (Windows ARM64): `python -m pip install -i https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/ORT-Nightly/pypi/simple/ ort-nightly-qnn` +- Nightly Python package (Windows ARM64): + - Requirements: + - Windows ARM64 + - Python 3.11.x + - Numpy 1.25.2 or >= 1.26.4 + - Install: `python -m pip install -i https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/ORT-Nightly/pypi/simple/ ort-nightly-qnn` ## Configuration Options The QNN Execution Provider supports a number of configuration options. These provider options are specified as key-value string pairs. @@ -113,7 +119,7 @@ The QNN HTP backend only supports quantized models. Models with 32-bit floating- This section provides instructions for quantizing a model and then running the quantized model on QNN EP's HTP backend using Python APIs. Please refer to the [quantization page](../performance/model-optimizations/quantization.md) for a broader overview of quantization concepts. ### Generating a quantized model (x64) -The ONNX Runtime python package provides utilities for quantizing ONNX models via the `onnxruntime.quantization` import. The quantization utilities are currently only supported on x86_64. +The ONNX Runtime python package provides utilities for quantizing ONNX models via the `onnxruntime.quantization` import. The quantization utilities are currently only supported on x86_64 due to issues install the `onnx` package on ARM64. Therefore, it is recommended to either use an x64 machine to quantize models or, alternatively, use a separate x64 python installation on Windows ARM64 machines. Install the nightly ONNX Runtime x64 python package. @@ -228,7 +234,7 @@ Add the `QNN_SDK\lib\aarch64-windows-msvc\` directory to your Windows PATH envir - Add a new entry that points to `QNN_SDK\lib\aarch64-windows-msvc\` ``` -Install the nightly ONNX Runtime ARM64 python package for QNN EP: +Install the nightly ONNX Runtime ARM64 python package for QNN EP (requires Python 3.11.x and Numpy 1.25.2 or >= 1.26.4): ```shell python -m pip install -i https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/ORT-Nightly/pypi/simple/ ort-nightly-qnn ``` From 5ed9b17e13a97464ed38cdd91643b506035265a4 Mon Sep 17 00:00:00 2001 From: adrianlizarraga Date: Tue, 6 Feb 2024 10:20:50 -0800 Subject: [PATCH 10/14] Fix typo --- docs/execution-providers/QNN-ExecutionProvider.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/execution-providers/QNN-ExecutionProvider.md b/docs/execution-providers/QNN-ExecutionProvider.md index e5e27deb8ad50..1a97e6ae22ecd 100644 --- a/docs/execution-providers/QNN-ExecutionProvider.md +++ b/docs/execution-providers/QNN-ExecutionProvider.md @@ -119,7 +119,7 @@ The QNN HTP backend only supports quantized models. Models with 32-bit floating- This section provides instructions for quantizing a model and then running the quantized model on QNN EP's HTP backend using Python APIs. Please refer to the [quantization page](../performance/model-optimizations/quantization.md) for a broader overview of quantization concepts. ### Generating a quantized model (x64) -The ONNX Runtime python package provides utilities for quantizing ONNX models via the `onnxruntime.quantization` import. The quantization utilities are currently only supported on x86_64 due to issues install the `onnx` package on ARM64. +The ONNX Runtime python package provides utilities for quantizing ONNX models via the `onnxruntime.quantization` import. The quantization utilities are currently only supported on x86_64 due to issues installing the `onnx` package on ARM64. Therefore, it is recommended to either use an x64 machine to quantize models or, alternatively, use a separate x64 python installation on Windows ARM64 machines. Install the nightly ONNX Runtime x64 python package. From f1b0a99cfd73e5df382a73273478f98ea3cc2a13 Mon Sep 17 00:00:00 2001 From: adrianlizarraga Date: Tue, 6 Feb 2024 11:30:53 -0800 Subject: [PATCH 11/14] Add list of supported operators; Add info about dynamic shapes --- .../QNN-ExecutionProvider.md | 94 ++++++++++++++++++- 1 file changed, 91 insertions(+), 3 deletions(-) diff --git a/docs/execution-providers/QNN-ExecutionProvider.md b/docs/execution-providers/QNN-ExecutionProvider.md index 1a97e6ae22ecd..e46c03c4cbead 100644 --- a/docs/execution-providers/QNN-ExecutionProvider.md +++ b/docs/execution-providers/QNN-ExecutionProvider.md @@ -111,6 +111,90 @@ The QNN Execution Provider supports a number of configuration options. These pro |---|---| |Device ID (string)|The ID of the device to use when setting `htp_arch`. Defaults to "0" (for single device).| +## Supported ONNX operators +|Operator|Notes| +|---|---| +|ai.onnx:Abs|| +|ai.onnx:Add|| +|ai.onnx:And|| +|ai.onnx:ArgMax|| +|ai.onnx:ArgMin|| +|ai.onnx:Asin|| +|ai.onnx:Atan|| +|ai.onnx:AveragePool|| +|ai.onnx:BatchNormalization|| +|ai.onnx:Cast|| +|ai.onnx:Clip|| +|ai.onnx:Concat|| +|ai.onnx:Conv|| +|ai.onnx:ConvTranspose|| +|ai.onnx:Cos|| +|ai.onnx:DepthToSpace|| +|ai.onnx:DequantizeLinear|| +|ai.onnx:Div|| +|ai.onnx:Elu|| +|ai.onnx:Equal|| +|ai.onnx:Exp|| +|ai.onnx:Expand|| +|ai.onnx:Flatten|| +|ai.onnx:Floor|| +|ai.onnx:Gather|Only support positive indices| +|ai.onnx:Gemm|| +|ai.onnx:GlobalAveragePool|| +|ai.onnx:Greater|| +|ai.onnx:GreaterOrEqual|| +|ai.onnx:GridSample|| +|ai.onnx:HardSwish|| +|ai.onnx:InstanceNormalization|| +|ai.onnx:LRN|| +|ai.onnx:LayerNormalization|| +|ai.onnx:LeakyRelu|| +|ai.onnx:Less|| +|ai.onnx:LessOrEqual|| +|ai.onnx:Log|| +|ai.onnx:LogSoftmax|| +|ai.onnx:LpNormalization|p == 2| +|ai.onnx:MatMul|Supported input data types on HTP backend: (uint8, uint8), (uint8, uint16), (uint16, uint8)| +|ai.onnx:Max|| +|ai.onnx:MaxPool|| +|ai.onnx:Min|| +|ai.onnx:Mul|| +|ai.onnx:Neg|| +|ai.onnx:Not|| +|ai.onnx:Or|| +|ai.onnx:Prelu|| +|ai.onnx:Pad|| +|ai.onnx:Pow|| +|ai.onnx:QuantizeLinear|| +|ai.onnx:ReduceMax|| +|ai.onnx:ReduceMean|| +|ai.onnx:ReduceMin|| +|ai.onnx:ReduceProd|| +|ai.onnx:ReduceSum|| +|ai.onnx:Relu|| +|ai.onnx:Resize|| +|ai.onnx:Round|| +|ai.onnx:Sigmoid|| +|ai.onnx:Sign|| +|ai.onnx:Sin|| +|ai.onnx:Slice|| +|ai.onnx:Softmax|| +|ai.onnx:SpaceToDepth|| +|ai.onnx:Split|| +|ai.onnx:Sqrt|| +|ai.onnx:Squeeze|| +|ai.onnx:Sub|| +|ai.onnx:Tanh|| +|ai.onnx:Tile|| +|ai.onnx:TopK|| +|ai.onnx:Transpose|| +|ai.onnx:Unsqueeze|| +|ai.onnx:Where|| +|com.microsoft:DequantizeLinear|Provides 16-bit integer dequantization support| +|com.microsoft:QuantizeLinear|Provides 16-bit integer quantization support| + +Supported data types vary by operator and QNN backend. Refer to the [QNN SDK documentation](https://docs.qualcomm.com/bundle/publicresource/topics/80-63442-50/operations.html) for more information. + ## Running a model with QNN EP's HTP backend (Python)

Offline workflow for quantizing an ONNX model for use on QNN EP

@@ -118,6 +202,10 @@ The QNN HTP backend only supports quantized models. Models with 32-bit floating- This section provides instructions for quantizing a model and then running the quantized model on QNN EP's HTP backend using Python APIs. Please refer to the [quantization page](../performance/model-optimizations/quantization.md) for a broader overview of quantization concepts. +### Model requirements +QNN EP does not support models with dynamic shapes (e.g., a dynamic batch size). Dynamic shapes must be fixed to a specific value. Refer to the documentation for [making dynamic input shapes fixed](../tutorials/mobile/helpers/make-dynamic-shape-fixed.md) for more information. + +Additionally, QNN EP supports a subset of ONNX operators (e.g., Loops and Ifs are not supported). Refer to the [list of supported ONNX operators](./QNN-ExecutionProvider.md#supported-onnx-operators). ### Generating a quantized model (x64) The ONNX Runtime python package provides utilities for quantizing ONNX models via the `onnxruntime.quantization` import. The quantization utilities are currently only supported on x86_64 due to issues installing the `onnx` package on ARM64. Therefore, it is recommended to either use an x64 machine to quantize models or, alternatively, use a separate x64 python installation on Windows ARM64 machines. @@ -191,7 +279,7 @@ if __name__ == "__main__": # Pre-process the original float32 model. preproc_model_path = "model.preproc.onnx" - model_changed = qnn_preprocess_model(input_model_path, prepoc_model_path) + model_changed = qnn_preprocess_model(input_model_path, preproc_model_path) model_to_quantize = preproc_model_path if model_changed else input_model_path # Generate a suitable quantization configuration for this model. @@ -262,8 +350,8 @@ session = onnxruntime.InferenceSession("model.qdq.onnx", # Run the model with your input. # TODO: Use numpy to load your actual input from a file or generate random input. -input0 = np.ones((1,2,1,4), dtype=np.float32) -result = session.run(None, {"input0": input0}) +input0 = np.ones((1,3,224,224), dtype=np.float32) +result = session.run(None, {"input": input0}) # Print output. print(result) From 6804732650373103d82956deba38eba70dc472ad Mon Sep 17 00:00:00 2001 From: adrianlizarraga Date: Tue, 6 Feb 2024 11:36:26 -0800 Subject: [PATCH 12/14] Add new line --- docs/execution-providers/QNN-ExecutionProvider.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/execution-providers/QNN-ExecutionProvider.md b/docs/execution-providers/QNN-ExecutionProvider.md index e46c03c4cbead..ae0a4bb5d2d4c 100644 --- a/docs/execution-providers/QNN-ExecutionProvider.md +++ b/docs/execution-providers/QNN-ExecutionProvider.md @@ -112,6 +112,7 @@ The QNN Execution Provider supports a number of configuration options. These pro |Device ID (string)|The ID of the device to use when setting `htp_arch`. Defaults to "0" (for single device).| ## Supported ONNX operators + |Operator|Notes| |---|---| |ai.onnx:Abs|| From 155923782dc4be53874e7e7445418ced3cddd2a4 Mon Sep 17 00:00:00 2001 From: adrianlizarraga Date: Tue, 6 Feb 2024 11:44:07 -0800 Subject: [PATCH 13/14] Typo --- docs/execution-providers/QNN-ExecutionProvider.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/execution-providers/QNN-ExecutionProvider.md b/docs/execution-providers/QNN-ExecutionProvider.md index ae0a4bb5d2d4c..b2c4d7e3dde6a 100644 --- a/docs/execution-providers/QNN-ExecutionProvider.md +++ b/docs/execution-providers/QNN-ExecutionProvider.md @@ -139,7 +139,7 @@ The QNN Execution Provider supports a number of configuration options. These pro |ai.onnx:Expand|| |ai.onnx:Flatten|| |ai.onnx:Floor|| -|ai.onnx:Gather|Only support positive indices| +|ai.onnx:Gather|Only supports positive indices| |ai.onnx:Gemm|| |ai.onnx:GlobalAveragePool|| |ai.onnx:Greater|| From 5af86cc86586abd0d629339a9f5dbb3fbcd3f5fb Mon Sep 17 00:00:00 2001 From: adrianlizarraga Date: Tue, 6 Feb 2024 11:47:32 -0800 Subject: [PATCH 14/14] Add Gelu --- docs/execution-providers/QNN-ExecutionProvider.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/execution-providers/QNN-ExecutionProvider.md b/docs/execution-providers/QNN-ExecutionProvider.md index b2c4d7e3dde6a..6ad125c231bef 100644 --- a/docs/execution-providers/QNN-ExecutionProvider.md +++ b/docs/execution-providers/QNN-ExecutionProvider.md @@ -140,6 +140,7 @@ The QNN Execution Provider supports a number of configuration options. These pro |ai.onnx:Flatten|| |ai.onnx:Floor|| |ai.onnx:Gather|Only supports positive indices| +|ai.onnx:Gelu|| |ai.onnx:Gemm|| |ai.onnx:GlobalAveragePool|| |ai.onnx:Greater|| @@ -192,6 +193,7 @@ The QNN Execution Provider supports a number of configuration options. These pro |ai.onnx:Unsqueeze|| |ai.onnx:Where|| |com.microsoft:DequantizeLinear|Provides 16-bit integer dequantization support| +|com.microsoft:Gelu|| |com.microsoft:QuantizeLinear|Provides 16-bit integer quantization support| Supported data types vary by operator and QNN backend. Refer to the [QNN SDK documentation](https://docs.qualcomm.com/bundle/publicresource/topics/80-63442-50/operations.html) for more information.