Add changes for b797272

Reference/sr3_options.7.html

@@ -1062,6 +1062,16 @@ <h4>LogFormat ( default: %(asctime)s [%(levelname)s] %(name)s %(funcName)s %(mes
 <li><p><a class="reference external" href="https://docs.python.org/3/library/logging.html#logrecord-attributes">https://docs.python.org/3/library/logging.html#logrecord-attributes</a></p></li>
 </ul>
 </section>
+<section id="logjson-flag-default-false-experimental">
+<h4>logJson &lt;flag&gt; (default: False) EXPERIMENTAL<a class="headerlink" href="#logjson-flag-default-false-experimental" title="Link to this heading"></a></h4>
+<p>when <em>logJson on</em> is set, a second log file with the .json extension is made beside the
+normal .log file.  Each line of the .json logs is a .json structure, containing
+a message written by the flow log. It does not contain unformatted output of
+sub-shells, and plugins which may produce arbitrary output.</p>
+<p>The .log file will contain the output of sub-programs launched by the flow,
+and the .json will only contain properly formatted log messages from the application itself
+and properly written callbacks (that use normal python logging mechanisms.)</p>
+</section>
 <section id="loglevel-default-info">
 <h4>logLevel ( default: info )<a class="headerlink" href="#loglevel-default-info" title="Link to this heading"></a></h4>
 <p>The level of logging as expressed by python’s logging. Possible values are :  critical, error, info, warning, debug.</p>

_modules/sarracenia/config.html

@@ -193,6 +193,7 @@ <h1>Source code for sarracenia.config</h1><div class="highlight"><pre>
     <span class="s1">&#39;identity_method&#39;</span><span class="p">:</span> <span class="s1">&#39;sha512&#39;</span><span class="p">,</span>
     <span class="s1">&#39;logDuplicates&#39;</span><span class="p">:</span> <span class="kc">False</span><span class="p">,</span>
     <span class="s1">&#39;logFormat&#39;</span><span class="p">:</span> <span class="s1">&#39;</span><span class="si">%(asctime)s</span><span class="s1"> [</span><span class="si">%(levelname)s</span><span class="s1">] </span><span class="si">%(name)s</span><span class="s1"> </span><span class="si">%(funcName)s</span><span class="s1"> </span><span class="si">%(message)s</span><span class="s1">&#39;</span><span class="p">,</span>
+    <span class="s1">&#39;logJson&#39;</span><span class="p">:</span> <span class="kc">False</span><span class="p">,</span>
     <span class="s1">&#39;logMetrics&#39;</span><span class="p">:</span> <span class="kc">False</span><span class="p">,</span>
     <span class="s1">&#39;logStdout&#39;</span><span class="p">:</span> <span class="kc">False</span><span class="p">,</span>
     <span class="s1">&#39;metrics_writeInterval&#39;</span><span class="p">:</span> <span class="mi">5</span><span class="p">,</span>
@@ -236,14 +237,16 @@ <h1>Source code for sarracenia.config</h1><div class="highlight"><pre>
 
 
 <span class="c1"># all the boolean settings.</span>
-<span class="n">flag_options</span> <span class="o">=</span> <span class="p">[</span> <span class="s1">&#39;acceptSizeWrong&#39;</span><span class="p">,</span> <span class="s1">&#39;acceptUnmatched&#39;</span><span class="p">,</span> <span class="s1">&#39;amqp_consumer&#39;</span><span class="p">,</span> <span class="s1">&#39;baseUrl_relPath&#39;</span><span class="p">,</span> <span class="s1">&#39;debug&#39;</span><span class="p">,</span> \
-    <span class="s1">&#39;delete&#39;</span><span class="p">,</span> <span class="s1">&#39;discard&#39;</span><span class="p">,</span> <span class="s1">&#39;download&#39;</span><span class="p">,</span> <span class="s1">&#39;dry_run&#39;</span><span class="p">,</span> <span class="s1">&#39;durable&#39;</span><span class="p">,</span> <span class="s1">&#39;exchangeDeclare&#39;</span><span class="p">,</span> <span class="s1">&#39;exchangeSplit&#39;</span><span class="p">,</span> <span class="s1">&#39;logReject&#39;</span><span class="p">,</span> <span class="s1">&#39;realpathFilter&#39;</span><span class="p">,</span> \
-    <span class="s1">&#39;follow_symlinks&#39;</span><span class="p">,</span> <span class="s1">&#39;force_polling&#39;</span><span class="p">,</span> <span class="s1">&#39;inline&#39;</span><span class="p">,</span> <span class="s1">&#39;inlineOnly&#39;</span><span class="p">,</span> <span class="s1">&#39;inplace&#39;</span><span class="p">,</span> <span class="s1">&#39;logDuplicates&#39;</span><span class="p">,</span> <span class="s1">&#39;logMetrics&#39;</span><span class="p">,</span> <span class="s1">&#39;logStdout&#39;</span><span class="p">,</span> <span class="s1">&#39;logReject&#39;</span><span class="p">,</span> <span class="s1">&#39;restore&#39;</span><span class="p">,</span> \
-    <span class="s1">&#39;messageDebugDump&#39;</span><span class="p">,</span> <span class="s1">&#39;mirror&#39;</span><span class="p">,</span> <span class="s1">&#39;timeCopy&#39;</span><span class="p">,</span> <span class="s1">&#39;notify_only&#39;</span><span class="p">,</span> <span class="s1">&#39;overwrite&#39;</span><span class="p">,</span> <span class="s1">&#39;post_on_start&#39;</span><span class="p">,</span> \
-    <span class="s1">&#39;permCopy&#39;</span><span class="p">,</span> <span class="s1">&#39;persistent&#39;</span><span class="p">,</span> <span class="s1">&#39;queueBind&#39;</span><span class="p">,</span> <span class="s1">&#39;queueDeclare&#39;</span><span class="p">,</span> <span class="s1">&#39;randomize&#39;</span><span class="p">,</span> <span class="s1">&#39;recursive&#39;</span><span class="p">,</span> <span class="s1">&#39;realpathPost&#39;</span><span class="p">,</span> \
+
+<span class="n">flag_options</span> <span class="o">=</span> <span class="p">[</span> <span class="s1">&#39;acceptSizeWrong&#39;</span><span class="p">,</span> <span class="s1">&#39;acceptUnmatched&#39;</span><span class="p">,</span> <span class="s1">&#39;amqp_consumer&#39;</span><span class="p">,</span> <span class="s1">&#39;baseUrl_relPath&#39;</span><span class="p">,</span> <span class="s1">&#39;debug&#39;</span><span class="p">,</span> 
+    <span class="s1">&#39;delete&#39;</span><span class="p">,</span> <span class="s1">&#39;discard&#39;</span><span class="p">,</span> <span class="s1">&#39;download&#39;</span><span class="p">,</span> <span class="s1">&#39;dry_run&#39;</span><span class="p">,</span> <span class="s1">&#39;durable&#39;</span><span class="p">,</span> <span class="s1">&#39;exchangeDeclare&#39;</span><span class="p">,</span> <span class="s1">&#39;exchangeSplit&#39;</span><span class="p">,</span> 
+    <span class="s1">&#39;follow_symlinks&#39;</span><span class="p">,</span> <span class="s1">&#39;force_polling&#39;</span><span class="p">,</span> <span class="s1">&#39;inline&#39;</span><span class="p">,</span> <span class="s1">&#39;inlineOnly&#39;</span><span class="p">,</span> <span class="s1">&#39;inplace&#39;</span><span class="p">,</span> <span class="s1">&#39;logJson&#39;</span><span class="p">,</span> 
+    <span class="s1">&#39;logMetrics&#39;</span><span class="p">,</span> <span class="s1">&#39;logReject&#39;</span><span class="p">,</span> <span class="s1">&#39;logStdout&#39;</span><span class="p">,</span> <span class="s1">&#39;logReject&#39;</span><span class="p">,</span> <span class="s1">&#39;restore&#39;</span><span class="p">,</span> <span class="s1">&#39;messageDebugDump&#39;</span><span class="p">,</span> 
+    <span class="s1">&#39;mirror&#39;</span><span class="p">,</span> <span class="s1">&#39;notify_only&#39;</span><span class="p">,</span> <span class="s1">&#39;overwrite&#39;</span><span class="p">,</span> <span class="s1">&#39;post_on_start&#39;</span><span class="p">,</span> <span class="s1">&#39;permCopy&#39;</span><span class="p">,</span> <span class="s1">&#39;persistent&#39;</span><span class="p">,</span> 
+    <span class="s1">&#39;queueBind&#39;</span><span class="p">,</span> <span class="s1">&#39;queueDeclare&#39;</span><span class="p">,</span> <span class="s1">&#39;randomize&#39;</span><span class="p">,</span> <span class="s1">&#39;recursive&#39;</span><span class="p">,</span> <span class="s1">&#39;realpathFilter&#39;</span><span class="p">,</span> <span class="s1">&#39;realpathPost&#39;</span><span class="p">,</span> 
     <span class="s1">&#39;reconnect&#39;</span><span class="p">,</span> <span class="s1">&#39;report&#39;</span><span class="p">,</span> <span class="s1">&#39;reset&#39;</span><span class="p">,</span> <span class="s1">&#39;retry_refilter&#39;</span><span class="p">,</span> <span class="s1">&#39;retryEmptyBeforeExit&#39;</span><span class="p">,</span> <span class="s1">&#39;save&#39;</span><span class="p">,</span> 
-    <span class="s1">&#39;sundew_compat_regex_first_match_is_zero&#39;</span><span class="p">,</span> <span class="s1">&#39;sourceFromExchange&#39;</span><span class="p">,</span> <span class="s1">&#39;sourceFromMessage&#39;</span><span class="p">,</span> <span class="s1">&#39;topicCopy&#39;</span><span class="p">,</span> 
-    <span class="s1">&#39;statehost&#39;</span><span class="p">,</span> <span class="s1">&#39;users&#39;</span><span class="p">,</span> <span class="s1">&#39;v2compatRenameDoublePost&#39;</span><span class="p">,</span> <span class="s1">&#39;wololo&#39;</span>
+    <span class="s1">&#39;sundew_compat_regex_first_match_is_zero&#39;</span><span class="p">,</span> <span class="s1">&#39;sourceFromExchange&#39;</span><span class="p">,</span> <span class="s1">&#39;sourceFromMessage&#39;</span><span class="p">,</span> 
+    <span class="s1">&#39;statehost&#39;</span><span class="p">,</span> <span class="s1">&#39;timeCopy&#39;</span><span class="p">,</span> <span class="s1">&#39;topicCopy&#39;</span><span class="p">,</span> <span class="s1">&#39;users&#39;</span><span class="p">,</span> <span class="s1">&#39;v2compatRenameDoublePost&#39;</span><span class="p">,</span> <span class="s1">&#39;wololo&#39;</span>
                 <span class="p">]</span>
 
 <span class="n">float_options</span> <span class="o">=</span> <span class="p">[</span> <span class="s1">&#39;messageRateMax&#39;</span><span class="p">,</span> <span class="s1">&#39;messageRateMin&#39;</span> <span class="p">]</span>

_modules/sarracenia/instance.html

@@ -94,6 +94,8 @@ <h1>Source code for sarracenia.instance</h1><div class="highlight"><pre>
 <span class="kn">import</span> <span class="nn">os</span>
 <span class="kn">import</span> <span class="nn">pathlib</span>
 <span class="kn">from</span> <span class="nn">sarracenia.moth</span> <span class="kn">import</span> <span class="n">Moth</span>
+<span class="kn">from</span> <span class="nn">sarracenia.featuredetection</span> <span class="kn">import</span> <span class="n">features</span>
+
 <span class="kn">import</span> <span class="nn">signal</span>
 <span class="kn">import</span> <span class="nn">sys</span>
 <span class="kn">import</span> <span class="nn">threading</span>
@@ -108,11 +110,14 @@ <h1>Source code for sarracenia.instance</h1><div class="highlight"><pre>
 
 <span class="n">logger</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">getLogger</span><span class="p">(</span><span class="vm">__name__</span><span class="p">)</span>
 
+<span class="k">if</span> <span class="n">features</span><span class="p">[</span><span class="s1">&#39;jsonlogs&#39;</span><span class="p">][</span><span class="s1">&#39;present&#39;</span><span class="p">]:</span>
+    <span class="kn">from</span> <span class="nn">pythonjsonlogger</span> <span class="kn">import</span> <span class="n">jsonlogger</span>
+
 
 <div class="viewcode-block" id="RedirectedTimedRotatingFileHandler">
 <a class="viewcode-back" href="../../Reference/code.html#sarracenia.instance.RedirectedTimedRotatingFileHandler">[docs]</a>
-<span class="k">class</span> <span class="nc">RedirectedTimedRotatingFileHandler</span><span class="p">(</span>
-        <span class="n">logging</span><span class="o">.</span><span class="n">handlers</span><span class="o">.</span><span class="n">TimedRotatingFileHandler</span><span class="p">):</span>
+<span class="k">class</span> <span class="nc">RedirectedTimedRotatingFileHandler</span><span class="p">(</span><span class="n">logging</span><span class="o">.</span><span class="n">handlers</span><span class="o">.</span><span class="n">TimedRotatingFileHandler</span><span class="p">):</span>
+
 <div class="viewcode-block" id="RedirectedTimedRotatingFileHandler.doRollover">
 <a class="viewcode-back" href="../../Reference/code.html#sarracenia.instance.RedirectedTimedRotatingFileHandler.doRollover">[docs]</a>
     <span class="k">def</span> <span class="nf">doRollover</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
@@ -307,6 +312,15 @@ <h1>Source code for sarracenia.instance</h1><div class="highlight"><pre>
 
                 <span class="n">logger</span><span class="o">.</span><span class="n">addHandler</span><span class="p">(</span><span class="n">handler</span><span class="p">)</span>
 
+                <span class="k">if</span> <span class="n">sarracenia</span><span class="o">.</span><span class="n">features</span><span class="p">[</span><span class="s1">&#39;jsonlogs&#39;</span><span class="p">][</span><span class="s1">&#39;present&#39;</span><span class="p">]</span> <span class="ow">and</span> <span class="n">cfg_preparse</span><span class="o">.</span><span class="n">logJson</span><span class="p">:</span>
+                    <span class="n">jsonHandler</span> <span class="o">=</span> <span class="n">RedirectedTimedRotatingFileHandler</span><span class="p">(</span>
+                        <span class="n">logfilename</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s1">&#39;.log&#39;</span><span class="p">,</span><span class="s1">&#39;.json&#39;</span><span class="p">),</span>
+                        <span class="n">when</span><span class="o">=</span><span class="n">lr_when</span><span class="p">,</span>
+                        <span class="n">interval</span><span class="o">=</span><span class="n">logRotateInterval</span><span class="p">,</span>
+                        <span class="n">backupCount</span><span class="o">=</span><span class="n">cfg_preparse</span><span class="o">.</span><span class="n">logRotateCount</span><span class="p">)</span>
+                    <span class="n">jsonHandler</span><span class="o">.</span><span class="n">setFormatter</span><span class="p">(</span><span class="n">jsonlogger</span><span class="o">.</span><span class="n">JsonFormatter</span><span class="p">(</span><span class="n">log_format</span><span class="p">))</span>
+                    <span class="n">logger</span><span class="o">.</span><span class="n">addHandler</span><span class="p">(</span><span class="n">jsonHandler</span><span class="p">)</span>
+
                 <span class="k">if</span> <span class="nb">hasattr</span><span class="p">(</span><span class="n">cfg_preparse</span><span class="p">,</span> <span class="s1">&#39;permLog&#39;</span><span class="p">):</span>
                     <span class="n">os</span><span class="o">.</span><span class="n">chmod</span><span class="p">(</span><span class="n">logfilename</span><span class="p">,</span> <span class="n">cfg_preparse</span><span class="o">.</span><span class="n">permLog</span><span class="p">)</span>
 

_sources/Reference/sr3_options.7.rst.txt

@@ -1089,6 +1089,19 @@ to control what is written to log files.  The format is documented here:
 * https://docs.python.org/3/library/logging.html#logrecord-attributes
 
 
+logJson <flag> (default: False) EXPERIMENTAL
+--------------------------------------------
+
+when *logJson on* is set, a second log file with the .json extension is made beside the
+normal .log file.  Each line of the .json logs is a .json structure, containing 
+a message written by the flow log. It does not contain unformatted output of
+sub-shells, and plugins which may produce arbitrary output.
+
+The .log file will contain the output of sub-programs launched by the flow, 
+and the .json will only contain properly formatted log messages from the application itself
+and properly written callbacks (that use normal python logging mechanisms.)
+
+
 logLevel ( default: info )
 --------------------------
 

_sources/fr/Reference/sr3_options.7.rst.txt

@@ -1083,6 +1083,17 @@ de python. Le format est documenté ici:
 
 * https://docs.python.org/fr/3/library/logging.html#logrecord-attributes
 
+logJson <flag> (par défaut : faux) EXPÉRIMENTAL
+------------------------------------------------
+
+lorsque *logJson on* est défini, un deuxième fichier journal avec l'extension .json est créé à côté du
+fichier .log normal. Chaque ligne des journaux .json est une structure .json, contenant
+un message écrit par le journal de flux. Il ne contient pas de sortie non formatée
+des sous-shell et des plugins qui peuvent produire une sortie arbitraire.
+
+Le fichier .log contiendra la sortie des sous-programmes lancés par le flux,
+et le .json ne contiendra que les messages de journal correctement formatés provenant de l'application elle-même
+et des rappels correctement écrits (qui utilisent des mécanismes de journalisation python normaux.)
 
 logLevel ( défaut: info )
 -------------------------

fr/Reference/sr3_options.7.html

@@ -1072,6 +1072,16 @@ <h4>LogFormat ( default: %(asctime)s [%(levelname)s] %(name)s %(funcName)s %(mes
 <li><p><a class="reference external" href="https://docs.python.org/fr/3/library/logging.html#logrecord-attributes">https://docs.python.org/fr/3/library/logging.html#logrecord-attributes</a></p></li>
 </ul>
 </section>
+<section id="logjson-flag-par-defaut-faux-experimental">
+<h4>logJson &lt;flag&gt; (par défaut : faux) EXPÉRIMENTAL<a class="headerlink" href="#logjson-flag-par-defaut-faux-experimental" title="Link to this heading"></a></h4>
+<p>lorsque <em>logJson on</em> est défini, un deuxième fichier journal avec l’extension .json est créé à côté du
+fichier .log normal. Chaque ligne des journaux .json est une structure .json, contenant
+un message écrit par le journal de flux. Il ne contient pas de sortie non formatée
+des sous-shell et des plugins qui peuvent produire une sortie arbitraire.</p>
+<p>Le fichier .log contiendra la sortie des sous-programmes lancés par le flux,
+et le .json ne contiendra que les messages de journal correctement formatés provenant de l’application elle-même
+et des rappels correctement écrits (qui utilisent des mécanismes de journalisation python normaux.)</p>
+</section>
 <section id="loglevel-defaut-info">
 <h4>logLevel ( défaut: info )<a class="headerlink" href="#loglevel-defaut-info" title="Link to this heading"></a></h4>
 <p>Niveau de journalisation exprimé par la journalisation de python. Les valeurs possibles sont :