Add python otfautohint into afdko

.flake8

@@ -5,15 +5,14 @@
 select = E,F,W,B,B901,B902,B903
 
 # ignore W504 line break before binary operator
-ignore = W504,B017
+ignore = W504,B017,E731,E741
 
 # explicitly set line length limitation
 max-line-length = 79
 
 # exclude files that haven't had linting fixes applied yet
 exclude = 
     agd.py,
-    beztools.py,
     comparefamily.py,
     otf2otc.py,
     proofpdf.py,

README.md

@@ -29,6 +29,13 @@ Please see the
 for additional information, such as links to reference materials and related
 projects.
 
+📣 News
+------------
+* **The Python port of psautohint will be (re)integrated into the AFDKO repository as "otfautohint":**
+  * A release will be made on August 28th, 2023
+  * Please feel free to try pre-release v4.0.0a in the meantime
+  * More information can be found in [docs/otfautohint_Notes.md](docs/otfautohint_Notes.md)
+
 Installation
 ------------
 

docs/AFDKO-Overview.md

@@ -17,9 +17,9 @@ The tools fall into several different functional groups.
 
 ## 2.1 Making/editing fonts
 
-### `psautohint`
+### `otfautohint`
 
-This program is the Adobe auto-hinter and lives in [its own project repo](https://github.com/adobe-type-tools/psautohint). It is used by several AFDKO tools and is installed automatically via AFDKO's project requirements.
+This program is the Adobe auto-hinter. It is used by several AFDKO tools.
 
 It can be applied to both OpenType/CFF and Type 1 fonts. Works with Type 1 and OpenType/CFF fonts only.
 
@@ -63,14 +63,14 @@ This tool will rotate and translate glyphs in a font, including the hints. Howev
 
 This allows you to cut and paste the entire binary block of a font table from one font to another. You do this by first using it on a source font with the `-x` option to copy a table from the source font to a separate file, and then using it with the `-a` option to add that table to a target font. It can also be used to simply delete a table, and to fix the font table checksums.
 
-### `psstemhist`
+### `otfstemhist`
 
-This program is actually the same tool as `psautohint` but with a different
-face. It provides reports which help in selecting the global hint data and
-alignment zones for Type 1 hinting. You should look at the reports from this
+This program is actually the same tool as `otfautohint` but with a different
+"face". It provides reports which help in selecting the global hint data and
+alignment zones for CFF hinting. You should look at the reports from this
 tool in order to select the most common stem widths, and then use a program such
 as FontLab or RoboFont to set the values in the font. This should be done before
-hinting the font. Works with Type 1 and OpenType/CFF fonts only.
+hinting the font.
 
 ### `ttfcomponentizer`
 
@@ -130,7 +130,7 @@ Note that the tools ending in “plot” are all simply small command-file scrip
 
 ## 2.3 Validation
 
-### `psautohint`
+### `otfautohint`
 
 The auto-hinting program will report at length about hinting issues. Some of these you can ignore, such as reports about near misses when a stem could be controlled by a hint-zone but is just a little too wide or too narrow. By adjusting either the stem widths or the hint-zones according to these reports, you can include more stems in the set that are controlled by hints, but you can also reasonably decide that is not worth the effort. However, many complaints do need fixing, such not having nodes at vertical or horizontal extremes of a curve.
 

docs/CommandLineHowTo.md

@@ -90,7 +90,7 @@ Commands which operate on files require that you specify the directory path of t
 ```
 open /Users/rroberts/.bash_profile # Mac
 start C:\adobe\FDK\FDKReleaseNotes.txt # Windows
-psautohint -a MyFont.pfa # any system
+otfautohint -a MyFont.pfa # any system
 ```
 
 The easiest way to get a file path into the command line is to drag its icon from a *Finder/Explorer* window onto the *Terminal/Command Prompt* window. When you do this, the absolute path — the complete path from the computer’s root directory — is copied. However, a command window always has a “current” directory. If the file is in the current directory, then you only need to type the file’s name, as in the third example above. To see the absolute path of the current directory, type the following command:
@@ -107,13 +107,13 @@ If you leave off the initial slash (/), then the path is assumed to be relative
 
 Some commands produce a lot of text output, so much that it would be more convenient to look at the output in a text editor with good search functions. To send the output of a command to a file, add a greater-than sign (`>`) followed by a file path, to the command line. For example:
 ```
-psautohint -a MyFont.pfa
+otfautohint -a MyFont.pfa
 ```
 is likely to produce several hundred lines of output. To browse this more easily, enter:
 ```
-psautohint -a MyFont.pfa > MyFont_autohint.txt
+otfautohint -a MyFont.pfa > MyFont_autohint.txt
 ```
-This will “re-direct” the output of the psautohint command into the file `MyFont_autohint.txt`. You
+This will “re-direct” the output of the otfautohint command into the file `MyFont_autohint.txt`. You
 can then open this file in your favorite text editor, and search for interesting notes.
 
 ## **Favorite AFDKO (Adobe® Font Development Kit for OpenType®) commands**
@@ -131,13 +131,13 @@ Run `checkOutlines` QA tool on the font MinionPro-Bold.otf present in the Bold s
 ```
 checkOutlines Bold/MinionPro-Bold.otf
 ```
-Autohint only unhinted glyphs in a font: (This allows you to manually hint some glyphs in FontLab without overwriting that work when using the psautohint program)
+Autohint only unhinted glyphs in a font: (This allows you to manually hint some glyphs in FontLab without overwriting that work when using the otfautohint program)
 ```
-psautohint font.pfa
+otfautohint font.pfa
 ```
 Autohint all glyphs in a font: (This will remove any hints that existed before)
 ```
-psautohint -a font.pfa
+otfautohint -a font.pfa
 ```
 Build and OpenType CFF font in release mode, assuming that all the input files (font.pfa, features, fontinfo, FontMenuNameDB and GlyphOrderAndAliasDB) have default names, and default locations relative to the current directory: (The resulting OpenType font file, will be named according to the font’s PostScript® name)
 ```

docs/README.md

@@ -7,6 +7,7 @@
 ---
 ##### • [Issues with OpenType/CFF and TrueType fonts in MS Font Validator](./MSFontValidatorIssues.md)
 ##### • [Practical issues in weight setting and style linking (Windows only)](./WinWeights.md)
+##### • [A few notes on otfautohint](./otfautohint_Notes.md)
 ---
 ##### • [FDK Build Notes](./FDK_Build_Notes.md)
 ##### • [MakeOTFEXE Feature File Parser Notes](./Feature_Parser_Notes.md)

docs/otfautohint_Notes.md

@@ -0,0 +1,154 @@
+# Notes on the OTF autohinter
+
+`otfautohint` contains a Python port of the "Automatic Coloring" code
+originally written in C by Bill Paxton. That code was most recently distributed
+as "psautohint" in a PyPI package of the same name.
+
+Changes Summary:
+* [Name-Change] The name-change better reflects how hinted input is now published, 
+  and lets the new version coexist with the old version when needed.
+* [AFDKO Tools Updated] Other tools in AFDKO have been updated to call oftautohint 
+  instead of psautohint, and the dependency on the latter's repository has been removed.
+* [Stopping Psautohint Development] We expect to stop development of psautohint once
+  the 3.9.8 release is made.
+* [Improvements] The new code fixes a number of bugs and in our judgement produces 
+  better results on average. It also uses a slightly different encoding in
+  UFO glif files. Accordingly, users should expect that running the new code
+  results in many differences compared with psautohint, but this should be a one-time change.
+* [Variable CFF Hinting] The new code also supports hinting of variable CFF-based fonts,
+  including directly hinting a built CFF2 font. Glyphs that include overlap will typically 
+  be hinted close to how they would have been hinted with overlap removed.
+* [Hinting Time] Because psautohint was partly written in (very old) C code,
+   and otfautohint is written entirely in Python, the new code takes significantly
+   longer to hint an individual glyph. However, we have also enhanced otfautohint
+   to hint different glyphs on multiple CPU cores by default. As a result the tool
+   will be 5-8 times slower when running on a single core but will typically be
+   slightly faster when running on 8 cores.
+
+Most of the algorithms are unchanged but not all. Some notable differences are:
+
+1.  Because all code is now in Python there is no role for the `bez` charstring
+    interchange format. CharStrings are now directly unpacked into `glyphData`
+    objects. These are capable of representing any (decompiled) CharString with
+    the caveats that the particular spline operator (e.g. `rCurveTo` vs
+    `curveTo`) is not recorded.
+2.  Intermediate hint state is now maintained in `hintState` objects, one per
+    glyph, rather than in globals.
+3.  The C code algorithms for vertical and horizontal hinting were mostly
+    implemented as separate functions. In the port there is typically one
+    function shared by both dimensions. The `glyphData/pt` 2-tuple object has
+    special `a` and `o` value accessors and a class-level switch as part of
+    this unification. (`a` is meant to suggest "aligned" and `o` is meant to
+    suggest "opposite", referring to the relation between the value and the
+    chosen alignment; `a` is `x` and `o` is `y` when horizontal alignment is
+    chosen, and vice-versa when vertical alignment is chosen.) Some bug fixes
+    and improvements that were only added to one dimension in the past now work
+    in both.
+4.  In the C code spline characteristics such as bounding boxes and measures of
+    flatness were calculated by approximating spline curves with line segments.
+    In the Python code these calculations are closed-form and implemented with
+    fontTools' quadratic and cubic root finders.
+5.  The C code had special functions for handling a spline with an inflection
+    point. The new code copies and splits such splines at their inflection
+    points, analyzes the copies, and copies the resulting analysis back to the
+    inflected splines.
+6.  The mask distribution algorithm (which is equivalent to what used to be
+    called "hint substitution" is implemeneted somewhat differently.
+
+There are also some features that are not (yet) ported:
+
+1.  The C code for hint substitution "promoted" hints to earlier splines under
+    some circumstances. This was not included because testing did not indicate
+    it was of noticable benefit in current renderers.
+2.  Previous versions of psautohint would clean up duplicate `moveTo` operators
+    and warn about duplicate subpaths or unusually large glyphs. It is now more
+    appropriate to check for these characteristics using sanitizers at earlier
+    stages in the development process.
+3.  Under some circumstances the C autohinter would divide curved splines at *t*
+    == .5 when the ``allow-changes`` option was active. The primary reason for
+    doing so was when a single spline "wanted" conflicting hints at its start
+    and end points. This is a relatively rare circumstance and the Adobe
+    maintainers are evaluating what, if anything, should be done in this case.
+4.  The C autohinter would add a new hint mask before each subpath in the
+    "percent" and "perthousand" glyphs but did not offer this option for other
+    glyphs. This code was not ported.
+
+Most functions are now documented in-line. Adapter code in `autohint.py` calls
+`hint()` on `glyphHinter` in `hinter.py`, which in turn calls into the
+dimension-specific `hhinter` and `vhinter` objects in the same file.
+
+The Python code is slower when hinting individual glyphs, often 5 or more times
+as slow or more compared with the C code on the same machine. It also uses more
+memory.  However, by default glyphs are now hinted in parallel using the Python
+`multiprocessing` module when multiple CPU cores are available. As part of this
+change the glyphs are also unpacked just before hinting and updated right after
+hinting in order to lower the total memory used by the process at a given time.
+As a result the overall hinting process is often slightly faster on
+contemporary machines.
+
+# Some notes on hinting glyphs in a variable font
+
+## Stem ordering
+
+The initial CFF2 specification, and all revisions at the time of writing,
+require that stems are defined in increasing order by their (lower) starting
+locations and then, if two stems start at the same place, their ending
+locations (which are higher except in the case of ghost hints).  Duplicate
+stems are disallowed.  Stems that would be specified out of order in a
+particular master (relative to the default master ordering) must therefore be
+removed.
+
+<!---
+As we hope to eliminate these restrictions at a future point the variable font
+autohinter supports two modes: a default mode in which stems are removed until
+all are in order across all masters and an experimental mode in which stems
+that change order are retained but treated as conflicting with all stems they
+"cross" anywhere in design space. 
+--->
+
+As long as a design space is defined by interpolation only (rather than
+extrapolation) the extremes of stem ordering are represented by the (sorted)
+orders in the individual masters. Consider the bottom edge of stem *i* in a
+variable glyph with *n* masters. It's location at some point in design space
+can be represented as
+
+c<sub>1</sub>\*s<sub>*i*1</sub> + c<sub>2</sub>\*s<sub>*i*2</sub> + ... + c<sub>*n*</sub>\*s<sub>*in*</sub>
+
+where s<sub>*ik*</sub> is the position of the edge in that glyph in master *k*
+and each `c` value is some interpolation coefficient, so that c<sub>1</sub> +
+c<sub>2</sub> + ... + c<sub>*n*</sub> == 1 and 0 <= c<sub>*k*</sub> <= 1
+
+The signed distance between the bottom edges of two stems *i* and *j* is accordingly
+
+c<sub>1</sub>\*s<sub>*i*1</sub> + c<sub>2</sub>\*s<sub>*i*2</sub> + ... + c<sub>*n*</sub>\*s<sub>*in*</sub> - c<sub>1</sub>\*s<sub>*j*1</sub> + c<sub>2</sub>\*s<sub>*j*2</sub> + ... + c<sub>*n*</sub>\*s<sub>*jn*</sub>
+
+or
+
+c<sub>1</sub>\*(s<sub>*i*1</sub> - s<sub>*j*1</sub>) + c<sub>2</sub>\*(s<sub>*i*2</sub> - s<sub>*j*2</sub>) + ... + c<sub>*n*</sub>\*(s<sub>*in*</sub> - s<sub>*jn*</sub>)
+
+The minimum/maximum distance in the space will therefore be *s*<sub>*ik*</sub>
+- *s*<sub>*jk*</sub> for whatever master *k* minimizes/maximizes this value.
+The question of whether *i* and *j* change order anywhere in design space is
+therefore equivalent to the question of whether *i* and *j* change order in any
+master, which is further equivalent to the question of whether they appear in a
+different order in any master besides the default.
+
+## Stem overlap
+
+Suppose that stems *i* and *j* are in the same order across all masters with
+stem *i* before stem *j* (so either s<sub>*ik*</sub> < s<sub>*jk*</sub> or
+(s<sub>*ik*</sub> == s<sub>*jk*</sub> and e<sub>*ik*</sub> <
+e<sub>*jk*</sub>)). Whether *i* overlaps with *j* in a given master *k* is
+defined by whether s<sub>*jk*</sub> - e<sub>*ik*</sub> < O<sub>m</sub> (the
+overlap margin). Therefore, by reasoning analogous to the above, the question
+of whether two consistently ordered stems overlap in design space is equivalent
+to whether they overlap in at least one master.
+
+Any two stems that change order between two masters overlap at some point in design
+space interpolated between those masters. The question of whether two stems overlap
+in the general is therefore equivalent to:
+
+1.  Do the stems change order in any master relative to the default master? If yes,
+    then they overlap.
+2.  If no, then is there any master *k* in which s<sub>*jk*</sub> -
+    e<sub>*ik*</sub> < O<sub>m</sub>? If yes, then they overlap.

pyproject.toml

@@ -8,5 +8,3 @@ requires = [
 	"ninja"
 ]
 build-backend = "setuptools.build_meta"
-
-[tool.setuptools_scm]

python/afdko/convertfonttocid.py

@@ -4,7 +4,7 @@
 convertfonttocid.py. v 1.13.2 Jul 30 2020
 
 Convert a Type 1 font to CID, given multiple hint dict defs in the
-"fontinfo" file. See psautohint help, with the "--doc-fddict" option,
+"fontinfo" file. See otfautohint help, with the "--doc-fddict" option,
 or the makeotf user guide for details on this format. The output file
 produced by convertFontToCID() is a CID Type 1 font, no matter what
 the input is.
@@ -508,11 +508,11 @@ def parseFontInfoFile(fontDictList, fontInfoData, glyphList, maxY, minY,
         # e.g outside of the Font BBox. We do this because if there are
         # glyphs which are not assigned to a user specified font dict,
         # it is because it doesn't make sense to provide alignment zones
-        # for the glyph. Since psautohint does require at least one bottom zone
-        # and one top zone, we add one bottom and one top zone that are
+        # for the glyph. Since otfautohint does require at least one bottom
+        # zone # and one top zone, we add one bottom and one top zone that are
         # outside the font BBox, so that hinting won't be affected by them.
         # NOTE: The FDDict receives a special name "No Alignment Zones" which
-        # psautohint recognizes.
+        # otfautohint recognizes.
         defaultFDDict = fontDictList[0]
 
         for key in kBlueValueKeys + kOtherBlueValueKeys:

python/afdko/makeinstancesufo.py

@@ -20,7 +20,7 @@
 )
 
 from defcon import Font
-from psautohint.__main__ import main as psautohint
+from .otfautohint.__main__ import main as otfautohint
 from ufonormalizer import normalizeUFO
 from ufoProcessor import build as ufoProcessorBuild
 
@@ -86,7 +86,7 @@ def filterDesignspaceInstances(dsDoc, options):
 
 def updateInstance(fontInstancePath, options):
     """
-    Run checkoutlinesufo and psautohint, unless explicitly suppressed.
+    Run checkoutlinesufo and otfautohint, unless explicitly suppressed.
     """
     if options.doOverlapRemoval:
         logger.info("Doing overlap removal with checkoutlinesufo on %s ..." %
@@ -100,12 +100,12 @@ def updateInstance(fontInstancePath, options):
             raise
 
     if options.doAutoHint:
-        logger.info("Running psautohint on %s ..." % fontInstancePath)
+        logger.info("Running otfautohint on %s ..." % fontInstancePath)
         ah_args = ['--no-zones-stems', fontInstancePath]
         if options.no_round:
             ah_args.insert(0, '-d')
         try:
-            psautohint(ah_args)
+            otfautohint(ah_args)
         except (Exception, SystemExit):
             raise
 

python/afdko/otfautohint/__init__.py

@@ -0,0 +1,41 @@
+import os
+
+
+class FontParseError(Exception):
+    pass
+
+
+def _font_is_ufo(path):
+    from fontTools.ufoLib import UFOReader
+    from fontTools.ufoLib.errors import UFOLibError
+    try:
+        UFOReader(path, validate=False)
+        return True
+    except (UFOLibError, KeyError, TypeError):
+        return False
+
+
+def get_font_format(font_file_path):
+    if _font_is_ufo(font_file_path):
+        return "UFO"
+    elif os.path.isfile(font_file_path):
+        with open(font_file_path, 'rb') as f:
+            head = f.read(4)
+            if head == b'OTTO':
+                return 'OTF'
+            elif head[0:2] == b'\x01\x00':
+                return 'CFF'
+            elif head[0:2] == b'\x80\x01':
+                return 'PFB'
+            elif head in (b'%!PS', b'%!Fo'):
+                for fullhead in (b'%!PS-AdobeFont', b'%!FontType1',
+                                 b'%!PS-Adobe-3.0 Resource-CIDFont'):
+                    f.seek(0)
+                    if f.read(len(fullhead)) == fullhead:
+                        if b"CID" not in fullhead:
+                            return 'PFA'
+                        else:
+                            return 'PFC'
+        return None
+    else:
+        return None