0.  Pre-Checkin Checklist

    Performed from top of repo, default branch, head:

    [ ]  Is tag 'current' at or near head of 'vendor'?

         hg heads
         hg tags

    [ ]  Expected differences between vendor and default?  Very
         little of the original source should be modified.

         hg diff -rcurrent openssl

    [ ]  Are the 'vendor' and 'default' branch source directories
         'openssl' and not 'openssl-<version>'?

    [ ]  Have you reverted/blocked checkin of files processed
         by copy-windows-links.pl?

    [ ]  Structure of the '$stage' directory tree look sane after the
         build?

    [ ]  Is '$stage/bin/openssl' statically linked with only a few
         dynamic system libraries?  (No libz/zlib but will dynamically
         link to crypto/ssl.)

    [ ]  Unit tests run during development builds?  Did they perform
         zlib compression checks as well?

    [ ]  Did everything build PIC on *nix?

    [ ]  Do the built shared libraries have u+w permissions on Linux?

    [ ]  Did the libraries pick up the correct version of zlib?  You
         can run 'strings' over them (e.g. stage/lib/release/
         libcrypto.1.0.0.dylib) and grep for the zlib version string
         (currently "1.2.8").

         $ strings stage/lib/release/libcrypto.*  | grep 1\\.[0-9]\\.
         OpenSSL 1.0.1g 7 Apr 2014
         MD4 part of OpenSSL 1.0.1g 7 Apr 2014
         ...
         TXT_DB part of OpenSSL 1.0.1g 7 Apr 2014
         1.2.8
         libcrypto.so.1.0.0
         OpenSSL 1.0.1g 7 Apr 2014
         MD4 part of OpenSSL 1.0.1g 7 Apr 2014
         ...
         TXT_DB part of OpenSSL 1.0.1g 7 Apr 2014
         1.2.8
          deflate 1.2.8 Copyright 1995-2013 Jean-loup Gailly and Mark Adler 
          inflate 1.2.8 Copyright 1995-2013 Mark Adler 
         libcrypto.so.1.0.0
         OpenSSL 1.0.1g 7 Apr 2014
         MD4 part of OpenSSL 1.0.1g 7 Apr 2014
         ...
         TXT_DB part of OpenSSL 1.0.1g 7 Apr 2014
         1.2.8
          deflate 1.2.8 Copyright 1995-2013 Jean-loup Gailly and Mark Adler 
          inflate 1.2.8 Copyright 1995-2013 Mark Adler 

    [ ]  Does the openssl program (stage/bin/openssl) run as expected
         after any necessary library setup?  On Mac, you'll need to do
         some special staging setup to run it as the .dylib's now
         expect an .app bundle structure:

        $ stage/bin/openssl version
        dyld: Library not loaded: @executable_path/../Resources/libcrypto.1.0.0.dylib
        Referenced from: /Users/monty/3P/3p-openssl-update/stage/lib/release/libssl.1.0.0.dylib
        Reason: image not found
        Trace/BPT trap: 5
        $ mkdir stage/Resources
        $ pushd stage/Resources
        $ ln -s ../lib/debug/* .     # or release/*
        $ popd
        $ stage/bin/openssl version
        OpenSSL 1.0.1g 7 Apr 2014
        $ 


1.  Introduction

    Simple build of openssl library from openssl.org.

    Repo structure *now* mostly follows standard conventions (see
    section at end).

    Prior to 1.0.1e, repo didn't follow standard convention and each
    release was disconnected from previous releases.  Modifications
    to branch 'default' were re-implemented on every update.

    Vendor distributions make use of symlinks so updates and other
    repo work should generally be done on non-Windows systems.  A Perl
    script, copy-windows-links.pl, converts pseudo-symlinks to
    concrete files.  These conversions should not be checked in.


2.  Modifications

    Essential modifications to openssl distributions.

    * The 'default' branch was damaged at some point probably on a
    Windows machine.  The following symlink files became concrete
    copies of earlier file versions.  This has been restored but
    someone may encounter merge problems as a result.  Pay attention
    to these: ideatest.c, jpaketest.c, md2test.c, rc5test.c and
    idea.h.  Creating concrete files where symlinks are expected will
    break Mercurial merges.

    * Created ms/linden_test.bat from ms/test.bat to run those tests
    compatible with the configuration options and to generate a build
    failure if the test run doesn't pass.

    Mac build constrained by llqtwebkit choice.

    * Mac is still building *old* llqtwebkit (based on the
    http://bitbucket.org/lindenlab/3p-llqtwebit repo) on 10.5.  To use
    this library with a 10.5 build of llqtwebkit, the
    -mmacosx-version-min must be kept at 10.5 or __bzero() will be
    used for memset operations.  We'll take the optimistic path for
    now based on the reborn repo
    (http://bitbucket.org/monty_linden/llqtwebkit2) which builds on
    more modern releases.


3.  Source Origin

    1.0.1h:

      http://www.openssl.org/source/openssl-1.0.1h.tar.gz
      MD5:  8d6d684a9430d5cc98a62a5d8fbda8cf

    1.0.1g:

      http://www.openssl.org/source/openssl-1.0.1g.tar.gz
      MD5:  de62b43dfcd858e66a74bee1c834e959

    1.0.1e:

      http://www.openssl.org/source/openssl-1.0.1e.tar.gz
      MD5:  66bf6f10f060d561929de96f9dfe5b8c


4.  Package Contents

    Common:
    * include/openssl/*

    Windows (debug and release):
    (DLLs built with zlib(d).lib.)
    * lib/debug/libeay32.lib
    * lib/debug/libeay32.dll
    * lib/debug/libeay32.pdb
    * lib/debug/ssleay32.lib
    * lib/debug/ssleay32.dll
    * lib/debug/ssleay32.pdb
    * lib/release/libeay32.lib
    * lib/release/libeay32.dll
    * lib/release/libeay32.pdb
    * lib/release/ssleay32.lib
    * lib/release/ssleay32.dll
    * lib/release/ssleay32.pdb

    Mac OS X (debug and release):
    * lib/release/libcrypto.a
    * lib/release/libssl.a
    * lib/debug/libcrypto.a
    * lib/debug/libssl.a

    Mac OS X (build but not packaged):
    (Dylibs built against a libz.a archive.)
    * lib/release/libcrypto.1.0.0.dylib (dylib - link target - install
      path:  @executable_path/../Release/libcrypto.1.0.0.dylib)
    * lib/release/libcrypto.dylib (symlink)
    * lib/release/libssl.1.0.0.dylib
    * lib/release/libssl.dylib
    * lib/debug/libcrypto.1.0.0.dylib
    * lib/debug/libcrypto.dylib
    * lib/debug/libssl.1.0.0.dylib
    * lib/debug/libssl.dylib

    Linux (debug and release):
    * lib/release/libcrypto.a
    * lib/release/libssl.a
    * lib/debug/libcrypto.a
    * lib/debug/libssl.a

    Linux (built but not packaged):
    (Shared libraries build against a libz.a archive.)
    * lib/release/libcrypto.so (symlink)
    * lib/release/libcrypto.so.1.0.0 (shared library)
    * lib/release/libssl.so
    * lib/release/libssl.so.1.0.0
    * lib/debug/libcrypto.so
    * lib/debug/libcrypto.so.1.0.0
    * lib/debug/libssl.so
    * lib/debug/libssl.so.1.0.0


5.  Consumers/Dependents

    Packages dependent on OpenSSL which will need attention
    (autobuild.xml) after changes.  This is not authoritative, use
    appropriate build tools to find all dependents.

    * curl

    * llqtwebkit
    * openjpeg (*can* be a dependency via curl when building
      certain tools)

    * viewer


===================================================================

               Third-Party Library Repo Structure


Introduction

We want to have a way to capture local modifications to a third-party
open-source project, such as libcurl, without needing write access to
their public repository.  We want to be able to carry forward such
modifications to newer versions of the public project.  All this
should be independent of the organizational decision as to whether
it's even desirable to try to submit our local modifications upstream.

Fortunately, the Subversion folks articulated a process years ago that
addresses this very requirement.  They call it "Vendor Branches."  The
same tactic, suitably adapted, works with Mercurial too.

The essence of the idea is that we capture and tag a particular
snapshot of the open-source project.  We develop our local
modifications to that, and the repository tip incorporates them.  But
when we want to update to a newer version of the public project, we
bring it into the repository in such a way that we can discover the
changes from the original snapshot and the new one -- and then have
Mercurial apply those deltas to the ''combined'' source.

The following material is adapted from
http://svnbook.red-bean.com/en/1.1/ch07s05.html, the Red Bean
Subversion book, but recast for Mercurial.  The Linden source for this
material is an internal wiki.  There may be superceding documentation
on the public wiki when you read this.  We recommend searching there
for updates to conventions below.  And note that each particular
library may implement variations of this scheme.


General Vendor Branch Management Procedure

Managing vendor branches generally works like this.  You create a
named branch ("vendor") to store the vendor source snapshots.  Then
you import the third party code into that branch.  Your modified
branch (named "default") is based on "vendor".  You always make your
local changes to the default branch.  With each new release of the
code you are tracking you bring it into the "vendor" branch and merge
the changes into "default", resolving whatever conflicts occur between
your local changes and the upstream changes.

Perhaps an example will help to clarify this algorithm.  We'll use a
scenario where your development team is creating a calculator program
that links against a third-party complex number arithmetic library,
libcomplex.  We'll construct a repository specifically for our
locally-modified version of that library.  To begin, we must
initialize our repository and create at least one file in our
"default" branch.

 $ hg init ourcomplex
 $ cd ourcomplex
 $ touch README.txt
 $ hg commit README.txt

Now we can create the vendor branch and do the import of the first
vendor drop.  We'll call our vendor branch "vendor", and each
successive code drop will be tagged "current".

 $ hg branch vendor
 $ tar -xjf ../libcomplex-1.0.tar.bz2
 $ mv libcomplex-1.0 libcomplex
 $ hg addremove
 $ hg commit -m "1.0 source drop"
 $ hg tag -r tip current
 $ hg tag -r current 1.0

We now have the current version of the libcomplex source code in
branch "vendor", tagged "current" and in a non-version-specific source
code subdirectory ("libcomplex").  Next, we merge it into the default
branch.  It is in the default branch that we will make our
customizations.

 $ hg update default
 $ hg merge vendor
 $ hg commit -m "initial: 1.0"

We get to work customizing the libcomplex code.  Before we know it,
our modified version of libcomplex is now completely integrated into
our calculator program.

A few weeks later, the developers of libcomplex release a new version
of their library, version 1.1, which contains some features and
functionality that we really want.  We'd like to upgrade to this new
version, but without losing the customizations we made to the existing
version.  What we essentially would like to do is to replace our
current baseline version of libcomplex 1.0 with a copy of libcomplex
1.1, and then have Mercurial re-apply the custom modifications we
previously made to that library to the new version.  But we actually
approach the problem from the other direction, applying the changes
made to libcomplex between versions 1.0 and 1.1 to our modified copy
of it.

To perform this upgrade, we update our repository to our vendor
branch, and update the "current" tag with the new libcomplex 1.1
source code.  We quite literally replace the existing files with the
new files, clearing out the whole tree and exploding the libcomplex
1.1 release tarball in its place.  The goal here is to make the tip of
our vendor branch contain only the libcomplex 1.1 code, and to ensure
that all that code is under version control.  Oh, and we want to do
this with as little version control history disturbance as possible.

 $ hg update vendor
 $ rm -rf *
 $ tar -xjf ../libcomplex-1.1.tar.bz2
 $ mv libcomplex-1.1 libcomplex
 $ hg addremove -s 60
 $ # Additional 'hg add' and 'hg rm' commands if needed
 $ hg commit -m "1.1 source drop"

After unpacking the 1.1 tarball, hg status will show files with local
modifications as well as, perhaps, some unversioned or missing files.
If we did what we were supposed to do, the unversioned files are only
those new files introduced in the 1.1 release of libcomplex.  The
missing files are files that were in 1.0 but not in 1.1.  The 'hg
addremove' command deals with both, and more: the '-s 60' switch
directs Mercurial to compare added files to deleted files, recognizing
any file at least 60% similar as a move/rename.

For simple or stable libraries, the 'hg addremove' command should be
reliable.  For more complicated libraries subject to refactoring or
large gaps of time between updates (e.g. libcurl), it can get a little
lost trying to match files in the old release with files in the new
release.  Pay attention to the output of the command or better still,
do dry runs.  Files erroneously moved can be excluded with the '-X'
option and then dealt with individually with 'hg add' and 'hg rm'
commands after 'hg addremove'.  (The readme file in the curl library
should document a particularly challenging case.)

Finally, once our current working copy contains only the libcomplex
1.1 code, we commit the changes we made to get it looking that way.

Our current vendor branch now contains the new vendor drop.  We move
the 'current' tag to the new version (in the same way we previously
tagged the version 1.0 vendor drop), and then merge the differences
between the version 1.0 and version 1.1 into our default branch.

 $ hg tag -f -r tip current
 $ hg tag -r current 1.1
 $ hg update default
 $ hg merge vendor
 # resolve all the conflicts between their changes and our changes
 # if you will have conflicts in .hgtags, simply take *all* lines
 ...
 $ hg commit -m "update with 1.1"

Any additional work needed to get the merged library working can
now be done on the default branch.


Revision Tags

We don't currently make use of Mercurial tags in the build and release
process for 3rd-party libraries.  But we would like to establish a
convention to document update and release points.  The tags we would
like to establish are:

 * 'current' Points to a succession of vendor releases checked into
   the 'vendor' branch.  Will almost always be at or close to branch
   head.

 * '<version>' Tag on the 'vendor' branch pointing to a verbatim
   checkin of a 3rd-party's <version> release.  Example:  '7.21.1' for
   a particular version of libcurl we have used.

 * Release-type tags on the default branch aren't as useful given how
   Mercurial handles tags and how autobuild works.


Schematic of a Third-Party Repository

Below is the output of the 'hg glog' command showing a library project
going through an initial 1.0 release and an update from the vendor to
1.1.  Significant revisions in the repository lifecycle are as
follows:

 0  Creation of the repo with an initial file.
 1  1.0 code drop on branch 'vendor'
 4  Merge of 1.0 code onto branch 'default'
 5  Modifications to library we wish to keep over time.  Released.
 6  1.1 code drop on branch 'vendor'
 9  Merge of 1.1 code onto branch 'default'
10  Fixes to merge yielding production 1.1 library.  Released.
 

@  changeset:   10:888229641f6e
|  tag:         tip
|  user:        Monty Brandenberg <[email protected]>
|  date:        Wed Oct 30 13:35:51 2013 -0400
|  summary:     Work to get 1.1 merge working.  Release.
|
o    changeset:   9:925ccdf09f50
|\   parent:      5:83c5775c23dc
| |  parent:      8:977001a08e48
| |  user:        Monty Brandenberg <[email protected]>
| |  date:        Wed Oct 30 13:35:20 2013 -0400
| |  summary:     update with 1.1
| |
| o  changeset:   8:977001a08e48
| |  branch:      vendor
| |  user:        Monty Brandenberg <[email protected]>
| |  date:        Wed Oct 30 13:33:49 2013 -0400
| |  summary:     Added tag 1.1 for changeset 5f6cb89add91
| |
| o  changeset:   7:59bce0f6d12f
| |  branch:      vendor
| |  user:        Monty Brandenberg <[email protected]>
| |  date:        Wed Oct 30 13:33:41 2013 -0400
| |  summary:     Added tag current for changeset 5f6cb89add91
| |
| o  changeset:   6:5f6cb89add91
| |  branch:      vendor
| |  tag:         current
| |  tag:         1.1
| |  parent:      3:8525ad934ecd
| |  user:        Monty Brandenberg <[email protected]>
| |  date:        Wed Oct 30 13:33:29 2013 -0400
| |  summary:     1.1 source drop
| |
o |  changeset:   5:83c5775c23dc
| |  tag:         1.0
| |  user:        Monty Brandenberg <[email protected]>
| |  date:        Wed Oct 30 13:32:31 2013 -0400
| |  summary:     Linden-specific changes to the library.  Release
| |
o |  changeset:   4:bccb736585f4
|\|  parent:      0:400e4516c406
| |  parent:      3:8525ad934ecd
| |  user:        Monty Brandenberg <[email protected]>
| |  date:        Wed Oct 30 13:31:40 2013 -0400
| |  summary:     initial:  1.0
| |
| o  changeset:   3:8525ad934ecd
| |  branch:      vendor
| |  user:        Monty Brandenberg <[email protected]>
| |  date:        Wed Oct 30 13:30:21 2013 -0400
| |  summary:     Added tag 1.0 for changeset 8ac3828d03bb
| |
| o  changeset:   2:7aa1a1cb62d9
| |  branch:      vendor
| |  user:        Monty Brandenberg <[email protected]>
| |  date:        Wed Oct 30 13:30:14 2013 -0400
| |  summary:     Added tag current for changeset 8ac3828d03bb
| |
| o  changeset:   1:8ac3828d03bb
|/   branch:      vendor
|    tag:         1.0
|    user:        Monty Brandenberg <[email protected]>
|    date:        Wed Oct 30 13:30:09 2013 -0400
|    summary:     1.0 source drop
|
o  changeset:   0:400e4516c406
   user:        Monty Brandenberg <[email protected]>
   date:        Wed Oct 30 13:29:16 2013 -0400
   summary:     Created repo with initial readme file