ex28.tex

\chapter{Exercise 28: Intermediate Makefiles}

In the next three Exercises you'll create a skeleton project directory to use
in building your C programs later.  This skeleton directory will be used in the
rest of the book, and in this exercise I'll cover just the \file{Makefile} so
you can understand it.

The purpose of this structure is to make it easy to build medium sized programs
without having to resort to configure tools.  If done right you can get very far
with just GNU make and some small shell scripts.

\section{The Basic Project Structure}

The first thing to do is make a \file{c-skeleton} directory and then put
a set of basic files and directories in it that many projects have.  Here's
my starter:

\begin{code}{Initial C Project Skeleton}
<< d['code/ex28.sh-session|pyg|l'] >>
\end{code}

At the end you see me do an \verb|ls -l| so you can see the final results.

Here's what each of these does:

\begin{description}
\item[LICENSE] If you release the source of your projects you'll want to 
    include a license.  If you don't though, the code is copyright by you
    and nobody has rights to it by default.
\item[README.md] Basic instructions for using your project go here.  It ends
    in \file{.md} so that it will be interpreted as markdown.
\item[Makefile] The main build file for the project.
\item[bin/] Where programs that users can run go. This is usually empty and the Makefile will create it if it's not there.
\item[build/] Where libraries and other build artifacts go. Also empty and the Makefile will create it if it's not there.
\item[src/] Where the source code goes, usually \file{.c} and \file{.h} files.
\item[tests/] Where automated tests go.
\item[src/dbg.h] I copied the \file{dbg.h} from Exercise 20 into \file{src/} for later.
\end{description}

I'll now break down each of the components of this skeleton project
so you can understand how it works.

\section{Makefile}

The first thing I'll cover is the Makefile because from that you can understand
how everything else works.  The Makefile in this exercise is much more detailed
than ones you've used so far, so I'm going to break it down after you type it
in:

\begin{code}{c-skeleton/Makefile}
\begin{lstlisting}
<< d['code/c-skeleton/Makefile'] >>
\end{lstlisting}
\end{code}

Remember that you need to indent the Makefile consistently with tab characters.
Your editor should know that and do the right thing, but if it doesn't then get
a different text editor.  No programmer should use an editor that fails at 
something so simple.

\subsection{The Header}

This makefile is designed to build a library we'll be working on later and 
to do so reliably on almost any platform by using special features of \program{GNU make}.  I'll break down each part in sections, starting with the header.

\begin{description}
\item[Makefile:1] These are the usual \ident{CFLAGS} that you set in all of your
    projects, but with a few others that may be needed to build libraries.
    \emph{You may need to adjust these for different platforms}.  Notice
    the \ident{OPTFLAGS} variable at the end which lets people augment the
    build options as needed.
\item[Makefile:2] Options used when linking a library, and allows someone else
    to augment the linking options using the \ident{OPTLIBS} variable.
\item[Makefile:3] Setting an \emph{optional} variable called \ident{PREFIX} that
    will only have this value if the person running the Makefile didn't
    already give a \ident{PREFIX} setting.  That's what the \verb|?=| does.
\item[Makefile:5] This fancy line of awesome \emph{dynamically} creates the \ident{SOURCES} variable by doing a \ident{wildcard} search for all \file{*.c} files in the
    \file{src/} directory. You have to give both \file{src/**/*.c} and 
    \file{src/*.c} so that GNU make will include the files in \file{src} and
    also the ones below it.
\item[Makefile:6] Once you have the list of source files, you can then use the
    \ident{patsubst} to take the \ident{SOURCES} list of \file{*.c} files and
    make a \emph{new} list of all the object files.  You do this by telling
    \ident{patsubst} to change all \file{\%.c} extensions to \file{\%.o} and
    then those are assigned to \ident{OBJECTS}.
\item[Makefile:8] Using the \ident{wildcard} again to find all the test
    source files for the unit tests.  These are separate from the library's
    source files.
\item[Makefile:9] Then using the same \ident{patsubst} trick to dynamically
    get all the \ident{TEST} targets.  In this case I'm stripping away the
    \file{.c} extension so that a full program will be made with the same
    name.  Previously I had replaced the \file{.c} with {.o} so an object
    file is created.
\item[Makefile:11] Finally, we say the ultimate target is \file{build/libYOUR\_LIBRARY.a}, which you will change to be whatever library you are actually trying to build.
\end{description}

This completes the top of the Makefile, but I should explain what I mean by
"lets people augment the build".  When you run make you can do this:

\begin{code}{A Changing A Make Build}
\begin{Verbatim}
# WARNING! Just a demonstration, won't really work right now.
# this installs the library into /tmp
$ make PREFIX=/tmp install
# this tells it to add pthreads
$ make OPTFLAGS=-pthread
\end{Verbatim}
\end{code}

If you pass in options that match the same kind of variables you have in
your \file{Makefile}, then those will show up in your build.  You can then
use this to change how the \file{Makefile} runs.  The first one alters the
\ident{PREFIX} so that it installs into \file{/tmp} instead.  The second
one sets \ident{OPTFLAGS} so that the \ident{-pthread} option is present.

\subsection{The Target Build}

Continuing with the breakdown of the \file{Makefile} I have actually building
the object files and targets:

\begin{description}
\item[Makefile:14] Remember that the first target is what \program{make} will
    run by default when no target is given.  In this case it's called \ident{all:}
    and it gives \verb|$(TARGET) tests| as the targets to build.  Look up at
    the \ident{TARGET} variable and you see that's the library, so \ident{all:}
    will first build the library.  The \ident{tests} target is then further
    down in the \ident{Makefile} and builds the unit tests.
\item[Makefile:16] Another target for making "developer builds" that introduces
    a technique for changing options for just one target.  If I do a "dev build"
    I want the \ident{CFLAGS} to include options like \ident{-Wextra} that are
    useful for finding bugs.  If you place them on the target line as options
    like this, then give another line that says the original target (in this
    case \ident{all}) then it will change the options you set.  I use this for
    setting different flags on different platforms that need it.
\item[Makefile:19] Builds the \ident{TARGET} library, whatever that is, and also
    uses the same trick from line 15 of giving a target with just options changes
    to alter them for this run.  In this case I'm adding \ident{-fPIC} just for
    the library build using the \verb|+=| syntax to add it on.
\item[Makefile:20] Now the real target where I say first make the \file{build}
    directory, then compile all the \ident{OBJECTS}.
\item[Makefile:21] Runs the \program{ar} command which actually makes the
    \ident{TARGET}.  The syntax \verb|$@ $(OBJECTS)| is a way of saying,
    "put the target for this Makefile source here and all the OBJECTS after that".
    In this case the \verb|$@| maps back to the \verb|$(TARGET)| on line 19,
    which maps to \file{build/libYOUR\_LIBRARY.a}.  It seems like a lot to 
    keep track of this indirection, and it can be, but once you get it working
    this means you just change \ident{TARGET} at the top and build a whole
    new library.
\item[Makefile:22] Finally, to make the library you run \program{ranlib} on the
    \ident{TARGET} and it's built.
\item[Makefile:24-24] This just makes the \file{build/} or \file{bin/} directories
    if they don't exist.  This is then referenced from line 19 when it gives the
    \ident{build} target to make sure the \file{build/} directory is made.
\end{description}

You now have all the stuff you need to build the software, so we'll create a
way to build and run unit tests to do automated testing.

\subsection{The Unit Tests}

C is different from other languages because it's easier to create one tiny little
program for each thing you're testing.  Some testing frameworks try to emulate
the module concept other languages have and do dynamic loading, but this doesn't
work well in C.  It's also unnecessary because you can just make a single program
that's run for each test instead.

I'll cover this part of the Makefile, and then later you'll see the contents of
the \file{tests/} directory that make it actually work.

\begin{description}
\item[Makefile:29] If you have a target that's not "real", but there is a directory
    or file with that name, then you need to tag the target with \ident{.PHONY:}
    so \program{make} will ignore the file and always run.
\item[Makefile:30] I use the same trick for modifying the \ident{CFLAGS} variable
    to add the \ident{TARGET} to the build so that each of the test programs
    will be linked with the \ident{TARGET} library.  In this case it will add
    \file{build/libYOUR\_LIBRARY.a} to the linking.
\item[Makefile:31] Then I have the actual \ident{tests:} target which depends 
    on all the programs listed in the \ident{TESTS} variable we created 
    in the header.  This one line actually says, "Make, use what you know
    about building programs and the current CFLAGS settings to build each
    program in \ident{TESTS}."
\item[Makefile:32] Finally, when all of the \ident{TESTS} are built, there's
    a simple shell script I'll create later that knows how to run them all
    and report their output.  This line actually runs it so you can see
    the test results.
\item[Makefile:34-35] In order to be able to dynamically rerun the tests 
    with Valgrind there's a \ident{valgrind:} target that sets the right
    variable and runs itself again.  This puts the valgrind logs into
    \file{/tmp/valgrind-*.log} so you can go look and see what might be
    going on.  The \file{tests/runtests.sh} then knows to run the
    test programs under Valgrind when it sees this \ident{VALGRIND} variable.
\end{description}

For the unit testing to work you'll need to create a little shell script
that knows how to run the programs.  Go ahead and create this \file{tests/runtests.sh} script:


\begin{code}{tests/runtests.sh}
<< d['code/c-skeleton/tests/runtests.sh|pyg|l'] >>
\end{code}

I'll be using this later when I cover how unit tests work.

\subsection{The Cleaner}

I now have fully working unit tests, so next up is making things clean when
I need to reset everything.

\begin{description}
\item[Makefile:38] The \ident{clean:} target starts things off whenever we
    need to clean up the project.
\item[Makefile:39-42] This cleans out most of the junk that various compilers
    and tools leave behind.  It also gets rid of the \file{build/} directory
    and uses a trick at the end to cleanly erase the weird \file{*.dSYM}
    directories Apple's XCode leaves behind for debugging purposes.
\end{description}

If you run into junk that you need to clean out, simply augment the list of things
being deleted in this target.

\subsection{The Install}

After that I'll need a way to install the project, and for a \file{Makefile} that's
building a library I just need to put something in the common \ident{PREFIX}
directory, which is usually \file{/usr/local/lib}.

\begin{description}
\item[Makefile:45] This makes \ident{install:} depend on the \ident{all:} target
    so that when you run \verb|make install| it will be sure to build everything.
\item[Makefile:46] I then use the program \program{install} to create the target
    \file{lib} directory if it doesn't exist.  In this case I'm trying to make
    the install as flexible as possible by using two variables that are conventions
    for installers.  \ident{DESTDIR} is handed to make by installers that 
    do their builds in secure or odd locations to build packages.  \ident{PREFIX}
    is used when people want the project to be installed in someplace other
    than \file{/usr/local}.
\item[Makefile:47] After that I'm just using \program{install} to actually install
    the library where it needs to go.
\end{description}

The purpose of the \program{install} program is to make sure things have the 
right permissions set.  When you run \verb|make install| you usually have to
do it as the root user, so the typical build process is \verb|make && sudo make install|.

\subsection{The Checker}

The very last part of this \file{Makefile} is a bonus that I include in my 
C projects to help me dig out any attempts to use the "bad" functions in C.
Namely the string functions and other "unprotected buffer" functions.

\begin{description}
\item[Makefile:50] Sets a variable which is a big regex looking for bad
    functions like \ident{strcpy}.
\item[Makefile:51] The \ident{check:} target so you can run a check whenever
     you need.
\item[Makefile:52] Just a way to print a message, but doing \program{@echo} tells
    \program{make} to not print the command, just its output.
\item[Makefile:53] Run the \program{egrep} command on the source files
    looking for any bad patterns.  The \verb,|| true, at the end is a way
    to prevent \program{make} from thinking that \program{egrep} not finding
    things is a failure.
\end{description}

When you run this it will have the odd effect that you'll get an error when there
is nothing bad going on.

\section{What You Should See}

I have two more exercises to go before I'm done building the project skeleton
directory, but here's me testing out the features of the \file{Makefile}.

\begin{code}{Checking The Makefile}
<< d['code/ex28.wyss.sh-session|pyg|l'] >>
\end{code}

When I run the \ident{clean:} target that works, but because I don't have
any source files in the \file{src/} directory none of the other commands
really work.  I'll finish that up in the next exercises.

\section{Extra Credit}

\begin{enumerate}
\item Try to get the \file{Makefile} to actually work by putting a source
    and header file in \file{src/} and making the library.  You shouldn't
    need a \ident{main} function in the source file.
\item Research what functions the \ident{check:} target is looking for in the
    \ident{BADFUNCS} regular expression it's using.
\item If you don't do automated unit testing, then go read about it so you're
    prepared later.
\end{enumerate}