Skip to content

Commit

Permalink
Tensorflow can be built from standard repo
Browse files Browse the repository at this point in the history
+ VS 2017 patches applied by script before build
-> removes necessity of extra tf fork
+ builds take place in cache folder
+ resolved duplicated include folders in lib/
  • Loading branch information
Jens committed Apr 17, 2019
1 parent b88cebe commit de76081
Show file tree
Hide file tree
Showing 10 changed files with 538 additions and 45 deletions.
1 change: 0 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -267,4 +267,3 @@ __pycache__/

# ignore generated files from build
/tensorflow*
/*/
27 changes: 16 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,26 +20,33 @@ Adding the build customisation file to a project automatically enables access to
Download OpenCV 4.0.1 into the lib directory and rename the folder to "OpenCV 4.0.1".


### Building Tensorflow 1.10 with cmake and Visual Studio 2017
### Building Tensorflow 1.10 with CMake and Visual Studio 2017

1. Install Visual Studio 2017 Community Edition, Python 3.66, Git, Cuda 10

2. Enable long filename for Git, otherwise you'll might not get very far.
2. Enable long filenames for Git, otherwise you'll might not get very far:
```
git config --system core.longpaths true
```

3. Enable long file names in Windows 10.
3. Enable long filenames in Windows 10 for the same reason :
- https://www.howtogeek.com/266621/how-to-make-windows-10-accept-file-paths-over-260-characters/
- https://docs.microsoft.com/en-us/windows/desktop/fileio/naming-a-file

Currently, only TensorFlow r1.10 is supported, for a couple of reasons:

1. It is the last version that that officially supports CUDA Compute Caps 3.5, and therefore runs with my slightly outdated 2013 GPU (NVidia Titan).
2. The cmake build supports Cuda 10.
3. The bazel Windows build (as of tf r1.10) has lots of issues.
4. Only 2 patches were necessary to build with Visual Studio 2017.
4. Builds with Visual Studio 2017 after applying two patches.
5. Debug build is fully supported - won't work with Bazel build either.

Open a Visual Studio 2017 x64 command prompt and execute the build script
Open a Visual Studio 2017 x64 command prompt and execute the build script that matches your hardware configuration
```
`build_tensorflow_r1.10-cmake-all.bat`.
build_tensorflow_r1.10-cmake-cpu.bat
build_tensorflow_r1.10-cmake-cuda.bat
```
The script downloads the patched fork of the TensorFlow r.1.10 branch, creates Visual Studio Solutions via CMake, and then invokes MSBuild to build targets for several CPU/GPU configurations. The build should take about half a day or so and leaves you with a couple of TensorFlow dlls in `lib\tensorflow\r1.10\vc15` for Debug, avx, avx2, cuda, cuda-avx & avx2-fma configurations.
The script clones TensorFlow r.1.10 branch, applies patches to enable building with Visual Studio 2017, creates Visual Studio solutions via CMake, and then invokes MSBuild to build targets for several CPU/GPU configurations. The build should take about half a day or so and leaves you with a couple of TensorFlow dlls in `lib\tensorflow\r1.10\vc15` for Debug, avx, avx2, cuda, cuda-avx & avx2-fma configurations.

You can build a single configuration with `build_tensorflow_with_cmake.bat "VS Configuration"`, for instance
```
Expand All @@ -50,15 +57,13 @@ The special fma build will be included into the avx2 build later on, but I'm int

There's no /GL build yet because it exceeds the 4G size limit of the COFF file format.

Include files are duplicated in each configuration on install, that's because each cpu/gpu variant is built as a single CMake solution.

The cpu/gpu arch is simply matched via the configuration name, so the project references Debug/Release (both built with AVX) automatically. To match additional build-config library, just name them after the folder that contains the specific version of Tensorflow. (For instance `Release-cuda-avx2`)
The cpu/gpu arch is simply matched via the configuration name, so the project references Debug/Release (both built with AVX) automatically. To match a build configuration, just name it after the folder that contains the specific version of Tensorflow. (For instance `Release-cuda-avx2`)


### Running the test project "Hello Tensorflow"
A hello-world example derived from https://joe-antognini.github.io/machine-learning/windows-tf-project, plus a Google test suite to assert that the library works.

If you build and run the Hello-Word example, the console should display the expected result
When you build and run the Hello-Word example, the console should display the expected result
```
7 17
-1 -3
Expand Down
2 changes: 1 addition & 1 deletion Tensorflow.props
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<_PropertySheetDisplayName>TensorFlow</_PropertySheetDisplayName>
<TENSORFLOW_DIR>$(MSBuildThisFileDirectory)tensorflow\r1.10\</TENSORFLOW_DIR>
<ExecutablePath>$(TENSORFLOW_DIR)$(VS_VERSION)\$(Configuration)\bin\;$(ExecutablePath)</ExecutablePath>
<IncludePath>$(TENSORFLOW_DIR)$(VS_VERSION)\$(Configuration)\include\;$(IncludePath)</IncludePath>
<IncludePath>$(TENSORFLOW_DIR)include\;$(IncludePath)</IncludePath>
<LibraryPath>$(TENSORFLOW_DIR)$(VS_VERSION)\$(Configuration)\lib\;$(LibraryPath)</LibraryPath>
</PropertyGroup>
<ItemDefinitionGroup>
Expand Down
262 changes: 262 additions & 0 deletions build/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,262 @@
## Ignore Visual Studio temporary files, build results, and
## files generated by popular Visual Studio add-ons.

# User-specific files
*.suo
*.user
*.userosscache
*.sln.docstates

# User-specific files (MonoDevelop/Xamarin Studio)
*.userprefs

# Build results
[Dd]ebug/
[Dd]ebugPublic/
[Rr]elease/
[Rr]eleases/
x64/
x86/
bld/
[Bb]in/
[Oo]bj/
[Ll]og/

# Visual Studio 2015 cache/options directory
.vs/
# Uncomment if you have tasks that create the project's static files in wwwroot
#wwwroot/

# MSTest test Results
[Tt]est[Rr]esult*/
[Bb]uild[Ll]og.*

# NUNIT
*.VisualState.xml
TestResult.xml

# Build Results of an ATL Project
[Dd]ebugPS/
[Rr]eleasePS/
dlldata.c

# DNX
project.lock.json
project.fragment.lock.json
artifacts/

*_i.c
*_p.c
*_i.h
*.ilk
*.meta
*.obj
*.pch
*.pdb
*.pgc
*.pgd
*.rsp
*.sbr
*.tlb
*.tli
*.tlh
*.tmp
*.tmp_proj
*.log
*.vspscc
*.vssscc
.builds
*.pidb
*.svclog
*.scc

# Chutzpah Test files
_Chutzpah*

# Visual C++ cache files
ipch/
*.aps
*.ncb
*.opendb
*.opensdf
*.sdf
*.cachefile
*.VC.db
*.VC.VC.opendb

# Visual Studio profiler
*.psess
*.vsp
*.vspx
*.sap

# TFS 2012 Local Workspace
$tf/

# Guidance Automation Toolkit
*.gpState

# ReSharper is a .NET coding add-in
_ReSharper*/
*.[Rr]e[Ss]harper
*.DotSettings.user

# JustCode is a .NET coding add-in
.JustCode

# TeamCity is a build add-in
_TeamCity*

# DotCover is a Code Coverage Tool
*.dotCover

# NCrunch
_NCrunch_*
.*crunch*.local.xml
nCrunchTemp_*

# MightyMoose
*.mm.*
AutoTest.Net/

# Web workbench (sass)
.sass-cache/

# Installshield output folder
[Ee]xpress/

# DocProject is a documentation generator add-in
DocProject/buildhelp/
DocProject/Help/*.HxT
DocProject/Help/*.HxC
DocProject/Help/*.hhc
DocProject/Help/*.hhk
DocProject/Help/*.hhp
DocProject/Help/Html2
DocProject/Help/html

# Click-Once directory
publish/

# Publish Web Output
*.[Pp]ublish.xml
*.azurePubxml
# TODO: Comment the next line if you want to checkin your web deploy settings
# but database connection strings (with potential passwords) will be unencrypted
#*.pubxml
*.publishproj

# Microsoft Azure Web App publish settings. Comment the next line if you want to
# checkin your Azure Web App publish settings, but sensitive information contained
# in these scripts will be unencrypted
PublishScripts/

# NuGet Packages
*.nupkg
# The packages folder can be ignored because of Package Restore
**/packages/*
# except build/, which is used as an MSBuild target.
!**/packages/build/
# Uncomment if necessary however generally it will be regenerated when needed
#!**/packages/repositories.config
# NuGet v3's project.json files produces more ignoreable files
*.nuget.props
*.nuget.targets

# Microsoft Azure Build Output
csx/
*.build.csdef

# Microsoft Azure Emulator
ecf/
rcf/

# Windows Store app package directories and files
AppPackages/
BundleArtifacts/
Package.StoreAssociation.xml
_pkginfo.txt

# Visual Studio cache files
# files ending in .cache can be ignored
*.[Cc]ache
# but keep track of directories ending in .cache
!*.[Cc]ache/

# Others
ClientBin/
~$*
*~
*.dbmdl
*.dbproj.schemaview
*.jfm
*.pfx
*.publishsettings
node_modules/
orleans.codegen.cs

# Since there are multiple workflows, uncomment next line to ignore bower_components
# (https://github.com/github/gitignore/pull/1529#issuecomment-104372622)
#bower_components/

# RIA/Silverlight projects
Generated_Code/

# Backup & report files from converting an old project file
# to a newer Visual Studio version. Backup files are not needed,
# because we have git ;-)
_UpgradeReport_Files/
Backup*/
UpgradeLog*.XML
UpgradeLog*.htm

# SQL Server files
*.mdf
*.ldf

# Business Intelligence projects
*.rdl.data
*.bim.layout
*.bim_*.settings

# Microsoft Fakes
FakesAssemblies/

# GhostDoc plugin setting file
*.GhostDoc.xml

# Node.js Tools for Visual Studio
.ntvs_analysis.dat

# Visual Studio 6 build log
*.plg

# Visual Studio 6 workspace options file
*.opt

# Visual Studio LightSwitch build output
**/*.HTMLClient/GeneratedArtifacts
**/*.DesktopClient/GeneratedArtifacts
**/*.DesktopClient/ModelManifest.xml
**/*.Server/GeneratedArtifacts
**/*.Server/ModelManifest.xml
_Pvt_Extensions

# Paket dependency manager
.paket/paket.exe
paket-files/

# FAKE - F# Make
.fake/

# JetBrains Rider
.idea/
*.sln.iml

# CodeRush
.cr/

# Python Tools for Visual Studio (PTVS)
__pycache__/
*.pyc
/cache
Loading

0 comments on commit de76081

Please sign in to comment.