Chinaunix首页 | 论坛 | 博客
  • 博客访问: 189323
  • 博文数量: 67
  • 博客积分: 375
  • 博客等级: 一等列兵
  • 技术积分: 525
  • 用 户 组: 普通用户
  • 注册时间: 2011-04-29 22:28
文章分类

全部博文(67)

文章存档

2021年(2)

2019年(1)

2018年(19)

2017年(23)

2016年(4)

2011年(18)

我的朋友

分类: 其他平台

2018-10-09 16:45:29

Build from source

Build a TensorFlow pip package from source and install it on Ubuntu Linux and macOS. While the instructions might work for other systems, it is only tested and supported for Ubuntu and macOS.

Note: We already provide well-tested, pre-built TensorFlow packages for Linux and macOS systems.

Setup for Linux and macOS

Install the following build tools to configure your development environment.

Install Python and the TensorFlow package dependencies


UBUNTU

MAC OS

		
sudo apt install python-dev python-pip  # or python3-dev python3-pip


Install the TensorFlow pip package dependencies (if using a virtual environment, omit the --user argument):

	
pip install -U --user pip six numpy wheel mock
pip install -U --user keras_applications==1.0.5 --no-deps 
pip install -U --user keras_preprocessing==1.0.3 --no-deps

The dependencies are listed in the  file under REQUIRED_PACKAGES.

Install Bazel

, the build tool used to compile TensorFlow.

Add the location of the Bazel executable to your PATH environment variable.

Install GPU support (optional, Linux only)

There is no GPU support for macOS.

Read the  guide to install the drivers and additional software required to run TensorFlow on a GPU.

Note: It is easier to set up one of TensorFlow's GPU-enabled Docker images.

Download the TensorFlow source code

Use  to clone the :

	
git clone cd tensorflow

The repo defaults to the master development branch. You can also checkout a  to build:

	
git checkout branch_name  # r1.9, r1.10, etc.

To test your copy of the source tree, run the following test (this may take a while):

	
bazel test -c opt -- //tensorflow/... -//tensorflow/compiler/... -//tensorflow/contrib/lite/...
Key Point: If you're having build problems on the latest development branch, try a release branch that is known to work.

Configure the build

Configure your system build by running the following at the root of your TensorFlow source tree:

	
./configure

This script prompts you for the location of TensorFlow dependencies and asks for additional build configuration options (compiler flags, for example). The following shows a sample run of ./configure (your session may differ):

?

View sample configuration session

Configuration options

For , specify the versions of CUDA and cuDNN. If your system has multiple versions of CUDA or cuDNN installed, explicitly set the version instead of relying on the default. ./configure creates symbolic links to your system's CUDA libraries—so if you update your CUDA library paths, this configuration step must be run again before building.

For compilation optimization flags, the default (-march=native) optimizes the generated code for your machine's CPU type. However, if building TensorFlow for a different CPU type, consider a more specific optimization flag. See the  for examples.

There are some preconfigured build configs available that can be added to the bazel build command, for example:

  • --config=mk1 —Support for the .
  • --config=monolithic —Configuration for a mostly static, monolithic build.
Note: Starting with TensorFlow 1.6, binaries use AVX instructions which may not run on older CPUs.

Build the pip package

Bazel build

CPU-only

Use bazel to make the TensorFlow package builder with CPU-only support:

	
bazel build --config=opt //tensorflow/tools/pip_package:build_pip_package

GPU support

To make the TensorFlow package builder with GPU support:

	
bazel build --config=opt --config=cuda //tensorflow/tools/pip_package:build_pip_package

Bazel build options

Building TensorFlow from source can use a lot of RAM. If your system is memory-constrained, limit Bazel's RAM usage with: --local_resources 2048,.5,1.0.

The  are built with GCC 4 and use the older ABI. For GCC 5 and later, make your build compatible with the older ABI using: --cxxopt="-D_GLIBCXX_USE_CXX11_ABI=0". ABI compatibility ensures that custom ops built against the official TensorFlow package continue to work with the GCC 5 built package.

Build the package

The bazel build command creates an executable named build_pip_package—this is the program that builds the pip package. For example, the following builds a .whl package in the /tmp/tensorflow_pkg directory:

	
./bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg

Although it is possible to build both CUDA and non-CUDA configurations under the same source tree, it's recommended to run bazel clean when switching between these two configurations in the same source tree.

Install the package

The filename of the generated .whl file depends on the TensorFlow version and your platform. Use pip install to install the package, for example:

	
pip install /tmp/tensorflow_pkg/tensorflow-version-cp27-cp27mu-linux_x86_64.whl
Success: TensorFlow is now installed.

Docker Linux builds

TensorFlow's Docker development images are an easy way to set up an environment to build Linux packages from source. These images already contain the source code and dependencies required to build TensorFlow. See the TensorFlow  for installation and the list of available image tags.

CPU-only

The following example uses the :nightly-devel image to build a CPU-only Python 2 package from the latest TensorFlow source code. See the  for available TensorFlow -devel tags.

Download the latest development image and start a Docker container that we'll use to build the pip package:

	
docker pull tensorflow/tensorflow:nightly-devel
docker run -it -w /tensorflow -v $PWD:/mnt -e HOST_PERMS="$(id -u):$(id -g)" \     tensorflow/tensorflow:nightly-devel bash 
git pull  # within the container, download the latest source code

The above docker run command starts a shell in the /tensorflow directory—the root of the source tree. It mounts the host's current directory in the container's /mnt directory, and passes the host user's information to the container through an environmental variable (used to set permissions—Docker can make this tricky).

Alternatively, to build a host copy of TensorFlow within a container, mount the host source tree at the container's /tensorflow directory:

	
docker run -it -w /tensorflow -v /path/to/tensorflow:/tensorflow -v $PWD:/mnt \     -e HOST_PERMS="$(id -u):$(id -g)" tensorflow/tensorflow:nightly-devel bash

With the source tree set up, build the TensorFlow package within the container's virtual environment:

  1. Configure the build—this prompts the user to answer build configuration questions.
  2. Build the tool used to create the pip package.
  3. Run the tool to create the pip package.
  4. Adjust the ownership permissions of the file for outside the container.
	
./configure  # answer prompts or use defaults
bazel build --config=opt //tensorflow/tools/pip_package:build_pip_package 
./bazel-bin/tensorflow/tools/pip_package/build_pip_package /mnt  # create package 
chown $HOST_PERMS /mnt/tensorflow-version-cp27-cp27mu-linux_x86_64.whl

Install and verify the package within the container:

pip uninstall tensorflow  # remove current version
pip install /mnt/tensorflow-version-cp27-cp27mu-linux_x86_64.whl 
cd /tmp  # don't import from source directory 
python -c "import tensorflow as tf; print(tf.__version__)"
Success: TensorFlow is now installed.

On your host machine, the TensorFlow pip package is in the current directory (with host user permissions): ./tensorflow-version-cp27-cp27mu-linux_x86_64.whl

GPU support

Docker is the easiest way to build GPU support for TensorFlow since the host machine only requires the (the NVIDIA? CUDA? Toolkit doesn't have to be installed). See the  and the TensorFlow  to set up  (Linux only).

The following example downloads the TensorFlow :nightly-devel-gpu-py3 image and uses nvidia-docker to run the GPU-enabled container. This development image is configured to build a Python 3 pip package with GPU support:

docker pull tensorflow/tensorflow:nightly-devel-gpu-py3
docker run --runtime=nvidia -it -w /tensorflow -v $PWD:/mnt -e HOST_PERMS="$(id -u):$(id -g)" \     tensorflow/tensorflow:nightly-devel-gpu-py3 bash

Then, within the container's virtual environment, build the TensorFlow package with GPU support:

./configure  # answer prompts or use defaults bazel build --config=opt --config=cuda //tensorflow/tools/pip_package:build_pip_package ./bazel-bin/tensorflow/tools/pip_package/build_pip_package /mnt  # create package chown $HOST_PERMS /mnt/tensorflow-version-cp35-cp35m-linux_x86_64.whl

Install and verify the package within the container and check for a GPU:

	
pip uninstall tensorflow  # remove current version
pip install /mnt/tensorflow-version-cp27-cp27mu-linux_x86_64.whl 
cd /tmp  # don't import from source directory 
python -c "import tensorflow as tf; print(tf.contrib.eager.num_gpus())"
Success: TensorFlow is now installed.

Tested build configurations

Linux

Version Python version Compiler Build tools
tensorflow-1.11.0 2.7, 3.3-3.6 GCC 4.8 Bazel 0.15.0
tensorflow-1.10.0 2.7, 3.3-3.6 GCC 4.8 Bazel 0.15.0
tensorflow-1.9.0 2.7, 3.3-3.6 GCC 4.8 Bazel 0.11.0
tensorflow-1.8.0 2.7, 3.3-3.6 GCC 4.8 Bazel 0.10.0
tensorflow-1.7.0 2.7, 3.3-3.6 GCC 4.8 Bazel 0.10.0
tensorflow-1.6.0 2.7, 3.3-3.6 GCC 4.8 Bazel 0.9.0
tensorflow-1.5.0 2.7, 3.3-3.6 GCC 4.8 Bazel 0.8.0
tensorflow-1.4.0 2.7, 3.3-3.6 GCC 4.8 Bazel 0.5.4
tensorflow-1.3.0 2.7, 3.3-3.6 GCC 4.8 Bazel 0.4.5
tensorflow-1.2.0 2.7, 3.3-3.6 GCC 4.8 Bazel 0.4.5
tensorflow-1.1.0 2.7, 3.3-3.6 GCC 4.8 Bazel 0.4.2
tensorflow-1.0.0 2.7, 3.3-3.6 GCC 4.8 Bazel 0.4.2
Version Python version Compiler Build tools cuDNN CUDA
tensorflow_gpu-1.11.0 2.7, 3.3-3.6 GCC 4.8 Bazel 0.15.0 7 9
tensorflow_gpu-1.10.0 2.7, 3.3-3.6 GCC 4.8 Bazel 0.15.0 7 9
tensorflow_gpu-1.9.0 2.7, 3.3-3.6 GCC 4.8 Bazel 0.11.0 7 9
tensorflow_gpu-1.8.0 2.7, 3.3-3.6 GCC 4.8 Bazel 0.10.0 7 9
tensorflow_gpu-1.7.0 2.7, 3.3-3.6 GCC 4.8 Bazel 0.9.0 7 9
tensorflow_gpu-1.6.0 2.7, 3.3-3.6 GCC 4.8 Bazel 0.9.0 7 9
tensorflow_gpu-1.5.0 2.7, 3.3-3.6 GCC 4.8 Bazel 0.8.0 7 9
tensorflow_gpu-1.4.0 2.7, 3.3-3.6 GCC 4.8 Bazel 0.5.4 6 8
tensorflow_gpu-1.3.0 2.7, 3.3-3.6 GCC 4.8 Bazel 0.4.5 6 8
tensorflow_gpu-1.2.0 2.7, 3.3-3.6 GCC 4.8 Bazel 0.4.5 5.1 8
tensorflow_gpu-1.1.0 2.7, 3.3-3.6 GCC 4.8 Bazel 0.4.2 5.1 8
tensorflow_gpu-1.0.0 2.7, 3.3-3.6 GCC 4.8 Bazel 0.4.2 5.1 8

macOS

CPU

Version Python version Compiler Build tools
tensorflow-1.11.0 2.7, 3.3-3.6 Clang from xcode Bazel 0.15.0
tensorflow-1.10.0 2.7, 3.3-3.6 Clang from xcode Bazel 0.15.0
tensorflow-1.9.0 2.7, 3.3-3.6 Clang from xcode Bazel 0.11.0
tensorflow-1.8.0 2.7, 3.3-3.6 Clang from xcode Bazel 0.10.1
tensorflow-1.7.0 2.7, 3.3-3.6 Clang from xcode Bazel 0.10.1
tensorflow-1.6.0 2.7, 3.3-3.6 Clang from xcode Bazel 0.8.1
tensorflow-1.5.0 2.7, 3.3-3.6 Clang from xcode Bazel 0.8.1
tensorflow-1.4.0 2.7, 3.3-3.6 Clang from xcode Bazel 0.5.4
tensorflow-1.3.0 2.7, 3.3-3.6 Clang from xcode Bazel 0.4.5
tensorflow-1.2.0 2.7, 3.3-3.6 Clang from xcode Bazel 0.4.5
tensorflow-1.1.0 2.7, 3.3-3.6 Clang from xcode Bazel 0.4.2
tensorflow-1.0.0 2.7, 3.3-3.6 Clang from xcode Bazel 0.4.2

GPU

Version Python version Compiler Build tools cuDNN CUDA
tensorflow_gpu-1.1.0 2.7, 3.3-3.6 Clang from xcode Bazel 0.4.2 5.1 8
tensorflow_gpu-1.0.0 2.7, 3.3-3.6 Clang from xcode Bazel 0.4.2 5.1 8

阅读(1032) | 评论(0) | 转发(0) |
给主人留下些什么吧!~~