ycm-superbuild(7)¶
Contents
YCM Superbuild¶
A YCM Superbuild is a CMake project whose only goal is to download and build several other projects.
The superbuild will check if a package is available on the system, and, if it is not, it will download its source code build and install it.
A YCM superbuild supports out of the box:
User mode
Developer mode
Demos
Automatic integration with CDash.
Automatic documentation generation using doxygen.
Generation of dependency graphs using dot.
Depending on your reason for using a YCM superbuild, you should skip to the appropriate section:
If you just want to build the YCM superbuild project with all the sub-projects and start using it, read the YCM Superbuild Manual for Basic Users.
If you want to download, build and modify the source code of one or some of the YCM superbuild sub-projects, read YCM Superbuild Manual for Developers.
If you want to create a new superbuild or to modify an existing one, or you are interested in the internals of a YCM superbuild read YCM Superbuild Manual for Maintainers.
YCM Superbuild Manual for Basic Users¶
A YCM superbuild can be built like any other CMake project.
Linux¶
Suppose <YOUR_PROJECT>
is the directory in which you downloaded the superbuild project
you want to build:
cd <YOUR_PROJECT>
mkdir build
cd build
cmake ..
Now, if you run
make
the superbuild will download and install all the required projects that cannot be found on the system.
After the build, all the subprojects will be installed inside the
${YCM_EP_INSTALL_DIR}
folder (by default ${CMAKE_BINARY_DIR}/install
,
i.e. build/install
), therefore in order to use it you will have to adjust
some environment variables
export PATH=$PATH:${PROJECT_SOURCE_DIR}/build/install/bin/
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:${PROJECT_SOURCE_DIR}/build/install/lib/
export CMAKE_PREFIX_PATH=$CMAKE_PREFIX_PATH:${PROJECT_SOURCE_DIR}/build/install/
You can add these lines (replacing ${PROJECT_SOURCE_DIR}
with the
folder where you downloaded your project) to your ~/.bashrc
file if you
don’t want to have to execute them manually every time.
In order to compile just one project (and all the projects on which this project depends) you can just run instead
make <project>
In order to update the external projects, you will have to run
make update-all
or if you want to update just one project, you can run
make <project>-update
After updating some project, you should then rebuild.
make
OS X¶
On OS X you have the option to generate a GNU Makefile or an Xcode project. If you choose to use the Makefile then you can follow the same steps of the Linux installation guide. Only the environmental variables change, as explained later.
If you choose to generate the Xcode project you have to follow the following steps:
mkdir build
cd build
cmake .. -G Xcode
Now, if you run
xcodebuild
the superbuild will download and install all the required projects that cannot be found on the system. The above command builds the project with the default configuration of Xcode. You can also open the project into the Xcode IDE and build it from there, or explicitly specify the configuration at command line:
#Debug
xcodebuild -configuration Debug
#Release
xcodebuild -configuration Release
After the build, all the subprojects will be installed inside the
${YCM_EP_INSTALL_DIR}
folder (by default ${CMAKE_BINARY_DIR}/install
,
i.e. build/install
), therefore in order to use it you will have to adjust
some environment variables
export PATH=$PATH:${PROJECT_SOURCE_DIR}/build/install/bin/
export DYLD_LIBRARY_PATH=$DYLD_LIBRARY_PATH:${PROJECT_SOURCE_DIR}/build/install/lib/
export CMAKE_PREFIX_PATH=$CMAKE_PREFIX_PATH:${PROJECT_SOURCE_DIR}/build/install/
You can add these lines (replacing ${PROJECT_SOURCE_DIR}
with the
folder where you downloaded your project) to your ~/.bashrc
file (or the
correct file for your shell) if you don’t want to have to execute them manually
every time.
In order to compile just one project (and all the projects on which this project depends) you can just run instead
xcodebuild -target <project>
In order to update the external projects, you will have to run
xcodebuild -target ALL_UPDATE
or if you want to update just one project, you can run
xcodebuild -target <project>-update
After updating some project, you should then rebuild.
xcodebuild
If you don’t remember the name of the targets you can type
xcodebuild -list
for a list of the targets in the project.
Todo
Add Xcode screenshot and instructions for using the GUI
YCM Superbuild Manual for Developers¶
A developer is someone that does not want just to build the superbuild but also wants to modify some of the subprojects.
Note
If a package should be built with a YCM Superbuild, this should not be
available on the system. If it is, then the source code will not even
be downloaded.
In order to keep 2 different versions, the superbuild should ignore
the system version, and download and build it instead.
This can be done by setting the USE_SYSTEM_<PACKAGE>
variable to
FALSE
or by setting the YCM_DISABLE_SYSTEM_PACKAGES
variable to
TRUE
to do it for all the packages of the superbuild.
This can be done by running ccmake
or
cmake-gui
and changing the value, or by
running adding -DUSE_SYSTEM_<PACKAGE>:BOOL=FALSE
to the
cmake
command line.
Directories¶
The superbuild will download each subproject inside the build tree of the project in the folder:
${PROJECT_SOURCE_DIR}/<component>/<project name>
The <component>
is assigned by the maintainer, and has the main
purpose to split the source code into conceptual units
The <project name>
should be the name that is used in
find_package()
calls to find that package.
Each project will be configured and built in the folder:
${PROJECT_BINARY_DIR}/<component>/<project name>
${PROJECT_BINARY_DIR}
is the folder where you are building the
YCM superbuild project, usually ${PROJECT_SOURCE_DIR}/build/
The superbuild will run the configure
, build
and install
step for each project.
Each project will be installed in ${YCM_EP_INSTALL_DIR}
(by default
${PROJECT_BINARY_DIR}/install
).
You should not change the YCM_EP_INSTALL_DIR
variable, unless you wish to
build the superbuild only once, and discard the build directory. In this case
you should change this variable to the final destination of the build since, for
many projects, the build is not relocatable.
Please also note that, if you change it to a folder that is not writable by
current user, you will have to run the whole build as superuser.
Developer Mode¶
A developer usually works on a limited set of projects.
For each the superproject that the developer will modify, he should
enable the YCM_EP_DEVEL_MODE_<PROJECT>
CMake cached variable.
This can be done by running ccmake
or
cmake-gui
and changing the value, or by
running adding -DYCM_EP_DEVEL_MODE_<PACKAGE>:BOOL=FALSE
to the
cmake
command line.
Note that the update target will be disabled for
projects in YCM_EP_DEVEL_MODE_<PROJECT>
, and they will not be
updated unless the user updates them manually.
An automatic update in a modified project, would require a manual intervention
anyway.
Also when working with branches the update performed by
ExternalProject
could switch branch or checkout a specific tag
or commit, and, even though no work should be lost, the user might not know how
to recover it.
For this reason the update target is disabled for the projects that the user
will modify, and it is enabled only if the YCM_EP_EXPERT_MODE
variable is enabled.
Expert Mode¶
The YCM_EP_EXPERT_MODE
variable will set the YCM superbuild in
“expert mode”. This is disabled by default.
This means that all the projects that are in “developer mode” will have
all the targets enabled (including the update step) and that the
update and similar targets will keep
these targets as well.
Targets¶
Global Targets¶
These targets influence all the whole YCM superbuild. In brackets is the name of the target on IDEs like Visual Studio and Xcode.
test
(TEST
)¶
Run tests for all sub-projects (only if tests are enabled, see
enable_testing()
.
update-all
(ALL_UPDATE
)¶
Update all sub-projects, except for those in
YCM_EP_DEVEL_MODE_<PROJECT>
.
fetch-all
(ALL_FETCH
)¶
Runs git fetch for all the sub-projects in
YCM_EP_DEVEL_MODE_<PROJECT>
(git sub-projects only).
status-all
(ALL_STATUS
)¶
Prints the status (using the appropriate SCM command) for all the sub-projects
in YCM_EP_DEVEL_MODE_<PROJECT>
.
clean-all
(ALL_CLEAN
)¶
Todo
Missing docs
print-directories-all
(ALL_PRINT_DIRECTORIES
)¶
Prints the source and binary directories for all the sub-projects in
YCM_EP_DEVEL_MODE_<PROJECT>
.
install
¶
Todo
Add a proper installation.
Warning
YCM does not create an install
target (yet). In some cases, i.e. if you
include some CMake module that installs some file, you might find an
install
target, but it will not install the whole superbuild, but just
these files.
One known module that adds the install
target is
catkinConfig.cmake
distributed with ROS Hydro, and usually included
by running find_package(catkin)
.
Component Targets¶
These targets influence a specific COMPONENT
, for example
external
Todo
Component targets.
<COMPONENT>
¶
Todo
Missing docs
<COMPONENT>-update
¶
Todo
Missing docs
Project Targets - Common¶
These targets are always available for all the sub-projects:
<PROJECT>-test
¶
Builds a sub-project and all its dependees and executes its tests (only
if tests are enabled, see enable_testing()
.
Project Targets - Basic Mode¶
These targets are available only for the sub-projects that are not in
YCM_EP_DEVEL_MODE_<PROJECT>
.
<PROJECT>-update
¶
Update a sub-project.
Project Targets - Development Mode¶
These targets are available only for the sub-projects that are in
YCM_EP_DEVEL_MODE_<PROJECT>
.
<PROJECT>-configure
¶
Configure a sub-project.
<PROJECT>-fetch
¶
Runs git fetch a sub-project (git sub-projects only).
<PROJECT>-status
¶
Prints the status (using the appropriate SCM command) for a sub-project.
<PROJECT>-clean
¶
Todo
Missing docs
<PROJECT>-edit-cache
¶
Edit CMake cache for a sub-project (CMake sub-projects only).
<PROJECT>-print-directories
¶
Prints the source and binary directories for a sub-project
<PROJECT>-dependees
¶
Builds all the sub-projects required by this sub-project.
<PROJECT>-dependees-update
¶
Update all the sub-projects that are not in
YCM_EP_DEVEL_MODE_<PROJECT>
and that are required by this
sub-project.
<PROJECT>-dependers
¶
Builds all the sub-projects that require this sub-project.
<PROJECT>-dependers-update
¶
Update all the sub-projects that are not in
YCM_EP_DEVEL_MODE_<PROJECT>
and that require this sub-project.
Project Targets - Special Components¶
Projects in some special components behave in a different way
documentation
Projects¶
These projects usually don’t have build
, configure
, or
install
step.
The only target enabled is the <PROJECT> target.
The update
step for these projects is always performed with the
<PROJECT> target.
If one of the steps is added manually, this step is performed with the <PROJECT> target.
These projects are not added to the global targets.
examples
Projects¶
These projects are not added to the global targets.
templates
Projects¶
These projects are not added to the global targets.
YCM Superbuild Manual for Maintainers¶
A YCM superbuild is based on the ExternalProject
CMake module.
The ExternalProject
module included in YCM is basically the
same, but includes a few extra patches to improve its functionalities,
and to fix a few issues in order to be able to work with the code
downloaded by ExternalProject
, without risking to lose work.
The reason why these patches are applied here, is because they can be
tested and used, before submitting them upstream. In the future, the
goal is to merge all the required features in the module upstream.
The CMake modified modules are not included automatically when you
include YCM. In order use the version supplied with YCM, you have to set
the YCM_USE_CMAKE_PROPOSED
variable to ON
before
searching for YCM using find_package(YCM)
or
bootstrapping it.
# Enable cmake-proposed modules
set(YCM_USE_CMAKE_PROPOSED ON)
# Now bootstrap YCM
include(YCMBootstrap)
Since the superbuild will download all the external project from the net, using the bootstrap will consider YCM like any other external project, but with the difference that it is downloaded and built at configure time, instead of at compile time. This allows you to use all YCM modules right after the bootstrap.
The other important modules for making a superbuild
YCMEPHelper
, a helper forExternalProject
that does some extra setup, and add some extra targets
FindOrBuildPackage
, that ensures that a package is available and eventually downloads and builds it.
See also
Note
A superbuild usually does not contain source code, but just the CMake files to build all the subprojects. It can build code, but the target dependencies should be handled properly.
Warning
CMake modules installed by packages built by the superbuild will
not be available during the configure phase of the superbuild,
therefore you cannot include them in the CMakeLists.txt
file, and
you cannot use the functions
and
macros
that these files declare. This is
often a good reason for not including source code in the superbuild.
Build Modules¶
Each subproject “Project” to be built, should have a
BuildProject.cmake
file in one of the folders in
CMAKE_MODULE_PATH
.
YCM has a few of them, see Build Package Modules, that are found
automatically after YCM was found by find_package(YCM)
or bootstrapped.
You can add more Build modules in your superbuild, by putting them in
some folder and adding that folder to the
CMAKE_MODULE_PATH
variable. You can also use this
variable to replace one of the Build files included in YCM, but in this
case your folder must be in CMAKE_MODULE_PATH
before you search for YCM:
# Build modules are in cmake folder
list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/cmake")
# Now search (or bootstrap) YCM
find_package(YCM REQUIRED)
# or
# include(YCMBootstrap)
Each build file should handle properly the dependencies required by the
project that will be built. By managing the dependencies for each
project, the superbuild can ensure that the dependencies are available
when the build starts. This is especially important when you run
parallel builds (i.e. make -j8
) because builds
This is a basic example for a project “Bar” that depends on project
“Foo” that uses YCMEPHelper
and FindOrBuildPackage
to build the code.
include(YCMEPHelper)
include(FindOrBuildPackage)
# Ensures that the superbuild knows about the Foo package
find_or_build_package(Foo QUIET)
ycm_ep_helper(Bar TYPE GIT
SYLE FOO_STYLE
REPOSITORY foo/bar.git
TAG master
DEPENDS Foo) # Explicitly declare that Bar depends on Foo
The main CMakeLists.txt
will then include
include(FindOrBuildPackage)
# Build the Bar package. Foo will be handled automatically
# You don't need a "find_or_build_package(Foo)" call here if the Foo
# package is not conceptually part of your superbuild
find_or_build_package(Bar)
Non CMake Projects¶
Projects written with CMake can be included in a superbuild in a few minutes. Other projects using for example autotools or other build systems, can still be included, but they require some more effort to add them to the build system.
In practice, the configure
step is the one that usually requires to
be modified. You can do it by passing the CONFIGURE_COMMAND
argument
to the ycm_ep_helper()
command.
An important thing that should be configured is the prefix
where
the package will be installed, otherwise it will be installed on the
system default (usually in /usr/local/
) and that folder might not be
writable by the user.
The following strings (including the <
and >
characters) can be
used to configure the steps:
<SOURCE_DIR> # Source directory
<BINARY_DIR> # Binary directory
<INSTALL_DIR> # Install directory
<TMP_DIR> # Directory for temporary files.
For example, for an automake project, the ycm_ep_helper()
call could be something similar:
ycm_ep_helper(Foo TYPE GIT
STYLE FOO_STYLE
REPOSITORY foo/foo.git
TAG v1.0
CONFIGURE_COMMAND <SOURCE_DIR>/configure --prefix=<INSTALL_DIR>)
Overriding Parameters¶
Each parameter of the ycm_ep_helper()
function can be
overridden using cmake variables, for example to overwrite the TAG
tag for project FOO
, and the COMPONENT
for project BAR
you
can just add somewhere, before the relative ycm_ep_helper()
call:
set(FOO_TAG v1.0)
set(BAR_COMPONENT important)
See YCMEPHelper
module documentation for other details about
the parameters that can be modified with a variable.
Specifying additional CMake arguments for all subprojects¶
In some situations, it may be convenient to be able to specify some
additional command line arguments that are passed during the cmake
invocation of all CMake-based subprojects. This is possible thanks
to the YCM_EP_ADDITIONAL_CMAKE_ARGS
CMake cache variable, that
can be set as command line argument passed to the superbuild cmake
invocation:
or as a CMake variable set in the CMake code:
set(YCM_EP_ADDITIONAL_CMAKE_ARGS "-DBOOL_OPTION:BOOL=ON -DLIST_VARIABLE:STRING=foo;bar")
Note that in the latter case, you need to make sure to set the
YCM_EP_ADDITIONAL_CMAKE_ARGS
before the first inclusion
of the YCMEPHelper
module in your project.
Components¶
YCMEPHelper
assigns to each sub-project a COMPONENT
(the
default component is external
.
This is useful to separate your project in conceptual units.
You can add any other component for your superbuild.
This will influence the superbuild in some ways:
The superbuild will create targets for each component.
See also
Component Targets for details.
The component will influence the folders where the project is downloaded and built.
See also
Directories for details.
Some special components are handled in a slightly different way:
external
component is for packages that the users of the superbuild will not modify.documentation
component contains only documentation and is not necessary for building the superbuild.example
andtemplate
components are not necessary for building the superbuild.
See also
Project Targets - Special Components for details.
Changing TAG (git repository only)¶
Todo
Perhaps this should be in the YCMEPHelper
module documentation?
ycm_ep_helper()
allows you to set a TAG
for git
repositories.
This TAG
can be any ref known by the git repository, i.e. a commit
hash, a tag or a branch.
ExternalProject handles this tag in different ways depending if this is a head or a fixed commit.
In the first case, git will perform a checkout of the tag the first time, and then it will perform a rebase when the update target is executed.
This is the recommended mode to use when you need to work on one project inside your superbuild.
Todo
Changing branch issues
In the latter case, git always performs a checkout of the specific commit, leaving the user in ‘detached HEAD’ state.
This is the recommended mode to use for projects that are just dependencies for projects inside your superbuild, and that developers will not modify.
Todo
Changing tag issues
CDash Integration¶
Todo
Unit tests are not well integrated yet, see YCM issue #17
Non Interactive Builds¶
For build machines the NON_INTERACTIVE_BUILD
variable
should be set to true.
Note
This should either be set by running
cmake -DNON_INTERACTIVE_BUILD:BOOL=TRUE
, or using an initial cache
file and running cmake -C <file>
Install Step¶
An important thing to notice is that, if the subprojects are written
in a proper way, the user will have all the files that he needs to
use the projects in ${PROJECT_BINARY_DIR}/install
This means that it is quite important for your subproject to install
the files in the install
step, and that all the files are installed
inside the install prefix
. For CMake projects this usually means
that the DESTINATION
argument for the install()
command should be a relative path instead of absoulute
Maintainer Mode¶
If the YCM_EP_MAINTAINER_MODE
CMake variable is
enabled, all the targets for all the projects will be enabled, including
the update
step.
This is an useful variable for maintainers, but is not recommended for developers.
Examples¶
This is a list of known projects using YCM.
WALK-MAN FP7 EU project ( WALK-MAN Superbuild Repository)
CoDyCo FP7 EU project (CoDyCo Superbuild Repository)