|
|
|---|
-- Working model -- |
CVS repository (the same as for SRT development) is used for source code management. We do not use CVS "wrapper" commands, provided by CMT. Development unit is a package. Package is built against or "uses" other packages, i.e. it inherits a number of macro definitions from used packages and provides it's own macros for the packages that uses it. For example, useful macro can be an include path for compiler or some environment variable. The very base package in the tree is "TDAQCPolicy" package. It provides all common macros, definitions and patterns for other packages. Macros provided by this package includes compiler flags and other essential build parameters and shall not be changed. Release is a number of tagged packages, checked out in some area and built. Release is organized as another special package, "DAQRelease" which just enumerates all packages included in the release along with their version tags. Policy and Release packages are provided by the librarian. Users develop their packages against some base release area (CMTPATH or CMTPROJECTPATH (in recent releases) env. var. points to the locations of other packages/releases/projects) A typical release area looks like following (TDAQ_DIR points to /afs/cern.ch/atlas/project/tdaq):
|
|
$TDAQ_DIR/cmt/tdaq-01-02-00 <-- some stable TDAQ release. /DAQRelease/.. <-- packages that checked-out and built in the release. /owl/owl-00-00-05/src/.. /cmt/project.cmt <-- CMT "project" file /external <-- External libraries used by the release $TDAQ_DIR/cmt/nightly <-- Nighlty release where latest versions of all packages are built (may not be stable) |
-- Installation Area -- |
| .../dev
<-- release area /DAQRelease/.. <-- checked-out packages /owl/owl-00-00-05/src/.. /oks ...
/external
<-- release external area with (links to) third-party binaries
/databases
<-- link to version of databases used in this release
/man
<-- Unix manual pages
/i686-slc3-gcc323-opt |
-- Developer working area -- |
/home-users/kazarov/working/online/ <-- root of user's CMT working area /installed <-- locally built packages are installed here ...
/dvs/dvs-00-06-12/cmt |
/obsolete from tdaq-01-08-01 on/ To "use" an installation area of
the release or some other
(working)
area, user must source cmt/setup.sh script from any
package
from that area.
The follwoing variables are prepended with the path to the installation area: CMTPATH, PATH, TDAQ_CLASSPATH, LD_LIBRARY_PATH, MAN_PATH, IDLINCLUDE and may be others.
TDAQ_INST_PATH is always set to the release installation area.
Each package defines what to be installed in it's requirements file,
by
applying some patterns defined in the policy package. Some patterns are
applied
automatically. See section for description of installation patterns
from TDAQPolicy package.
NEW:
starting from release tdaq-01-08-01, each release (tdaq, tdaq-common,
dqm-common, LCG) is organized as a 'CMT
project'. It implies that user's working area is also a
'project' which uses some other projects (e.g. tdaq-01-08-01). CMTPATH
variable is not taken into account, and CMTPROJECTPATH should contain pathes
to the projects in use. On afs, the defautl value is
"/afs/cern.ch/atlas/offline/external:/afs/cern.ch/atlas/project/tdaq/cmt"
This value is set for you if
you source cmtsetup script in afs environment.
To make your working area to
act as a project, one need to create in it cmt/project.cmt file
with content 'use tdaq tdaq-01-08-01' (it should correspond to the release
you want to work with).
-- Creating a CMT package -- |
<package_root>/cmt/requirements /Makefile /src/... /test |
<package_root> can be any place, e.g. ~/working/online. <package-version> normally has the form package_mnemonic-rr-vv-pp , and may be equal to the current CVS tag of a package, for instance 'ilu-02-21-04'.
-- SVN operations -- |
Our project uses CERN IT central SVN service. General information
about accessing IT SVN server: How
To (our project is "atlastdaq").
http://svn.web.cern.ch/svn/howto.php#accessing-clients
|
SSH-based access: SVNROOT=svn+ssh://svn.cern.ch/reps/atlastdaq |
$SVNROOT/<path-to-package>/<package>/trunk/ <- this is HEAD of the package (src, cmt directories are there)you may access this structure like on the usual filesystem (which is maintained on the SVN server), e.g.
/src/
/cmt/requirements
/tags/tag1/
/tag2/ <- this looks like a complete copy of the package, tagged
/tag3/src/...
...
/branches/branch1/
/branch2/ <- another named 'copy' which you can commit to and tag
..... /src/...
> svn ls $SVNROOT/DAQ/online/dvs/branches/tdaq-02-00-01_patches/The plain SVN checkout command looks like
> svn co $SVNROOT/<path-to-package>/<package>/tags/<tag> <package>which checks-out aprticular <tag> of a <package>, and
> svn co $SVNROOT/<path-to-package>/<package>/trunk/ <package>checks-out HEAD (main trunk) of a <package> into the directory $CWD/<package>. For instance:
> svn co $SVNROOT/DAQ/online/tmgr/trunk/ tmgrThere is a recommended helper script getpkg, which checks out a package and creates important CMT version file:
> $TDAQ_DIR/cmt/bin/getpkg [-l|-ll] <package> [[-b] tag]
Usage: getpkg [-l | -ll] <package> [[-b] tag]It is affected by user-side SVNOFFSET variable, which specifies path to packages in the SVN tree (default is "DAQ/online").
Checks-out a package <package> from SVN in the current directory.
If [tag] is not specified, checks-out HEAD version from main trunk.
-l switch gives the list of last tags in the repository.
-ll is -l long version, lists also timestamps and authors of the tags.
-b specifies the branch tag to check-out.
> svn update
> svn ci [-m "message"]
> svn copy $SVNROOT/<path-to-package>/<package>/trunk/ $SVNROOT/<path-to-package>/<package>/tags/<tag> -m "comments"
> svn copy . $SVNROOT/<path-to-package>/<package>/tags/<tag> -m "comments"
> svn.tag <package> <tag> "comments"
> svn.rtag <package> <tag> "comments"
> svn help
> svn delprop 'svn:eol-style' <your file>
-- How to patch a package in a release -- |
- it is to fix bugs or feature requests,
essential for data taking and reported as Savannah bug entry. The
patch description must contain
a reference to Savannah and the application of a patch should close the
entry.
- a patch shall not change (public) package's API
and require rebuilding of client packages. Even changes in private API
(header files) usually requires recompilation of client code.
- it may include binaries (applications & libraries), scripts but
not public header files.
- patch must have a tag in CVS in order to be reproducable. Typically
this is a tag in the patch branch.
- patch must be tested before it is applied on afs and at P1. The
'RepositoryRoot' feature of Partition objetc is the best way to test
the new libraries.
1. Clean your local installation area. Do not forget to modify there
cmt/project.cmt file according to the release you work with.
2. Check-out the version of your package
corresponding to the release branch
(e.g. tdaq-01-08-01_patches).
> /afs/cern.ch/atlas/project/tdaq/cmt/bin/getpkg DAQ/DataFlow/MyPackage -b tdaq-01-09-01_patches
3. Apply patching changes, build, install and test your package for all CMTCONFIGs supported by the release. The script $TDAQ_DIR/cmt/adm/bin/build_patch may help you. You can copy it locally, slightly modify according to your needs and run.
NB: it is useful to modify cmt/requirements file to build and install only the binaries which are changed
4. Commit changes (to the SVN branch) and set a SVN tag to the
patched version of your package. Use normal tagging scheme, i.e.
<package>-mm-nn-pp or vXrYpZ, just increase the last
("patch level"), i.e. pp or Z digit.
5. remove the files which are not changed (e.g. header files, data
file) from your local installation area, leaving there only the
binaries
and
libraries that were patched.
6. cd to the installation area and run the make_patch script, e.g.
> cd ~/working/df/installed
> /afs/cern.ch/atlas/project/tdaq/cmt/bin/make_patch.sh TDAQ-01-09-01_NN.sh
where
TDAQ-01-09-01_NN is the patch name and _NN is the patch number (it does
not
really matter, the name can be changed later). It will produce a patch
file TDAQ-01-09-01_NN.sh in the same directory.
7. send a reference to the
resulting file to the release
testing list along with
all following information:
- CVS tag for the package
- short description about the fixed problem
and
the reference to Svannah bug entry
- list of files in the patch
- required user actions (e.g. restaring of applications etc.)
8. / optional / If
appropriate,
merge the changes (you have just commited to branch) with the HEAD
version of the package:
{TO BE COMPLETED}
Conflicts may be
created by merging, that need to be sloved before the commit.
The probability of having conflicts increases if the merge is postponed
and then done once both the branch and the main trunk have been changed
substantially.
Therefore, except if there are good reasons not to merge, try to do it
regularly.
See examples of this information in README files for patches already
available for releases.
-- How to develop a package against a release -- |
NOTE: This section is relevant to "afs" development environment, available for releases in /afs/cern.ch/atlas/project/tdaq/cmt. Refer to section "export releases" section for development instructions for outside users.
0. Set up your local working area,
see here.
1. Setup the release you want to work with, sourcing the setup script with release name as required parameter:
> source /afs/cern.ch/atlas/project/tdaq/cmt/bin/cmtsetup.[c]sh <release> [CMTCONFIG]
| NOTE 1: If you skip CMTCONFIG parameter, the script detects your system type and sets it to a default value according to the table below in this section. The default tag is calculated by the script 'getcmtconfig'. If you want to work with a particular configuration (e.g. profiled or debug version), you have to provide this parameter explicitly to the script. | |
| NOTE 2: cmtsetup.sh script invokes real setup scripts from releases installed in /afs/cern.ch/atlas/project/tdaq/cmt. Those scripts were generated during release build. They are "CMT-independent", i.e. they do not invoke any CMT command. |
2. Check-out your package using getpkg helper script and cd to the package cmt directory:
> cd ~/working/online/This cmt is the directory where you should issue all cmt commands, because CMT expects a requirements file here. For instance, you can ask cmt to list all used packages and check if everything is OK:
> $TDAQ_DIR/cmt/bin/getpkg ipc
> cd ipc/ipc-00-00-00/cmt
> cmt show uses
# Selection :
use CMT v1r10 /afs/cern.ch/sw/contrib
use OnlinePolicy OnlinePolicy-00-00-01 /afs/cern.ch/atlas/project/tdaq/cmt/dev
use cmdl cmdl-01-04-06 /afs/cern.ch/atlas/project/tdaq/cmt/dev
use owl owl-00-00-05 /afs/cern.ch/atlas/project/tdaq/cmt/dev
use ilu ilu-02-12-25 /afs/cern.ch/atlas/project/tdaq/cmt/dev
3. Configure your package:
> cmt configThis step has to be done once when you checked-out the package and later each time when you change version of CMT, version of the package ot it's location.
Creating setup scripts.
Creating cleanup scripts.
4. Setup your working area.
Source setup.sh from the cmt directory
of
your package:
> source ./setup.[c]sh
This will set (i.e. prepend) properly all
vital
environment
variables, like CMTPATH, PATH, LD_LIBRARY_PATH etc. i.e.
establishing
development and run-time environment to work with your working area.
5. Build your package.
Just invoke 'gmake' to build everything and then 'gmake install' to install your files.
> gmake # builds all your constituentsCMTCONFIG env. variable is
automatically defined
in the release setup script (step 1).
It is also called "primary" tag or "binary tag" or "binary
target". Like SRT_TARGET, it specifies an alias
to the platform/compiler
specific options and defines binary branch (referred in requirements
file
as $(bin) macro) in which all object files, applications and libraries
for
this target are held.
-- Installation patterns -- |
Here:
<package>
- name of the package
$(bin)
- CMT macro, referring to
the
current working area, set to ../${CMTCONFIG}
|
|
|
|
| install_headers_auto - platform independent header files | <package>/*.{h, hh, hpp, hxx, idl, inc, i, ii, inl} | include/<package>/ |
| install_scripts_auto - executable scripts | scripts/*.{sh, csh, pl} | $(CMTCONFIG)/bin |
| install_headers_bin_auto - platform dependent header files | $(bin)/<package>/*.{h, hh, hpp, hxx, idl, inc, i, ii, inl} | $(CMTCONFIG)/include/<package>/ |
| install_idl_auto
- IDL interface files |
idl/*.idl |
include/<package>/ |
| install_docs_auto - documentation | doc/*.{ps, pdf, txt, html} | share/doc/<package>/ |
apply_pattern <pattern_name> <pattern_parameters>
for example
apply_pattern
install_apps
files="ipc_server ipc_ls"
apply_pattern
install_docs
files="../data/oo-schemes/*.ps"
apply_pattern
install_libs
files="libis.so libisdoc.so"
apply_pattern
install_headers
src_dir="../Xm" files="*.h" target_dir="Xm"
|
|
|
|
| install_apps - applications | files | $(CMTCONFIG)/bin/ |
| install_scripts - scripts | src_dir, files | share/bin/ |
| install_libs - libraries | files | $(CMTCONFIG)/lib/ |
| install_headers - header files | name, src_dir, files, target_dir | include/<package>/<target_dir>/ |
| install_headers_bin - platform-dependent header files | name, src_dir, files, target_dir |
$(CMTCONFIG)/include/<package>/<target_dir>/ |
| install_docs - documentation | name, src_dir, files | share/doc/<package>/ |
| install_jar - installs JAR file, also prepends CLASSPATH | name, files | share/lib |
| install_data - arbitrary data files | name, src_dir, target_dir, files | share/data/<package>/<target_dir>/ |
| install_examples - code examples | name, src_dir, files | share/examples/<package>/ |
"files" and "src_dir" are required parameters. If you need to install files in a subdirectory, supply "tagret_dir" parameter. If you apply a pattern more than once, you must also to use "name" parameter to make each pattern usage unique: here is an example of usage of install_data from confdb package:
apply_pattern install_data name=foo1
src_dir=$(bin)
files=*.schema.xml
apply_pattern install_data name=foo2
src_dir=../gui
files=*.xml
apply_pattern install_data name=foo3
src_dir=../gui/bitmaps target_dir=gui/bitmaps
files=*.xbm
apply_pattern install_data name=foo4
src_dir=../gui/pixmaps target_dir=gui/pixmaps
files=*.xpm
apply_pattern install_data name=foo5
src_dir=../gui/online-help \
target_dir=gui/online-help \
files=*.html
-- Requirements file hints and HOWTO -- |
library
ipc
$(link_opts) ../src/*.cpp
library
ipc
$(link_opts) ../src/*.cpp
macro ipc_shlibflags
"-lowl -lomniORB4 -lomnithread
$(socket_libs)"
application ipc_server ../bin/server.cc
macro ipc_server_linkopts "-lipc -lowl -lomniORB4 -lomnithread $(socket_libs)"
library LC $(link_opts) src/libc/*.cpp
library LB $(link_opts) src/libb/*.cpp
macro AA_dependencies LA
datamacro LA_dependencies "LB LC"
private
macro
constituents
"app1 app2 lib1"
\
<- list of targets for all platforms
i686&linux
"app1 app2 app3
lib2"
\ <- targets to be
built
only on intel linux platform
slc3
"app1 app_root"
<- to be
build
only on SLC3 platform
NOTE: this must be in "private" section othervise client packages would be affected.
You may also use "remove_macro" instriction:
macro_remove
constituents
""
\
icc8
"ipc.jar"
<-- do not build java stuff for icc8
compiler
<package_root>/jsrc/igui
/igui/*.java
/tile/*.java
/images/*.gif
/idl/rc.idl
And you also need to use idl interfaces from other packages.
Then, to build a jar, apply the build_jar
pattern:
apply_pattern build_jar
\
name=igui
\ # name for jar file
src_dir=../jsrc
\ # source dir
sources="igui/*.java
df/*.java"
\ # java sources, relative to src_dir
idl_sources="../idl/igui.idl
\ # idl sources, absolute or
$(RCROOT)/comms/rc.idl
\ # relative to cmt dir.
$(DSAROOT)/src/pmg_daqSupervisorUI.idl \
$(PMGROOT)/src/pmg.idl"
\
others="images/*.gif"
# other files, relative to src_dir
Resulting igui.jar can later install using pattern
apply_pattern install_jar files="igui.jar"
Installation is not included in the build_jar
pattern, because it may be used to build test libraries not to be
installed.
> export CMTEXTRATAGS=debug
That's it.
How to implement a check target for a package
To execute a script in "gmake check", apply the following patern:
apply_pattern check_target
name=dvs
# some name (needed if you have several check targets)
file=../bin/check_dvs.sh
# script to run
args="$(bin)"
# script parameters
to build an application during check target, use -group option when specifying application constituent:
application
my_check
-group=check ../tests/my_check.cpp
How to refer to installation area from within requirements file
The following macros are defined (with the same meaning as in SRT):
macro prefix
<path>/installed
# installation area,
macro datadir
<path>/installed/share/data # data area
macro docdir
<path>/installed/share/doc # documentation area
macro top_srcdir
<path>/<package>/<version> #
package's
sources
<path> is set either to user's working area or to release
area,
according to context.
-- Release exportation -- |
What is exported:
- release install area, splitted into common part and
platform-dependent
parts.
- databases
- TDAQPolicy package which is needed to develop you packages
against
downloaded and installed release according to Online working model.
- Sources needed to build the release on remote site with CMT
-- How to use a release -- |
> source /afs/cern.ch/atlas/project/tdaq/cmt/nightly/installed/setup.[c]sh
(both -csh and -sh scripts are available)
This is also applicable for the release which was downloaded and
installed
elsewhere. See releases download
page for more details.
-- How to build a release -- |
That's it.
There is a number of scripts available in $TDAQ_DIR/cmt/adm/bin which are
used for release builds on afs:
co_release_common <release_name>
[<dir>]
- checks out a release to <dir> (not obligitary parameter) and
create necessary directories and links. It also copies external stuff
from $TDAQ_DIR/cmt/external.
build_release_common <release_name>
<cmtconfig> [<dir>]
- builds a release for a particular <cmtconfig> locally on this
machine.
build_package <package_name>
<release_name> ["<cmtconfig+>"]
- builds a prtitcular package in the release for all (or some)
CMTCONFIGs. It is to be used if a package is updated and needs to be
rebuilt directly in the release.
do_setup_scripts <release_name>
- produces a set of setup scripts in 'installed' dir in the release.
This script is run from co_release_common, so you need it only if some
settings in requirements files are changed and you need to update setup
scripts.
do_tarballs
<release_name>
- makes tarballs (but not the HTML download page) for download at
$TDAQ_DIR/cmt/download
gen_status_table
<release_name>
- produces HTML page with table
The followin directories are needed by the above scripts:
$TDAQ_DIR/cmt/adm/targets/<release>/
- the list of CMTCONFIGs used in this release is there in form of a
file per target with machine name inside used for build.
$TDAQ_DIR/cmt/adm/packages/common/<release>/
- 'tag collector', file per package.
The latest TDAQPolicy
and
TDAQCPolicy
requirements files with comments - for a librarian who wants to
understand details
of our CMT working model.
-- Related Links -- |
ATLAS Offline
CMT page
ATLAS Data
Collection CMT page