D0Merge
Overview
_D0Merge_
configurator for merging files, using either 'copydoom.py' or 'evcopy' executable. As it uses a framework executable, 'copyd0om.py' uses a stager to load the input files one by one. With 'evcopy', all the input files should be on accessible disks when the merging starts. Configurator performs several checks to insure a successful merge. To create the medatata of the output file, one needs either the list of the metadata of the inputfiles or list of their provenance files (XML format), or access to SAM when the input files have been previously declared.
configurator for merging files, using either 'copydoom.py' or 'evcopy' executable. As it uses a framework executable, 'copyd0om.py' uses a stager to load the input files one by one. With 'evcopy', all the input files should be on accessible disks when the merging starts. Configurator performs several checks to insure a successful merge. To create the medatata of the output file, one needs either the list of the metadata of the inputfiles or list of their provenance files (XML format), or access to SAM when the input files have been previously declared.
Schema Element Documentation
| Parameter | Default Value | Description |
|---|---|---|
| AppFamily | None | SAM Application family (The tuple AppFamily, AppName, AppVersion ) has to be declared to SAM |
| AppName | None | SAM Application name |
| AppVersion | None | SAM Application version ident. to d0release version |
| BasicScript | copyd0om.py | Name of the script which performs the copy (evcopy or copyd0om.py) |
| CommandLineArgs | None | Arguments passed to the Executable eg. myoutput.dat |
| CommandLineOpts | Options passed to the Executable eg. --verbose | |
| CustomExecutable | None | Name of a custom executable binary, supercede Executable |
| D0Release | None | Version of D0 Software to use for this executable |
| DataStream | None | DataStream of the output stream, 'notstreamed' for MC files |
| DataTier | None | DataTier of the output stream |
| DistroDirectory | None | Path to runtime 'releases' directory, setup script will be constructed with os.path.join(self['DistroDirectory'], 'releases', self['ProjectVersion'], self['DistroSetup']) |
| DistroPackage | None | If using an executable tarball, list it's full (runtime) path here and the setup script will unpack it. |
| DistroSetup | run.sh | Name of exe setup script to source |
| Executable | None | Name of executable binary, set automatically for explicitly supported programs in D0Runjob |
| FrameworkRCPName | None | Name of framework RCP file to use for the executable. D0runjob has a default name if not set. |
| FrameworkRCPPackage | None | Package to get framework RCP from, D0runjob has a default name if not set. |
| InputFiles | None | List of input files |
| InputFilesLocation | None | D0runjob working variable, set automatically |
| MaxSeed | 500000000 | Max. seed value |
| MDInputFiles | None | List of MetaData for the input files |
| MergeSamMetadata | None | Type of SamMetadataNode (i.e. thumbnail) |
| MinSeed | 0 | Min. seed value (protection of generator seeds) |
| noFPE | on | by default, executables (except D0gstar) are run with -fpe option i.e. noFPE is off |
| NumRecords | None | No description |
| OutputFile | None | No description |
| OutputFilePrefix | None | It allows to bypass the default prefix for the task as set in d0workflow" |
| OutputFilesLocation | None | D0runjob working variable, set automatically |
| ParamCategory | merge | Set by d0runjob for each configurator among the possible values : merge, generated, simulated, digitized, reconstructed, merge, root-tuple. |
| ProjectName | merge | Name of the project, set by d0runjob for each task i.e. pythia, alpgen, d0gstar,... |
| ProjectVersion | None | ident. to release version |
| ProvenanceFileLocation | None | D0runjob working variable, set automatically |
| ProvenanceFileName | None | D0runjob working variable, set automatically |
| ProvInputFiles | None | List of provenance files (instead of MDInputFiles ) |
| RcpCustomDir | None | Local directory for custom RCPs |
| RcpDirectory | None |
D0runjob working variable defined as $SRT_PUBLIC_CONTEXT/ |
| SamFileType | None | for reconstructed data it is 'derivedDetector' for MC files it is 'importedSimulated' |
| SamSchema | [] | List of directories to appear in SAM metadata, set for each configurator |
| SamStoreOutput | None | Output stream to store in Enstore i.e. thumbnail |
| StagerCmd | None | Stager used by the framework to download input files |
| Strace | None | Switch to strace the command |
| TaskDirectoryName | None | D0runjob working variable |
| TaskScriptName | None | D0runjob working variable |
| UseMaxopt | None | Use the Maxopt build of this executable if possible. (1: use maxopt, 0: use normal debug build |
| UserExeDir | None | User directory for customized executables in 1st position in PATH |
| WorkGroup | dzero | Working group name |