Access to offline data on TWIST cluster

The standard method to access offline data on the TWIST cluster is through the directory

It contains sub directories named like 2600, 2700 etc. Directory 2600, for example, holds symlinks to runs 2600 to 2699. (Those of them that are on disk, of course.)

All analysis jobs should read data through this central location, (/home/e614/datalinks) and never refer a file by it's physical pathname. For example, to analyse run 2809 do

This allows to make disk space management (at least a part of it) transparent for the user. A data file can be moved from one disk to another; nothing has to be done by user who runs analysis jobs on the file since the symbolic link will be updated to point to a new location.

It's also easy to see if a run you're interested in is on disk. Just use

command to see all runs currently available on disk. Hint: you can use the "-L" switch of the ls command to see statistics on data files. I.e., shows the symlink, while shows size and other parameters of the actual file.

A script is automatically executed every 5 minutes to make sure all links in /home/e614/datalinks/ are up to date. It removes hanging links and adds links if new files are available.

The "datalinks" directory is supposed to keep a collection of symbolic links only; not actual files. If you need to put a data file on an offline disk (for example, you are on shift and want to save for analysis a run you have taken) please copy it to any of the  /twist/data1/rawdata, /twist/data2/rawdata, ....   directories. The automatic script will pick up your file and make a link to it from a /home/e614/datalinks/ subdir within 5 minutes.

October, 2002: the "master" directory used to be on /twist/data3. Since the disk is no longer exists it's been moved to e614's home directory.

Back to TWIST software documentation page