Difference between revisions of "Accessing Swestore with the ARC client"
| Line 16: | Line 16: | ||
If your system doesn't have the ARC client installed you can look at | If your system doesn't have the ARC client installed you can look at | ||
| − | the [[ARC_client_installation|ARC client installation]] page in this wiki or the official | + | the [[ARC_client_installation|ARC client installation]] page in this |
| − | + | wiki or the official Nordugrid | |
| − | the windows and MacOS clients are a bit unstable. | + | [http://www.nordugrid.org/documents/arc-client-install.html ARC |
| + | installation] page for more information. Please note that the windows | ||
| + | and MacOS clients are a bit unstable. | ||
A complete user guide for the ARC client can be found in | A complete user guide for the ARC client can be found in | ||
http://www.nordugrid.org/documents/arc-ui.pdf. | http://www.nordugrid.org/documents/arc-ui.pdf. | ||
| + | |||
| + | ARC can handle many different file transfer protocols, such as: | ||
| + | ''http, https, httpg, ftp, gsiftp, lfc, rls, srm''. But only the srm | ||
| + | protocol is currently is supported when accessing SweStore using the | ||
| + | ARC client. | ||
== ARC 0.8 versus 1.0 == | == ARC 0.8 versus 1.0 == | ||
| Line 38: | Line 45: | ||
= Copying files = | = Copying files = | ||
| − | Copying files to and from resources is accomplished using the | + | Copying files to and from resources is accomplished using the '''arccp''' command. |
| − | '''arccp''' command | ||
| − | |||
| − | |||
== Copying single files == | == Copying single files == | ||
| − | Copying single files is accomplished in the same way as using the normal '''cp''' command as shown in the following example: | + | Copying single files is accomplished in the same way as using the |
| + | normal '''cp''' command as shown in the following example: | ||
| + | |||
| + | $ arccp archive.tar.gz srm://srm.swegrid.se/ops/ | ||
| − | + | Please note the trailing / which marks the destination as a directory. | |
| + | Without a / the destination will be a file, which may or may not be | ||
| + | what you wanted. All required directories are created when needed so | ||
| + | the destination may be a nonexisting directory. | ||
== Recursive copying == | == Recursive copying == | ||
| − | Recursive copying is accomplished using the '''--recursive''' option. The argument to the option determines the depth of the recursive copy. | + | Recursive copying is accomplished using the '''--recursive''' option |
| + | to arccp. The argument to the option determines the depth of the | ||
| + | recursive copy. | ||
| − | + | $ arccp --recursive=3 foobar/ srm://srm.swegrid.se/ops/foobar/ | |
= Listing files = | = Listing files = | ||
| − | Listing files on a resources is done using the '''arcls''' command. In the simplest form the command just takes a URL as input and displays names and directories without any extra information as shown in the following example: | + | Listing files on a resources is done using the '''arcls''' command. In |
| + | the simplest form the command just takes a URL as input and displays | ||
| + | names and directories without any extra information as shown in the | ||
| + | following example: | ||
| − | + | $ arcls srm://srm.swegrid.se/ops/ | |
motd.1 | motd.1 | ||
bla | bla | ||
| Line 65: | Line 80: | ||
ops | ops | ||
motd.f343 | motd.f343 | ||
| − | |||
| − | Additional information can be listed by adding the '''--long''' option | + | Additional information can be listed by adding the '''--long''' option: |
| − | + | $ arcls --long srm://srm.swegrid.se/ops | |
<Name> <Type> <Size> <Creation> <Validity> <CheckSum> <Latency> | <Name> <Type> <Size> <Creation> <Validity> <CheckSum> <Latency> | ||
motd.1 file 354 2008-06-05 12:28:23 (n/a) adler32:582d7718 NEARLINE | motd.1 file 354 2008-06-05 12:28:23 (n/a) adler32:582d7718 NEARLINE | ||
bla dir 512 2008-08-22 12:23:49 (n/a) (n/a) NEARLINE | bla dir 512 2008-08-22 12:23:49 (n/a) (n/a) NEARLINE | ||
| − | + | foobar dir 512 2008-11-17 15:07:39 (n/a) (n/a) NEARLINE | |
ops dir 512 2010-01-21 11:26:00 (n/a) (n/a) NEARLINE | ops dir 512 2010-01-21 11:26:00 (n/a) (n/a) NEARLINE | ||
motd.f343 file 436 2010-01-08 14:35:40 (n/a) adler32:0fed94f2 ONLINE | motd.f343 file 436 2010-01-08 14:35:40 (n/a) adler32:0fed94f2 ONLINE | ||
| − | |||
Metadatainformation on a specific file can be listed by specifying the '''-m''' or '''--metadata''' option. In the following example the metadata information of the '''motd.1''' file is shown: | Metadatainformation on a specific file can be listed by specifying the '''-m''' or '''--metadata''' option. In the following example the metadata information of the '''motd.1''' file is shown: | ||
| − | + | $ arcls --metadata srm://srm.swegrid.se/ops/motd.1 | |
/ops/motd.1 | /ops/motd.1 | ||
accessperm:rw-r--r-- | accessperm:rw-r--r-- | ||
| Line 99: | Line 112: | ||
= Creating directories = | = Creating directories = | ||
| − | There is no command line tool for creating directories. This command will probably be added in | + | There is no command line tool for creating directories. This command |
| + | will probably be added in an upcoming release. In the meantime the | ||
| + | following procedure can be used: | ||
| − | + | $ touch dummyfile | |
| − | + | $ arccp dummyfile srm://srm.swegrid.se/ops/newdir/ | |
| − | + | $ arcls srm://srm.swegrid.se/ops/newdir/ | |
| − | |||
dummyfile | dummyfile | ||
| − | The | + | The dummmyfile can be removed using the arcrm command described in the following section. |
= Removing files or directories = | = Removing files or directories = | ||
| − | + | $ arcrm srm://srm.swegrid.se/ops/newdir/dummyfile | |
| − | + | $ arcrm srm://srm.swegrid.se/ops/newdir/ | |
To remove directories they have to be empty. | To remove directories they have to be empty. | ||
Revision as of 14:05, 10 November 2011
< Getting started with SweGrid
< Swestore National Storage
(PAGE UNDER DEVELOPMENT)
Contents
The ARC client
This guide describes how to use the Nordugrid ARC client for storing and retrieving files from SweStore National Storage. The ARC client is usually used for sending grid jobs to grid clusters, but it also contain commands for data management.
If your system doesn't have the ARC client installed you can look at the ARC client installation page in this wiki or the official Nordugrid [http://www.nordugrid.org/documents/arc-client-install.html ARC installation] page for more information. Please note that the windows and MacOS clients are a bit unstable.
A complete user guide for the ARC client can be found in http://www.nordugrid.org/documents/arc-ui.pdf.
ARC can handle many different file transfer protocols, such as: http, https, httpg, ftp, gsiftp, lfc, rls, srm. But only the srm protocol is currently is supported when accessing SweStore using the ARC client.
ARC 0.8 versus 1.0
In late spring 2011 Nordugrid release the 1.0 version of ARC (sometimes called 11.05). One of the new features of 1.0 compared to the previous 0.8 release was a new command set. Basically most of the ng* commands was replaced with the new arc* commands. Some functionality moved between commands (ngstat became arcinfo and arcstat) and some new commands was introduced (arcproxy as an replacement for grid-proxy-init, which wasn't an arc command at all but a part of the Globus Toolkit). There are still legacy compat binaries in place for the old ng* commands, but I strongly suggest that you use arc* when available.
Copying files
Copying files to and from resources is accomplished using the arccp command.
Copying single files
Copying single files is accomplished in the same way as using the normal cp command as shown in the following example:
$ arccp archive.tar.gz srm://srm.swegrid.se/ops/
Please note the trailing / which marks the destination as a directory. Without a / the destination will be a file, which may or may not be what you wanted. All required directories are created when needed so the destination may be a nonexisting directory.
Recursive copying
Recursive copying is accomplished using the --recursive option to arccp. The argument to the option determines the depth of the recursive copy.
$ arccp --recursive=3 foobar/ srm://srm.swegrid.se/ops/foobar/
Listing files
Listing files on a resources is done using the arcls command. In the simplest form the command just takes a URL as input and displays names and directories without any extra information as shown in the following example:
$ arcls srm://srm.swegrid.se/ops/ motd.1 bla generated ops motd.f343
Additional information can be listed by adding the --long option:
$ arcls --long srm://srm.swegrid.se/ops <Name> <Type> <Size> <Creation> <Validity> <CheckSum> <Latency> motd.1 file 354 2008-06-05 12:28:23 (n/a) adler32:582d7718 NEARLINE bla dir 512 2008-08-22 12:23:49 (n/a) (n/a) NEARLINE foobar dir 512 2008-11-17 15:07:39 (n/a) (n/a) NEARLINE ops dir 512 2010-01-21 11:26:00 (n/a) (n/a) NEARLINE motd.f343 file 436 2010-01-08 14:35:40 (n/a) adler32:0fed94f2 ONLINE
Metadatainformation on a specific file can be listed by specifying the -m or --metadata option. In the following example the metadata information of the motd.1 file is shown:
$ arcls --metadata srm://srm.swegrid.se/ops/motd.1 /ops/motd.1 accessperm:rw-r--r-- checksum:adler32:582d7718 ctime:2008-06-05 12:28:23 filestoragetype:PERMANENT group:25001 latency:NEARLINE lifetimeassigned:PT1S lifetimeleft:PT1S mtime:2011-03-24 12:57:42 owner:25001 path:/ops/motd.1 size:354 spacetokens: type:file
Creating directories
There is no command line tool for creating directories. This command will probably be added in an upcoming release. In the meantime the following procedure can be used:
$ touch dummyfile $ arccp dummyfile srm://srm.swegrid.se/ops/newdir/ $ arcls srm://srm.swegrid.se/ops/newdir/ dummyfile
The dummmyfile can be removed using the arcrm command described in the following section.
Removing files or directories
$ arcrm srm://srm.swegrid.se/ops/newdir/dummyfile $ arcrm srm://srm.swegrid.se/ops/newdir/
To remove directories they have to be empty.
