Difference between revisions of "SweStore/iRODS icommand"

From SNIC Documentation
Jump to: navigation, search
(Created page with "For accessing the Swestore national storage, we use (since Nov 2013 iRODS, and more specifically the iRODS client called "icommands", which is a set of commands similar to the un...")
 
Line 13: Line 13:
 
After activating the iRODS icommands, you will be placed in one of your projects. if you run "ils", you will then see a listing of files and folders in that project, something like this:
 
After activating the iRODS icommands, you will be placed in one of your projects. if you run "ils", you will then see a listing of files and folders in that project, something like this:
  
[samuel@kalkyl4 ~]$ ils
+
[samuel@kalkyl4 ~]$ ils
 
   /ssUppnexZone/proj/b2011221:
 
   /ssUppnexZone/proj/b2011221:
 
     C- /ssUppnexZone/proj/b2011221/firstRun
 
     C- /ssUppnexZone/proj/b2011221/firstRun
Line 19: Line 19:
 
To enter one of the (or the only) folder(s), do:
 
To enter one of the (or the only) folder(s), do:
  
icd [proj-id]
+
icd [proj-id]
 
... in this case:
 
... in this case:
  
icd firstrun
+
icd firstrun
 
To switch to another project, use
 
To switch to another project, use
  
icd ..
+
icd ..
 
to back up a level. After that, you can change to another project or folder using icd as previously
 
to back up a level. After that, you can change to another project or folder using icd as previously
  
icd b2011222
+
icd b2011222
 
Upload files
 
Upload files
 
To upload the above mentioned folder, do (-r is needed to recurse into directories):
 
To upload the above mentioned folder, do (-r is needed to recurse into directories):
  
iput -r [a local folder]
+
iput -r [a local folder]
 
For single files, the -r flag is not needed:
 
For single files, the -r flag is not needed:
  
iput [a local file]
+
iput [a local file]
 
To list the newly uploaded file/directory:
 
To list the newly uploaded file/directory:
  
ils
+
ils
 
To create folders, do:
 
To create folders, do:
  
imkdir [folder-name]
+
imkdir [folder-name]
 
If you want to verify the upload outside of iRODS, there's a utility called ssverify.sh that will let you do that.
 
If you want to verify the upload outside of iRODS, there's a utility called ssverify.sh that will let you do that.
  
$ ssverify.sh test /ssUppnexZone/proj/p2061001/test
 
Checking if test/a matches /ssUppnexZone/proj/p2061001/test/a... seems good!
 
Checking if test/b matches /ssUppnexZone/proj/p2061001/test/b... seems good!
 
Checking if test/c matches /ssUppnexZone/proj/p2061001/test/c... seems good!
 
Checking if test/du matches /ssUppnexZone/proj/p2061001/test/du... seems good!
 
Checking if test/foo matches /ssUppnexZone/proj/p2061001/test/foo... seems good!
 
It seems test matches /ssUppnexZone/proj/p2061001/test on Swestore.
 
$  echo femtiofyra > test/c
 
$ ssverify.sh test /ssUppnexZone/proj/p2061001/test
 
Checking if test/a matches /ssUppnexZone/proj/p2061001/test/a... seems good!
 
Checking if test/b matches /ssUppnexZone/proj/p2061001/test/b... seems good!
 
Checking if test/c matches /ssUppnexZone/proj/p2061001/test/c... NO! Something is wrong!
 
Either the files don't match or something went wrong, please contact
 
staff@uppmax.uu.se to investigate.
 
 
Downloading files
 
Downloading files
 
To download a file again, do:
 
To download a file again, do:
  
iget [a file in iRODS]
+
iget [a file in iRODS]
 
... or, for folders, do it recursively:
 
... or, for folders, do it recursively:
  
iget -r [a folder in iRODS]
+
iget -r [a folder in iRODS]
 
File removal
 
File removal
 
To remove a file, you'll like need to use -f to irm in order to bypass the use of a trash area, which is currently not supported at Uppmax.
 
To remove a file, you'll like need to use -f to irm in order to bypass the use of a trash area, which is currently not supported at Uppmax.
Line 71: Line 57:
 
You can have iput show the progress with the -P flag
 
You can have iput show the progress with the -P flag
 
If you want to be placed in a particular directory when you load the irods module, you can create a file called
 
If you want to be placed in a particular directory when you load the irods module, you can create a file called
$HOME/.irods/.irodsEnv
+
$HOME/.irods/.irodsEnv
 
containing
 
containing
irodsHome '/ssUppnexZone/proj/myproj'
+
irodsHome '/ssUppnexZone/proj/myproj'
irodsCwd '/ssUppnexZone/proj/myproj'
+
irodsCwd '/ssUppnexZone/proj/myproj'
 
where
 
where
myproj
+
myproj
 
is replaced by your project id.
 
is replaced by your project id.
 
Read more info about the respective i-commands with "[command] -h"
 
Read more info about the respective i-commands with "[command] -h"

Revision as of 10:39, 31 October 2013

For accessing the Swestore national storage, we use (since Nov 2013 iRODS, and more specifically the iRODS client called "icommands", which is a set of commands similar to the unix commands "ls", "cd", "imkdir" etc, but with an "i" in front, plus two FTP-like commands: "iput" and "iget" (plus some more, but those are the ones you need most).

Note that data put on Swestore via iRODS is not accessible through other means.

Activate the iRODS icommands The iRODS icommands client is activated through the module system, as so many other things on SNIC clusters.

1. Log in to SNIC cluster 2. Execute:

module load irods iinit After activating the iRODS icommands, you will be placed in one of your projects. if you run "ils", you will then see a listing of files and folders in that project, something like this:

[samuel@kalkyl4 ~]$ ils
 /ssUppnexZone/proj/b2011221:
   C- /ssUppnexZone/proj/b2011221/firstRun

Navigate around To enter one of the (or the only) folder(s), do:

icd [proj-id]

... in this case:

icd firstrun

To switch to another project, use

icd ..

to back up a level. After that, you can change to another project or folder using icd as previously

icd b2011222

Upload files To upload the above mentioned folder, do (-r is needed to recurse into directories):

iput -r [a local folder]

For single files, the -r flag is not needed:

iput [a local file]

To list the newly uploaded file/directory:

ils

To create folders, do:

imkdir [folder-name]

If you want to verify the upload outside of iRODS, there's a utility called ssverify.sh that will let you do that.

Downloading files To download a file again, do:

iget [a file in iRODS]

... or, for folders, do it recursively:

iget -r [a folder in iRODS]

File removal To remove a file, you'll like need to use -f to irm in order to bypass the use of a trash area, which is currently not supported at Uppmax.

More hints You can have iput show the progress with the -P flag If you want to be placed in a particular directory when you load the irods module, you can create a file called

$HOME/.irods/.irodsEnv

containing

irodsHome '/ssUppnexZone/proj/myproj'
irodsCwd '/ssUppnexZone/proj/myproj'

where

myproj

is replaced by your project id. Read more info about the respective i-commands with "[command] -h"