Difference between revisions of "Swestore-irods"

From SNIC Documentation
Jump to: navigation, search
(Swestore iRODS is decomissioned)
(Tag: Redirect target changed)
 
(12 intermediate revisions by 3 users not shown)
Line 1: Line 1:
= National Storage using iRODS =
+
#REDIRECT[[Swestore iRODS is decommissioned]]
[[Category:Storage]]
 
[[Category:iRODS]]
 
 
 
NOTA: THIS PAGE IS OUT OF DATE !
 
 
 
== Getting access ==
 
; Apply for storage
 
: Please follow the instructions on the [[Apply for storage on SweStore]] page.
 
 
 
;iRODS user authentication
 
:SweStore's iRODS system uses [http://www.yubico.com/products/yubikey-hardware/yubikey/ Yubikey] one-time passwords (OTP). With a simple touch of a button, a 44 character one-time password is generated and sent to the system. The user will be provided with a SweStore yubikey.
 
:Yubikey has a status as pilot now. It can be changed in the future.
 
 
 
; iRODS usage <span style="color:#FF0000"> Pilot. </span><br> - How to acquire a SweStore yubikey
 
:Please send an email to [mailto:support@swestore.se?subject=Yubikey support@swestore.se] and provide the shipping address to where the yubikey should be sent.<br>
 
:Yubikey has a status as pilot now. It can be changed in the future.
 
 
 
== Support ==
 
 
 
If you have any issues using SweStore please do not hesitate to contact [mailto:support@swestore.se support@swestore.se].
 
 
 
<span style="color:#FF0000"> Better not to use filename with single quotes. (There were problems with these but they had been fixed.</span><br>
 
 
 
== Usage monitoring ==
 
* [http://status.swestore.se/munin/monitor/monitor/ Per Project Monitoring of Swestore usage]
 
 
 
== Supported clients ==
 
 
 
: iDrop web - Point your Web browser to [https://iweb.swestore.se iweb.swestore.se]
 
: E-iRODS iCommands 3.0 - Command line client [ftp://ftp.renci.org/pub/irods/releases/3.0.1 Download E-iRODS icommands]
 
 
 
SweStore iRODS uses PAM authentication and SweStore yubikeys. With a simple touch of a button, a 44 character one-time password is generated and sent to the system.
 
 
 
 
 
=== Web GUI (iDrop web) ===
 
Please see the specific documentation for [[iDrop web]].
 
 
 
=== Community iRODS version 3.3 ===
 
The community iRODS client version 3.3 also should work, with PAM authentication.<br>
 
It is available from [http://irods.sdsc.edu/download.html SDSC].
 
Please install the OpenSSL include files and libraries:
 
<pre>
 
$ sudo apt-get install libssl-dev (debian based system)
 
# yum install openssl-devel (redhat-based systems)
 
</pre>
 
Download irods 3.3 from http://irods.sdsc.edu/download.html and unpack the tar.gz archive.
 
 
 
Please enable the following defines in the Makefile iRODS/config/config.mk.in
 
<pre>
 
PAM_AUTH = 1
 
PAM_AUTH_NO_EXTEND = 1
 
USE_SSL = 1
 
</pre>
 
Please run irodssetup to compile the irods community client with PAM authentication.
 
 
 
== SweStore iRODS usage documentation  ==
 
 
 
To use the system you need to have the E-iRODS command line client installed or using iDROP web.
 
 
 
=== Command line client ===
 
 
 
For Linux systems the iRODS command line client is available as an installable package for various
 
Linux platforms from the e-iRODS website downloads section.
 
 
 
The command line client is natural to use for Unix users.
 
There are versions of the usual ls, rm, mv, mkdir, pwd, rsync
 
commands prefixed with an i for iRODS, i.e. irm, imv, imkdir etc.
 
 
 
As expected iput and iget move files to and from the irods system.
 
All these commands print short help when using the -h option.
 
 
 
==== iCommands environment file ====
 
 
 
There is an environment file .irodsEnv in the .irods subdirectory
 
of the home directory ($HOME/.irods/.irodsEnv) which contains information where and how
 
to access the iRODS metadata (iCAT) server.
 
 
 
It looks like (placeholders are in <>):
 
<pre>
 
irodsHost 'irods.swestore.se'
 
irodsPort 1247
 
irodsDefResource 'snicdefResc'
 
irodsHome '/snicZone/proj/<PROJECT_NAME>'
 
irodsCwd '/snicZone/proj/<PROJECT_NAME>'
 
irodsUserName '<USERNAME>'
 
irodsZone 'snicZone'
 
irodsAuthScheme 'PAM'
 
</pre>
 
 
 
The iCAT server is irods.swestore.se.
 
The default irods zone name is snicZone.
 
The default resource is snicdefResc.
 
It is best to set the home directory to the same as the
 
project directory, which would be a subdirectory under
 
the /snicZone/proj directory tree.
 
 
 
==== Yubikey instructions ====
 
 
 
Prerequisite: A correct iCommands environment file, see above for instructions.
 
 
 
# Insert the yubikey in an available USB-slot in your computer.
 
# Type iinit
 
# Touch the conductive surface on the yubikey to send an one-time password to the system.
 
 
 
<pre>
 
 
 
$ iinit
 
Enter your current PAM (system) password:
 
$ ils
 
/snicZone/proj/<projectname>:
 
$
 
</pre>
 
 
 
After that we can use the usual iCommands for 8 hours.
 
 
 
More details on the iCommands are available at
 
https://www.irods.org/index.php/icommands
 
 
 
==== iCommands ====
 
 
 
Having initialized the session as described above we can use tie iRODS versions
 
of the basic Unix commands. The project directory is under /snicZone/proj, all
 
members of the project should have write access to this directory. We can use
 
the command
 
<pre>
 
icd /snicZone/proj/projectname
 
</pre>
 
to move to the project directory, or to change to an another project directory
 
when we are members of more than one project.
 
 
 
All commands give short help when invoked with the -h flag.
 
 
 
To put files files into the iRODS system we can use:
 
<pre>
 
iput localfile irodsfile
 
</pre>
 
or, to put a whole directory tree:
 
<pre>
 
iput -r localdirectory irodscollection
 
</pre>
 
 
 
To load large amout of data it might be more advantageous to use
 
<pre>
 
irsync -r localdirectory irodscollection
 
</pre>
 
It might be a good idea to use -K so then checksums will be computed,
 
stored and checked.
 
 
 
To create directories (collections in iRODSspeak) we use:
 
<pre>
 
imkdir collection
 
</pre>
 
as it would be expected.
 
 
 
To get those files back we can use
 
<pre>
 
iget irodsfile localfile
 
</pre>
 
or
 
<pre>
 
irsync -r irodscollection localdirectory
 
</pre>
 
 
 
To remove files we use:
 
<pre>
 
irm
 
</pre>
 
or
 
<pre>
 
irm -r
 
</pre>
 
 
 
Removing files like that would put the files into the trashcan (path: /snicZone/trash/).
 
Time to time we would need to empty the trashcan, using
 
<pre>
 
irmtrash
 
</pre>
 
 
 
==== Using iCommands on SNIC HPC clusters ====
 
 
 
On SNIC-clusters the icommands command line tools are either available in the PATH or by adding the irods module, e.g.
 
: module load irods
 
:If the irods commands are not available at the SNIC HPC cluster, please contact support@swestore.se
 
We also need to setup the iCommands environment file $HOME/.irods/.irodsEnv
 
 
 
=== Storage Project directory structure ===
 
 
 
Your storage project is available at /snicZone/proj/<PROJECT NAME>
 
 
 
/snicZone/home/<USERNAME> is just a small home directory.
 
 
 
=== iDROP web client ===
 
 
 
See the [[iDrop web]] specific page.
 
 
 
=== Upstream documentation ===
 
Detailed documentation, papers and resources are available from
 
the [http://www.eirods.org E-iRODS web site]
 
 
 
[http://www.irods.org Community iRODS]
 
 
 
[https://groups.google.com/d/forum/irod-chat‎ User forum]
 

Latest revision as of 13:14, 8 February 2023