====== OSG Setup ======
Follow this link to view the [[https://twiki.grid.iu.edu/twiki/bin/view/ReleaseDocumentation/GetGridCertificates|OSG Site Certificate Guide]].

We are placing our OSG certificate packages in the export directory which will be mounted in all nodes as /share/osg.

<fc #800000># Edit the auto.share file</fc>\\
''**[user@grow-prod ~]$** sudo vi /etc/auto.share''\\

<fc #800000># Add this line.</fc>\\
<file>
osg grid-0-0.local:/export/&
</file>

<fc #800000># Update by executing:</fc>\\
''**[user@grow-prod ~]$** cd /var/411; make''
===== Pacman Setup =====
<fc #800000># Login to the grid node and as root user execute the following:</fc>\\
<fc #800000># Download the latest pacman</fc>\\
''%%**[user@grid-0-0 ~]$** wget "http://physics.bu.edu/pacman/sample_cache/tarballs/pacman-latest.tar.gz"%%''\\

<fc #800000># Unpackage pacman into /opt directory</fc>\\
''**[user@grid-0-0 ~]$** sudo tar xzvf pacman-latest.tar.gz -C /opt''\\

<fc #800000># Change directory to /opt/pacman-3.29</fc>\\
''**[user@grid-0-0 ~]$** cd /opt/pacman-3.29''\\

<fc #800000># Source pacman for the first time by executing:</fc>\\
''**[user@grid-0-0 ~]$** sudo . /opt/pacman-3.29/setup.sh''\\

<fc #800000># Add these lines to /root/.bashrc</fc>\\
''**[user@grid-0-0 ~]$** sudo vi /root/.bashrc''\\
<file>
# Source pacman
if [ -f /opt/pacman-3.29/setup.sh ]; then
    . /opt/pacman-3.29/setup.sh
fi
</file>

===== Install Certificate Scripts Package =====

<fc #800000># Retrieve and install the scripts package (Optional)</fc>\\
''**[user@grid-0-0 /opt/osg-1.2.20]$** sudo pacman -get http://vdt.cs.wisc.edu/vdt_200_cache:PPDG-Cert-Scripts''\\

<fc #800000># Setup the environment</fc>\\
''**[user@grid-0-0 /opt/osg-1.2.20]$** sudo source setup.sh''\\

===== Setup Initial Certificate Authority (CA) Package =====
**The commands that are in red notate where you will have to enter you own host name or serial numbers if not for the GROW cluster.**\\

<fc #800000># On the grid node execute the following:</fc>\\
''**[user@grid-0-0 /opt/osg-1.2.20]$** sudo vdt-ca-manage setupCA --location local --url osg''\\

<fc #800000># The results should look like so:</fc>\\
//Setting up CA Certificates for VDT installation at '/export/osg'\\
CA Certificates will be installed into /export/osg/globus/share/certificates
//
Setup completed successfully.

===== Host Certificate / Container Certificate =====
==== Request ====
<fc #800000># Change directory to the /opt/osg-1.2.20, source the setup, and execute this command.</fc>\\
''**[root@grid-0-0 ~]#** cd /opt/osg-1.2.20''\\
''**[root@grid-0-0 osg-1.2.20]#** source setup.sh''\\
''**[root@grid-0-0 osg-1.2.20]#** cert-request -ou s -dir . -label <fc #FF0000>grow-grid.its.uiowa.edu</fc>''\\


==== Retrieve and Install ====
<fc #800000># Once you have a reply from DOEGrids with a serial number in 0xYYYY format you can retrieve the certificate.</fc>\\
''**[root@grid-0-0 osg-1.2.20]#** cert-retrieve  -serial <fc #FF0000>0xYYYY</fc> -label <fc #FF0000>grow-grid.its.uiowa.edu</fc> -dir . -prefix <fc #FF0000>grow-grid.its.uiowa.edu</fc>''\\
<file>
checking CertLib version, V2-7,  This is the latest version, released 18 May 2009.
 using CA doegrids
Using URL https://pki1.doegrids.org/displayBySerial?op=displayBySerial&serialNumber=0xYYYY 
Checking that the certificate and ./grow-grid.its.uiowa.edukey.pem match
writing RSA key
./grow-grid.its.uiowa.educert.pem and ./grow-grid.its.uiowa.edukey.pem now contain your new credential
</file>

==== Verify ====
<fc #800000># Check to make sure the certificate matches your machine hostname.</fc>\\
''**[root@grid-0-0 osg-1.2.20]#** grid-cert-info -file ./<fc #FF0000>grow-grid.its.uiowa.edu</fc>cert.pem -subject''\\
<file>/DC=org/DC=doegrids/OU=Services/CN=grow-grid.its.uiowa.edu</file>

<fc #800000># Now execute:</fc>\\
''**[root@grid-0-0 osg-1.2.20]#** hostname -f''\\
<file>grow-grid.its.uiowa.edu</file>


<fc #800000># Copy the certificate to the default certificate directory and edit the file permissions.</fc>\\
''**[root@grid-0-0 osg-1.2.20]#** cp ./<fc #FF0000>grow-grid.its.uiowa.edu</fc>cert.pem /etc/grid-security/hostcert.pem''\\
''**[root@grid-0-0 osg-1.2.20]#** chmod 444 /etc/grid-security/hostcert.pem''\\
''**[root@grid-0-0 osg-1.2.20]#** cp ./<fc #FF0000>grow-grid.its.uiowa.edu</fc>key.pem /etc/grid-security/hostkey.pem''\\
''**[root@grid-0-0 osg-1.2.20]#** chmod 400 /etc/grid-security/hostkey.pem''\\

<fc #800000># We will use the host certificate as the container certificate. This is the only instance where using a copy of the host certificate is allowed.</fc>\\
''**[root@grid-0-0 osg-1.2.20]#** cp <fc #FF0000>grow-grid.its.uiowa.edu</fc>cert.pem /etc/grid-security/containercert.pem''\\
''**[root@grid-0-0 osg-1.2.20]#** cp <fc #FF0000>grow-grid.its.uiowa.edu</fc>key.pem /etc/grid-security/containerkey.pem''\\

<fc #800000># Change ownership on the container certificate.</fc>\\
''**[root@grid-0-0 osg-1.2.20]#** chown daemon:daemon /etc/grid-security/containercert.pem''\\
''**[root@grid-0-0 osg-1.2.20]#** chown daemon:daemon /etc/grid-security/containerkey.pem''\\
===== HTTP Service Certificate =====
==== Request ====
<fc #800000># Change directory to the /opt/osg-1.2.20, source the setup, and execute this command.</fc>\\
''**[root@grid-0-0 ~]#** cd /opt/osg-1.2.20''\\
''**[root@grid-0-0 osg-1.2.20]#** source setup.sh''\\
''**[root@grid-0-0 osg-1.2.20]#** cert-request -ou s -service http -host <fc #FF0000>grow-grid.its.uiowa.edu</fc> -dir . -label <fc #FF0000>grow-grid.its.uiowa.edu</fc>-http''\\


==== Retrieve and Install ====

<fc #800000># Create a new directory for the http service certificate.</fc>\\
''**[root@grid-0-0 osg-1.2.20]#** mkdir /etc/grid-security/http''\\

<fc #800000># Once you have the serial number, retrieve the certificate.</fc>\\
''**[root@grid-0-0 osg-1.2.20]#** cert-retrieve  -serial <fc #FF0000>0xYYYY</fc> -label <fc #FF0000>grow-grid.its.uiowa.edu</fc>-http -dir . -prefix <fc #FF0000>grow-grid.its.uiowa.edu</fc>-http''\\

<fc #800000># Copy the certificate to the default directory and edit the permissions.</fc>\\
''**[root@grid-0-0 osg-1.2.20]#** cp ./<fc #FF0000>grow-grid.its.uiowa.edu</fc>-httpcert.pem /etc/grid-security/http/httpcert.pem''\\
''**[root@grid-0-0 osg-1.2.20]#** chmod 444 /etc/grid-security/http/httpcert.pem''\\
''**[root@grid-0-0 osg-1.2.20]#** cp ./<fc #FF0000>grow-grid.its.uiowa.edu</fc>-httpkey.pem /etc/grid-security/http/httpkey.pem''\\
''**[root@grid-0-0 osg-1.2.20]#** chmod 400 /etc/grid-security/http/httpkey.pem''\\

<fc #800000># Change ownership of the certificate.</fc>\\
''**[root@grid-0-0 osg-1.2.20]#** chown daemon:daemon /etc/grid-security/http/httpcert.pem''\\
''**[root@grid-0-0 osg-1.2.20]#** chown daemon:daemon /etc/grid-security/http/httpkey.pem''\\

==== Verify ====
<fc #800000># Check to make sure the certificate matches your machine hostname.</fc>\\
''**[root@grid-0-0 osg-1.2.20]#** grid-cert-info -file ./<fc #FF0000>grow-grid.its.uiowa.edu-http</fc>cert.pem -subject''\\
<file>/DC=org/DC=doegrids/OU=Services/CN=grow-grid.its.uiowa.edu</file>

<fc #800000># Now execute:</fc>\\
''**[root@grid-0-0 osg-1.2.20]#** hostname -f''\\
<file>grow-grid.its.uiowa.edu</file>

===== RSV Service Certificate =====
We will create an rsvuser for this certificate.

<fc #800000># On the head node.</fc>\\
''**[root@grow-prod user]#** useradd -c "RSV monitoring user" -n rsvuser''\\
''**[root@grow-prod user]#** passwd rsvuser''\\
<fc #800000># Contact admin for password</fc>\\

''**[root@grow-prod user]#** ssh-agent $SHELL''\\
''**[root@grow-prod user]#** ssh-add''\\
''**[root@grow-prod user]#** rocks sync config''\\
''**[root@grow-prod user]#** rocks sync users''\\
==== Request ====
<fc #800000># Change directory to the /opt/osg-1.2.20, source the setup, and execute this command.</fc>\\
''**[root@grid-0-0 ~]#** cd /opt/osg-1.2.20''\\
''**[root@grid-0-0 osg-1.2.20]#** source setup.sh''\\
''**[root@grid-0-0 osg-1.2.20]#** cert-request -ou s -service rsv -host <fc #FF0000>grow-grid.its.uiowa.edu</fc> -dir . -label <fc #FF0000>grow-grid.its.uiowa.edu</fc>-rsv''\\

==== Retrieve and Install ====

<fc #800000># Once you have gotten the serial number, retrieve the certificate.</fc>\\
''**[root@grid-0-0 osg-1.2.20]#** cert-retrieve  -serial <fc #FF0000>0xYYYY</fc> -label <fc #FF0000>grow-grid.its.uiowa.edu</fc>-rsv -dir . -prefix <fc #FF0000>grow-grid.its.uiowa.edu</fc>-rsv''\\

<fc #800000># Copy the files to the default location and edit the file permissions.</fc>\\
''**[root@grid-0-0 osg-1.2.20]#** cp ./<fc #FF0000>grow-grid.its.uiowa.edu</fc>-rsvcert.pem /etc/grid-security/rsvcert.pem''\\
''**[root@grid-0-0 osg-1.2.20]#** chmod 444 /etc/grid-security/rsvcert.pem''\\
''**[root@grid-0-0 osg-1.2.20]#** cp ./<fc #FF0000>grow-grid.its.uiowa.edu</fc>-rsvkey.pem /etc/grid-security/rsvkey.pem''\\
''**[root@grid-0-0 osg-1.2.20]#** chmod 400 /etc/grid-security/rsvkey.pem''\\

<fc #800000># Change ownership on the certificate.</fc>\\
''**[root@grid-0-0 osg-1.2.20]#** chown rsvuser:users /etc/grid-security/rsvcert.pem''\\
''**[root@grid-0-0 osg-1.2.20]#** chown rsvuser:users /etc/grid-security/rsvkey.pem''\\

==== Verify ====
<fc #800000># Check to make sure the certificate matches your machine hostname.</fc>\\
''**[root@grid-0-0 osg-1.2.20]#** grid-cert-info -file ./<fc #FF0000>grow-grid.its.uiowa.edu-rsv</fc>cert.pem -subject''\\
<file>/DC=org/DC=doegrids/OU=Services/CN=grow-grid.its.uiowa.edu</file>

<fc #800000># Now execute:</fc>\\
''**[root@grid-0-0 osg-1.2.20]#** hostname -f''\\
<file>grow-grid.its.uiowa.edu</file>

===== Prepare the Environment =====
Before installing OSG software we must prepare the environment.
==== Create Directories ====
<fc #800000># On the grid node, create the directories which will be network mounted, change permissions, and change ownership on the certificates.</fc>\\
''**[root@grid-0-0 osg]#** mkdir wnclient-1.2 ce-1.2''\\
''**[root@grid-0-0 osg]#** ln -s wnclient-1.2 wnclient''\\
''**[root@grid-0-0 osg]#** ln -s ce-1.2 ce''\\
''**[root@grid-0-0 osg]#** ln -s ce-1.2 se''\\
''**[root@grid-0-0 osg]#** mkdir -p app/etc''\\
''**[root@grid-0-0 osg]#** chmod 777 app app/etc''\\
''**[root@grid-0-0 osg]#** mkdir /data/se/osg''\\
''**[root@grid-0-0 osg]#** chown root:users /data/se/osg''\\
''**[root@grid-0-0 osg]#** chmod 775 /data/se/osg''\\

==== Garbage Collection ====
=== Compute node ===
<fc #800000># We added an extend-compute.xml that creates a cron job to take care of garbage collection.</fc>\\
''**[root@grow-prod ~]#** vi /export/rocks/install/site-profiles/5.4/nodes/extend-compute.xml''\\
Click [[http://highenergy.physics.uiowa.edu/~dsquires/grow/backup_files/extend-compute.xml|here]] to see our extend-compute.xml file.

<fc #800000># Make sure to create a new distro when adding or editing a kickstart file.</fc>\\
''**[root@grow-prod ~]#** cd /export/rocks/install''\\
''**[root@grow-prod ~]#** rocks create distro''\\

<fc #800000># Check to see if the xml format is correct.</fc>\\
''**[root@grow-prod ~]#** rocks list appliance xml compute''\\
<fc #800000># If there is a mistake this will tell you which line has an error.</fc>\\

=== Login Node ===
?
==== Install xinetd ====
<fc #800000># Install xinetd service on the grid node which is needed by OSG.</fc>\\
''**[root@grid-0-0 osg]#** yum install xinetd''\\
''**[root@grid-0-0 osg]#** /etc/rc.d/init.d/xinetd restart''\\
''**[root@grid-0-0 osg]#** chkconfig --add xinetd''\\

==== Reconfigure grid node hostname ====
<fc #800000># Edit the /etc/sysconfig/network file.</fc>\\
''**[root@grow-grid ~]#** vi /etc/sysconfig/network''\\
<file>
HOSTNAME=grid-0-0.local
</file>
with
<file>
HOSTNAME=grow-grid.its.uiowa.edu
</file>

<fc #800000># In /etc/hosts, add:</fc>\\
<file>
128.255.88.50   grow-grid.its.uiowa.edu
</file>
<fc #800000># Then tell hostname to print the true FQHN:</fc>\\
''**[root@grow-grid ~]#** hostname grow-grid.its.uiowa.edu''\\

<fc #800000># Restart the network:</fc>\\
''**[root@grow-grid ~]#** service network restart''\\
<fc #800000># Make sure to logout from the grid node and log back in before continuing.</fc>

==== Enable Automatic Updates of CA Certificates ====
''[root@grow-grid ~]# cd /export/osg''\\
''[root@grow-grid osg]# vdt-control --enable vdt-update-certs''\\
<file>
running 'vdt-register-service --name vdt-update-certs --enable'... ok
</file>

==== Enable Automatic Update of Revocation List ====
''[root@grow-grid osg]# cd /export/osg''\\
''[root@grow-grid osg]# vdt-control --enable fetch-crl''\\
<file>
running 'vdt-register-service --name fetch-crl --enable'... ok
</file>
===== CE =====
''[root@grow-grid ce]# cd /share/osg/ce''\\
''[root@grow-grid ce]# pacman -get %%http://software.grid.iu.edu/osg-1.2:ce%%
''\\
''[root@grow-grid ce]# . setup.sh ''\\
''[root@grow-grid ce]# vdt-ca-manage setupca --location root --url osg''\\
<file>
Setting up CA Certificates for VDT installation at '/share/osg/ce-1.2'
CA Certificates will be installed into /etc/grid-security/certificates

Setup completed successfully.
</file>
''[root@grow-grid ce]# export VDTSETUP_CONDOR_LOCATION=/opt/condor''\\
''[root@grow-grid ce]# pacman  allow trust-all-caches -get %%http://software.grid.iu.edu/osg-1.2:Globus-Condor-Setup%%''\\
''[root@grow-grid ce]# pacman -allow trust-all-caches -get %%http://software.grid.iu.edu/osg-1.2:ManagedFork%%''\\
''[root@grow-grid ce]# $VDT_LOCATION/vdt/setup/configure_globus_gatekeeper --managed-fork y --server y''\\


==== Change Port Numbers ====
<fc #800000># Because the CE and SE are on the same node they need to differentiate port numbers for some services.</fc>\\

''[root@grow-grid ce]# vi $VDT_LOCATION/tomcat/v55/conf/server.xml''\\

<fc #800000># Change the following line to read "7443"</fc>\\
<file>
enableLookups="false" redirectPort="8443" protocol="AJP/1
</file>

<fc #800000># File should now read:</fc>\\
<file>
enableLookups="false" redirectPort="7443" protocol="AJP/1
</file>

<fc #800000># Now edit this file:</fc>
''[root@grow-grid ce]# vi $VDT_LOCATION/apache/conf/extra/httpd-ssl.conf''\\
<fc #800000># There will be three lines that need to be edited.</fc>\\
<file>
Listen 8443
RewriteRule (.*) https://%{SERVER_NAME}:8443$1
<VirtualHost _default_:8443>
ServerName www.example.com:8443
</file>
<fc #800000># Change to:</fc>\\
<file>
Listen 7443
RewriteRule (.*) https://%{SERVER_NAME}:7443$1
<VirtualHost _default_:7443>
ServerName www.example.com:7443
</file>
''[root@grow-grid ce]# vdt-post-install''

''[root@grow-grid ce]# vdt-ca-manage setupca --location local --url osg''\\
<file>
Setting up CA Certificates for VDT installation at '/share/osg/ce-1.2'
CA Certificates will be installed into /share/osg/ce-1.2/globus/share/certificates

Setup completed successfully.
</file>

''[root@grow-grid ce]# cd /etc/grid-security''\\
''[root@grow-grid grid-security]# ln -s /share/osg/ce/globus/share/certificates''\\
<file>
ln: creating symbolic link `./certificates' to `/share/osg/ce/globus/share/certificates': File exists
</file>

==== Edit rsvuser bashrc file ====

''[root@grow-prod rsvuser]# vi .bashrc''\\
<fc #800000># Add the following lines</fc>\\
<file>
. /share/osg/ce/setup.sh
. $VDT_LOCATION/vdt/etc/condor-cron-env.sh
</file>

==== OSG Environment ====
''[root@grow-prod ~]# cd /etc/profile.d''\\
''[root@grow-prod profile.d]#  vi osg.sh''\\
<fc #800000># Add this line to the file.</fc>\\
<file>
. /share/osg/ce/setup.sh
</file>
''[root@grow-prod profile.d]#  vi osg.csh''\\
<fc #800000># Add this line to the file.</fc>\\
<file>
source /share/osg/ce/setup.csh
</file>


==== Setup Condor Environment Variables ====
=== Edit the Firewall ===

''[root@grow-grid etc]# vi /etc/sysconfig/iptables''\\
<fc #800000># Add the following lines:</fc>\\
<file>
# GLOBUS_TCP_PORT_RANGE
-A INPUT  -m state --state NEW -p tcp -m tcp --dport 20000:25000 -j ACCEPT
# Monalisa, grabs 3 ports from the following range
-A INPUT  -m state --state NEW -p tcp -m tcp --dport 9000:9010 -j ACCEPT
-A INPUT  -m state --state NEW -p udp -m udp --dport 9000 -j ACCEPT
# GRAM
-A INPUT  -m state --state NEW -p tcp -m tcp --dport 2119 -j ACCEPT
# Gridftp
-A INPUT  -m state --state NEW -p tcp -m tcp --dport 2811 -j ACCEPT
# Optional Services
# RLS Server
-A INPUT  -m state --state NEW -p tcp -m tcp --dport 39281 -j ACCEPT
# MyProxy
-A INPUT  -m state --state NEW -p tcp -m tcp --dport 7512 -j ACCEPT
# MDS
-A INPUT  -m state --state NEW -p tcp -m tcp --dport 2135 -j ACCEPT
# GIIS
-A INPUT  -m state --state NEW -p tcp -m tcp --dport 2136 -j ACCEPT
# GUMS/VOMS
</file>

''[root@grow-grid etc]# /etc/rc.d/init.d/iptables restart''\\
<file>
Flushing firewall rules: [  OK  ]
Setting chains to policy ACCEPT: filter nat [  OK  ]
Unloading iptables modules: [  OK  ]
Applying iptables firewall rules: [  OK  ]
Loading additional iptables modules: ip_conntrack_netbios_ns [  OK  ]
</file>
''[root@grow-grid etc]#  /etc/rc.d/init.d/xinetd reload''\\
<file>
  Reloading configuration:                                   [  OK  ] 
</file>

=== Edit Stateful Firewalls ===

''[root@grow-grid etc]# cd /export/osg/vdt/etc/''\\
''[root@grow-grid etc]# vi vdt-local-setup.sh''\\
<file>
# This file is sourced by setup.sh.  Use it for any custom setup for this site.
# This file will be preserved across VDT installations if OLD_VDT_LOCATION is set.

# Set GLOBUS_TCP_PORT_RANGE to define communication ports for outbound connections.
export GLOBUS_TCP_PORT_RANGE=20000,25000

# Set GLOBUS_TCP_SOURCE_RANGE to define communication ports for inbound connections.
export GLOBUS_TCP_SOURCE_RANGE=20000,25000

# Set GLOBUS_TCP_PORT_RANGE_STATE_FILE to the location where Globus should record
# TCP port usage for outbound connections in case of a stateful firewall.
export GLOBUS_TCP_PORT_RANGE_STATE_FILE=/var/cache/iptables/globus

# Set GLOBUS_TCP_SOURCE_RANGE_STATE_FILE to the location where Globus should record
# TCP port usage for inbound connections in case of a stateful firewall.
export GLOBUS_TCP_SOURCE_RANGE_STATE_FILE=/var/cache/iptables/globus
</file>

''[root@grow-grid etc]# vi vdt-local-setup.csh''\\
<file>
# This file is sourced by setup.sh.  Use it for any custom setup for this site.
# This file will be preserved across VDT installations if OLD_VDT_LOCATION is set.

# Set GLOBUS_TCP_PORT_RANGE to define communication ports for outbound connections.
setenv GLOBUS_TCP_PORT_RANGE 20000,25000

# Set GLOBUS_TCP_SOURCE_RANGE to define communication ports for inbound connections.
setenv GLOBUS_TCP_SOURCE_RANGE 20000,25000

# Set GLOBUS_TCP_PORT_RANGE_STATE_FILE to the location where Globus should record
# TCP port usage for outbound connections in case of a stateful firewall.
setenv GLOBUS_TCP_PORT_RANGE_STATE_FILE /var/cache/iptables/globus

# Set GLOBUS_TCP_SOURCE_RANGE_STATE_FILE to the location where Globus should record
# TCP port usage for inbound connections in case of a stateful firewall.
setenv GLOBUS_TCP_SOURCE_RANGE_STATE_FILE /var/cache/iptables/globus
</file>


<fc #800000># It may be necessary to limit the Linux ephemeral port range to avoid the Globus ports defined above. Please check the /etc/sysctl.conf file for the following lines; insert if needed:</fc>\\
''[root@grow-grid etc]# vi /etc/sysctl.conf''\\
<file>
# Limit ephemeral ports to avoid globus TCP port range
# See OSG CE install guide
net.ipv4.ip_local_port_range = 10240 19999
</file>


<fc #800000># Execute sysctl as the root user for these settings to take effect:</fc>\\
''[root@grow-grid etc]# sysctl -p''\\


==== Additional Steps for Scientific Linux ====
<fc #800000># Scientific Linux uses /etc/hosts.deny to block all connections requests. Please add following lines to /etc/hosts.allow to open access to GridFTP and GRAM:</fc>\\
''[root@grow-grid etc]# vi /etc/hosts.allow''\\
<file>
/export/osg/vdt/services/vdt-run-gsiftp.sh : ALL
/export/osg/vdt/services/vdt-run-globus-gatekeeper.sh : ALL
</file>


==== Install Managedfork ====
<fc #800000># Install managedfork</fc>\\
''[root@grow-grid osg]# export VDTSETUP_CONDOR_LOCATION=/opt/condor''\\
''[root@grow-grid osg]# export VDTSETUP_CONDOR_CONFIG=/opt/condor/condor_configure''\\
''[root@grow-grid osg]# pacman -get %%http://software.grid.iu.edu/osg-1.2:ManagedFork%%''\\
<file>
Do you want to add http://software.grid.iu.edu/osg-1.2 to trusted.caches? (y/n/yall): yall
INFO: The Globus-Base-Info-Server package is not supported on this platform  
</file>

<fc #FF0000># This section needs to be done</fc>\\
<file>
By default managedfork does not restrict the execution of jobs forked on the gatekeeper. Restrictions can be defined using the Condor configuration file $CONDOR_CONFIG:

START_LOCAL_UNIVERSE = TotalLocalJobsRunning < 5 || GridMonitorJob =?= TRUE
 NOTE
Match TotalLocalJobsRunning to the capabilities of your gatekeeper. You must execute condor_reconfig as root for the changes to take effect.
</file>

==== Install Condor Job Manager ====
''[root@grow-grid ~]# cd /export/osg''\\
''[root@grow-grid osg]# pacman -get %%http://software.grid.iu.edu/osg-1.2:Globus-Condor-Setup%%''\\

<fc #800000># Next, optimize Gratia probe performance by changing the directory to record the job history. The location can be changed in the file $VDTSETUP_CONDOR_CONFIG using the variable PER_JOB_HISTORY_DIR:</fc>\\
''[root@grow-grid etc]# vi /opt/condor/condor_configure''\\
<file>
PER_JOB_HISTORY_DIR = /export/osg/gratia/var/data
</file>
''[root@grow-grid etc]# condor_reconfig''\\

==== Enable Log-file Rotation (Optional) ====

<fc #800000># Optionally enable the rotation of all log-files using vdt-control:</fc>\\
''[root@grid-grid /export/osg]$ vdt-control --enable vdt-rotate-logs''\\

==== Headline ====
''[root@grow-grid osg]# vdt-post-install''\\
<file>
Starting...
Done.
Making log symlinks in /export/osg/logs
</file>

===== Configure the grid-mapfile service =====
''[root@grow-grid osg]# . /export/osg/ce/setup.sh''\\
''[root@grow-grid osg]# vdt-control --enable edg-mkgridmap''\\
<file>
running 'vdt-register-service --name edg-mkgridmap --enable'... ok
</file>

===== Configuration Procedure =====
<fc #800000># Below is the changes we made to our config.ini file. We only are listing the variables that were changed.</fc>\\
<fc #800000># Edit the /share/osg/ce-1.2/osg/etc/config.ini file</fc>\\
''[root@grow-grid ce-1.2]# vi /share/osg/ce-1.2/osg/etc/config.ini''\\
<file>
;==================
;    IMPORTANT
;==================
localhost = grow-grid.its.uiowa.edu
admin_email = daniel-squires@uiowa.edu
osg_location = /share/osg/ce

;==================
; Site Information
;==================
group = OSG
host_name = grow-grid.its.uiowa.edu
resource = GROW-GRID
resource_group = GROW-UIOWA
sponsor = CMS
contact = %(admin_email)s
email = %(admin_email)s
city = "Iowa City"
country = "United States"
longitude = -91.5459099
latitude = 41.660154

;==================
;     Condor
;==================
enabled = True
condor_location = /opt/condor
condor_config = /opt/condor/condor_configure
job_contact = %(localhost)s/jobmanager-condor
util_contact = %(localhost)s/jobmanager
wsgram = %(enable)s

;==================
;   Managed Fork
;==================
enabled = %(enable)s

;==================
;  Misc Services
;==================
use_cert_updater = %(enable)s
authorization_method = gridmap
enable_webpage_creation = %(enable)s
enable_cleanup = %(disable)s

;==================
;      RSV
;==================
enabled = True
rsv_user = rsvuser
ce_hosts = %(localhost)s
enable_gridftp_probes = True
gridftp_hosts = %(localhost)s
gridftp_dir = /tmp
gratia_probes = %(default)s
enable_srm_probes = True
srm_hosts = %(localhost)s
srm_dir = /tmp
srm_webservice_path = srm/v2/server

;==================
;      Storage
;==================
se_available = %(enable)s
default_se = %(localhost)s
grid_dir = /share/osg/wnclient
app_dir = /share/osg/app
data_dir = /data/se/osg
worker_node_temp = /tmp

;==================
;      GIP
;==================
advertise_gsiftp = %(disable)s

;==================
;   Subclusters
;==================
[Subcluster grow-grid.its.uiowa.edu-ce1]
name = %(localhost)-ce1
node_count = 12
cpu_speed_mhz = 1000
cpu_platform = x86_64
ram_mb = 3952
cpu_model = Dual-Core AMD Opteron(tm) Processor 2212
cpus_per_node = 4
cores_per_node = 8
cpu_vendor = AMD

[Subcluster grow-grid.its.uiowa.edu-ce2]
name = %(localhost)-ce2
node_count = 1
ram_mb = 3270
cpu_model = Dual Core AMD Opteron(tm) Processor 290
cpu_vendor = AMD
cpu_speed_mhz =  2813 
cpu_platform = x86_64
cpus_per_node = 4
cores_per_node = 8
inbound_network = FALSE
outbound_network = TRUE

;==================
;       SE
;==================
[SE grow-grid.its.uiowa.edu-se1]
name = GROW-GRID
srm_endpoint = srm://%(localhost)s:8443/srm/v2/server
provider_implementation = bestman
version = 2.2.1.2.i7.p3
implementation = bestman
default_path = /data/se/osg/VONAME
use_df = True

</file>

<fc #800000># We are only using Condor so we deleted out the sections for PBS, SGE, and LSF</fc>\\

====== Edit Vomses ======
<fc #800000># Remove the following line from the vomses file to use only one proxy server for cms.</fc>\\
''[root@grow-grid etc]# vi /share/osg/ce/glite/etc/vomses''\\
<file>
"cms" "voms.cern.ch" "15002" "/DC=ch/DC=cern/OU=computers/CN=voms.cern.ch" "cms"
</file>

====== Verify Configuration ======
<fc #800000># Verify that the configuration script is valid.</fc>\\
''[root@grow-grid osg]# configure-osg -v''\\

====== Run Configuration Script ======
''[root@grow-grid osg]# configure-osg -c''\\
<file>
Using /share/osg/ce-1.2/osg/etc/config.ini for configuration information
running 'vdt-register-service --name vdt-rotate-logs --enable'... ok
running 'vdt-register-service --name globus-gatekeeper --enable'... ok
running 'vdt-register-service --name globus-ws --enable'... ok
running 'vdt-register-service --name fetch-crl --enable'... ok
Running /share/osg/ce-1.2/fetch-crl/share/doc/fetch-crl-2.8.5/fetch-crl.cron, this process make take some time to fetch all the crl updates
running 'vdt-register-service --name vdt-update-certs --enable'... ok
running 'vdt-register-service --name edg-mkgridmap --enable'... ok
running 'vdt-register-service --name gums-host-cron --disable'... ok
PRIMA for GT4 web services has been disabled
You will now be using a grid-mapfile for authorization.
Modifications to the /etc/sudoers file are still required.
You will need to restart the /etc/init.d/globus-ws container
to effect the changes.
Running /share/osg/ce-1.2/edg/sbin/edg-mkgridmap, this process may take some time to query vo and gums servers
INFO: Attempting to configure Apache to serve OSG site index page
 Adding site alias.
 Apache setup properly to serve the site information page.
 Restart Apache for changes to take effect.
 Enabling the Apache service using vdt-control ...
 Page can be viewed at https://HOSTNAME:8443/site
INFO: Resetting all metrics and consumers to disabled because --reset was supplied
INFO: Configuring RSV to use a service certificate.
Enabling consumers: html-consumer gratia-consumer
Enabling default metrics of type 'OSG-Local-Monitor' for host 'grow-grid.its.uiowa.edu'
Enabling default metrics of type 'OSG-CE' for host 'grow-grid.its.uiowa.edu'
Enabling default metrics of type 'OSG-GridFTP' for host 'grow-grid.its.uiowa.edu'
Enabling default metrics of type 'OSG-SRM' for host 'grow-grid.its.uiowa.edu'
running 'vdt-register-service --name condor-cron --enable'... ok
The following consumer subscription has been installed:
        HOST:    http://is2.grid.iu.edu:14001
        TOPIC:   OSG_CE
        DIALECT: RAW

The following consumer subscription has been installed:
        HOST:    http://is1.grid.iu.edu:14001
        TOPIC:   OSG_CE
        DIALECT: RAW

The following consumer subscription has been installed:
        HOST:    https://osg-ress-1.fnal.gov:8443/ig/services/CEInfoCollector
        TOPIC:   OSG_CE
        DIALECT: OLD_CLASSAD

running 'vdt-register-service --name tomcat-55 --enable'... ok
running 'vdt-register-service --name apache --enable'... ok
running 'vdt-register-service --name gratia-condor --enable'... ok
running 'vdt-register-service --name mysql5 --enable'... ok
running 'vdt-register-service --name gsiftp --enable'... ok
Configure-osg completed successfully
</file>

====== Configure grid-mapfile service ======
''[root@grow-grid osg]# vdt-control --enable edg-mkgridmap''\\
<file>
running 'vdt-register-service --name edg-mkgridmap --enable'... ok
</file>

# Copy the lines from sudo-example.txt into /etc/sudoers
''[root@grow-grid etc]# vi /share/osg/ce-1.2/osg/etc/sudo-example.txt''\\
<fc #800000># Copy the following lines.</fc>\\
<file>
Runas_Alias GLOBUSUSERS = ALL, !root

daemon   ALL=(GLOBUSUSERS) \
     NOPASSWD: /share/osg/ce-1.2/globus/libexec/globus-gridmap-and-execute \
     -g /etc/grid-security/grid-mapfile \
     /share/osg/ce-1.2/globus/libexec/globus-job-manager-script.pl *

daemon   ALL=(GLOBUSUSERS) \
     NOPASSWD: /share/osg/ce-1.2/globus/libexec/globus-gridmap-and-execute \
     -g /etc/grid-security/grid-mapfile \
     /share/osg/ce-1.2/globus/libexec/globus-gram-local-proxy-tool *
</file>

<fc #800000># Paste</fc>\\
''[root@grow-prod etc]# vi /etc/sudoers''\\

<fc #800000># Add user accounts for approved VOs</fc>\\
''[root@grow-prod ~]# useradd -c "Monitoring information service" -n mis -s /bin/true''\\
''[root@grow-prod ~]# useradd -c "CMS grid jobs" -n uscms01 -s /bin/true''\\
''[root@grow-prod ~]# useradd -c "Fermilab Service Desk" -n fermilab -s /bin/true''\\
''[root@grow-prod ~]# useradd -c "Open Science Grid" -n osg -s /bin/true''\\
''[root@grow-prod ~]# useradd -c "OSG Education Activity" -n osgedu -s /bin/true''\\
''[root@grow-prod dsquires]# useradd -c "International Linear Collider" -n ilc -s /bin/true''\\

<fc #800000># Add the following lines to the edg-mkgridmap.conf to make sure local users are not locked out.</fc>\\
''[root@grow-grid ce]# vi /share/osg/ce/edg/etc/edg-mkgridmap.conf''\\
<file>
#### GMF_LOCAL: gmf_local grid-mapfile-local
gmf_local   /etc/grid-security/grid-mapfile-local
</file>

<fc #800000># Sync Rocks users.</fc>\\
''[root@grow-prod ~]# ssh-agent $SHELL''\\
''[root@grow-prod ~]# ssh-add''\\
''[root@grow-prod ~]# rocks sync config''\\
''[root@grow-prod ~]# rocks sync users''\\

<fc #800000># Now create the grid-mapfile-local file which will map users to accounts. The last two lines are added to allow Bockjoo Kim access to setup CMSSW.</fc>\\
<file>
"/DC=org/DC=doegrids/OU=Services/CN=http/grow-grid.its.uiowa.edu" uscms01
"/DC=org/DC=doegrids/OU=Services/CN=rsv/grow-grid.its.uiowa.edu" rsvuser
"/DC=org/DC=doegrids/OU=People/CN=Bockjoo Kim (UFlorida T2 Service) 606361" cmssoft
"/DC=org/DC=doegrids/OU=People/CN=Bockjoo Kim 740786" cmssoft
</file>

====== Install the SE ======
<fc #800000># Change to SE directory</fc>\\
''[dsquires@grow-grid ~]$ cd /share/osg/se''\\

<fc #800000># Source the OSG environment.</fc>\\
''[dsquires@grow-grid se]$ . setup.sh''\\

<fc #800000># Use pacman to install Bestman. Make sure to copy your config.ini file to a safe location as Bestman will overwrite this file.</fc>\\

''[root@grow-grid se]# pacman -get %%http://software.grid.iu.edu/osg-1.2:Bestman%%''\\

<fc #800000># Replace the config.ini file with the original file.</fc>\\
<fc #800000># Source the environment again.</fc>\\
''[root@grow-grid se]# . setup.sh''\\
<file>
[root@grow-grid se]# vdt/setup/configure_bestman --server y --user best --cert /etc/grid-security/bestmancert.pem --key /etc/grid-security/bestmankey.pem --http-port 7070 --https-port 8443 --globus-tcp-port-range 20000,25000 --enable-gateway --with-allowed-paths "/tmp;/home;/data" --with-transfer-servers gsiftp://grow-grid.its.uiowa.edu
</file>

<fc #800000># Edit the /etc/sudoers file by adding these lines and commenting out the first line.</fc>\\
<file>
#Defaults    requiretty

Cmnd_Alias SRM_CMD = /bin/rm, /bin/mkdir, /bin/rmdir, /bin/mv, /bin/ls 
Runas_Alias SRM_USR = ALL, !root 
daemon ALL=(SRM_USR) NOPASSWD: SRM_CMD
</file>

====== Install worker node client ======
''[root@grow-grid dsquires]# cd /share/osg/wnclient''\\

<fc #800000># Use pacman to install worker node client.</fc>\\
''[root@grow-grid wnclient]# pacman -allow trust-all-caches -get %%http://software.grid.iu.edu/osg-1.2:wn-client%%''\\

<fc #800000># Setup the new environment.</fc>\\
''[root@grow-grid wnclient]# . setup.sh''\\


''[root@grow-grid wnclient]# vdt-ca-manage setupca --location local --url osg''\\
<file>
Setting up CA Certificates for VDT installation at '/share/osg/wnclient-1.2'
CA Certificates will be installed into /share/osg/wnclient-1.2/globus/share/certificates

Setup completed successfully.
</file>

''[root@grow-grid wnclient]# rm globus/TRUSTED_CA''\\
<file>
rm: remove symbolic link `globus/TRUSTED_CA'? y
</file>
''[root@grow-grid wnclient]# ln -s /share/osg/ce/globus/TRUSTED_CA globus/TRUSTED_CA''\\

====== Start CE and SE ======
''[root@grow-grid wnclient]# cd /share/osg/ce''\\
''[root@grow-grid ce]# . setup.sh''\\
''[root@grow-grid ce]# vdt-control %%--%%on''\\
<file>
enabling init service bestman... ok
enabling init service mysql5... ok
enabling init service globus-ws... ok
enabling cron service vdt-update-certs... ok
enabling cron service edg-mkgridmap... ok
enabling cron service vdt-rotate-logs... ok
skipping init service 'MLD' -- marked as disabled
enabling cron service gratia-condor... ok
enabling init service apache... ok
enabling init service tomcat-55... ok
enabling init service condor-cron... ok
enabling inetd service gsiftp... ok
enabling inetd service globus-gatekeeper... ok
enabling cron service fetch-crl... ok
skipping cron service 'gums-host-cron' -- marked as disabled
skipping cron service 'vdt-cleanup' -- marked as disabled
skipping cron service 'gratia-gridftp-transfer' -- marked as disabled
enabling init service osg-rsv... ok
</file>
====== Verify ======
<fc #800000># Login as a user, source the environment, get a proxy and run the following perl script.</fc>\\
''[dsquires@grow-grid ce]$ grid-proxy-init''\\


====== Notes ======


====== Contact Info ======
This Dokuwiki page is maintained by:\\
Daniel Squires\\
University of Iowa\\
Department of Computer Science\\
Email: <daniel-squires@uiowa.edu>