OpenShift Origin
In this step of the tutorial let’s configure the system limits for the nodes, and also let’s configure GIT over SSH.
Let’s begin.
Node1
We’ll clarify some concepts. The clients will connect to the broker system and will configure their name space (DNS resolution), will create an application (at a node) and will choose which gear their will use. Now, the gear is chosen from broker, but the runtime environment will be created in the node which is responsible for the CPU, RAM and storage limits, thanks to a gear definition which defaults to “small, medium and large”. Then, if we have several nodes configured (i.e. node1 and node2), node1 defines small gear to 256MB of RAM and 1GB of HDD, but node2 could defines small gear to 512MB of RAM and 10GB of HDD. There is no control by the broker to this situation, because it only sees the “small” label; so our work should be to establish a logical rules to create a homogeneous environment.
PAM modules
OpenShift Origin uses some exclusive PAM modules, but the OS doesn’t use it so we must configure PAM system modules to use OpenShift PAM modules. Since there is a lot of config files to modify, let’s use Augeas, a project belonging to Red Hat. It is a configuration editing tool. It parses configuration files in their native formats and transforms them into a tree. Configuration changes are made by manipulating this tree and saving it back into native config files.
yum install -y augeas
I show the commands, but there is not enough tutorial to explain all what it does so anyone, with a lot of coffee and very calmly, very calmly, can study quietly 😉
cat <<EOF | augtool set /files/etc/pam.d/sshd/#comment[.='pam_selinux.so close should be the first session rule'] 'pam_openshift.so close should be the first session rule' ins 01 before /files/etc/pam.d/sshd/*[argument='close'] set /files/etc/pam.d/sshd/01/type session set /files/etc/pam.d/sshd/01/control required set /files/etc/pam.d/sshd/01/module pam_openshift.so set /files/etc/pam.d/sshd/01/argument close set /files/etc/pam.d/sshd/01/#comment 'Managed by openshift_origin' set /files/etc/pam.d/sshd/#comment[.='pam_selinux.so open should only be followed by sessions to be executed in the user context'] 'pam_openshift.so open should only be followed by sessions to be executed in the user context' ins 02 before /files/etc/pam.d/sshd/*[argument='open'] set /files/etc/pam.d/sshd/02/type session set /files/etc/pam.d/sshd/02/control required set /files/etc/pam.d/sshd/02/module pam_openshift.so set /files/etc/pam.d/sshd/02/argument[1] open set /files/etc/pam.d/sshd/02/argument[2] env_params set /files/etc/pam.d/sshd/02/#comment 'Managed by openshift_origin' rm /files/etc/pam.d/sshd/*[module='pam_selinux.so'] set /files/etc/pam.d/sshd/03/type session set /files/etc/pam.d/sshd/03/control required set /files/etc/pam.d/sshd/03/module pam_namespace.so set /files/etc/pam.d/sshd/03/argument[1] no_unmount_on_close set /files/etc/pam.d/sshd/03/#comment 'Managed by openshift_origin' set /files/etc/pam.d/sshd/04/type session set /files/etc/pam.d/sshd/04/control optional set /files/etc/pam.d/sshd/04/module pam_cgroup.so set /files/etc/pam.d/sshd/04/#comment 'Managed by openshift_origin' set /files/etc/pam.d/runuser/01/type session set /files/etc/pam.d/runuser/01/control required set /files/etc/pam.d/runuser/01/module pam_namespace.so set /files/etc/pam.d/runuser/01/argument[1] no_unmount_on_close set /files/etc/pam.d/runuser/01/#comment 'Managed by openshift_origin' set /files/etc/pam.d/runuser-l/01/type session set /files/etc/pam.d/runuser-l/01/control required set /files/etc/pam.d/runuser-l/01/module pam_namespace.so set /files/etc/pam.d/runuser-l/01/argument[1] no_unmount_on_close set /files/etc/pam.d/runuser-l/01/#comment 'Managed by openshift_origin' set /files/etc/pam.d/su/01/type session set /files/etc/pam.d/su/01/control required set /files/etc/pam.d/su/01/module pam_namespace.so set /files/etc/pam.d/su/01/argument[1] no_unmount_on_close set /files/etc/pam.d/su/01/#comment 'Managed by openshift_origin' set /files/etc/pam.d/system-auth-ac/01/type session set /files/etc/pam.d/system-auth-ac/01/control required set /files/etc/pam.d/system-auth-ac/01/module pam_namespace.so set /files/etc/pam.d/system-auth-ac/01/argument[1] no_unmount_on_close set /files/etc/pam.d/system-auth-ac/01/#comment 'Managed by openshift_origin' save EOF cat <<EOF > /etc/security/namespace.d/sandbox.conf # /sandbox \$HOME/.sandbox/ user:iscript=/usr/sbin/oo-namespace-init root,adm,apache EOF cat <<EOF > /etc/security/namespace.d/tmp.conf /tmp \$HOME/.tmp/ user:iscript=/usr/sbin/oo-namespace-init root,adm,apache EOF cat <<EOF > /etc/security/namespace.d/vartmp.conf /var/tmp \$HOME/.tmp/ user:iscript=/usr/sbin/oo-namespace-init root,adm,apache EOF
CGroups
CGroups sets the limits for CPU or RAM or network or all of them combined for a user task(process). So let’s use CGroups for our node. We have installed CGroups on Fedora 19 by default, so we’ll start the needed services. The starting order is important to avoid dependencies issues.
systemctl start cgconfig.service systemctl start cgred.service systemctl enable cgconfig.service systemctl enable cgred.service
Setting limits
In the file /etc/openshift/resource_limits.conf are set the gears limits. Of all the directives I’ll focus only in two, one set the storage limit and the other set the RAM limit:
quota_blocks=1048576 # 1 block = 1024byte --> 1048576 blocks= 1GB memory_limit_in_bytes=536870912 # 512MB
We’ll work with disk quotas, so let’s enable the mountpoint option for the file system, and we’ll do it modifying the file /etc/fstab
/dev/mapper/fedora_node1-root / ext4 defaults 1 1
This is the result
/dev/mapper/fedora_node1-root / ext4 defaults,usrquota 1 1
We’ll remount the FS, and let’s update the quota information
mount -o remount / quotacheck -cumgv /
SELinux
Once more we need to enable some SELinux booleans, and restore contexts.
setsebool -P httpd_unified=on httpd_can_network_connect=on httpd_can_network_relay=on httpd_read_user_content=on httpd_enable_homedirs=on httpd_run_stickshift=on allow_polyinstantiation=on httpd_run_stickshift=on httpd_execmem=on restorecon -RFvv /var/run restorecon -RFvv /var/lib/openshift /etc/openshift/node.conf /etc/httpd/conf.d/openshiftSystem Control
As our system will manage too much http requests, should allow more httpd processes are triggered so we need to set several kernel semaphores to avoid the collapse of processes. We'll modify the kernel.sem kernel param. Also we must increase the ports in use, and the connection tracking table too, so let's define net.ipv4.ip_local_port_range and net.netfilter.nf_conntrack_max. Again we'll use augeascat <<EOF | augtool set /files/etc/sysctl.conf/kernel.sem "250 32000 32 4096" set /files/etc/sysctl.conf/net.ipv4.ip_local_port_range "15000 35530" set /files/etc/sysctl.conf/net.netfilter.nf_conntrack_max "1048576" save EOF sysctl -p /etc/sysctl.confNow we need to define the minimum UID and GID for the node users (gears). And must be equals to GEAR_MIN_UID directive in the file /etc/openshift/node.conf
cat <SSH y GIT
If we want to GIT works with SSH we must modify the /etc/ssh/sshd_config adding the acceptance of GIT_SSH variable. Also wee need to increase the maximum SSH connections allowed
cat <> /etc/ssh/sshd_config AcceptEnv GIT_SSH EOF cat < Enabling services and firewall
Let's enable the services for the network traffic control, configure the forwarding of the apps.
lokkit --port=35531-65535:tcp systemctl enable httpd.service systemctl enable openshift-port-proxy.service systemctl enable openshift-tc.service systemctl enable openshift-gears.serviceConfiguring domain for the node
This is the end of this post. We need to define the domain params of OpenShift for the node. Let's do it in /etc/openshift/node.conf file with this directives:PUBLIC_HOSTNAME="node1.dmartin.es" # The node host's public hostname PUBLIC_IP="10.10.10.101" # The node host's public IP address BROKER_HOST="broker.dmartin.es" # IP or DNS name of broker host for REST API EXTERNAL_ETH_DEV='eth0' # Update to match name of external network device CLOUD_DOMAIN="dmartin.es"Finally we must initialize facter to recolect the MCollective metadata. This works is managed by cron, but we'll ensure correct operation of facter
/etc/cron.minutely/openshift-facts systemctl rebootNow we'll restart the node and... wait for the next post where we'll create the first application.
See you soon.
One thought on “OpenShift Origin installation over Fedora 19 (Mega Tutorial) – Part 7”