TCP tuning Details

The following are important for TCP performance, and the default values of 1 are fine:

net.ipv4.tcp_window_scaling
net.ipv4.tcp_timestamps
net.ipv4.tcp_sack

Notes:

some people recommend disabling tcp_timestamps. We do not recommend this for high-speed networks. It may help for home users on slow networks, as timestamps add an additional 10 bytes to each packet. But more accurate timestamp make TCP congestion control algorithms work better, and are recommended for fast networks.
some people recommend increasing net.tcp_mem. This is not usually needed. tcp_mem values are measured in memory pages, not bytes. The size of each memory page differs depending on hardware and configuration options in the kernel, but on standard i386 computers, this is 4 kilobyte or 4096 bytes. So the defaults values are fine for most cases.
For more information on TCP variables see: http://www.frozentux.net/ipsysctl-tutorial/ipsysctl-tutorial.html#TCPVARIABLES

Starting in Linux 2.6.7 (and back-ported to 2.4.27), Linux includes alternative congestion control algorithms beside the traditional 'reno' algorithm. These are designed to recover quickly from packet loss on high-speed WANs. Starting with version 2.6.13, Linux supports plugable congestion control algorithms. The congestion control algorithm used is set using the sysctl variable net.ipv4.tcp_congestion_control, which is set to bic/cubic or reno by default, depending on which version of the 2.6 kernel you are using.

To get a list of congestion control algorithms that are available in your kernel (if you are running 2.6.20 or higher), run:

sysctl net.ipv4.tcp_available_congestion_control
The choice of congestion control options is selected when you build the kernel. The following are some of the options are available in the 2.6.23 kernel:

reno: Traditional TCP used by almost all other OSes. (default)
cubic: CUBIC-TCP
bic: BIC-TCP
htcp: Hamilton TCP
vegas: TCP Vegas
westwood: optimized for lossy networks
If cubic and/or htcp are not listed when you do 'sysctl net.ipv4.tcp_available_congestion_control', try the following, as most distributions include them as loadable kernel modules:

/sbin/modprobe tcp_htcp
/sbin/modprobe tcp_cubic

NOTE: There seems to be bugs in both bic and cubic for a number of versions of the 2.6.18 kernel used by Redhat Enterprise Linux 5.3 - 5.5 and its variants (Centos, Scientific Linux, etc.) We recommend using htcp with a 2.6.18.x kernel to be safe.

For long fast paths, we highly recommend using cubic or htcp. Cubic is the default for a number of Linux distributions, but if is not the default on your system, you can do the following:

sysctl -w net.ipv4.tcp_congestion_control=cubic
On systems supporting RPMS, You can also try using the ktune RPM, which sets many of these as well.

More information on tuning parameters and defaults for Linux 2.6 are available in the file ip-sysctl.txt, which is part of the 2.6 source distribution.

Warning on Large MTUs: If you have configured your Linux host to use 9K MTUs, but the connection is using 1500 byte packets, then you actually need 9/1.5 = 6 times more buffer space in order to fill the pipe. In fact some device drivers only allocate memory in power of two sizes, so you may even need 16/1.5 = 11 times more buffer space!

And finally a warning: for very large BDP paths where the TCP window is > 20 MB, you may hit the Linux SACK implementation problem. If Linux has too many packets in flight when it gets a SACK event, it takes too long to locate the SACKed packet, and you get a TCP timeout and CWND goes back to 1 packet. Restricting the TCP buffer size to about 12 MB seems to avoid this problem, but clearly limits your total throughput. Another solution is to disable SACK. This appears to have been fixed in version 2.6.26.

Also, I've been told that for some network paths, using the Linux 'tc' (traffic control) system to pace traffic out of the host can help improve total throughput.

$?이게 무엇인고?

Shell Programming을 위해 웹서핑을 하다보니 예전에 모르고 넘어간 $?가 나와있었다.
아~ 이거구나~~~ 후후후 기쁘다.

다음 예제에서 $?는 명령 수행 후의 결과를 돌려주는 것을 볼 수 있다. 정상적일 경우 0, 뭔가 잘못 되었을 경우는 2를 반환하는 것을 볼 수 있다. ㅋ
# ls -ld /tmp
drwxrwxrwt 5 root root 4096 Aug 19 19:45 /tmp
[root@server01 ~]# echo $?
0 // Good command return of 0.
[root@server01 ~]# ls -l /junk
ls: /junk: No such file or directory
[root@server01 ~]# echo $?
2 // Something went wrong, there was an error, so return 2.


자세한 내용은 초보 시스템 관리자를 위한 Bash 스크립트 작성 참고

근데 나도 초보인거야???? 이구~

CentOS에 Apache Tomcat 6 설치 하기

Installing Apache Tomcat 6 on RHEL or CentOS

현재 CentOS의 최신 버전인 5.6도 아직까지 Tomcat 5까지만 패키지로 제공하고 있다. 하지만 문제는 이 패키지가 Oracle에서 제공하는 JDK와는 잘 맞지 않는다는데 있다.

여기서는 JDK 6 와 Tomcat 6 버전을 CentOS 5.6기반에 설치하는 것을 기술하도록 한다.

절차

1. Oracle JDK 6.0 설치(RPM버전)
2. Apache Tomcat6 설치
3. Apache Portable Runtime(APR) 기반 Native Library 생성
4. Native Daemon을 통한 Tomcat6 서비스 등록

Oracle JDK 6.0 설치

1. Oracle JDK 6 on CentOS 참조

Apache Tomcat 6 설치

1. Apache Tomcat 6.x 다운로드 하기
2. 압축 해제 및 소유권 할당

   cd /tmp
   tar -zxvf apache-tomcat-6.0.32.tar.gz -C /usr/share/
   ln -s /usr/share/apache-tomcat-6.0.32 /usr/share/tomcat6
   useradd -u 501 -M -d /usr/share/tomcat6 tomcat
   chown -R tomcat: /usr/share/apache-tomcat-6.0.32 
   chown -h tomcat: /usr/share/tomcat6

   여기서 다운로드 받은 apache-tomcat 파일은 /tmp 디렉토리에 있으며, 버전은 6.0.32를 사용하는 것으로 하겠다.

Apache Portable Runtime(APR) 기반 Native Library 생성(Optional)

1. Native Library 빌드
   1. 압축 해제

      cd /usr/share/tomcat6/bin
      tar -zxvf tomcat-native.tar.gz
      cd tomcat-native-1.1.20-src/jni/native

   2. Dependency Package 설치

      yum install openssl-devel apr-devel -y

   3. Native Library 빌드

      ./configure --with-apr=/usr/bin --with-ssl=/usr/include/openssl
      make && make install

2. Native Library 라이브러리 시스템 등록

   echo "/usr/local/apr/lib" > /etc/ld.so.conf.d/tomcat-native.conf
   ldconfig
   ldconfig -v |grep apr
   /usr/local/apr/lib:
           libaprutil-1.so.0 -> libaprutil-1.so.0.2.7
           libgstdataprotocol-0.10.so.0 -> libgstdataprotocol-0.10.so.0.17.0
           libapr-1.so.0 -> libapr-1.so.0.2.7
           libaprutil-1.so.0 -> libaprutil-1.so.0.2.7
           libgstdataprotocol-0.10.so.0 -> libgstdataprotocol-0.10.so.0.17.0
           libapr-1.so.0 -> libapr-1.so.0.2.7

3. Native Library 라이브러리 Tomcat에게 알리기

   echo "export CATALINA_OPTS=\"\$CATALINA_OPTS -Djava.library.path=/usr/local/apr/lib\"" >> /usr/share/tomcat6/bin/setenv.sh

Tomcat 6 Service 시작 스크립트 생성

#!/bin/bash
#
# Init script file for Tomcat Server
#
# chkconfig: 2345 55 25
# description: Apache Tomcat Server
#

# Source function library.
. /etc/init.d/functions

RUN_AS_USER=tomcat
CATALINA_HOME=/usr/share/tomcat6

start() {
        echo "Starting Tomcat: "
        if [ "x$USER" != "x$RUN_AS_USER" ]; then
          su - $RUN_AS_USER -c "$CATALINA_HOME/bin/startup.sh"
        else
          $CATALINA_HOME/bin/startup.sh
        fi
        echo "done."
}
stop() {
        echo "Shutting down Tomcat: "
        if [ "x$USER" != "x$RUN_AS_USER" ]; then
          su - $RUN_AS_USER -c "$CATALINA_HOME/bin/shutdown.sh"
        else
          $CATALINA_HOME/bin/shutdown.sh
        fi
        echo "done."
}

case "$1" in
  start)
        start
        ;;
  stop)
        stop
        ;;
  restart)
        stop
        sleep 10
        start
        ;;
  *)
        echo "Usage: $0 {start|stop|restart}"
esac

exit 0

Native Daemon용 시작 스크립트 생성(Optional)

1. The Java portion of Commons Daemon(jsvc) 컴파일 (64Bit)

   cd /usr/share/tomcat6/bin
   tar -zxvf commons-daemon-native.tar.gz
   cd commons-daemon-1.0.5-native-src/unix
   export CFLAGS=-m64
   export LDFLAGS=-m64
   ./configure --with-java=/usr/java/default
   make
   cd ../.. (tomcat의 bin디렉토리)
   cp commons-daemon-1.0.5-native-src/unix/jsvc ./
   chown tomcat: /usr/share/tomcat6/bin/jsvc
   rm -rf commons-daemon-1.0.5-native-src

   requirements

   gcc
2. Tomcat 6 Service 시작 스크립트 생성

   vi /etc/init.d/tomcat6

   #!/bin/sh
   #
   # Startup script for Tomcat 6.0.13, the Apache Servlet Engine
   #
   # chkconfig: 234 20 80
   # description: Tomcat 6.0.13 is the Apache Servlet Engine
   # processname: tomcat
   # pidfile: /var/run/tomcat6.pid
   #
   ##############################################################################
   
   #
   # Licensed to the Apache Software Foundation (ASF) under one or more
   # contributor license agreements.  See the NOTICE file distributed with
   # this work for additional information regarding copyright ownership.
   # The ASF licenses this file to You under the Apache License, Version 2.0
   # (the "License"); you may not use this file except in compliance with
   # the License.  You may obtain a copy of the License at
   #
   #     http://www.apache.org/licenses/LICENSE-2.0
   #
   # Unless required by applicable law or agreed to in writing, software
   # distributed under the License is distributed on an "AS IS" BASIS,
   # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
   # See the License for the specific language governing permissions and
   # limitations under the License.
   ##############################################################################
   #
   # Small shell script to show how to start/stop Tomcat using jsvc
   # If you want to have Tomcat running on port 80 please modify the server.xml
   # file:
   #
   #    <!-- Define a non-SSL HTTP/1.1 Connector on port 80 -->
   #    <Connector className="org.apache.catalina.connector.http.HttpConnector"
   #               port="80" minProcessors="5" maxProcessors="75"
   #               enableLookups="true" redirectPort="8443"
   #               acceptCount="10" debug="0" connectionTimeout="60000"/>
   #
   # Adapt the following lines to your configuration
   JAVA_HOME=/usr/java/default
   CATALINA_HOME=/usr/share/tomcat6
   DAEMON_HOME=/usr/share/tomcat6/bin
   TOMCAT_USER=tomcat
   
   # for multi instances adapt those lines.
   TMP_DIR=/var/tmp
   PID_FILE=/var/run/jsvc.pid
   CATALINA_BASE=/usr/share/tomcat6
   
   #CATALINA_OPTS="-Djava.library.path=/usr/share/tomcat6/bin:"
   if [ -r "$CATALINA_BASE"/bin/setenv.sh ]; then
     . "$CATALINA_BASE"/bin/setenv.sh
   elif [ -r "$CATALINA_HOME"/bin/setenv.sh ]; then
     . "$CATALINA_HOME"/bin/setenv.sh
   fi
   
   CLASSPATH=\
   $JAVA_HOME/lib/tools.jar:\
   $CATALINA_HOME/bin/commons-daemon.jar:\
   $CATALINA_HOME/bin/bootstrap.jar
   
   case "$1" in
     start)
       #
       # Start Tomcat
       #
       $DAEMON_HOME/jsvc \
       -user $TOMCAT_USER \
       -home $JAVA_HOME \
       -Dcatalina.home=$CATALINA_HOME \
       -Dcatalina.base=$CATALINA_BASE \
       -Djava.io.tmpdir=$TMP_DIR \
       -jvm server \
       -procname Tomcat6 \
       -wait 10 \
       -pidfile $PID_FILE \
       -outfile $CATALINA_HOME/logs/catalina.out \
       -errfile '&1' \
       $CATALINA_OPTS \
       -cp $CLASSPATH \
       org.apache.catalina.startup.Bootstrap
       #
       # To get a verbose JVM
       #-verbose \
       # To get a debug of jsvc.
       #-debug \
       exit $?
       ;;
   
     stop)
       #
       # Stop Tomcat
       #
       $DAEMON_HOME/jsvc \
       -stop \
       -pidfile $PID_FILE \
       org.apache.catalina.startup.Bootstrap
       exit $?
       ;;
   
     restart)
       #
       # Restart Tomcat
       #
       $0 stop
       sleep 5
       $0 start
       ;;
   
     *)
       echo "Usage tomcat.sh start/stop"
       exit 1;;
   esac

   과부하를 발생시킬수 있는 웹 컨텍스를 실행하고자 하는 경우 이 방법을 가급적 사용하지 말기 바람

서비스 등록 및 시작

• 서비스 등록

  chmod u+x /etc/init.d/tomcat6
  chkconfig --add tomcat6
  service tomcat6 start

• 서비스 구동 확인

  pgrep -u tomcat -l
  11900 jsvc

xinetd를 통하여 80포트를 8080 포트로 리다이렉트하기

일반적인 Unix계열의 시스템에서는 1024포트 이하의 포트를 사용하기 위해서는 반드시 root privileges를 가지고 있어야 한다.

문제는 tomcat과 같은 서비스는 디폴트 포트가 8080를 사용하는데, 이를 80포트로 사용하기위해서는 반드시 서비스를 root 계정으로 실행 시켜야 하는 보안 문제가 발생할 수 있다. 또한 80포트로 변경하려면 server.xml을 수정해야만 한다. 이를 수행하지 않고 수퍼 데몬인 xinetd을 통하여 port redirecting을 하도록 하겠다.

xinetd 설정 파일 생성

먼저 http와 https 포트를 tomcat에 연결하기 위해 각각의 서비스 설정 파일을 만들어야한다.
* http 설정 파일 생성

# vi /etc/xinetd.d/http
service http
{
    disable = no
    flags = REUSE
    socket_type = stream
    wait = no
    user = root
    redirect = 127.0.0.1 8080
    log_on_failure += USERID
}

* http 설정 파일 생성

# vi /etc/xinetd.d/https
service https
{
    disable = no
    flags = REUSE
    socket_type = stream
    wait = no
    user = root
    redirect = 127.0.0.1 8443
    log_on_failure += USERID
}

xinetd 재시작

# service xinetd restart
웹브라우저에서 80포트를 통해 8080로 리다이렉트 됨을 확인한다.

리눅스에서 Memory 정리하기

많은 작업을 수행하다보면 메모리가 회수(?)되지 않는 상황이 생긴다.
이럴 때 다음 명령을 수행하여 회수를 할 수 있다.

[이 전]
[root@jpnips tmp]$ free
total used free shared buffers cached
Mem: 32932100 25916432 7015668 0 1535684 22394696
-/+ buffers/cache: 1986052 30946048
Swap: 65537156 532 65536624

[root@jpnips ~]# sync;echo 3 > /proc/sys/vm/drop_caches

[이 후]
[root@jpnips ~]# free
total used free shared buffers cached
Mem: 32932100 4894320 28037780 0 1392 4331752
-/+ buffers/cache: 561176 32370924
Swap: 65537156 532 65536624

rm 명령 수행시 Argument list too long 에러 해결법

모든 tmp_로 시작하는 파일을 삭제하라는 명령을 일반적으로 다음과 같이 수행한다.
rm tmp_*
하지만 문제는 파일이 너무 많게 될 경우 실제로 rm tmp_첫번째 tmp_두번째 tmp_세번째 .... 이런식으로 수행이 되게되는데 여기서 가장 큰 문제는 파일이 너무 많을 경우 rm이 받을 수 있는 인자 개수가 제한이 되어있는데 무수하게 지울 파일 이름을 넣는데 있다.

이럴 경우는 다음과 같은 오류 메시지를 출력하며 rm 명령을 수행하지 못하게 된다.
-bash: /bin/rm: Argument list too long

어떻게 해야할까? 해결법은 하나씩 찾아서 rm에 인자로 넣어주는 것이다.
find . -name "tmp_*" | xargs rm

간단하지 않은가? 아주 단순한 진리가 들어있는 로직이다. ㅋ

Howto Create Software RAID 5 with Grow option in Linux

원문: http://www.electromech.info/software-raid-rhel-5-linux.html

Create RAID 5 with GROW Option

Description :- We have created this exercise @Electromech Corporation to Create Software RAID with GROW option on RHEL 5. Perform below steps for Creating RAID 5 with Grow Option. We have performed this Exercise on RHEL 5 Linux, but You can perform this exercise on any linux distribution like Fedora Ubuntu OpenSUSE Ubuntu Desktop Edition Ubuntu Server Edition.
1. Let's create 5 partition with size of 500MB each.

[root@electromech-corporation ~]# fdisk /dev/sda

The number of cylinders for this disk is set to 19457. There is nothing wrong with that, but this is larger than 1024, and could in certain setups cause problems with: 1) software that runs at boot time (e.g., old versions of LILO) 2) booting and partitioning software from other OSs (e.g., DOS FDISK, OS/2 FDISK)


Command (m for help): n
First cylinder (1544-19457, default 1544):
Using default value 1544
Last cylinder or +size or +sizeM or +sizeK (1544-19457, default 19457): +500M

Command (m for help): n
First cylinder (1557-19457, default 1557):
Using default value 1557
Last cylinder or +size or +sizeM or +sizeK (1557-19457, default 19457): +500M

Command (m for help): n
First cylinder (1570-19457, default 1570):
Using default value 1570
Last cylinder or +size or +sizeM or +sizeK (1570-19457, default 19457): +500M

Command (m for help): n
First cylinder (1583-19457, default 1583):
Using default value 1583
Last cylinder or +size or +sizeM or +sizeK (1583-19457, default 19457): +500M

Command (m for help): n
First cylinder (1596-19457, default 1596):
Using default value 1596
Last cylinder or +size or +sizeM or +sizeK (1596-19457, default 19457): +500M

====> Let's confirm partition which we have created recently.

Command (m for help): p


Disk /dev/sda: 160.0 GB, 160041885696 bytes
255 heads, 63 sectors/track, 19457 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sda1 * 1 13 104391 83 Linux
/dev/sda2 14 1288 10241437+ 83 Linux
/dev/sda3 1289 1543 2048287+ 82 Linux swap / Solaris
/dev/sda4 1544 19457 143894205 5 Extended
/dev/sda5 1544 1556 104391 83 Linux
/dev/sda6 1557 1569 104391 83 Linux
/dev/sda7 1570 1582 104391 83 Linux
/dev/sda8 1583 1595 104391 83 Linux
/dev/sda9 1596 1608 104391 83 Linux

Now Let's convert Normal Partition code to RAID Partition.

Command (m for help): t
Partition number (1-9): 5
Hex code (type L to list codes): fd
Changed system type of partition 5 to fd (Linux raid autodetect)

Command (m for help): t
Partition number (1-9): 6
Hex code (type L to list codes): fd
Changed system type of partition 6 to fd (Linux raid autodetect)

Command (m for help): t
Partition number (1-9): 7
Hex code (type L to list codes): fd
Changed system type of partition 7 to fd (Linux raid autodetect)

Command (m for help): t
Partition number (1-9): 8
Hex code (type L to list codes): fd
Changed system type of partition 8 to fd (Linux raid autodetect)

Command (m for help): t
Partition number (1-9): 9
Hex code (type L to list codes): fd
Changed system type of partition 9 to fd (Linux raid autodetect)

Let's Save Partition Scheme.

Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table.
The new table will be used at the next reboot.
Syncing disks.
2. Let's Update Partition scheme using Partprobe.


[root@electromech-corporation ~]# partprobe
3. Creating RAID 5

=> Adding Created Partition to RAID.

[root@electromech-corporation ~]# mdadm --create -a yes /dev/md5 --level=5 --raid-devices=5 /dev/sda{5,6,7,8,9}
mdadm: /dev/sda5 appears to contain an ext2fs file system
size=1020096K mtime=Wed Dec 30 10:34:18 2009
Continue creating array? y
mdadm: array /dev/md5 started.
=> Formating Software RAID Device

[root@electromech-corporation ~]# mkfs.ext3 /dev/md5
mke2fs 1.39 (29-May-2006)
Filesystem label=
OS type: Linux
Block size=1024 (log=0)
Fragment size=1024 (log=0)
104448 inodes, 417280 blocks
20864 blocks (5.00%) reserved for the super user
First data block=1
Maximum filesystem blocks=67633152
51 block groups
8192 blocks per group, 8192 fragments per group
2048 inodes per group
Superblock backups stored on blocks:
8193, 24577, 40961, 57345, 73729, 204801, 221185, 401409

Writing inode tables: done
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information:

This filesystem will be automatically checked every 27 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
=> Mounting Sofware RAID Device and See details of Software RAID Device.

[root@electromech-corporation ~]# mkdir /raid5
[root@electromech-corporation ~]# mount /dev/md5 /raid5/
[root@electromech-corporation ~]# mdadm --detail /dev/md5
/dev/md5:
Version : 00.90.03
Creation Time : Wed Dec 30 18:59:49 2009
Raid Level : raid5
Array Size : 417280 (407.57 MiB 427.29 MB)
Used Dev Size : 104320 (101.89 MiB 106.82 MB)
Raid Devices : 5
Total Devices : 5
Preferred Minor : 5
Persistence : Superblock is persistent

Update Time : Wed Dec 30 19:00:14 2009
State : clean, degraded, recovering
Active Devices : 4
Working Devices : 5
Failed Devices : 0
Spare Devices : 1

Layout : left-symmetric
Chunk Size : 64K

Rebuild Status : 37% complete

UUID : 523e09aa:9fc68431:ac2e0711:7dc7fdbc
Events : 0.6

Number Major Minor RaidDevice State
0 8 5 0 active sync /dev/sda5
1 8 6 1 active sync /dev/sda6
2 8 7 2 active sync /dev/sda7
3 8 8 3 active sync /dev/sda8
5 8 9 4 spare rebuilding /dev/sda9

[root@electromech-corporation ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda2 9.5G 2.1G 7.0G 23% /
/dev/sda1 99M 19M 75M 21% /boot
tmpfs 982M 0 982M 0% /dev/shm
/dev/md5 1.5G 35M 1.3G 3% /raid5
=> Adding New Spares to Software RAID.


[root@electromech-corporation ~]# mdadm --manage /dev/md5 -a /dev/sda10
mdadm: added /dev/sda10
[root@electromech-corporation ~]# mdadm --manage /dev/md5 -a /dev/sda11
mdadm: added /dev/sda11
[root@electromech-corporation ~]# mdadm --detail /dev/md5
/dev/md5:
Version : 0.90
Creation Time : Sat Jun 12 11:07:00 2010
Raid Level : raid5
Array Size : 1493568 (1458.81 MiB 1529.41 MB)
Used Dev Size : 497856 (486.27 MiB 509.80 MB)
Raid Devices : 4
Total Devices : 6
Preferred Minor : 5
Persistence : Superblock is persistent

Update Time : Sat Jun 12 11:08:27 2010
State : clea n
Active Devices : 4
Working Devices : 6
Failed Devices : 0
Spare Devices : 2

Layout : left-symmetric
Chunk Size : 64K

UUID : e97460c5:76313186:a478a71b:2c154df0
Events : 0.6

Number Major Minor RaidDevice State
0 8 5 0 active sync /dev/sda5
1 8 6 1 active sync /dev/sda6
2 8 7 2 active sync /dev/sda7
3 8 8 3 active sync /dev/sda8
4 8 9 4 active sync /dev/sda9

5 8 10 - spare /dev/sda10
6 8 11 - spare /dev/sda11
=> Adding Spare devices to Software RAID.

[root@electromech-corporation ~]# mdadm --grow /dev/md5 --raid-devices=6
mdadm: Need to backup 768K of critical section..
mdadm: ... critical section passed.
[root@station17 ~]# mdadm --detail /dev/md5
/dev/md5:
Version : 0.91
Creation Time : Sat Jun 12 11:07:00 2010
Raid Level : raid5
Array Size : 1493568 (1458.81 MiB 1529.41 MB)
Used Dev Size : 497856 (486.27 MiB 509.80 MB)
Raid Devices : 5
Total Devices : 6
Preferred Minor : 5
Persistence : Superblock is persistent
Update Time : Sat Jun 12 11:09:30 2010
State : clean, recovering
Active Devices : 5
Working Devices : 6
Failed Devices : 0
Spare Devices : 1

Layout : left-symmetric
Chunk Size : 64K

Rebuild Status : 1% complete

UUID : e97460c5:76313186:a478a71b:2c154df0
Events : 0.6
Number Major Minor RaidDevice State
0 8 5 0 active sync /dev/sda5
1 8 6 1 active sync /dev/sda6
2 8 7 2 active sync /dev/sda7
3 8 8 3 active sync /dev/sda8
4 8 9 4 active sync /dev/sda9
4 8 10 4 active sync /dev/sda10

5 8 11 - spare /dev/sda11

[root@electromech-corporation ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda2 9.5G 2.1G 7.0G 23% /
/dev/sda1 99M 19M 75M 21% /boot
tmpfs 982M 0 982M 0% /dev/shm
/dev/md5 1.5G 35M 1.3G 3% /raid5

[root@electromech-corporation ~]# resize2fs /dev/md5
resize2fs 1.39 (29-May-2006)
Filesystem at /dev/md5 is mounted on /raid5; on-line resizing required
Performing an on-line resize of /dev/md5 to 497856 (4k) blocks.
The filesystem on /dev/md5 is now 497856 blocks long.

[root@electromech-corporation ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda2 9.5G 2.1G 7.0G 23% /
/dev/sda1 99M 19M 75M 21% /boot
tmpfs 982M 0 982M 0% /dev/shm
/dev/md5 1.9G 35M 1.8G 2% /raid5


[root@electromech-corporation ~]# watch cat /proc/mdstat


Personalities : [raid6] [raid5] [raid4]
md5 : active raid5 sda5[4] sda6[5] sda7[3] sda8[2] sda9[1] sda10[0]
1991424 blocks super 0.91 level 5, 64k chunk, algorithm 2 [6/6] [UUUUUU]
[===========>.........] reshape = 59.1% (295296/497856) finish=2.9min spe
ed=1139K/sec

unused devices:

That's It.
Thanks to Tejal Mam for creating such a Wonderful Exercise.

Bash Cheat sheets

!!	Last command
!foo	Run most recent command starting with 'foo...' (ex. !ps, !mysqladmin)
!foo:p	Print command that !foo would run, and add it as the latest to command history
!$	Last 'word' of last command ('/path/to/file' in the command 'ls -lAFh /path/to/file', '-uroot' in 'mysql -uroot')
!$:p	Print word that !$ would substitute
!*o	All but first word of last command ('-lAFh /path/to/file' in the command 'ls -lAFh /path/to/file', '-uroot' in 'mysql -uroot')
!*:p	Print words that !* would substitute
^foo^bar	Replace 'foo' in last command with 'bar', print the result, then run. ('mysqladmni -uroot', run '^ni^in', results in 'mysqladmin -uroot')
{a,b,c}	passes words to the command, substituting a, b, and c sequentially (`cp file{,.bk}` runs `cp file file.bk`)
Ctrl + a	Jump to the start of the line
Ctrl + b	Move back a char
Ctrl + c	Terminate the command
Ctrl + d	Delete from under the cursor
Ctrl + e	Jump to the end of the line
Ctrl + f	Move forward a char
Ctrl + k	Delete to EOL
Ctrl + l	Clear the screen
Ctrl + r	Search the history backwards
Ctrl + R	Search the history backwards with multi occurrence
Ctrl + t	Transpose the current char with the previous
Ctrl + u	Delete backward from cursor
Ctrl + w	Delete backward a word
Ctrl + xx	Move between EOL and current cursor position
Ctrl + x @	Show possible hostname completions
Ctrl + z	Suspend/ Stop the command
Ctrl + x; Ctrl + e	Edit line into your favorite editor
Alt + >	Move to the first line in the history
Alt + *lt;	Move to the last line in the history
Alt + ?	Show current completion list
Alt + *	Insert all possible completions
Alt + /	Attempt to complete filename
Alt + .	Yank last argument to previous command
Alt + b	Move backward
Alt + c	Capitalize the word
Alt + d	Delete word
Alt + f	Move forward
Alt + l	Make word lowercase
Alt + n	Search the history forwards non-incremental
Alt + p	Search the history backwards non-incremental
Alt + r	Recall command
Alt + t	Transpose the current word with the previous
Alt + u	Make word uppercase
Alt + back-space	Delete backward from cursor
(Here "2T" means Press TAB twice) $ 2T - All available commands(common) $ (string)2T - All available commands starting with (string) $ /2T - Entire directory structure including Hidden one $ (dir)2T - Only Sub Dirs inside (dir) including Hidden one $ *2T - Only Sub Dirs inside without Hidden one $ ~2T - All Present Users on system from "/etc/passwd" $ $2T - All Sys variables $ @2T - Entries from "/etc/hosts" $ =2T - Output like ls or dir .bash_profile = sourced by login shell, .bashrc = sourced by all shells, .bash_aliases = should be sourced by .bashrc
Run something: for i in a b c; do $i 'hello'; done Do something on a bunch of files: for i in *.rb; do echo $i; done If syntax: if [ -e .ssh ]; then echo "hi"; fi file check flags: -e: file exists -f: regular file (non directory) -d: directory -s: non-zero file -x: execute permission Avoid duplicates in your history: export HISTIGNORE="&:ls:ls *:[bf]g:exit"

리눅스 상에서 큰 텍스트 파일을 특정 라인 수 만큼 분할하기

split --lines [분할할 라인수] [분할할 파일] [분할된 파일의 Prefix]

이렇게 하면 특정 라인 수 만큼의 여러파일로 분할되어 나온다.
간단하지만 강력한 기능의 split ^^

[Apache+SVN] SVNParentPath 사용 시 forbidden 문제 해결

Web(HTTP)를 통하여 SVN을 접속시키려 한다면 DAV_SVN 모듈을 사용하여 연동을 시키게 되는데 이때 CentOS나 Redhat Enterprise Linux 경우 /etc/httpd/conf.d/subversion.conf 에 설정을 하게 된다.

문제는 기본 설정으로 대부분  SVNParentPath <디렉토리> 와 같은 형식의 문장이 들어가 있게 되는데 이대로 사용한다면 절대로 여러 프로젝트를 웹페이지 상에서 볼 수 없을 것이다.
 이때는  SVNListParentPath on 라는 라인을 추가하고 HTTPD를 다시 시작해 본다.

그럼 짜잔하고 나타나는 경우가 있을 것이다.

How to patch and rebuild an RPM package

The following describes how to derive a patch for a modification you need to make to the source code of a program, and apply that patch during the RPM building process.

Setup

By default, at least on a Red Hat box, rpm uses /usr/src/redhat as the location of the %_topdir macro, which specifies where most of the work involved in building an RPM takes place. You can and should change this; it is a good idea to make this a directory that you can write to with a non-privileged account, to avoid compiling and building packages as root. Why? A lot of commands get executed when building a package. Sometimes things go wrong. If you're root, important things may be damaged. A big mess may be made. I once (foolishly) rebuilt a proftpd package as root, and the "make install" stage blew up and left newly compiled files all over the place, whereas if I'd been a regular user, I'd have simply gotten a bunch of "permission denied" messages. Anyway, the macro is easily changed by adding something like the following to ~/.rpmmacros:

# Path to top of build area
%_topdir    /home/you/src/rpm

If you have never worked on RPMs in this manner, you will need to create a few directories in which to work. I use a subdirectory in my homedir:

linmiri% mkdir -p ~/src/rpm
linmiri% cd ~/src/rpm
linmiri% mkdir BUILD RPMS SOURCES SPECS SRPMS
linmiri% mkdir RPMS/{i386,i486,i586,i686,noarch,athlon}

The BUILD directory is where sources will be extracted and compiled, and as such is more or less a temporary working directory. SOURCES is for source tarballs, patches, and additional files, which are extracted from SRPM packages you install, and used and included in packages you build. The RPMS and SRPMS directories are where packages you build will end up, so don't use them as a storage area, as you may end up overwriting something unintentionally. The SPECS directory is where the specfile included with an SRPM will be placed. Again, be careful leaving work in this directory; if you later install another SRPM of the same package, it will overwrite your modified specfile!

Rebuilding an RPM from SRPM

If you just want to recompile, without making any changes to the source, all you have to do is run:

linmiri% rpmbuild --rebuild package-1.0.src.rpm

You'll still need the writeable %_topdir as described above, but otherwise, that's it. Your new package will be in %_topdir/RPMS/.

Installing the SRPM

The first thing you will need to do is install the source RPM (SRPM). If you have set your %_topdir as described, you will be able to install the SRPM as yourself:

linmiri% rpm -ivh package-1.0.src.rpm

That will put a specfile in %_topdir/SPECS, and a source tarball (plus any other included patches or additional files) in %_topdir/SOURCES.

Changing `configure` flags

If all you need to do is change the configure flags, just edit the specfile. You don't need to touch the source at all. There's usually a %configure section, where it should be fairly self-evident what to do. Then skip to "Rebuilding the package."

Patching the source

Since the rpmbuild process will delete the original extracted source and start from scratch when building packages, you'll want to do your work on a copy of the source with a different name. So if extracting package-1.0.tar.gz creates a directory named package-1.0/, make a copy named something like package-1.0p/ and make your changes there.

To create a patch containing your changes, cd to %_topdir/BUILD -- one directory above the source tree -- and use diff to create a patchfile in the SOURCES directory. There may be other patches to apply to this source, so give your patchfile a unique name to quickly describe the purpose of the patch. Run diff with "-u" to get the unified diff format that is standard for patches, "-N" to include any new files from your modified source, and "-r" to operate recursively, listing the original first, then your version:

linmiri% diff -uNr package-1.0/ package-1.0p/ > ../SOURCES/package-1.0-my.patch

The top of the patchfile will look something like this:

--- package-1.0/file Thu Aug 14 16:24:49 2003
+++ package-1.0p/file Mon Aug 18 01:24:17 2003

For another approach at making diffs, `man gendiff`. That tool lets you change individual files, leaving the originals with a unique filename suffix, and generates a diff based on those, all from a single source directory. I usually prefer keeping my changes in a separate directory, to prevent accidental deletion of my work by rpmbuild.

Adding the patch to the package

Next, add the patch to the specfile, so it will be applied when building the package. There may be other patches already, and they are applied in order of their number in the specfile, so number yours appropriately. Add to %_topdir/SPECS/package.spec, in the top section where the name, version, and source lines are:

Patch0: package-1.0-my.patch

Farther down, there will be a section that deals with preparing for the build. Add a patch command that corresponds with the patch line above, typically right after %setup:

%prep
%setup ...
%patch0 -p1

The "-p1" in the patch command above is how many directories to strip off the beginning of the filenames listed in the patch. Since, at this stage of building an RPM, the current directory is the top level of the source tree -- one level lower than where the patch was made -- we need to strip off one directory level.

Again, there may be other patches already being applied to this package, so give yours a unique number in the specfile, and be careful that your patch doesn't break others, and that others don't break yours. Usually the next available number will work fine, and your patch will be applied after all others. If the package is already installed, make the version number of your package higher so it will be considered newer when you go to upgrade it, and update the changelog with a quick note on what you did.

Rebuilding the package

You should be able to rebuild tha package with your changes now:

linmiri% rpmbuild -ba SPECS/package.spec

That will patch, configure, and compile the source, build a binary RPM package in %_topdir/RPMS//, and an SRPM package in %_topdir/SRPMS containing the original source (checksums for which can still be used), plus your additional patch and new specfile, so that the package can be rebuilt again on top of the work you've already done. Use "-bb" to make just a binary RPM, or "-bs" to make only an SRPM.

The easy way

"Jesus tapdancing christ, Brad," you might be saying, "that's a lot of work." Yes, it is. If you were hoping you could just go into the source after doing a "make" and fiddle with a couple of things before rolling the RPM, sorry, you're shit outta luck. The rpmbuild command won't let you do it. You can "short-circuit" to skip various parts of the overall process, but not all the way to the "create an RPM out of this" stage.

Part of the philosophy behind RPM is that the build process should be repeatable, as explained here. It may be small consolation when you just want make a simple change on your own system, where building once is good enough, but that's the way it is. Hopefully you now know enough to get by.

Origin : http://bradthemad.org/tech/notes/patching_rpms.php 

CentOS 5에서 NTFS 파티션 마운트하기

일반적인 리눅스 배포본이라면 ntfs-3g라는 패키지를 대부분 설치하여 사용 가능하다.

하지만 Redhat Enterprise Linux(RHEL) 의 Free(?) 판인 CentOS는 기본 설정 만으로는 ntfs-3g를 설치할 수 없다. 이 RHEL과 CentOS는 서버에 특화된 리눅스라고 할 수 있다. 사실 본인도 리눅스 서버에서 NTFS를 특별히 지원할 필요는 없다고 생각된다.

이 논쟁이 될 수 있는 소지는 뒤로하고 NTFS를 마운트 할 수 있는 환경을 만들어보자.

1. Repository에 RPMForge 등록하기 
   
   RPMForge Repository는 일반적인 /etc/yum.repos.d/ 에 추가하는 방식이 아닌 RPM을 통하여 등록하도록 하고 있다.
   
   
   다운로드 URL은 다음과 같다.
   http://wiki.centos.org/AdditionalResources/Repositories/RPMForge
   
   
   • i386 http://packages.sw.be/rpmforge-release/rpmforge-release-0.5.2-2.el5.rf.i386.rpm
   • x86_64 http://packages.sw.be/rpmforge-release/rpmforge-release-0.5.2-2.el5.rf.x86_64.rpm
   자신의 architecture에 해당하는 파일을 다운로드를 받은 후 rpm 명령을 통하여 설치한다. 다운로드 파일 검증이라는 과정이 있지만 이 부분은 생략한다. 자세한 사항은 위에서 언급한 URL에서 확인한다.
   
   
   
   
2. ntfs-3g 패키지 설치하기이제 yum을 통하여 ntfs-3g 패키지를 설치할 수 있을 것이다.
   
   # yum install fuse fuse-ntfs-3g
   
3. NTFS 마운트 하기
   마운트는 별반 다른게 없으며 단지 fstype을 ntfs-3g로 하면 된다. 

SSH Brute forcing 방지 방화벽 정책 2

1. Here I will show you how to add the first rule in iptables...
   • The 1st rule we add is where we tell iptables to create a list called ssh_attempt and store the source ip of every recent ssh attempt on port 22 using tcp on interface eth0.
   • "iptables -A INPUT -i eth0 -p tcp -m tcp --dport 22 -m state --state NEW -m recent --set --name ssh_attempt --rsource"
     
   • step by step explanation..
   • iptables is the command that you use to enter the firewall rules in.
   • "-A INPUT" means APPEND to the INPUT chain
   • "-i eth0" means this rule will use the interface eth0
   • "-p tcp" means we are using the TCP protocol
   • "-m tcp" means we are matching the TCP protocol
   • "--dport 22" means we are matching based on the destination port 22
   • "-m state --state NEW" This rule will only apply to NEW incoming ssh connections not ESTABLISHED or RELATED.
   • "-m recent --set --name ssh_attempt --rsource" allows us to match packets based on recent events that we have previously matched and sets the name of the list , while saving the source ip address and port.
In this step we will show you how to deny those bots..
• "iptables -A INPUT -i eth0 -p tcp -m tcp --dport 22 -m state --state NEW -m recent --update --seconds 10 --hitcount 2 --name ssh_attempt --rsource -j DROP"
• The only difference in this rule is these options.
• "--update --seconds 10 --hitcount 1 --name ssh_attempt" This will match true if the source is available in the specified list and it also updates the last-seen time in the list. The "--seconds 10" match is used to specify how long since the "last seen". The "--hitcount 2" will limit the match to only include packets that have seen at least the hitcount amount of packets.

출처 : SSH Blocking How To using Iptables

Screen HowTo Part 2

Linux HowTo's - Advance Linux HowTo's

Written by Allen Sanabria

Saturday, 17 May 2008 10:39

In the the first part of this HowTo,  I gave you the bare minimum to get Screen working and how to use it on a day to day basis.... This time around I will show you some of the advanced features of screen that may make your life easier in the terminal.. Before continuing this HowTo please read Part 1 if you do not have previous experience with screen. So lets start this HowTo with some more Screen Commands...   1- To start a split screen session in you current screen session...     ((ctrl+a) S)  2- To switch between split screens...    ((ctrl+a) )... 3- To copy output from one split screen to the next...     ((ctrl+a) [) now using the up and down arrow find the output you want to copy and from where you want to start copying hit the and keep using the arrows to where you want to stop copying and hit the  again. 4- To paste the output from the previous command...     ((ctrl+a) ]) Now if you want to paste it in the split terminal you would have to do step 2 then step 4. I use the above commands especially to copy output from one terminal to next without a mouse. 5- Now lets say you want to lock our screen session from private eyes...    ((ctrl+a) x)    To unlock your session just use your password for you account. The following below will make your screen session show all of your sessions in a nice bar below your terminal, show you the current date and time, as well as tell you the host you are in. Check the screenshot above to see what it would look like. Add the following below to .screenrc in your home directory..... Example /home/dynasty/.screenrc hardstatus alwayslastline hardstatus string '%{b}[ %{B}%H %{g}][%= %{wk}%?%-Lw%?%{=b kR} (%{W}%n*%f %t%?(%u)%?%{=b kR})%{= kw}%?%+Lw%?%?%= %{g}][%{Y}%l%{g}]%{=b C}[ %m/%d %c ]%{W}' startup_message off defscrollback 200000 shelltitle dynasty caption always

Screen HowTo Part 1

Linux HowTo's - Advance Linux HowTo's

Written by Allen Sanabria

Sunday, 11 May 2008 17:46

I've been using screen for the past 3 years and for the first time I've decided to seek out more functionality out of it.  So last week I spent about 2 hours looking up some of the different functionality that I can use in the day to day. Let me tell you I found a bunch of goodies that I use now religiously (For a week now ;) ). So for those of you who do not know what a screen session is... (STRAIGHT FROM THE MAN PAGES) Screen is a full-screen window manager that multiplexes a physical terminal between several processes (typically interactive shells).  Essentially you can have 30 terminal sessions inside one terminal and much more.....  To run Screen all you need to do is type "screen" in your terminal. You more then likely do not have it installed by default. So for those of you who have the distributions below...  Fedora/CentOS/RHE/ or any RedHat based system, you will need to run "yum install screen" Ubuntu/Debian based systems, you will need to run "apt-get install screen" Gentoo based systems, you will need to run "emerge screen" To start a new screen session all you have to do is type "screen" Now that you are in a screen session, the fun begins now... 1- To start a new sub session (Virtual Session) type    ((ctrl+a) c)   Please ignore the plus (+)symbol, this essentially means "ctrl a then c" 2- To switch between sessions...    ((ctrl+a) ) 3- To switch between your previous session and the current one..    ((ctrl+a) a) 4- To kill a frozen session...    ((ctrl+a) k)  5- To see a list of available commands...    ((ctrl+a) ?) 6- To see a list of virtual terminals you have open..    ((ctrl+a) ") 7- To Detach from a screen session and not lose any of your virtual terminals...    ((ctrl+a) d)( This is one of the best features of screen!! ) 8- To view  current screen sessions once you are logged out of screen..    "screen -list" 9- To reattach to an Attached session (Session that was not detached properly)    "screen -D -r <11546.pts-5.hostname>" (session name from the output of the screen -list command) 10- To reattach to a detached session..    "screen -r <11546.pts-5.hostname>" (session name from the output of the screen -list command) 10- To end that session you can either type   "exit" or a (ctrl+d)