Friday 22 April 2011

리눅스에서 Memory 정리하기

많은 작업을 수행하다보면 메모리가 회수(?)되지 않는 상황이 생긴다.
이럴 때 다음 명령을 수행하여 회수를 할 수 있다.

[이 전]
[root@jpnips tmp]$ free
total used free shared buffers cached
Mem: 32932100 25916432 7015668 0 1535684 22394696
-/+ buffers/cache: 1986052 30946048
Swap: 65537156 532 65536624

[root@jpnips ~]# sync;echo 3 > /proc/sys/vm/drop_caches

[이 후]
[root@jpnips ~]# free
total used free shared buffers cached
Mem: 32932100 4894320 28037780 0 1392 4331752
-/+ buffers/cache: 561176 32370924
Swap: 65537156 532 65536624

rm 명령 수행시 Argument list too long 에러 해결법

모든 tmp_로 시작하는 파일을 삭제하라는 명령을 일반적으로 다음과 같이 수행한다.
rm tmp_*
하지만 문제는 파일이 너무 많게 될 경우 실제로 rm tmp_첫번째 tmp_두번째 tmp_세번째 .... 이런식으로 수행이 되게되는데 여기서 가장 큰 문제는 파일이 너무 많을 경우 rm이 받을 수 있는 인자 개수가 제한이 되어있는데 무수하게 지울 파일 이름을 넣는데 있다.

이럴 경우는 다음과 같은 오류 메시지를 출력하며 rm 명령을 수행하지 못하게 된다.
-bash: /bin/rm: Argument list too long

어떻게 해야할까? 해결법은 하나씩 찾아서 rm에 인자로 넣어주는 것이다.
find . -name "tmp_*" | xargs rm

간단하지 않은가? 아주 단순한 진리가 들어있는 로직이다. ㅋ

Wednesday 6 April 2011

Howto Create Software RAID 5 with Grow option in Linux

원문: http://www.electromech.info/software-raid-rhel-5-linux.html

Create RAID 5 with GROW Option

Description :- We have created this exercise @Electromech Corporation to Create Software RAID with GROW option on RHEL 5. Perform below steps for Creating RAID 5 with Grow Option. We have performed this Exercise on RHEL 5 Linux, but You can perform this exercise on any linux distribution like Fedora Ubuntu OpenSUSE Ubuntu Desktop Edition Ubuntu Server Edition.
1. Let's create 5 partition with size of 500MB each.

[root@electromech-corporation ~]# fdisk /dev/sda

The number of cylinders for this disk is set to 19457. There is nothing wrong with that, but this is larger than 1024, and could in certain setups cause problems with: 1) software that runs at boot time (e.g., old versions of LILO) 2) booting and partitioning software from other OSs (e.g., DOS FDISK, OS/2 FDISK)


Command (m for help): n
First cylinder (1544-19457, default 1544):
Using default value 1544
Last cylinder or +size or +sizeM or +sizeK (1544-19457, default 19457): +500M

Command (m for help): n
First cylinder (1557-19457, default 1557):
Using default value 1557
Last cylinder or +size or +sizeM or +sizeK (1557-19457, default 19457): +500M

Command (m for help): n
First cylinder (1570-19457, default 1570):
Using default value 1570
Last cylinder or +size or +sizeM or +sizeK (1570-19457, default 19457): +500M

Command (m for help): n
First cylinder (1583-19457, default 1583):
Using default value 1583
Last cylinder or +size or +sizeM or +sizeK (1583-19457, default 19457): +500M

Command (m for help): n
First cylinder (1596-19457, default 1596):
Using default value 1596
Last cylinder or +size or +sizeM or +sizeK (1596-19457, default 19457): +500M

====> Let's confirm partition which we have created recently.

Command (m for help): p


Disk /dev/sda: 160.0 GB, 160041885696 bytes
255 heads, 63 sectors/track, 19457 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sda1 * 1 13 104391 83 Linux
/dev/sda2 14 1288 10241437+ 83 Linux
/dev/sda3 1289 1543 2048287+ 82 Linux swap / Solaris
/dev/sda4 1544 19457 143894205 5 Extended
/dev/sda5 1544 1556 104391 83 Linux
/dev/sda6 1557 1569 104391 83 Linux
/dev/sda7 1570 1582 104391 83 Linux
/dev/sda8 1583 1595 104391 83 Linux
/dev/sda9 1596 1608 104391 83 Linux

Now Let's convert Normal Partition code to RAID Partition.

Command (m for help): t
Partition number (1-9): 5
Hex code (type L to list codes): fd
Changed system type of partition 5 to fd (Linux raid autodetect)

Command (m for help): t
Partition number (1-9): 6
Hex code (type L to list codes): fd
Changed system type of partition 6 to fd (Linux raid autodetect)

Command (m for help): t
Partition number (1-9): 7
Hex code (type L to list codes): fd
Changed system type of partition 7 to fd (Linux raid autodetect)

Command (m for help): t
Partition number (1-9): 8
Hex code (type L to list codes): fd
Changed system type of partition 8 to fd (Linux raid autodetect)

Command (m for help): t
Partition number (1-9): 9
Hex code (type L to list codes): fd
Changed system type of partition 9 to fd (Linux raid autodetect)

Let's Save Partition Scheme.

Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table.
The new table will be used at the next reboot.
Syncing disks.
2. Let's Update Partition scheme using Partprobe.


[root@electromech-corporation ~]# partprobe
3. Creating RAID 5

=> Adding Created Partition to RAID.

[root@electromech-corporation ~]# mdadm --create -a yes /dev/md5 --level=5 --raid-devices=5 /dev/sda{5,6,7,8,9}
mdadm: /dev/sda5 appears to contain an ext2fs file system
size=1020096K mtime=Wed Dec 30 10:34:18 2009
Continue creating array? y
mdadm: array /dev/md5 started.
=> Formating Software RAID Device

[root@electromech-corporation ~]# mkfs.ext3 /dev/md5
mke2fs 1.39 (29-May-2006)
Filesystem label=
OS type: Linux
Block size=1024 (log=0)
Fragment size=1024 (log=0)
104448 inodes, 417280 blocks
20864 blocks (5.00%) reserved for the super user
First data block=1
Maximum filesystem blocks=67633152
51 block groups
8192 blocks per group, 8192 fragments per group
2048 inodes per group
Superblock backups stored on blocks:
8193, 24577, 40961, 57345, 73729, 204801, 221185, 401409

Writing inode tables: done
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information:

This filesystem will be automatically checked every 27 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
=> Mounting Sofware RAID Device and See details of Software RAID Device.

[root@electromech-corporation ~]# mkdir /raid5
[root@electromech-corporation ~]# mount /dev/md5 /raid5/
[root@electromech-corporation ~]# mdadm --detail /dev/md5
/dev/md5:
Version : 00.90.03
Creation Time : Wed Dec 30 18:59:49 2009
Raid Level : raid5
Array Size : 417280 (407.57 MiB 427.29 MB)
Used Dev Size : 104320 (101.89 MiB 106.82 MB)
Raid Devices : 5
Total Devices : 5
Preferred Minor : 5
Persistence : Superblock is persistent

Update Time : Wed Dec 30 19:00:14 2009
State : clean, degraded, recovering
Active Devices : 4
Working Devices : 5
Failed Devices : 0
Spare Devices : 1

Layout : left-symmetric
Chunk Size : 64K

Rebuild Status : 37% complete

UUID : 523e09aa:9fc68431:ac2e0711:7dc7fdbc
Events : 0.6

Number Major Minor RaidDevice State
0 8 5 0 active sync /dev/sda5
1 8 6 1 active sync /dev/sda6
2 8 7 2 active sync /dev/sda7
3 8 8 3 active sync /dev/sda8
5 8 9 4 spare rebuilding /dev/sda9

[root@electromech-corporation ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda2 9.5G 2.1G 7.0G 23% /
/dev/sda1 99M 19M 75M 21% /boot
tmpfs 982M 0 982M 0% /dev/shm
/dev/md5 1.5G 35M 1.3G 3% /raid5
=> Adding New Spares to Software RAID.


[root@electromech-corporation ~]# mdadm --manage /dev/md5 -a /dev/sda10
mdadm: added /dev/sda10
[root@electromech-corporation ~]# mdadm --manage /dev/md5 -a /dev/sda11
mdadm: added /dev/sda11
[root@electromech-corporation ~]# mdadm --detail /dev/md5
/dev/md5:
Version : 0.90
Creation Time : Sat Jun 12 11:07:00 2010
Raid Level : raid5
Array Size : 1493568 (1458.81 MiB 1529.41 MB)
Used Dev Size : 497856 (486.27 MiB 509.80 MB)
Raid Devices : 4
Total Devices : 6
Preferred Minor : 5
Persistence : Superblock is persistent

Update Time : Sat Jun 12 11:08:27 2010
State : clea n
Active Devices : 4
Working Devices : 6
Failed Devices : 0
Spare Devices : 2

Layout : left-symmetric
Chunk Size : 64K

UUID : e97460c5:76313186:a478a71b:2c154df0
Events : 0.6

Number Major Minor RaidDevice State
0 8 5 0 active sync /dev/sda5
1 8 6 1 active sync /dev/sda6
2 8 7 2 active sync /dev/sda7
3 8 8 3 active sync /dev/sda8
4 8 9 4 active sync /dev/sda9

5 8 10 - spare /dev/sda10
6 8 11 - spare /dev/sda11
=> Adding Spare devices to Software RAID.

[root@electromech-corporation ~]# mdadm --grow /dev/md5 --raid-devices=6
mdadm: Need to backup 768K of critical section..
mdadm: ... critical section passed.
[root@station17 ~]# mdadm --detail /dev/md5
/dev/md5:
Version : 0.91
Creation Time : Sat Jun 12 11:07:00 2010
Raid Level : raid5
Array Size : 1493568 (1458.81 MiB 1529.41 MB)
Used Dev Size : 497856 (486.27 MiB 509.80 MB)
Raid Devices : 5
Total Devices : 6
Preferred Minor : 5
Persistence : Superblock is persistent
Update Time : Sat Jun 12 11:09:30 2010
State : clean, recovering
Active Devices : 5
Working Devices : 6
Failed Devices : 0
Spare Devices : 1

Layout : left-symmetric
Chunk Size : 64K

Rebuild Status : 1% complete

UUID : e97460c5:76313186:a478a71b:2c154df0
Events : 0.6
Number Major Minor RaidDevice State
0 8 5 0 active sync /dev/sda5
1 8 6 1 active sync /dev/sda6
2 8 7 2 active sync /dev/sda7
3 8 8 3 active sync /dev/sda8
4 8 9 4 active sync /dev/sda9
4 8 10 4 active sync /dev/sda10

5 8 11 - spare /dev/sda11

[root@electromech-corporation ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda2 9.5G 2.1G 7.0G 23% /
/dev/sda1 99M 19M 75M 21% /boot
tmpfs 982M 0 982M 0% /dev/shm
/dev/md5 1.5G 35M 1.3G 3% /raid5

[root@electromech-corporation ~]# resize2fs /dev/md5
resize2fs 1.39 (29-May-2006)
Filesystem at /dev/md5 is mounted on /raid5; on-line resizing required
Performing an on-line resize of /dev/md5 to 497856 (4k) blocks.
The filesystem on /dev/md5 is now 497856 blocks long.

[root@electromech-corporation ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda2 9.5G 2.1G 7.0G 23% /
/dev/sda1 99M 19M 75M 21% /boot
tmpfs 982M 0 982M 0% /dev/shm
/dev/md5 1.9G 35M 1.8G 2% /raid5


[root@electromech-corporation ~]# watch cat /proc/mdstat


Personalities : [raid6] [raid5] [raid4]
md5 : active raid5 sda5[4] sda6[5] sda7[3] sda8[2] sda9[1] sda10[0]
1991424 blocks super 0.91 level 5, 64k chunk, algorithm 2 [6/6] [UUUUUU]
[===========>.........] reshape = 59.1% (295296/497856) finish=2.9min spe
ed=1139K/sec

unused devices:

That's It.
Thanks to Tejal Mam for creating such a Wonderful Exercise.

Tuesday 5 April 2011

Bash Cheat sheets

!!Last command
!foo Run most recent command starting with 'foo...' (ex. !ps, !mysqladmin)
!foo:p Print command that !foo would run, and add it as the latest to
command history
!$ Last 'word' of last command ('/path/to/file' in the command 'ls -lAFh
/path/to/file', '-uroot' in 'mysql -uroot')
!$:p Print word that !$ would substitute
!*o All but first word of last command ('-lAFh /path/to/file' in the command
'ls -lAFh /path/to/file', '-uroot' in 'mysql -uroot')
!*:p Print words that !* would substitute
^foo^bar Replace 'foo' in last command with 'bar', print the result, then
run. ('mysqladmni -uroot', run '^ni^in', results in 'mysqladmin -uroot')
{a,b,c}passes words to the command, substituting a, b, and c sequentially
(`cp file{,.bk}` runs `cp file file.bk`)
Ctrl + aJump to the start of the line
Ctrl + bMove back a char
Ctrl + cTerminate the command
Ctrl + dDelete from under the cursor
Ctrl + eJump to the end of the line
Ctrl + fMove forward a char
Ctrl + kDelete to EOL
Ctrl + lClear the screen
Ctrl + rSearch the history backwards
Ctrl + RSearch the history backwards with multi occurrence
Ctrl + tTranspose the current char with the previous
Ctrl + uDelete backward from cursor
Ctrl + wDelete backward a word
Ctrl + xxMove between EOL and current cursor position
Ctrl + x @Show possible hostname completions
Ctrl + zSuspend/ Stop the command
Ctrl + x; Ctrl + eEdit line into your favorite editor
Alt + >Move to the first line in the history
Alt + *lt;Move to the last line in the history
Alt + ?Show current completion list
Alt + *Insert all possible completions
Alt + /Attempt to complete filename
Alt + .Yank last argument to previous command
Alt + bMove backward
Alt + cCapitalize the word
Alt + dDelete word
Alt + fMove forward
Alt + lMake word lowercase
Alt + nSearch the history forwards non-incremental
Alt + pSearch the history backwards non-incremental
Alt + rRecall command
Alt + tTranspose the current word with the previous
Alt + uMake word uppercase
Alt + back-spaceDelete backward from cursor
(Here "2T" means Press TAB twice)
  $ 2T - All available commands(common)
  $ (string)2T - All available commands starting with (string)
  $ /2T - Entire directory structure including Hidden one
  $ (dir)2T - Only Sub Dirs inside (dir) including Hidden one
  $ *2T - Only Sub Dirs inside without Hidden one 
  $ ~2T - All Present Users on system from "/etc/passwd"
  $ $2T - All Sys variables
  $ @2T - Entries from "/etc/hosts"
  $ =2T - Output like ls or dir
  .bash_profile = sourced by login shell, 
  .bashrc = sourced by all shells, 
  .bash_aliases = should be sourced by .bashrc

Run something:
  for i in a b c; do $i 'hello'; done

Do something on a bunch of files:
  for i in *.rb; do echo $i; done

If syntax:
  if [ -e .ssh ]; then echo "hi"; fi

file check flags:
  -e:  file exists
  -f:  regular file (non directory)
  -d:  directory
  -s:  non-zero file
  -x:  execute permission

Avoid duplicates in your history:
  export HISTIGNORE="&:ls:ls *:[bf]g:exit"