Black Hole Socket Problem

Problem:

• 1 client 1 server, connected with non-block tcp socket. Linux 2.6.*+.


• Client writes about 3k bytes into the socket, by calling 'write' system call.


• 'Write' returns with correct number of bytes written to the socket.


• Server side socket receives nothing.

Causes:

• Client side MTU is a bit larger than the MTU on some router(s) between the 2 machine


• ICMP message requesting package fragmentation are block somehow(maybe by firewall).


• Client insistently sends the same package again and again.

IP package with DF flag set is not fragmentated by routers into smaller ones if package size is larger than the routers. instead, router sends back an ICMP message telling the sender to fragmentate package.

But ICMP messages are blocked somewhere.


Circumstances And Phenomenon:
In this case, MTUs on client and server are both 1500.

I dumped the network traffic with tcpdump, and we can have a close look:

Phenomenon on client:
2 packages sent. The 1st package was resent time by time:

17:23:06.933574 IP (tos 0x0, ttl 64, id 57558, offset 0, flags [DF], proto: TCP (6), length: 1500) 10.54.40.43.43145 > 10.29.14.74.http: ., cksum 0x5096 (incorrect (-> 0x5c4e), 0:1448(1448) ack 1 win 46

17:23:06.933580 IP (tos 0x0, ttl 64, id 57559, offset 0, flags [DF], proto: TCP (6), length: 730) 10.54.40.43.43145 > 10.29.14.74.http: P, cksum 0x4d94 (incorrect (-> 0x3933), 1448:2126(678) ack 1 win 46

17:23:07.167049 IP (tos 0x0, ttl 64, id 57560, offset 0, flags [DF], proto: TCP (6), length: 1500) 10.54.40.43.43145 > 10.29.14.74.http: ., cksum 0x5096 (incorrect (-> 0x5b5b), 0:1448(1448) ack 1 win 46

17:23:07.634922 IP (tos 0x0, ttl 64, id 57561, offset 0, flags [DF], proto: TCP (6), length: 1500) 10.54.40.43.43145 > 10.29.14.74.http: ., cksum 0x5096 (incorrect (-> 0x5987), 0:1448(1448) ack 1 win 46

Phenomenon on server:
only the second package of size 730 received successfuly

17:23:08.605622 IP (tos 0x0, ttl 59, id 57559, offset 0, flags [DF], proto: TCP (6), length: 730) 202.108.3.204.43145 > 10.29.14.74.http: P, cksum 0x9d5b (correct), 1448:2126(678) ack 1 win 46

Solution:
As you may not have the privilege to telnet onto the router to take some adjustment, the simplest way is to change configuration on client in one of the following way:

At network level:
Decrease mtu on network adapter:

ifconfig eth* mtu 1400

At system level:
Clear the default 'MTU discovery' flag with sysctl:

net.ipv4.ip_no_pmtu_disc = 1

At user application level:
Set socket option 'IP_MTU_DISCOVER' with setsockopt(2) to clear 'DF' flag of IP package.


Reference:

1. DF flag of IP package Header
   


2. Internet Control Message Protocol


3. IP fragmentation


4. MTU or Maximum transmission unit


5. IP programming


6. Path MTU Discovery


7. sysctl

｀

LAMP starts

First it is ftp. That is always the way one deploy a site to server.

Download vsftp, ./configure and then make install.
Or install with yum or atp.
Edit /etc/vsftpd.conf(or maybe /etc/vsftpd/vsftpd.conf).
To enable anonymous access, add the 2 following line:

anon_root=/data
anon_upload_enable=YES

To add another user,

todo here...

Then we go with mysql 5.0(5.0.51a.)

Get mysql noninstall version, unpack it into /usr/local.
make a symbol link as /usr/local/mysql to /usr/local/mysql.real.path.
and execute the following command to create users and start mysql daemon:

shell> groupadd mysql
shell> useradd -g mysql mysql
shell> cd /usr/local
shell> gunzip < /PATH/TO/MYSQL-VERSION-OS.tar.gz | tar xvf - shell> ln -s FULL-PATH-TO-MYSQL-VERSION-OS mysql
shell> cd mysql
shell> chown -R mysql .
shell> chgrp -R mysql .
shell> scripts/mysql_install_db --user=mysql
shell> chown -R root .
shell> chown -R mysql data
shell> bin/mysqld_safe --user=mysql &

copy one of the support-files/my-*.cnf to /etc/my.cnf as the default mysql config.

To stop mysql : mysqladmin shutdown

If needed, use mysqladmin -u root -p shutdown

After mysql started, we need to add some account to it.

mysql> GRANT ALL PRIVILEGES ON *.* TO 'monty'@'localhost'
-> IDENTIFIED BY 'some_pass' WITH GRANT OPTION;

mysql> GRANT ALL PRIVILEGES ON *.* TO 'monty'@'%'
-> IDENTIFIED BY 'some_pass' WITH GRANT OPTION;

mysql> GRANT RELOAD,PROCESS ON *.* TO 'admin'@'localhost';

mysql> GRANT USAGE ON *.* TO 'dummy'@'localhost';

'all privileges' means this user can do anything.
*.* means all tables
'monty' is the user name, 'localhost' is the ip address from where this user can login.
'some_pass' is password.
'grant option' means this user can do grant operation.

To use old passward format for old mysql client :

mysql> SET PASSWORD FOR
 -> 'some_user'@'some_host' = OLD_PASSWORD('newpwd'); 

Alternative ：

mysql> UPDATE mysql.user SET Password = OLD_PASSWORD('newpwd')
 -> WHERE Host = 'some_host' AND User = 'some_user';

mysql> FLUSH PRIVILEGES; 

Or：

Start mysqld with the --old-passwords option.

Then we go with php5 (mysql 4.1 or later requires php 5).

download php source &

./configure --with-mysql & make install.

dynamic lib in linux

/etc/ld.so.conf
dynamic libs config file

command : ldconfig
Each time after changing that file, run 'ldconfig' to flush config data.

command : ldd
Check which lib has been linked by an executable.
example : ldd a.out

libpthread.so.0 => /lib/tls/libpthread.so.0 (0x00986000)
libc.so.6 => /lib/tls/libc.so.6 (0x0081c000)
/lib/ld-linux.so.2 (0x007fe000)

command : objdump
Print out many useful information about an executable file.
example : objdump -p a.out

a.out：     文件格式 elf32-i386

程序头：
    PHDR off    0x00000034 vaddr 0x08048034 paddr 0x08048034 align 2**2
         filesz 0x000000e0 memsz 0x000000e0 flags r-x
  INTERP off    0x00000114 vaddr 0x08048114 paddr 0x08048114 align 2**0
         filesz 0x00000013 memsz 0x00000013 flags r--
    LOAD off    0x00000000 vaddr 0x08048000 paddr 0x08048000 align 2**12
    ....
动态节：
  NEEDED      libpthread.so.0
  NEEDED      libc.so.6
    ....

版本引用：
  required from libpthread.so.0:
    0x0d696910 0x00 04 GLIBC_2.0
    0x0d696911 0x00 03 GLIBC_2.1
  required from libc.so.6:
    0x0d696910 0x00 02 GLIBC_2.0

begin sed at a sight

Sed is an Editor..

The normal way to use 'sed' is:
sed 'reg_pattern/commands' filename

sed processes each line in a file as a unit.
sed finds out each line matched the given regular expression, & deal with it by these 'commands'. commands could be a single command such as D, or sequence of commands quoted by '{}'.

the most used format of sed : "sed 's/pattern/replace/' file" is an abbreviation, 's' is a command. searching pattern reg is absent, so that 's' command applies to each line.
the full format is : sed '/.*/{s/pattern/replace/g}' file

there are some usually used examples:

# join all lines
sed ':x;/./{N;s/[\n\r]//g;bx}'

#remove single-line comment
sed 's/\/\*.*\*\///g'

#remove multi-line comment
sed '/^[ \t]*\/\*/, /.*\*\//d'

#remove dos-line-breaker in unix file
sed 's/\cm//g'