Sometime earlier (several hours ago), there’s a really high disk io then all services are down.
I found the file system was remount in RO mode. Something weird.
Anyway, fixed now.
`
feuvan@wonderland:~$ sudo fsck
fsck from util-linux-ng 2.17.2
e2fsck 1.41.11 (14-Mar-2010)
/dev/xvda contains a file system with errors, check forced.
Pass 1: Checking inodes, blocks, and sizes
Inodes that were part of a corrupted orphan linked list found. Fix? yes
Inode 114702 was part of the orphaned inode list. FIXED.
Inode 303107 was part of the orphaned inode list. FIXED.
Deleted inode 958465 has zero dtime. Fix? yes
Inode 958466 was part of the orphaned inode list. FIXED.
Inode 958467 was part of the orphaned inode list. FIXED.
Inode 958468 was part of the orphaned inode list. FIXED.
Inode 958470 was part of the orphaned inode list. FIXED.
Deleted inode 961482 has zero dtime. Fix? yes
Inode 961483 was part of the orphaned inode list. FIXED.
Inode 961484 was part of the orphaned inode list. FIXED.
Inode 961485 was part of the orphaned inode list. FIXED.
Inode 961486 was part of the orphaned inode list. FIXED.
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
Block bitmap differences: -583804 -(583806–583815) -(1232332–1232666)
Fix? yes
Free blocks count wrong for group #17 (4, counted=15).
Fix? yes
Free blocks count wrong for group #37 (13654, counted=13989).
Fix? yes
Free blocks count wrong (2522826, counted=2523172).
Fix? yes
Inode bitmap differences: -114702 -303107 -(958465–958468) -958470 -(961482–961486)
Fix? yes
Free inodes count wrong for group #14 (8130, counted=8131).
Fix? yes
Free inodes count wrong for group #37 (6348, counted=6349).
Fix? yes
Free inodes count wrong for group #117 (5170, counted=5180).
Fix? yes
Free inodes count wrong (910912, counted=910924).
Fix? yes
/dev/xvda: * FILE SYSTEM WAS MODIFIED *
/dev/xvda: * REBOOT LINUX *
/dev/xvda: 121268/1032192 files (2.6% non-contiguous), 1605596/4128768 blocks
feuvan@wonderland:~$ sudo reboot
Broadcast message from feuvan@wonderland
(/dev/pts/0) at 3:58 …
The system is going down for reboot NOW!
`