Home made NAS server with Ubuntu 8.04.1

HOWTO : Home made NAS server with Ubuntu 8.04.1

There are many NAS for home users in the market, such as Synology, Qnap, LinkStation and etc. They are not in good performance and not cheap in price. However, they are good in less power consumption. It is because I owned not only one brand of such products at home now.

Recently, I bought a VIA PC-1 PC2500E motherboard, which has VIA C7-D 1.5GHz CPU on board. It is cheap in price and use less power too. The maximum amount of RAM is 2 GB.

Testing it with Ubuntu 8.04.1 Desktop version for a while, I am very satisfied with the performance of the CPU, although it is not quiet fast indeed. I decided to build a home made NAS server with remote BitTorrent function.

Hardware
Motherboard – VIA PC-1 PC2500E with VIA C7-D 1.5GHz CPU
RAM – 2 X 1GB DDR2 667MHz (maximum)
Hard drive – 300GB Seagate SATA (The motherboard treats it as ATA drive)
Router – Planet WRT-401E (wired) (optional)

Software
Operating system – Ubuntu 8.04.1 Server Edition
File server – Samba
FTP server – vsFTPd
Remote access – OpenSSH
Web Server – Apache, PHP and MySQL
Remote BitTorrent – TorrentFlux (front-end) and BitTornado (back-end)
Security software – Fail2Ban

The performance of the Samba is quiet good and it can stream the video clips. You can remote control the server with OpenSSH and also can remote the BitTorrent anywhere.

I do not choose the GUI interface for server maintenance as it is not required after the server has been set up. I use command line mode indeed.

The server needs some fine tune for better performance and I will tell you how. The installation of the softwares is in Part II.


HOWTO : Home made NAS server with Ubuntu 8.04.1 – Part II

Here comes the software installation. The first thing is to install the operating system – Ubuntu 8.04.1 Server Edition. The installation is quiet straight forward and you should choose LAMP, OpenSSH and Samba (Print server is optional) during the installation. Use a more complicate but easy to remember password as the user password and MySQL password. I use same password for them. Please write it down and you will need it later and forever.

Once installed, you can install vsFTPd and Fail2Ban. BitTorrent feature will be discuss later.

sudo apt-get update
sudo apt-get upgrade
sudo apt-get install vsftpd fail2ban

Samba



The Samba is already installed and you are required to configure it to make it work. I will make the Samba working in this way : Every user has it owned home directory and every user can access to a public directory for sharing document and files.

sudo nano /etc/samba/smb.conf

To add the following lines to the [global] section :
# If you need to access Big5 filename, it is required; otherwise it is optional.
dos charset = cp950

At Authentication part, uncomment the “; security = user” to read as “security = user”

Go to [homes] section and add the following lines to it :
[homes]
comment = Home Directories
path = /home/%S
browseable = yes
read >
create mask = 0664
directory mask = 0664
valid users = %S

Add the following lines to the end of the file :
[public]
comment = Share to all
path = /home/public
browseable = yes
read >
create mask = 0664
directory mask = 0664
valid users = samiux,john,mary
admin users = samiux

To reload the configuration of your Samba server :
sudo /etc/init.d/samba reload

where “admin users = samiux”may be different from you.

Then made a “public” directory at /home :
sudo mkdir /home/public
chgrp users /home/public
chmod 0777 -R /home/public

Now, create normal users as Samba users. Only the users exist in the server can use the Samba.

sudo adduser john
sudo adduser mary

You can access the Samba at Windows system in this way :

Go to your home directory :
\\192.168.0.15\samiux

Go to the public directory :
\\192.168.0.15\public

* your IP address may be different from me

Beware that all home user directories and public directory are writable. Users can add or delete files therein. The home directory owners cannot read and writable another home user directories.

By the way, you can also access the Samba file server on your Ubuntu desktop.

We will discuss how to configure vsFTPd in next part.


Now, your Samba file server is up and working. It is high time to configure the vsFTP server. Anonymous users can download your file from your FTP server but they cannot upload and change anything in it. Local users can access their local home directories (read and write) by using FTP client, e.g. Filezilla or gFTP.

vsFTP

Edit the file at /etc/vsftpd.conf :
sudo nano /etc/vsftpd.conf

Uncomment and change the following lines (removes the “#”in front of the entry) :
anonymous_enable=YES
local_enable=YES
write_enable=YES
data_connection_timeout=120
chroot_local_user=YES

If you are behind a router or firewall, you need to add the following lines at the end of the file :
pasv_enable=YES
pasv_promiscuous=YES
pasv_min_port=50000
pasv_max_port=50100
pasv_address=192.168.0.15

Change the ports 50000-50100 at your desire and add this port range to the router’s port forwarding section or virtual server section. The pasv_address should be the IP address of your server (yours may different from me).

Remember to forward port 20 and 21 out at your router.

Anonymous users can download files from /home/ftp and you can put some files there for them to download.

To restart your vsFTPd :
sudo /etc/init.d/vsftpd restart






HOWTO : Home made NAS server with Ubuntu 8.04.1 – Part IV

Your Samba server and vsFTPd server are running smoothly. Now, we are discussing the installation of remote BitTorrent feature.

You can access your remote BitTorrent server anywhere at anytime. The server requires Apache, PHP and MySQL. They have been installed when you install your Ubuntu Server that selecting the LAMP option.

BitTorrent

sudo apt-get install torrentflux bittornado

You will asked for MySQL root password, it is the same as the one when you install the Ubuntu Server. TorrentFlux Administration password to be entered in the same as the MySQL and Ubuntu user password in my case. (for easy remembering).

Run the TorrentFlux server at your desktop computer with Firefox :


Enter the Log in name as “root” and the password as what you had entered during the installation (TorrentFlux Administration password).

You can replace the “your_server_ip” with your domain name or the IP that provided by your ISP (or say, the real IP).

Updated on 2008-AUG-13 :
Someone complained that the Torrentflux with Bittornado eats a lot of CPU resources. I confirmed that it is. I am looking for an alternative solution - BitTornado (backend) and Torrentflux-b4rt (front end). The title for the tutorial is at HOWTO : Home made NAS server with Ubuntu 8.04.1 - Part IV (a).



HOWTO : Home made NAS server with Ubuntu 8.04.1 – Part IV (a)

Since someone complained that Torrentflux with BitTornado eats a lot of CPU resources. I then switch to Torrentflux-b4rt with BitTornado. Torrentflux-b4rt is a fork of Torrentflux but it is completely rewritten by the developers for the current version. The current version is still in beta 2 at this writing.

I have tested the torrent download at 1134.80 KB/s (one torrent) and it eats not more than 25% of CPU resources (according to the TOP). The overall CPU usage of the system is not more than 35% for the speed of download. Quiet good indeed. It is better than Torrentflux.

Torrentflux-b4rt supports wget, torrent and nzb metafile type download method. The downloaded video clips can be used for streaming via VLC. B4rt also supports Transmission, Azureus, BitTorrent Mainline and BitTornado. However, I had tested the latest version of Transmission (1.32) unsuccessful – it is too slow to start the download and unable to stop the transfer. I think it is bugs. Azureus requires Java to run and I will not test it at all. BitTorrent Mainline is also not tested. Therefore, I selected BitTornado.

Torrentflux-b4rt

Install required libraries and softwares :
sudo apt-get install php5-cli unrar unzip vlc uudeview build-essential bittornado

Download and compile the cksfv that required by b4rt :
wget
tar -xjvf cksfv-1.3.12.tar.bz2

cd cksfv-1.3.12
./configure
make
sudo make install

Get and install the Torrentflux-b4rt (the current version at this writing) from the official site :
wget
tar -xjvf torrentflux-b4rt_1.0-beta2.tar.bz2

cd torrentflux-b4rt_1.0-beta2
sudo cp -R html /var/www/torrentflux

sudo chmod -R 0777 /var/torrentflux/inc/config

Create a directory for the download :
sudo mkdir /home/torrent
sudo chmod -R 0777 /home/torrent

Install the Torrentflux-b4rt with the browser. Type in “”

Follow the instructions on the screen. The username and password of MySQL is the administrator’s username and password of the MySQL (i.e. “root” and the password that you install the MySQL). Make sure to delete the “setup.php”at /var/www/torrentflux after the installation.

The Torrentflux-b4rt is already installed. You can login by the browser at “”. Beware, the username and password that you have key in is the administrator username and password. The program will create for you once you keyed in.

I nearly forgot to tell you to open or forward the (default) port 49160 to 49300 at router or firewall. You can change it as desire.

Samba configuration

If you want to share the downloaded files with the intranet, you just added the following lines at the end of the /etc/samba/smb.conf :
[torrent]
comment = Share to all
path = /home/torrent
browseable = yes
read >
create mask = 0664
directory mask = 0664
valid users = samiux,john,mary
admin users = samiux

Access to the torrent directory at Windows system :
\\your_server_ip\torrent

Bonus

Transmission installation procedure :
sudo apt-get install libcurl4-openssl-dev gettedxt libc6-dev libssl-dev pkg-config

Download and install the Transmission :
wget
tar -xjvf transmission-1.32.tar.bz2

cd transmission-1.32
./configure --without-gtk
make
sudo make install


HOWTO : Home made NAS server with Ubuntu 8.04.1 – Part V

This part is optional. The web server with PHP and MySQL is already running. Now, we are going to fine tune the Apache and PHP to make it running as faster as possible. The kernel can also be optimized too.

Apache

sudo a2enmod deflate
sudo /etc/init.d/apache2 force-reload

Edit the file /etc/apache2/conf.d/deflate.conf :
sudo nano /etc/apache2/conf.d/deflate.conf


DeflateCompressionLevel 6
AddOutputFilterByType DEFLATE text/plain
AddOutputFilterByType DEFLATE text/html
AddOutputFilterByType DEFLATE text/xml
AddOutputFilterByType DEFLATE text/css
AddOutputFilterByType DEFLATE application/xhtml+xml
AddOutputFilterByType DEFLATE application/xml
AddOutputFilterByType DEFLATE application/rss+xml
AddOutputFilterByType DEFLATE application/atom_xml
AddOutputFilterByType DEFLATE application/x-javascript
AddOutputFilterByType DEFLATE application/x-httpd-php
AddOutputFilterByType DEFLATE application/x-httpd-fastphp
AddOutputFilterByType DEFLATE application/x-httpd-eruby
AddOutputFilterByType DEFLATE image/svg+xml
AddOutputFilterByType DEFLATE application/postscript

Header append Vary User-Agent



Edit /etc/apache2/apache2.conf :
sudo nano /etc/apache2/apache2.conf

Then change the value as the following :
ServerSignature Off
ServerTokens Prod

Edit /etc/apache2/sites-available/default :
sudo nano /etc/apache2/sites-available/default

Then change the value as the following at /var/www section :
Options -Indexes FollowSymLinks

Restart your Apache server :
sudo /etc/init.d/apache2 restart

PHP
To accelerate PHP, you need Zend but I will show you how to use eAccelerator.

Download the eAccelerator :
wget
tar xvf eaccelerator-0.9.5.3.tar.bz2

sudo apt-get install build-essential php5-dev

cd eaccelerator-0.9.5.3
phpize
./configure
make
sudo make install

sudo mkdir /tmp/eaccelerator
sudo chmod 0777 /tmp/eaccelerator

Add the following lines at the end of /etc/php5/apache2/php.ini :
extension=”eaccelerator.so”
; shm_size default is 16, you may change to 64 or 128 depends on your RAM
eaccelerator.shm_size=”16”
eaccelerator.cache_dir=”/tmp/eaccelerator”
eaccelerator.enable=”1”
eaccelerator.optimizer=”1”
eaccelerator.check_mtime=”1”
eaccelerator.debug=”0”
eaccelerator.filter=””
eaccelerator.shm_max=”0”
eaccelerator.shm_ttl=”0”
eaccelerator.shm_prune_period=”0”
eaccelerator.shm_only=”0”
eaccelerator.compress=”1”
eaccelerator.compress_level=”9”

Restart the Apache :
sudo /etc/init.d/apache2 restart

Kernel

Please refer to my blog “Performance-tunning” at for details:


Performance tuning

Add the following lines to /etc/sysctl.conf if you have 512MB RAM or more :

kernel.sem = 250 32000 100 128
kernel.shmall = 2097152
kernel.shmmax = 2147483648
kernel.shmmni = 4096
# If you have more than 512MB RAM, use this setting (uncomment it and comment the setting just below)
#fs.file-max = 256000
# If you have 512MB RAM or less, use this setting
fs.file-max = 65535
vm.swappiness = 1
vm.vfs_cache_pressure = 50

If you have a fast broadband (UL 10Mbps / DL 10Mbps, my broadband connection speed) (other connection speed is not tested), use the following setting :

net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
net.ipv4.tcp_rmem = 4096 87380 16777216
net.ipv4.tcp_wmem = 4096 65536 16777216
net.ipv4.tcp_no_metrics_save = 1

If you have a Wifi (such as IEEE 802.11 a/b/g) or a 3.5G modem, use the following setting :

net.core.rmem_default = 524288
net.core.rmem_max = 524288
net.core.wmem_default = 524288
net.core.wmem_max = 524288
net.ipv4.tcp_wmem = 4096 87380 524288
net.ipv4.tcp_rmem = 4096 87380 524288
net.ipv4.tcp_mem = 524288 524288 524288
net.ipv4.tcp_rfc1337 = 1
net.ipv4.ip_no_pmtu_disc = 0
net.ipv4.tcp_sack = 1
net.ipv4.tcp_fack = 1
net.ipv4.tcp_window_scaling = 1
net.ipv4.tcp_timestamps = 1
net.ipv4.tcp_ecn = 0
net.ipv4.route.flush = 1

Then make the setting effective :

sudo /sbin/sysctl -p

Lightening Ubuntu!!!!





HOWTO : Home made NAS server with Ubuntu 8.04.1 – Part VI

This part is also optional. The Samba is shared with Windows desktops that files stored in the Samba will be infected by virus. To prevent this, you should install the ClamAV, the open source anti-virus program for Linux that kills Windows virus.

ClamAV

Edit /etc/apt/sources.list and append the following lines at the end of the file :
sudo nano /etc/apt/sources.list

# ClamAV PPA
deb hardy main
deb-src hardy main

sudo apt-get update
sudo apt-get install clamav clamav-daemon arj unzoo lha unrar

Make sure clamav-daemon is running :
ps ax | grep clamd

If not :
sudo /etc/init.d/clamav-daemon start
sudo /etc/init.d/clamav-freshclam start


Remarks : The current version of ClamAV at the time of this writing cannot scan RAR files. It is a bug and it will be fixed later.

Some personal NAS in the market comes with iTune music server. However, iTune is not an Open Source software.

Sockos is an Open Source personal music server that written in Java. She requires Sun’s Java Runtime. IcedTea is not compatible so far.

Sockos is a cross platform software and requires no installation. She can run on a standalone personal computer or on a server. For running on personal computer with GUI, please refer to her official site.

The client computer requires no mp3 player to play the music but may need Flash (optional). Flex player requires no installation of music player on the clinet computer. Other formats, such as WMV, OGG and Flac, require pre-installed music players that fit for the special formats.

The advantage of Sockos is that you can listen to your mp3 files at anytime and anywhere under the condition that fast internet connection is available. The disadvantage is that you should have at least IEEE 802.11g (54M) Wifi connection for smooth operation. GPRS and HSDPA do not work properly in my testing.

I am going to talk about running Sockos in Ubuntu Server 8.04.1 (without GUI).

Step 1 :

Install the Sun Java and runtime in the Ubuntu server (NAS).
sudo apt-get install sun-java6-bin sun-java6-fonts sun-java6-jre sun-java6-plugin

Download the Sockos from the official site. Extact it on your home directory of your Ubuntu server. If your server (NAS) is detached the keyboard and monitor, plug in them now.

wget
unzip sockso-latest.zip
cd sockos-1.0.9

Step 2 :

Create a directory at /var for storing the database files.
sudo mkdir /var/sockos
chmod -R 0755 /var/sockos

Step 3 :

Run the server as user (such as samiux) at the physcial server at tty1 (Ctrl+Alt+F1) at /home/samiux/sockos-1.0.9.
sudo sh linux.sh –nogui –datadir /var/suckso

A directory namely “covers” and files “database.lck”, “database.log”, “database.properties” and “database.script” will be created on /var/sockso.

A console will be prompted and telling you that your IP address and the port 4444 to be used for the music server.

Step 4 :

Now, create the paths for the mp3 that located in the music server (your NAS – Ubuntu Server).

For example, if some mp3 files are storing at 3 locations, such as :
/home/samiux/music, /home/john/mp3 and /home/mary/songs.

At the console (at the server), type the following commands :
coladd /home/samiux/music
coladd /home/john/mp3
coladd /home/mary/songs

collist to list all the paths that you just created. You can delete the path by coldel command.

Step 5 :

You can create a user, e.g. samiux by the following command or create at the web interface.
useradd samiux 9876543210

(where 9876543210 is password and the other is your email address)

Your Sockos Music Server is ready but you should not exit the console; otherwise, it will terminate the Sockos. Leave it alone and press Ctrl+Alt+F2. You can revisit to the console again by pressing Ctrl+Alt+F1. Type “help” for assistance at the console.

Step 6 :

Go to your personal computer (client) and open a browser, e.g. Firefox and type the following on the address.


(where 192.168.0.15 is your NAS address behind a router, 4444 is the port that Sockos to be used)

Warning

If you want to share your music files with others over the internet, you should beware the copyright law in the music industry or you may in serious trouble – lawsuit. You have been warned.

Enjoy your loving music at anywhere and anytime!!!