gnulogs.com

TCP , UDP ports and connections

2011-07-25T07:02:00.000-07:00

TCP vs UDP

Overview

TCP (Transmission Control Protocol) is the most commonly used protocol on the Internet. The reason for this is because TCP offers error correction. When the TCP protocol is used there is a "guaranteed delivery." This is due largely in part to a method called "flow control." Flow control determines when data needs to be re-sent, and stops the flow of data until previous packets are successfully transferred. This works because if a packet of data is sent, a collision may occur. When this happens, the client re-requests the packet from the server until the whole packet is complete and is identical to its original.

UDP (User Datagram Protocol) is anther commonly used protocol on the Internet. However, UDP is never used to send important data such as webpages, database information, etc; UDP is commonly used for streaming audio and video. Streaming media such as Windows Media audio files (.WMA) , Real Player (.RM), and others use UDP because it offers speed! The reason UDP is faster than TCP is because there is no form of flow control or error correction. The data sent over the Internet is affected by collisions, and errors will be present. Remember that UDP is only concerned with speed. This is the main reason why streaming media is not high quality.

On the contrary, UDP has been implemented among some trojan horse viruses. Hackers develop scripts and trojans to run over UDP in order to mask their activities. UDP packets are also used in DoS (Denial of Service) attacks. It is important to know the difference between TCP port 80 and UDP port 80. If you don't know what ports are go here.

Frame Structure

As data moves along a network, various attributes are added to the file to create a frame. This process is called encapsulation. There are different methods of encapsulation depending on which protocol and topology are being used. As a result, the frame structure of these packets differ as well. The images below show both the TCP and UDP frame structures.

TCP FRAME STRUCTURE

UDP FRAME STRUCTURE

The payload field contains the actually data. Notice that TCP has a more complex frame structure. This is largely due to the fact the TCP is a connection-oriented protocol. The extra fields are need to ensure the "guaranteed delivery" offered by TCP.

test

2011-03-21T13:08:00.001-07:00

test

Zombie and orphan

2010-12-11T13:07:00.001-08:00

On Unix operating systems, a zombie process or defunct process is a process that has completed execution but still has an entry in the process table, allowing the process that started it to read its exit status. In the term's colorful metaphor, the child process has died but has not yet been reaped.

When a process ends, all of the memory and resources associated with it are deallocated so they can be used by other processes. However, the process's entry in the process table remains. The parent is sent a SIGCHLD signal indicating that a child has died; the handler for this signal will typically execute the wait system call, which reads the exit status and removes the zombie. The zombie's process ID and entry in the process table can then be reused. However, if a parent ignores the SIGCHLD, the zombie will be left in the process table. In some situations this may be desirable, for example if the parent creates another child process it ensures that it will not be allocated the same process ID.

A zombie process is not the same as an orphan process. Orphan processes don't become zombie processes; instead, they are adopted by init (process ID 1), which waits on its children.

The term zombie process derives from the common definition of zombie�an undead person.

Zombies can be identified in the output from the Unix ps command by the presence of a "Z" in the STAT column. Zombies that exist for more than a short period of time typically indicate a bug in the parent program. As with other leaks, the presence of a few zombies isn't worrisome in itself, but may indicate a problem that would grow serious under heavier loads.

To remove zombies from a system, the SIGCHLD signal can be sent to the parent manually, using the kill command. If the parent process still refuses to reap the zombie, the next step would be to remove the parent process. When a process loses its parent, init becomes its new parent. Init periodically executes the wait system call to reap any zombies with init as parent.

An orphan process is a computer process whose parent process has finished or terminated.

A process can become orphaned during remote invocation when the client process crashes after making a request of the server.

Orphans waste server resources and can potentially leave a server in trouble. However there are several solutions to the orphan process problem:

1. Extermination is the most commonly used technique; in this case the orphan process is killed.

2. Reincarnation is a technique in which machines periodically try to locate the parents of any remote computations; at which point orphaned processes are killed.

3. Expiration is a technique where each process is allotted a certain amount of time to finish before being killed. If need be a process may "ask" for more time to finish before the allotted time expires.

A process can also be orphaned running on the same machine as its parent process. In a UNIX-like operating system any orphaned process will be immediately adopted by the special "init" system process. This operation is called re-parenting and occurs automatically. Even though technically the process has the "init" process as its parent, it is still called an orphan process since the process which originally created it no longer exists.

Apache Tomcat Integration

2010-03-29T01:11:00.001-07:00

Introduction tomcat and Apache web server

Apache httpd is very fastest web server for static contents. Apache httpd web server has biggest market share of more than 60% in web server in world. Today apache httpd web server supports dynamic contents through module connectors. It supports php, cgi, perl and many more scripting languages. Apache is widely used in the world and it is freely available for download. It is open source code by apache projects. Apache's biggest advantage is platform independent, can work on any platform unix, linux and windows without any problems. Apache is not only platform independent, it is very easy to configure and very easy to use. Tomcat is another open source project from apache. Tomcat is java based web server, which support java servlet, and servlet side scripting. Java is platform independent and day by day getting popularity in market.

Java has advantage of security and more secure than other programming languages. Servlet has advantage of precompiled and need not to compile on every request send by client on browser. All servlets and JSP runs under Java virtual machine. Tomcat is freely available from apache, and java is available from sun. Apache tomcat integration achieve through mod_jk connectors. It can communicate through AJP13 apache jserv protocol 13. This mo_jk connector interacts with tomcat web server and apache httpd, and send dynamic request to tomcat web server and response back to client. All static content handled by httpd server only. Let’s start integration of tomcat and apache step by step.

Installation of JDK in linux

Tomcat doesn't run alone, it needs java runtime environment JRE or JVM for running jsp and servlet pages. JDK is freely available from sun. JDK can be downloaded from sun's website. http://java.sun.com/javase/downloads/index.jsp . Download latest version of jdk or sdk and install on the machine. Sun provides different versions of jdk for different platform. There are different versions available from sun for linux, windows or sun solaris. Tomcat 5.5 is work only with JDK 1.5 version, so use at least jdk 1.5 versions. Installing JDK on linux is not difficult work; this can be done by double click on bin format file or simple RPM. If you are installing JDK from rpm then you can run this command to install.
rpm -ivh jdk-1_5_0_07-nb-5_0-linux-ml.rpm

If you are installing JDK from bin file you can run this command to install, and also change permissions to 777 to bin file.
sh jdk-1_5_0_07-nb-5_0-linux-ml.bin.

JDK installed at default path /opt directory in linux fedora core 5.

Installation Tomcat on linux
see more Installing tomcat on Windows

After successful installation of JDK, install tomcat. Tomcat is freely available from apache's website. Download latest version of tomcat from apache website. Tomcat is distributed in binary and source code. Binary code is precompiled of jar files and easy to install, and source code have to compile and install manually. Installation through source code need ant, this will compile automatically all files in sub directory or packages. Tomcat can be downloaded from http://archive.apache.org/dist/tomcat/tomcat-5/ Download tomcat in tar.gz format for Linux or any RPM form file and executable file or zipped file for windows platform. Unzip tar file
tar -xvf apache-tomcat-5.5.17.tar.gz

Copy this tomcat in root or anywhere. We used to install tomcat at root / in linux.

Tomcat need to know where is java installed on system, where to compile servlets and beans. Now set environment variables for java, jre and classpath for servlet-api. This can be done by export command manully. Export will keep environment setting until system is not shutdown or restarted. When system get restarted all environment setting will lost.
export JAVA_HOME=/opt/jdk

or we can call this automatically by putting export command in startup.sh script and shutdown.sh script of tomcat. This can be done easily, open startup.sh file in vi or any text editor from %TOMCAT_HOME%/bin/startup.sh /opt/jdk is folder where actual java is installed. Or specify the java home directory path

#!/bin/sh
# -----------------------------------------------------------------------------
# Start Script for the CATALINA Server
#
# $Id: startup.sh 385888 2006-03-14 21:04:40Z keith $
# -----------------------------------------------------------------------------

# Better OS/400 detection: see Bugzilla 31132
export JAVA_HOME=/opt/jdk/
os400=false
darwin=false
case "`uname`" in
CYGWIN*) cygwin=true;;
OS400*) os400=true;;
Darwin*) darwin=true;;
esac

# resolve links - $0 may be a softlink
PRG="$0"

while [ -h "$PRG" ] ; do
ls=`ls -ld "$PRG"`
link=`expr "$ls" : '.*-> $.*$$'`
if expr "$link" : '/.*' > /dev/null; then
PRG="$link"
else
PRG=`dirname "$PRG"`/"$link"
fi
done

It should be also defined in shutdown.sh script of tomcat. Open shutdown.sh script in any text editor, and include export command in shutdown.sh script.

#!/bin/sh
# -----------------------------------------------------------------------------
# Stop script for the CATALINA Server
#
# $Id: shutdown.sh 385888 2006-03-14 21:04:40Z keith $
# -----------------------------------------------------------------------------

# resolve links - $0 may be a softlink
export JAVA_HOME=/opt/jdk
PRG="$0"

while [ -h "$PRG" ] ; do
ls=`ls -ld "$PRG"`
link=`expr "$ls" : '.*-> $.*$$'`
if expr "$link" : '/.*' > /dev/null; then
PRG="$link"
else
PRG=`dirname "$PRG"`/"$link"
fi
done

PRGDIR=`dirname "$PRG"`
EXECUTABLE=catalina.sh

# Check that target executable exists
if [ ! -x "$PRGDIR"/"$EXECUTABLE" ]; then
echo "Cannot find $PRGDIR/$EXECUTABLE"
echo "This file is needed to run this program"
exit 1
fi

exec "$PRGDIR"/"$EXECUTABLE" stop "$@"

Test tomcat is installed properly. Run this command to startup tomcat through linux terminal.

%TOMCAT_HOME%/bin/startup.sh

e.g. /tomcatA/bin/startup.sh

And shutdown

%TOMCAT_HOME%/bin/shutdown.sh

e.g. /tomcatA/bin/shutdown.sh

If tomcat is throwing error, check logs folder %TOMCAT_HOME%/logs/catalina.out. If your tomcat does not have logs folder make it manually and restart tomcat again.

Open web browser and in address bar write http://localhost:8080/. If it opens apache's page, means tomcat is working properly.

Now you need to configure servlets and application context path. Servlet invoker is default invoker to load servlet into tomcat memory. No need to describe, any parameters in web.xml. Servlet and port configuration setting go through Servlet invoker
Apache installation

see more Installing Apache on Windows

Apache httpd also open source project by apache group. Apache easily available from http://archive.apache.org/dist/httpd/ . Download stable tar.gz versions of apache httpd. Apache httpd 2.0 more stable version then other versions.

First you have to remove previous installed httpd from your system. Otherwise it will conflict with new installed httpd services.

Commands to remove default rpm of httpd(apache).

Find your installed rpm packages in system by
rpm -qa | grep -i httpd

uninstall rpm
rpm -e packageName

e.g. rpm -e httpd-2.2.0-5.1.2
Uncompress downloaded httpd-2.0.55.tar.gz by tar command
tar -xvf httpd-2.0.55.tar.gz

Command to install apache manually.
Go in directory where apache is unzipped

[root@clustering ~] cd /httpd
[root@ clustering httpd]./configure --prefix=/usr/local/httpd
[root@ clustering httpd]make
[root@ clustering httpd] make install

Apache is now installed if in last it doesn't show any error.
Test apache by starting apache
[root@ clustering httpd] /usr/local/httpd/bin/apachect1 start

If it is started correctly you can check it on browser by opening localhost address bar.
Mod_jk Connector

This is very important step of tomcat apache integration. Mod_jk is connector module which helps to make connection between tomcat and apache web server. The request come from apache and apache web server send this request to tomcat web server through mod_jk. There are two type of mod_jk available

1. mod_jk.so module
2. mod_jk2.so module

mod_jk2.so's development is stopped. So we are using mod_jk.so module. This can be downloaded from http://archive.apache.org/dist/jakarta/tomcat-connectors/.
Download binary format module.
Copy mok_jk.so module in modules directory of apache web server
e.g. C:\apache\Apache2\modules
or in linux

e.g. /usr/local/httpd/modules
Configuration Tomcat and Apache

Apache Tomcat Configuration, Apache web server use a configuration file, httpd.conf. This file is in folder

/usr/local/httpd/conf/httpd.conf

Or %httpd%/conf/httpd.conf. Open this file in any text editor notepad or vi editor and do editing as show in this file.

Listen 80

#
# Dynamic Shared Object (DSO) Support
#
# To be able to use the functionality of a module which was built as a DSO you
# have to place corresponding `LoadModule' lines at this location so the
# directives contained in it are actually available _before_ they are used.
# Statically compiled modules (those listed by `httpd -l') do not need
# to be loaded here.
#
# Example:
# LoadModule foo_module modules/mod_foo.so
#

LoadModule jk_module modules/mod_jk.so
JkWorkersFile /usr/local/httpd/conf/workers.properties
JkLogFile "logs/mod_jk.log"
JkLogLevel error
JkMount /gnulogsbox loadbalancer
JkMount /gnulogsbox/* loadbalancer

#
# ExtendedStatus controls whether Apache will generate "full" status
# information (ExtendedStatus On) or just basic information (ExtendedStatus
# Off) when the "server-status" handler is called. The default is Off.
#
#ExtendedStatus On

### Section 2: 'Main' server configuration
#
# The directives in this section set up the values used by the 'main'
# server, which responds to any requests that aren't handled by a
# definition. These values also provide defaults for
# any containers you may define later in the file.
#
# All of these directives may appear inside containers,
# in which case these default settings will be overridden for the
# virtual host being defined.
#

#
# If you wish httpd to run as a different user or group, you must run
# httpd as root initially and it will switch.
#

Add this highlighted line in httpd.conf file. Create a file in /usr/local/httpd/log/ mod_jk.log. This log file refers to error thrown by apache web server. Errors can be read from this log file.

Create new file in /usr/local/httpd/conf/workers.properties. This workers.properties file contains all information about workers (Tomcat), their ports connector Port AJP apache javaserv port. A sample file is given here.

workers.tomcat_home=/tomcat
workers.java_home=$JAVA_HOME
ps=/
worker.list=tomcat,loadbalancer

worker.tomcat.port=8009
worker.tomcat.host=192.168.1.80
worker.tomcat.type=ajp13
worker.tomcat.lbfactor=1

worker.loadbalancer.type=lb
worker.loadbalancer.balanced_workers=tomcat
worker.loadbalancer.sticky_session=1

Worker.tomcat.host=192.168.1.80 is tomcat server ipaddress or use computer name instead of ipaddress like this

Worker.tomcat.host=gnulogsbox.

Now It is almost done. Just stop apache web server and again start. To start apache web server manually, should use command.
[root@ clustering httpd] /usr/local/httpd/bin/apachect1 stop

And start apache web server now
[root@ clustering httpd] /usr/local/httpd/bin/apachect1 start

If apache starts again without error, this means all things are going in right way. If apache doesn't start and throw an error then you need to change mod_jk.so connector according to your server hardware configuration. Download another mod_jk connector and again copy to module folder and stop and start apache httpd server. Otherwise, again and again find suitable connector for your hardware and server configuration, test it again apache. Error may be this kind of

[root@clustering httpd]# /usr/local/httpd/bin/apachectl start
Syntax error on line 234 of /usr/local/httpd/conf/httpd.conf:
Cannot load /usr/local/httpd/modules/mod_jk.so into server: /usr/local/httpd/modules/mod_jk.so: invalid ELF header

When apache httpd server start without any error,

[root@clustering httpd]# /usr/local/httpd/bin/apachectl start
httpd: Could not determine the server's fully qualified domain name, using 127.0.0.1 for ServerName

This is error free startup of apache.
Run Jsp and servlet

Now start tomcat server. After startup of tomcat server, you can test simple jsp page. http://192.168.1.80/ this will open apache home page.

1. What is a SAN?

2009-12-13T03:47:00.000-08:00

1. What is a SAN?
A SAN, or storage area network, is a dedicated network that is separate from LANs
and WANs. It generally serves to interconnect the storage-related resources that are
connected to one or more servers. It is often characterized by its high interconnection
data rates (Gigabits/sec) between member storage peripherals and by its highly
scalable architecture. Though typically spoken of in terms of hardware, SANs very
often include specialized software for their management, monitoring and
configuration.
SANs can provide many benefits. Centralizing data storage operations and their
management is certainly one of the chief reasons that SANs are being specified and
developed today. Administrating all the storage resources in high-growth and
mission-critical environments can be daunting and very expensive. SANs can
dramatically reduce the management costs and complexity of these environments
while providing significant technical advantages.
SANs can be based upon several different types of high-speed interfaces. In fact,
many SANs today use a combination of different interfaces. Currently, Fibre Channel
serves as the de facto standard being used in most SANs. Fibre Channel is an
industry-standard interconnect and high-performance serial I/O protocol that is media
independent and supports simultaneous transfer of many different protocols.
Additionally, SCSI interfaces are frequently used as sub-interfaces between internal
components of SAN members, such as between raw storage disks and a RAID
controller.
MSKL SAN Tutorial
Provding large increases in storage performance, state-of-the-art reliability and
scalability are primary SAN benefits. Storage performance of a SAN can be much
higher than traditional direct attached storage, largely because of the very high data
transfer rates of the electrical interfaces used to connect devices in a SAN (such as
Fibre Channel). Additionally, performance gains can come from opportunities
provided by a SAN’s flexible architecture, such as load balancing and LAN-free
backup. Even storage reliability can be greatly enhanced by special features made
possible within a SAN. Options like redundant I/O paths, server clustering, and runtime
data replication (local and/or remote) can ensure data and application
availability. Adding storage capacity and other storage resources can be
accomplished easily within a SAN, often without the need to shut down or even
quiese the server(s) or their client networks. These features can quickly add up to
large cost savings, fewer network outages, painless storage expansion, and reduced
network loading.
By providing these dedicated and “very high speed” networks for storage and backup
operations. SANs can quickly justify their implementation. Offloading tasks, such as
backup, from LANs and WANs is vital in today’s IT environments where networks
loads and bandwidth availability are critical metrics by which organizations measure
their own performance and even profits. Backup windows have shrunken dramatically
and some environments have no backup windows at all since entire data networks
and applications often require 24x365 availability.
As with many IT technologies, SANs depend on new and developing standards to
ensure seamless interoperability between their member components. SAN hardware
components such as Fibre Channel hubs, switches, host bus adapters, bridges and
RAID storage systems rely on many adopted standards for their connectivity. SAN
software, every bit as important its hardware, often provides many of the features and
benefits that SANs have come to be known for. SAN software can provide or enable
foundation features and capabilities, including:
· SAN Management
· SAN Monitoring (including “phone home” notification features)
· SAN Configuration
· Redundant I/O Path Management
· LUN Masking and Assignment
· Serverless Backup
· Data Replication (both local and remote)
· Shared Storage (including support for heterogeneous platform environments)

to be cntnd..

Apache Performance Tuning

2009-07-15T06:38:00.000-07:00

Apache Performance Tuning

General

RAM

The single biggest issue affecting webserver performance is RAM. Have as much RAM as your hardware, OS, and funds allow [within reason].

The more RAM your system has, the more processes [and threads] Apache can allocate and use; which directly translates into the amount of concurrent requests/clients Apache can serve.

Generally speaking, disk I/O is usually a close 2nd, followed by CPU speed and network link. Note that a single PII 400 Mhz with 128-256 Megs of RAM can saturate a T3 (45 Mbps) line.

Select MPM

Chose the right MPM for the right job:

prefork [default MPM for Apache 2.0 and 1.3]:

Apache 1.3-based.
Multiple processes, 1 thread per process, processes handle requests.
Used for security and stability.
Has higher memory consumption and lower performance over the newer Apache 2.0-based threaded MPMs.

worker:

Apache 2.0-based.
Multiple processes, many threads per process, threads handle requests.
Used for lower memory consumption and higher performance.
Does not provide the same level of isolation request-to-request, as a process-based MPM does.

winnt:

The only MPM choice under Windows.
1 parent process, exactly 1 child process with many threads, threads handle requests.
Best solution under Windows, as on this platform, threads are always "cheaper" to use over processes.

Configure MPM

Core Features and Multi-Processing Modules

Default Configuration


StartServers            8
MinSpareServers         5
MaxSpareServers        20
MaxClients            150
MaxRequestsPerChild  1000



StartServers            2
MaxClients            150
MinSpareThreads        25
MaxSpareThreads        75
ThreadsPerChild        25
MaxRequestsPerChild     0



ThreadsPerChild       250
MaxRequestsPerChild     0

Directives

MaxClients, for prefork MPM

MaxClients sets a limit on the number of simultaneous connections/requests that will be served.

I consider this directive to be the critical factor to a well functioning server. Set this number too low and resources will go to waste. Set this number too high and an influx of connections will bring the server to a stand still. Set this number just right and your server will fully utilize the available resources.

An approximation of this number should be derived by dividing the amount of system memory (physical RAM) available by the maximum size of an apache/httpd process; with a generous amount spared for all other processes.

MaxClients ≈ (RAM - size_all_other_processes)/(size_apache_process)

Use 'ps -ylC httpd --sort:rss' to find process size. Divide number by 1024 to get megabytes. Also try 'top'.

Use 'free -m' for a general overview. The key figure to look at is the buffers/cache used value.

Use 'vmstat 2 5' to display the number of runnable, blocked, and waiting processes; and swap in and swap out.

Example:

System: VPS (Virtual Private Server), CentOS 4.4, with 128MB RAM
Apache: v2.0, mpm_prefork, mod_php, mod_rewrite, mod_ssl, and other modules
Other Services: MySQL, Bind, SendMail
Reported System Memory: 120MB
Reported httpd process size: 7-13MB
Assumed memory available to Apache: 90MB

Optimal settings:

StartServers 5
MinSpareServers 5
MaxSpareServers 10
ServerLimit 15
MaxClients 15
MaxRequestsPerChild 2000

With the above configuration, we start with 5-10 processes and set a top limit of 15. Anything above this number will cause serious swapping and thrashing under a load; due to the low amount of RAM available to the [virtual] Server. With a dedicated Server, the default values [ServerLimit 256] will work with 1-2GB of RAM.

When calculating MaxClients, take into consideration that the reported size of a process and the effective size are two different values. In this setup, it might be safe to use 20 or more workers... Play with different values and check your system stats.

Note that when more connections are attempted than there are workers, the connections are placed into a queue. The default queue size value is 511 and can be adjusted with the ListenBackLog directive.

ThreadsPerChild, for winnt MPM

On the Windows side, the only useful directive is ThreadsPerChild, which is usually set to a value of 250 [defaults to 64 without a value]. If you expect more, or less, concurrent connections/requests, set this directive appropriately. Check process size with Task Manager, under different values and server load.

MaxRequestsPerChild

Directive MaxRequestsPerChild is used to recycle processes. When this directive is set to 0, an unlimited amount of requests are allowed per process.

While some might argue that this increases server performance by not burdening Apache with having to destroy and create new processes, there is the other side to the argument...

Setting this value to the amount of requests that a website generates per day, divided by the number of processes, will have the benefit of keeping memory leaks and process bloat to a minimum [both of which are a common problem]. The goal here is to recycle each process once per day, as apache threads gradually increase their memory allocation as they run.

Note that under the winnt MPM model, recycling the only request serving process that Apache contains, can present a problem for some sites with constant and heavy traffic.

Requests vs. Client Connections

On any given connection, to load a page, a client may request many URLs: page, site css files, javascript files, image files, etc.

Multiple requests from one client in rapid succession can have the same effect on a Server as "concurrent" connections [threaded MPMs and directive KeepAlive taken into consideration]. If a particular website requires 10 requests per page, 10 concurrent clients will require MPM settings that are geared more towards 20-70 clients. This issue manifests itself most under a process-based MPM [prefork].

Separate Static and Dynamic Content

Use separate servers for static and dynamic content. Apache processes serving dynamic content will carry overhead and swell to the size of the content being served, never decreasing in size. Each process will incur the size of any loaded PHP or Perl libraries. A 6MB-30MB process size [or 10% of server's memory] is not unusual, and becomes a waist of resources for serving static content.

For a more efficient use of system memory, either use mod_proxy to pass specific requests onto another Apache Server, or use a lightweight server to handle static requests:

lighttpd [has experimental win32 builds]
tux [patched into RedHat, runs inside the Linux kernel and is at the top of the charts in performance]

The Server handling the static content goes up front.

Note that configuration settings will be quite different between a dynamic content Server and a static content Server.

mod_deflate

Reduce bandwidth by 75% and improve response time by using mod_deflate.

 LoadModule deflate_module modules/mod_deflate.so

AddOutputFilterByType DEFLATE text/html text/plain text/css text/xml application/x-javascript

Loaded Modules

Reduce memory footprint by loading only the required modules.

Some also advise to statically compile in the needed modules, over building DSOs (Dynamic Shared Objects). Very bad advice. You will need to manually rebuild Apache every time a new version or security advisory for a module is put out, creating more work, more build related headaches, and more downtime.

mod_expires

Include mod_expires for the ability to set expiration dates for specific content; utilizing the 'If-Modified-Since' header cache control sent by the user's browser/proxy. Will save bandwidth and drastically speed up your site for [repeat] visitors.

Note that this can also be implemented with mod_headers.

KeepAlive

Enable HTTP persistent connections to improve latency times and reduce server load significantly [25% of original load is not uncommon].

prefork MPM:

 KeepAlive On
KeepAliveTimeout 2
MaxKeepAliveRequests 80

worker and winnt MPMs:

 KeepAlive On
KeepAliveTimeout 15
MaxKeepAliveRequests 80

With the prefork MPM, it is recommended to set 'KeepAlive' to 'Off'. Otherwise, a client will tie up an entire process for that span of time. Though in my experience, it is more useful to simply set the 'KeepAliveTimeout' value to something very low [2 seconds seems to be the ideal value]. This is not a problem with the worker MPM [thread-based], or under Windows [which only has the thread-based winnt MPM].

With the worker and winnt MPMs, the default 15 second timeout is setup to keep the connection open for the next page request; to better handle a client going from link to link. Check logs to see how long a client remains on each page before moving on to another link. Set value appropriately [do not set higher than 60 seconds].

SymLinks

Make sure 'Options +FollowSymLinks -SymLinksIfOwnerMatch' is set for all directories. Otherwise, Apache will issue an extra system call per filename component to substantiate that the filename is NOT a symlink; and more system calls to match an owner.

 
Options FollowSymLinks

AllowOverride

Set a default 'AllowOverride None' for your filesystem. Otherwise, for a given URL to path translation, Apache will attempt to detect an .htaccess file under every directory level of the given path.

 
AllowOverride None

ExtendedStatus

If mod_status is included, make sure that directive 'ExtendedStatus' is set to 'Off'. Otherwise, Apache will issue several extra time-related system calls on every request made.

ExtendedStatus Off

Timeout

Lower the amount of time the server will wait before failing a request.

Timeout 45

Other/Specific

Cache all PHP pages, using Squid, and/or a PHP Accelerator and Encoder application, such as APC. Also take a look at mod_cache under Apache 2.2.

Convert/pre-render all PHP pages that do not change request-to-request, to static HTML pages. Use 'wget' or 'HTTrack' to crawl your site and perform this task automatically.

Pre-compress content and pre-generate headers for static pages; send-as-is using mod_asis. Can use 'wget' or 'HTTrack' for this task. Make sure to set zlib Compression Level to a high value (6-9). This will take a considerable amount of load off the server.

Use output buffering under PHP to generate output and serve requests without pauses.

Avoid content negotiation for faster response times.

Make sure log files are being rotated. Apache will not handle large (2gb+) files very well.

Gain a significant performance improvement by using SSL session cache.

Outsource your images to Amazon's Simple Storage Service (S3).

Measuring Web Server Performance

Load Testing

Apache HTTP server benchmarking tool
httperf
The Grinder, a Java Load Testing Framework

Benchmarks

I have searched extensively for Apache, lighttpd, tux, and other webserver benchmarks. Sadly, just about every single benchmark I could locate appeared to have been performed completely without thought, or with great bias.

Do not trust any posted benchmarks, especially ones done with the 'ab' tool.

The only way to get a valid report is to perform the benchmark yourself.

For valid results, note to test under a system with limited resources, and maximum resources. But most importantly, configure each httpd server application for the specific situation.

DAG repo to yum

2009-06-27T20:54:00.001-07:00

You can make yum more robust by adding more repositories like DAG, UPDATE and RPMforge. For adding extra repositories to yum, please do the following.

cd /etc/yum.repos.d
vi dag.repo // the add the following lines in that file//

[dag]
name=Dag RPM Repository for Red Hat Enterprise Linux
baseurl=http://apt.sw.be/redhat/el$releasever/en/$basearch/dag
gpgcheck=1
rpm --import http://dag.wieers.com/rpm/packages/RPM-GPG-KEY.dag.txt

After this save this file and run the following command

yum check-update

Now yum will be having more repositories.

MySQL Tweak[core level]

2009-06-18T21:31:00.000-07:00

A my.cnf values run on a dual xeon with 2 GB's of ram, this is a shared hosting machine that runs MySQL and web, so all memory is not allocated to MySQL.
------------------------------------------------
/etc/my.cnf

datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
skip-locking
skip-innodb
query_cache_limit=1M
query_cache_size=32M
query_cache_type=1
max_connections=900
interactive_timeout=100
wait_timeout=100
connect_timeout=10
thread_cache_size=128
#key_buffer=16M
key_buffer=200M
join_buffer=1M
max_allowed_packet=16M
table_cache=1536
sort_buffer_size=1M
read_buffer_size=1M
read_rnd_buffer_size=1M
max_connect_errors=10
# Try number of CPU's*2 for thread_concurrency
thread_concurrency=4
myisam_sort_buffer_size=64M
#log-bin
server-id=1

Query caching was added as of MySQL version 4, the following three directives will greatly enhance mysql server performance.

query_cache_limit=1M
query_cache_size=32M
query_cache_type=1

Query caching is a server wide variable, so set these generous. I have found the above levels are generally best if you server has at least 512 ram. If you run a server just for DBs with a lot of ram, you can up these quite a bit, like 2m limit and a 64+M cache size.

The key buffer is a variable that is shared amongst all MySQL clients on the server. A large setting is recomended, particularly helpful with tables that have unique keys. (Most do)

key_buffer=150M

The next set of buffers are at a per client level. It is important to play around with these and get them just right for your machine. With the setting below, every active mysql client will have close to 3 MB's in buffers. So 100 clients = almost 300 MB. Giving too much to these buffers will be worse than giving too little. Nothing kills a server quite like memory swapping will.

sort_buffer_size=1M
read_buffer_size=1M
read_rnd_buffer_size=768K

The following directive should be set to 2X the number of processors in your machine for best performance.

thread_concurrency=2

Heres a few example configurations for servers running MySQL and web for common memory sizes. These are not perfect, but good starting points.

Server with 512MB RAM:

thread_cache_size=50
key_buffer=40M
table_cache=384
sort_buffer_size=768K
read_buffer_size=512K
read_rnd_buffer_size=512K
thread_concurrency=2

For servers with 1 GB ram:

thread_cache_size=80
key_buffer=150M
table_cache=512
sort_buffer_size=1M
read_buffer_size=1M
read_rnd_buffer_size=768K
thread_concurrency=2

########################################################

For optimizing mysql, first we need to know the values of mysql variables and status.
The following are some commands used for this purpose:
# mysqladmin processlist extended-status

or

mysql> show status;
mysql> show variables;

To get more specific answer, the commands can be enhanced a little more like as follows:

mysql> show status like '%Open%_tables';

mysql> show variables like 'table_cache';

1. The most important variables in mysql are table_cache and key_buffer_size

a) Run the above two commands and check Open_tables and Opened_tables
If Opened_tables is big, then your table_cache variable is probably
too small.

So increase the table_cache variable. Open /etc/my.cnf and change/add table_cache=newvalue

b) Run the following commands to check key_buffer_size, key_read_requests and key_reads

mysql> show variables like '%key_buffer_size%';
mysql> show status like '%key_read%';

If key_reads / key_read_requests is < 0.01, key_buffer_size is enough. Otherwise key_buffer_size should be increased.

Also run the following command to check key_write_requests and key_writes

mysql> show status like '%key_write%';

If key_writes / key_write_requests is not less than 1 (near 0.5 seems to be fine), increase key_buffer_size.

Check the total size of all .MYI files. If it is larger than key_buffer_size change key_buffer_size to total size of MYI files.

2. Wait_timeout, max_connection, thread_cache

If you want to allow more connections, reduce wait_timeout to 15 seconds and increase max_connection as you want.

Check the number of idle connections. If it is too high reduce the wait_timeout and use Thread_cache

How many threads we should keep in a cache for reuse. When a client disconnects, the client's threads are put in the cache if there aren't more than thread_cache_size threads from before. All new threads are first taken from the cache, and only when the cache is empty is a new thread created. This variable can be increased to improve performance if you have a lot of new connections. (Normally this doesn't give a notable performance improvement if you have a good thread implementation.) By examing the difference between the Connections and Threads_created you can see how efficient the current thread cache is for you.

If Threads_created is big, you may want to increase the
thread_cache_size variable. The cache hit rate can be calculated with
Threads_created/Connections.
Default thread_cache_size may be 0 if so increase it to 8.
You may try this formula : table_cache = opened table / max_used_connection

Hub, Switches, and Routers

2009-06-15T23:58:00.001-07:00

Hub, Switches, and Routers
---------------------------

Hub
A common connection point for devices in a network. Hubs are commonly used to connect segments of a LAN. A hub contains multiple ports. When a packet arrives at one port, it is copied to the other ports so that all segments of the LAN can see all packets.

Switch
In networks, a device that filters and forwards packets between LAN segments. Switches operate at the data link layer (layer 2) and sometimes the network layer (layer 3) of the OSI Reference Model and therefore support any packet protocol. LANs that use switches to join segments are called switched LANs or, in the case of Ethernet networks, switched Ethernet LANs.

Router
A device that forwards data packets along networks. A router is connected to at least two networks, commonly two LANs or WANs or a LAN and its ISP.s network. Routers are located at gateways, the places where two or more networks connect. Routers use headers and forwarding tables to determine the best path for forwarding the packets, and they use protocols such as ICMP to communicate with each other and configure the best route between any two hosts.

The Differences Between These Devices on the Network
Today most routers have become something of a Swiss Army knife, combining the features and functionality of a router and switch/hub into a single unit. So conversations regarding these devices can be a bit misleading — especially to someone new to computer networking.

The functions of a router, hub and a switch are all quite different from one another, even if at times they are all integrated into a single device. Let's start with the hub and the switch since these two devices have similar roles on the network. Each serves as a central connection for all of your network equipment and handles a data type known as frames. Frames carry your data. When a frame is received, it is amplified and then transmitted on to the port of the destination PC. The big difference between these two devices is in the method in which frames are being delivered.

In a hub, a frame is passed along or "broadcast" to every one of its ports. It doesn't matter that the frame is only destined for one port. The hub has no way of distinguishing which port a frame should be sent to. Passing it along to every port ensures that it will reach its intended destination. This places a lot of traffic on the network and can lead to poor network response times.

Additionally, a 10/100Mbps hub must share its bandwidth with each and every one of its ports. So when only one PC is broadcasting, it will have access to the maximum available bandwidth. If, however, multiple PCs are broadcasting, then that bandwidth will need to be divided among all of those systems, which will degrade performance.

A switch, however, keeps a record of the MAC addresses of all the devices connected to it. With this information, a switch can identify which system is sitting on which port. So when a frame is received, it knows exactly which port to send it to, without significantly increasing network response times. And, unlike a hub, a 10/100Mbps switch will allocate a full 10/100Mbps to each of its ports. So regardless of the number of PCs transmitting, users will always have access to the maximum amount of bandwidth. It's for these reasons why a switch is considered to be a much better choice then a hub.

Routers are completely different devices. Where a hub or switch is concerned with transmitting frames, a router's job, as its name implies, is to route packets to other networks until that packet ultimately reaches its destination. One of the key features of a packet is that it not only contains data, but the destination address of where it's going.

A router is typically connected to at least two networks, commonly two Local Area Networks (LANs) or Wide Area Networks (WAN) or a LAN and its ISP's network . for example, your PC or workgroup and EarthLink. Routers are located at gateways, the places where two or more networks connect. Using headers and forwarding tables, routers determine the best path for forwarding the packets. Router use protocols such as ICMP to communicate with each other and configure the best route between any two hosts.

Today, a wide variety of services are integrated into most broadband routers. A router will typically include a 4 - 8 port Ethernet switch (or hub) and a Network Address Translator (NAT). In addition, they usually include a Dynamic Host Configuration Protocol (DHCP) server, Domain Name Service (DNS) proxy server and a hardware firewall to protect the LAN from malicious intrusion from the Internet.

All routers have a WAN Port that connects to a DSL or cable modem for broadband Internet service and the integrated switch allows users to easily create a LAN. This allows all the PCs on the LAN to have access to the Internet and Windows file and printer sharing services.

Some routers have a single WAN port and a single LAN port and are designed to connect an existing LAN hub or switch to a WAN. Ethernet switches and hubs can be connected to a router with multiple PC ports to expand a LAN. Depending on the capabilities (kinds of available ports) of the router and the switches or hubs, the connection between the router and switches/hubs may require either straight-thru or crossover (null-modem) cables. Some routers even have USB ports, and more commonly, wireless access points built into them.

Some of the more high-end or business class routers will also incorporate a serial port that can be connected to an external dial-up modem, which is useful as a backup in the event that the primary broadband connection goes down, as well as a built in LAN printer server and printer port.

Besides the inherent protection features provided by the NAT, many routers will also have a built-in, configurable, hardware-based firewall. Firewall capabilities can range from the very basic to quite sophisticated devices. Among the capabilities found on leading routers are those that permit configuring TCP/UDP ports for games, chat services, and the like, on the LAN behind the firewall.

So, in short, a hub glues together an Ethernet network segment, a switch can connect multiple Ethernet segments more efficiently and a router can do those functions plus route TCP/IP packets between multiple LANs and/or WANs; and much more of course.

DNS(Domain Name Service)

2009-06-15T23:50:00.000-07:00

Domain Name Service
Host Names

Domain Name Service (DNS) is the service used to convert human readable names of hosts to IP addresses. Host names are not case sensitive and can contain alphabetic or numeric letters or the hyphen. Avoid the underscore. A fully qualified domain name (FQDN) consists of the host name plus domain name as in the following example:

computername.domain.com

The part of the system sending the queries is called the resolver and is the client side of the configuration. The nameserver answers the queries. Read RFCs 1034 and 1035. These contain the bulk of the DNS information and are superceded by RFCs 1535-1537. Naming is in RFC 1591. The main function of DNS is the mapping of IP addresses to human readable names.

Three main components of DNS

1. resolver
2. name server
3. database of resource records(RRs)

Domain Name System

The Domain Name System (DNS) is basically a large database which resides on various computers and it contains the names and IP addresses of various hosts on the internet and various domains. The Domain Name System is used to provide information to the Domain Name Service to use when queries are made. The service is the act of querying the database, and the system is the data structure and data itself. The Domain Name System is similar to a file system in Unix or DOS starting with a root. Branches attach to the root to create a huge set of paths. Each branch in the DNS is called a label. Each label can be 63 characters long, but most are less. Each text word between the dots can be 63 characters in length, with the total domain name (all the labels) limited to 255 bytes in overall length. The domain name system database is divided into sections called zones. The name servers in their respective zones are responsible for answering queries for their zones. A zone is a subtree of DNS and is administered separately. There are multiple name servers for a zone. There is usually one primary nameserver and one or more secondary name servers. A name server may be authoritative for more than one zone.

DNS names are assigned through the Internet Registries by the Internet Assigned Number Authority (IANA). The domain name is a name assigned to an internet domain. For example, mycollege.edu represents the domain name of an educational institution. The names microsoft.com and 3Com.com represent the domain names at those commercial companies. Naming hosts within the domain is up to individuals administer their domain.

Access to the Domain name database is through a resolver which may be a program or part of an operating system that resides on users workstations. In Unix the resolver is accessed by using the library functions "gethostbyname" and "gethostbyaddr". The resolver will send requests to the name servers to return information requested by the user. The requesting computer tries to connect to the name server using its IP address rather than the name.

Structure and message format

The drawing below shows a partial DNS hierarchy. At the top is what is called the root and it is the start of all other branches in the DNS tree. It is designated with a period. Each branch moves down from level to level. When referring to DNS addresses, they are referred to from the bottom up with the root designator (period) at the far right. Example: "myhost.mycompany.com.".

Partial DNS Hierarchy

DNS is hierarchical in structure. A domain is a subtree of the domain name space. From the root, the assigned top-level domains in the U.S. are:

* GOV - Government body.
* EDU - Educational body.
* INT - International organization
* NET - Networks
* COM - Commercial entity.
* MIL - U. S. Military.
* ORG - Any other organization not previously listed.

Outside this list are top level domains for various countries.

Each node on the domain name system is separated by a ".". Example: "mymachine.mycompany.com.". Note that any name ending in a "." is an absolute domain name since it goes back to root.
DNS Message format:

Bits Name Description
0-15 Identification Used to match responses to requests. Set by client and returned by server.
16-31 Flags Tells if query or response, type of query, if authoritative answer, if truncated, if recursion desired, and if recursion is available.
32-47 Number of questions
48-63 Number of answer RRs
64-79 Number of authority RRs
80-95 Number of additional RRs
96-?? Questions - variable lengths There can be variable numbers of questions sent.
??-?? Answers - variable lengths Answers are variable numbers of resource records.
??-?? Authority - variable lengths
??-?? Additional Information - variable lengths

Question format includes query name, query type and query class. The query name is the name being looked up. The query class is normally 1 for internet address. The query types are listed in the table below. They include NS, CNAME, A, etc.

The answers, authority and additional information are in resource record (RR) format which contains the following.

1. Domain name
2. Type - One of the RR codes listed below.
3. Class - Normally indicates internet data which is a 1.
4. Time to live field - The number of seconds the RR is saved by the client.
5. Resource data length specifies the amount of data. The data is dependent on its type such as CNAME, A, NS or others as shown in the table below. If the type is "A" the data is a 4 byte IP address.

The table below shows resource record types:

Type RR value Description
A 1 Host's IP address
NS 2 Host's or domain's name server(s)
CNAME 5 Host's canonical name, host identified by an alias domain name
PTR 12 Host's domain name, host identified by its IP address
HINFO 13 Host information
MX 15 Host's or domain's mail exchanger
AXFR 252 Request for zone transfer
ANY 255 Request for all records
Usage and file formats

If a domain name is not found when a query is made, the server may search for the name elsewhere and return the information to the requesting workstation, or return the address of a name server that the workstation can query to get more information. There are special servers on the Internet that provide guidance to all name servers. These are known as root name servers. They do not contain all information about every host on the Internet, but they do provide direction as to where domains are located (the IP address of the name server for the uppermost domain a server is requesting). The root name server is the starting point to find any domain on the Internet.
Name Server Types

There are three types of name servers:

1. The primary master builds its database from files that were preconfigured on its hosts, called zone or database files. The name server reads these files and builds a database for the zone it is authoritative for.
2. Secondary masters can provide information to resolvers just like the primary masters, but they get their information from the primary. Any updates to the database are provided by the primary.
3. Caching name server - It gets all its answers to queries from other name servers and saves (caches) the answers. It is a non-authoritative server.

The caching only name server generates no zone transfer traffic. A DNS Server that can communicate outside of the private network to resolve a DNS name query is referred to as forwarder.
DNS Query Types

There are two types of queries issued:

1. Recursive queries received by a server forces that server to find the information requested or post a message back to the querier that the information cannot be found.
2. Iterative queries allow the server to search for the information and pass back the best information it knows about. This is the type that is used between servers. Clients used the recursive query.
3. Reverse - The client provides the IP address and asks for the name. In other queries the name is provided, and the IP address is returned to the client. Reverse lookup entries for a network 192.168.100.0 is "100.168.192.in-addr arpa".

Generally (but not always), a server-to-server query is iterative and a client-resolver-to-server query is recursive. You should also note that a server can be queried or it can be the person placing a query. Therefore, a server contains both the server and client functions. A server can transmit either type of query. If it is handed a recursive query from a remote source, it must transmit other queries to find the specified name, or send a message back to the originator of the query that the name could not be found.
DNS Transport protocol

DNS resolvers first attempt to use UDP for transport, then use TCP if UDP fails.
The DNS Database

A database is made up of records and the DNS is a database. Therefore, common resource record types in the DNS database are:

* A - Host's IP address. Address record allowing a computer name to be translated into an IP address. Each computer must have this record for its IP address to be located. These names are not assigned for clients that have dynamically assigned IP addresses, but are a must for locating servers with static IP addresses.
* PTR - Host’s domain name, host identified by its IP address
* CNAME - Host’s canonical name allows additional names or aliases to be used to locate a computer.
* MX - Host’s or domain’s mail exchanger.
* NS - Host’s or domain’s name server(s).
* SOA - Indicates authority for the domain
* TXT - Generic text record
* SRV - Service location record
* RP - Responsible person
* HINFO - Host information record with CPU type and operating system.

When a resolver requests information from the server, the DNS query message indicates one of the preceding types.
DNS Files

* CACHE.DNS - The DNS Cache file. This file is used to resolve internet DNS queries. On Windows systems, it is located in the WINNTROOT\system32\DNS directory and is used to configure a DNS server to use a DNS server on the internet to resolve names not in the local domain.

Example Files

Below is a partial explanation of some records in the database on a Linux based system. The reader should view this information because it explains some important DNS settings that are common to all DNS servers. An example /var/named/db.mycompany.com.hosts file is listed below.

mycompany.com. IN SOA mymachine.mycompany.com. root.mymachine.mycompany.com. (
1999112701 ; Serial number as date and two digit number YYMMDDXX
10800 ; Refresh in seconds 28800=8H
3600 ; Retry in seconds 7200=2H
604800 ; Expire 3600000=1 week
86400 ) ; Minimum TTL 86400=24Hours
mycompany.com. IN NS mymachine.mycompany.com.
mycompany.com. IN MX 10 mailmachine.mycompany.com.
mymachine.mycompany.com. IN A 10.1.0.100
mailmachine.mycompany.com. IN A 10.1.0.4
george.mycompany.com. IN A 10.1.3.16

A Line by line description is as follows:

1. The entries on this line are:
1. mycompany.com. - Indicates this server is for the domain mycompany.com.
2. IN - Indicates Internet Name.
3. SOA - Indicates this server is the authority for its domain, mycompany.com.
4. mymachine.mycompany.com. - The primary nameserver for this domain.
5. root.mymachine.mycompany.com. - The person to contact for more information.
The lines in the parenthesis, listed below, are for the secondary nameserver(s) which run as slave(s) to this one (since it is the master).
2. 1999112701 - Serial number - If less than master's SN, the slave will get a new copy of this file from the master.
3. 10800 - Refresh - The time in seconds between when the slave compares this file's SN with the master.
4. 3600 - Retry - The time the server should wait before asking again if the master fails to respond to a file update (SOA request).
5. 604800 - Expire - Time in seconds the slave server can respond even though it cannot get an updated zone file.
6. 86400 - TTL - The time to live (TTL) in seconds that a resolver will use data received from a nameserver before it will ask for the same data again.
7. This line is the nameserver resource record. There may be several of these if there are slave name servers.

mycompany.com. IN NS mymachine.mycompany.com.

Add any slave server entries below this like:

mycompany.com. IN NS ournamesv1.mycompany.com.
mycompany.com. IN NS ournamesv2.mycompany.com.
mycompany.com. IN NS ournamesv3.mycompany.com.

8. This line indicates the mailserver record.

mycompany.com. IN MX 10 mailmachine.mycompany.com.

There can be several mailservers. The numeric value on the line indicates the preference or precedence for the use of that mail server. A lower number indicates a higher preference. The range of values is from 0 to 65535. To enter more mailservers, enter a new line for each one similar to the nameserver entries above, but be sure to set the preferences value correctly, at different values for each mailserver.
9. The rest of the lines are the name to IP mappings for the machines in the organization. Note that the nameserver and mailserver are listed here with IP addresses along with any other server machines required for your network.

mymachine.mycompany.com. IN A 10.1.0.100
mailmachine.mycompany.com. IN A 10.1.0.4
george.mycompany.com. IN A 10.1.3.16

Domain names written with a dot on the end are absolute names which specify a domain name exactly as it exists in the DNS hierarchy from the root. Names not ending with a dot may be a subdomain to some other domain.

Aliases are specified in lines like the following:

mymachine.mycompany.com IN CNAME nameserver.mycompany.com.
george.mycompany.com IN CNAME dataserver.mycompany.com.
Linux1.mycompany.com IN CNAME engserver.mycompany.com.
Linux2.mycompany.com IN CNAME mailserver.mycompany.com.

When a client (resolver) sends a request, if the nameserver finds a CNAME record, it replaces the requested name with the CNAME, then finds the address of the CNAME value, and return this value to the client.

A host that has more than one network card which is set to address two different subnets can have more than one address for a name.

mymachine.mycompany.com IN A 10.1.0.100
IN A 10.1.1.100

When a client queries the nameserver for the address of a multi homed host, the nameserver will return the address that is closest to the client address. If the client is on a different network than both the subnet addresses of the multi homed host, the server will return both addresses.

For more information on practical application of DNS, read the DNS section of the Linux User's Guide.

ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: NO)

2009-06-14T22:37:00.000-07:00

[root@sylesh ~]# mysql -u root
ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: NO)

>>disabling password authentication
service mysql stop

wait until MySQL shuts down. Then run

mysqld_safe --skip-grant-tables &

then you will be able to login as root with no password.

mysql -uroot mysql

In MySQL command line prompt issue the following command:
use databasename;

UPDATE user SET password=PASSWORD("abcd") WHERE user="root";
FLUSH PRIVILEGES;
EXIT

/etc/init.d/mysqld restart

At this time your root password is reset to "abcd" and MySQL will now
know the privileges and you'll be able to login with your new password:

mysql -uroot -p mysql

How to enable SSI On Your Server with .htaccess and XBitHack apache directive

2009-06-12T14:29:00.000-07:00

>>>

The below notes will demonstrates how to enable SSI on your server using .htaccess.

If you are paying for hosting services you may need to get permission from your host to make sure you are not violating their Terms of Service which could result in you getting the boot! Every decent host supports SSI but double-check to make sure.

To enable SSI either create a file simple called .htaccess or edit your existing .htaccess file and place the following code in it:

AddType text/html .shtml
AddHandler server-parsed .shtml
Options Indexes FollowSymLinks Includes

Note: to enable SSI for your full web site place the .htaccess in the root directory of your site; to enable it for just a certain directory place the .htaccess file only in that particular directory.

The first line of the code above tells the server that .shtml is a valid extension. The second line adds a handler to all pages with the .shtml extension which tells the server to parse (process) the document for server side includes.

If you prefer you can use a different file extension for your files which you want parsed for server side includes. Simply change the .shtml to .shtm etc. If you also want your .htm documents parsed by the server (so you don't need to rename all your files) simply add the following after the first line of the code above:

AddHandler server-parsed .htm

If you want to use SSI in your default directory page, such as index.shtml you may (but normally won't) need to add the following to the .htaccess file:

DirectoryIndex index.shtml index.htm

This means that index.shtml can be your default page. If this page is not found the server will look for index.htm etc. More on this in the .htaccess guides section.

>>SSI Without .shtml

In order to understand what this use of htaccess can do for you, you have to understand what SSI directives are. (SSI directives are covered in the How To Use Your CGI-BIN page.) You can put an SSI directive tag in your Web page, but that doesn't mean the server will look for it. Looking through an html file for SSI directives is called "parsing", and by default a server doesn't parse every html file. It only parses pages that have a .shtml extension.

Dilemma:

You want to start using SSI directives in your Web pages to call a script or display certain things on the pages. Your host requires that pages with SSI directives have a .shtml extension. However, over time all of your pages have been linked to and indexed by search engines using their current .html extensions. If you change the extensions to comply with your host, a lot of people will start getting 404 errors.

htaccess to the rescue! Certain htaccess statements allow you to tell the server to parse certain pages that don't have a .shtml extension.

If you created the htaccess.txt file above, simply add the statements given below to it and re-ftp/rename it. If you didn't, here are the steps:

1. Use a text editor to create an htaccess.txt file and enter the following statements into it:

AddType text/html .html
AddHandler server-parsed .html

replacing .html with .htm if that's what you are using for your pages.

2. Save the file and ftp it (using ASCII mode) to your Web root directory (or whatever directory your index.html file is in).

3. Rename the htaccess.txt file on the server to .htaccess

4. Try it out by entering a URL for one of the pages that contains an SSI directive and see if it's working.

The above can be thought of as the "directory method" method for enabling SSI parsing because all files in the directory with the specified extension will be parsed, including files in any sub-directories. SSI parsing does have a small performance price due to all this parsing. If your site has a lot of traffic and a lot of pages that performance price could add up. What if you have a lot of traffic and a lot of pages but you only have a few files that you want parsed? Then you'd want to use XBitHack which is covered in the next section.

Not all hosts allow you to use a .htaccess file. They have to use an AllowOverride statement in one of the global configuration files. Ask your host, or a potential host, if they allow the use of .htaccess files. If so, also ask if they allow the use of XbitHack. If they so 'No' to the question of htaccess, pleading with them to enable it on your server may work, especially if you sound like you know what you're talking about (which this page will help you to do).

A .htaccess file is a very powerful tool. You can use it to set up password-protected directories, change the way Apache responds to certain events, etc. The flip side of that is that you can really hose things up or give unintended access to visitors if you're not careful. You may want to try out your attempts with .htaccess during low-traffic times on your Website so that any problems can be corrected without affecting too many visitors.

Note also that the very fact that this is a very powerful tool may be reason enough for some hosting services not to allow you to use it. A hosting service sets up multiple "virtual" Web servers so multiple domains can be hosting on a single system (each domain having it's own virtual Web server). They do this by adding statements (aka directives) to the main Apache configuration file (named httpd.conf). When they add these virtual server directives they must include the directive to enable htaccess functionality. If you try the above and it doesn't work, chances are good your host doesn't have the htaccess function enabled.

What is XBitHack
----------------

XBitHack (pronounced "X bit hack") is simply one of those htaccess configuration statements mentioned above. If you're not willing to put up with the performance costs of the "directory method" for enabling parsing of non-.shtml pages covered above, think of XBitHack as a "file method". This is because you can specify on a file-by-file basis which non-.shtml files get parsed.

Using XBitHack for this "file method" has two steps:

* turn on XBitHack by adding the statement to your .htaccess file
* "flag" the html pages you want parsed by changing their permissions to something a little out of the ordinary

If you created the htaccess.txt file above, simply add the statement given below to it and re-ftp/rename it to enable XBitHack. If your .htaccess file contains the AddType and AddHandler statements from above, REMOVE THEM. If you didn't create the file earlier, here are the steps to enabling XBitHack:

1. Use a text editor to create an htaccess.txt file and enter the following statement into it:

XBitHack on

2. Save the file and ftp it (using ASCII mode) to your Web root directory (or whatever directory your index.html file is in).

3. Rename the htaccess.txt file to .htaccess

4. CHMOD the page files, and only the page files, that you want parsed (i.e. that will contain SSI directives) to 744 (instead of 644). This is what tells the server to parse the page.

5. Try it out by entering a URL for one of the pages that contains an SSI directive and see if it's working.

If it doesn't work, check your error log for a message like

XBitHack not allowed here

It is possible that your host allows htaccess but not XBitHack. If you don't find the above error, you'll have to contact your host's technical support operation. However, by knowing what htaccess and XBitHack are, you can ask them intelligent questions regarding your problem. When they realize you know what you are talking about, they will be less likely to feed you a line of BS. Also, don't be surprised if the support person you speak to doesn't know what you are talking about. First-line technical support and sales people are usually entry-level jobs in an organization. If you get the sense they don't know what you are talking about, ask to speak to a more senior support person who does.

dbmmanage - Manage user authentication files in DBM format(apache binary)

2009-06-10T07:50:00.000-07:00

DBM User Authentication

This week, we explain how to store user authentication information in DBM files for faster access when you have thousands of users.

The feature on User Authentication shows how to restrict pages to selected people. We showed how to use the htpasswd program to create the necessary .htpasswd files, and how to create group files to provide more control over the users. We also said that .htpasswd files and group files like this are not very efficient when a large number of users are involved. This is because these are plain text files and for every request in the authenticated area Apache has to read through the file looking for the user. A much faster way to store the user information is to use files in DBM format. This article explains how to create and manage DBM format user authentication files.

What is DBM?

DBM files are a simple and relatively standard method of storing information for quick retrieval. Each item of information stored in a DBM file consists of two parts: a key and a value. If you know the key you can access the value very quickly. The DBM file maintains an 'index' of the keys, each of which points to where the value is stored within the file, and the index is usually arranged such that values can be accessed with the minimum number of file system accesses even for very large numbers of keys.

In practice, on many systems a DBM 'file' is actually stored in two files on the disk. If, for example, a DBM file called 'users' is created, it will actually be stored in files called users.pag and users.dir. If you ever need to rename or delete a DBM from the command line, remember to change both the files, keeping the extensions (.pag and .dir) the same. Some newer versions of DBM only create one file.

Provided the key is known in advance DBM format files are a very efficient way of accessing information associated with that key. For web user authentication, the key will be the username, and the value will store their (encrypted) password. Looking up usernames and their passwords in a DBM file will be more efficient than using a plain text file when more than a few users are involved. This will be particularly important for sites with lots of users (say, over 10,000) or where there are lots of accesses to authenticated pages.

Preparing Apache for DBM Files

If you want to use DBM format files with Apache, you will need to make sure it is compiled with DBM support. By default, Apache cannot use DBM files for user authentication, so the optional DBM authentication module needs to be included. Note that this is included in addition to the normal user authentication module (which uses plain text files, as explained in the previous article). It is possible to have support for multiple file formats compiled into Apache at the same time.

To add the DBM authentication module, edit your Configuration file in the Apache src directory. Remove the comment from the line which currently says

  # Module dbm_auth_module     mod_auth_dbm.o

To remove the comment, delete the # and space character at the right-hand end of the line. Now update the Apache configuration by running ./Configure, then re-make the executable with make.

However, before compiling you might also need to tell Apache where to find the DBM functions. On some systems this is automatic. On others you will need to add the text -lndbm or -ldbm to the EXTRA_LIBS line in the Configuration file. (Apache 1.2 will attempt to do this automatically if needed, but you might still need to configure it manually in some cases). If you are not sure what your system requires, try leaving it blank and compiling. If at the end of the compilation you see errors about functions such as _dbm_fetch() not being found, try each of these choices in turn. (Remember to re-run ./Configure after changing Configuration). If you still cannot get it to compile, you might have a system where the DBM library is installed in a non-standard directory, or where the there is no DBM library available. You could either contact you system administrator, or download and compile your own copy of the DBM libraries.

Creating A DBM Users File

For standard (htpasswd) user authentication password files, the program htpasswd is used to add new users and set their passwords. To create and manage DBM format user files another program from the Apache support directory is used. The program is called dbmmanage and is written in perl (so you will need perl on your system, and it will need to have been compiled with support for the same DBM library you compiled into Apache. If you have only just installed DBM on your system you will might need to re-compile perl to build in DBM support).

This program can be used to create a new DBM file, add users and passwords to it, change passwords, or delete users. To start by creating a new DBM file and adding a user to it, run the command:

  dbmmanage /usr/local/etc/httpd/usersdbm adduser martin hamster

The creates the DBM file /usr/local/etc/httpd/usersdbm (which might actually consist of /usr/local/etc/httpd/usersdbm.dir and /usr/local/etc/httpd/usersdbm.pag), if it does not already exist. It then adds the user 'martin' with password 'hamster'. This command can be used with other usernames and passwords to add more users, or with an existing username to change that user's password. A user can be deleted from the password file with

   dbmmanage /usr/local/etc/httpd/usersdbm delete martin

You can get a list of all the users in the DBM file with

   dbmmanage /usr/local/etc/httpd/usersdbm view

Restricting a Directory

Now you have a DBM user authentication file with some users in it, you are ready to create an authenticated area. You can restrict a directory either using a section in access.conf or by using a .htaccess file. The feature on user authentication explained how you can set up a basic .htaccess file, using this example:

  AuthName "restricted stuff"
 AuthType Basic
 AuthUserFile /usr/local/etc/httpd/users

 require valid-user

To use DBM files, the only change is to replace the directive AuthUserFile line with

  AuthDBMUserFile /usr/local/etc/httpd/usersdbm

This single change tells Apache that the user file is now in a DBM format, rather than plain text. All the rest of the user authentication setup remains the same (so the authentication type is still Basic, and the syntax of require is the same as before).

Using Groups

Each user can be in one or more "groups", and you can restrict access to people just in a specified group. This makes it possible to manage all your users on your site in a single database, and customise the areas that each can access. The use of DBM files for storing group information is particularly efficient because you can use the same file to store both password and group information.

The dbmmanage command can be used to set group information for users. For example, to add the user "martin" to the group "staff", you would use

  dbmmanage /usr/local/etc/httpd/users adduser martin hamster staff

You put a user into multiple groups but listing them, separated by commas. For example,

  dbmmanage /usr/local/etc/httpd/users adduser martin hamster staff,admin

Note that dbmmanage has to be told the password as well, and there is no way to set or change group information for a user without knowing their password. This means in practice that dbmmanage is not suitable for managing users in groups, and you will have to write your own management scripts. Some help writing perl to manage DBM files is given later in this article.

After creating a user and group file containing details of which users are in which groups, you can restrict access by these groups. For example, to restrict access to an area to only people in the group staff, you could use:

  AuthName "restricted stuff"
 AuthType Basic
 AuthDBMUserFile /usr/local/etc/httpd/users
 AuthDBMGroupFile /usr/local/etc/httpd/users

 require group staff

Custom Management of DBM Files

The supplied dbmmanage script to manage DBM files is adequate for basic editing, but cannot handle advanced use, such as managing group information. It is also command line driven, while a Web interface might be a better choice in many situations. To do either of these things you will have to write programs to manage DBM files yourself. Using perl this is not too difficult.

As a simple example, say you have an existing .htpasswd file and you want to convert it to a DBM file, putting all the users in a specific group. We will introduce the concepts here, and there is a link below to the completed program for you to download. It will be written in Perl which is quick to write and easy to customise, although the principles of DBM use are the same whatever language is used.

The basic way to look in a DBM file is given here. DBM files are opened in Perl as 'hashed arrays'. The "key" is the user name, and the value is the encrypted password and optionally group information. A simple script to lookup all the keys and values in a DBM is:

  dbmopen(%DBM, "/usr/local/etc/httpd", 0644) ||
                        die "Cannot open file: $!\n";
 while (($key, $value) = each %DBM) {
   print "key=$key, value=$value\n";
 }
 dbmclose(%DBM);

Note that if the given DBM file does not exist, it will be created. This script will work with both perl 4 and perl 5 (although Perl 5 users might prefer to use the new tie facility instead of dbmopen). To lookup a known key you would use:

  $key = "martin";

 dbmopen(%DBM, "/usr/local/etc/httpd", 0644) ||
                                       die "Cannot open file: $!\n";
 $value = $DBM{$key};
 if (!defined($value)) {
   print "$key not stored\n";
 } else {
   print "key=$key, value=$value\n";
 }
 dbmclose(%DBM);

Now we can write a script to convert a htpasswd file into a DBM database, optionally putting each user into one or more groups. The script is htpasswd2dbm.pl , and is used like this:

  cd /usr/local/etc/httpd
 htpasswd2dbm.pl -htpasswd users usersdbm

The -htpasswd option specifies the htpasswd file to be read, the the final argument is the DBM file to create (or add to). To set a group, use the -group argument. For example, to put all the users from this file into the groups admin and staff, use

  htpasswd2dbm.pl -htpasswd users -group admin,staff usersdbm

The program will add users to an existing DBM database, so it can be used to merge multiple htpasswd files. If you give users from different files different groups, you will be able to set up access restrictions on a group-by-group basis, and manage all your users in one database. Note that if there is already a user with the same username in the DBM file it will be overwritten by the new information.

Group information stored in a DBM file as part of the value. If no group information is stored, the key associated with a username just consists of the encrypted password. To store group information, the encrypted password is followed by a colon, then a list of groups that the user is in, each separated by a comma. So a typical key might look like this:

  E7yT67YGht65:admin,staff

A program written in perl can easily extract the group information, for example:

  $value = $DBM{$key};
 ($enc, $groupfield) = split(/:/, $value);
 @groups = split(/,/, $groupfield);

It is also possible to store additional information in the DBM file, by following the groups list with a colon. Apache will ignore any data after a colon following the groups list, so it could be used, for example, to store the real name and contact details for the user, and an expiry date. This could be stored in the DBM like this:

  $DBM{$key} = join(":", $enc, join(",", @groups),
                          $realname, $company, $emailaddr,
                          $expdate);

Keeping all the user information together in a database like this, which Apache can also use for user authentication, can make administering a site with many users simpler.

TCP Wrapper

2009-05-28T22:38:00.000-07:00

TCP Wrapper is a host-based Networking ACL system, used to filter network access to Internet Protocol servers on (Unix-like) operating systems such as Linux or BSD. It allows host or subnetwork IP addresses, names and/or ident query replies, to be used as tokens on which to filter for access control purposes.

TCP Wrappers Configuration Files

(From redhat.com)

To determine if a client machine is allowed to connect to a service, TCP wrappers reference the following two files, which are commonly referred to as hosts access files:

/etc/hosts.allow
/etc/hosts.deny

When a client request is received by a TCP wrapped service, it takes the following basic steps:

The service references /etc/hosts.allow. — The TCP wrapped service sequentially parses the /etc/hosts.allow file and applies the first rule specified for that service. If it finds a matching rule, it allows the connection. If not, it moves on to step 2.
The service references /etc/hosts.deny. — The TCP wrapped service sequentially parses the /etc/hosts.deny file. If it finds a matching rule is denies the connection. If not, access to the service is granted.

The following are important points to consider when using TCP wrappers to protect network services:

Because access rules in hosts.allow are applied first, they take precedence over rules specified in hosts.deny. Therefore, if access to a service is allowed in hosts.allow, a rule denying access to that same service in hosts.deny is ignored.
Since the rules in each file are read from the top down and the first matching rule for a given service is the only one applied, the order of the rules is extremely important.
If no rules for the service are found in either file, or if neither file exists, access to the service is granted.
TCP wrapped services do not cache the rules from the hosts access files, so any changes to hosts.allow or hosts.deny take effect immediately without restarting network services.

15.2.1. Formatting Access Rules

The format for both /etc/hosts.allow and /etc/hosts.deny are identical. Any blank lines or lines that start with a hash mark (#) are ignored, and each rule must be on its own line.

Each rule uses the following basic format to control access to network services:

:  [: : : ...]

— A comma separated list of process names (not service names) or the ALL wildcard (see Section 15.2.1.1 Wildcards). The daemon list also accepts operators listed in Section 15.2.1.3 Operators to allow greater flexibility.
— A comma separated list of hostnames, host IP addresses, special patterns (see Section 15.2.1.2 Patterns), or special wildcards (see Section 15.2.1.1 Wildcards) which identify the hosts effected by the rule. The client list also accepts operators listed in Section 15.2.1.3 Operators to allow greater flexibility.
— An optional action or colon separated list of actions performed when the rule is triggered. Option fields support expansions (see Section 15.2.3.4 Expansions), launch shell commands, allow or deny access, and alter logging behavior (see Section 15.2.3 Option Fields).

The following is a basic sample hosts access rule:

vsftpd : .example.com

This rule instructs TCP wrappers to watch for connections to the FTP daemon (vsftpd) from any host in the example.com domain. If this rule appears in hosts.allow, the connection will be accepted. If this rule appears in hosts.deny, the connection will be rejected.

The next sample hosts access rule is more complex and uses two option fields:

sshd : .example.com  \
: spawn /bin/echo `/bin/date` access denied>>/var/log/sshd.log \
: deny

Note that in this example that each option field is preceded by the backslash (\). Use of the backslash prevents failure of the rule due to length.

Warning

If the last line of a hosts access file is not a newline character (created by pressing the [Enter] key), the last rule in the file will fail and an error will be logged to either /var/log/messages or /var/log/secure. This is also the case for a rule lines that span multiple lines without using the backslash. The following example illustrates the relevant portion of a log message for a rule failure due to either of these circumstances:

warning: /etc/hosts.allow, line 20: missing newline or line too long

This sample rule states that if a connection to the SSH daemon (sshd) is attempted from a host in the example.com domain, execute the echo command (which will log the attempt to a special file), and deny the connection. Because the optional deny directive is used, this line will deny access even if it appears in the hosts.allow file. For a more detailed look at available options, see Section 15.2.3 Option Fields.

15.2.1.1. Wildcards

Wildcards allow TCP wrappers to more easily match groups of daemons or hosts. They are used most frequently in the client list field of access rules.

The following wildcards may be used:

ALL — Matches everything. It can be used for both the daemon list and the client list.
LOCAL — Matches any host that does not contain a period (.), such as localhost.
KNOWN — Matches any host where the hostname and host address are known or where the user is known.
UNKNOWN — Matches any host where the hostname or host address are unknown or where the user is unknown.
PARANOID — Matches any host where the hostname does not match the host address.

	Caution
	The `KNOWN`, `UNKNOWN`, and `PARANOID` wildcards should be used with care as a disruption in name resolution may prevent legitimate users from gaining access to a service.

15.2.1.2. Patterns

Patterns can be used in the client list field of access rules to more precisely specify groups of client hosts.

The following is a list of the most common accepted patterns for a client list entry:

Hostname beginning with a period (.) — Placing a period at the beginning of a hostname, matches all hosts sharing the listed components of the name. The following example would apply to any host within the example.com domain:
ALL : .example.com
IP address ending with a period (.) — Placing a period at the end of an IP address matches all hosts sharing the initial numeric groups of an IP address. The following example would apply to any host within the 192.168.x.x network:
ALL : 192.168.
IP address/netmask pair — Netmask expressions can also be used as a pattern to control access to a particular group of IP addresses. The following example would apply to any host with an address of 192.168.0.0 through 192.168.1.255:
ALL : 192.168.0.0/255.255.254.0
The asterisk (*) — Asterisks can be used to match entire groups of hostnames or IP addresses, as long as they are not mixed in a client list containing other types of patterns. The following example would apply to any host within the example.com domain:
ALL : *.example.com
The slash (/) — If a client list begins with a slash, it is treated as a file name. This is useful if rules specifying large numbers of hosts are necessary. The following example refers TCP wrappers to the /etc/telnet.hosts file for all Telnet connections:
in.telnetd : /etc/telnet.hosts

Other, lesser used patterns are also accepted by TCP wrappers. See the hosts access man 5 page for more information.

	Warning
	Be very careful when creating rules requiring name resolution, such as hostnames and domain names. Attackers can use a variety of tricks to circumvent accurate name resolution. In addition, any disruption in DNS service would prevent even authorized users from using network services. It is best to use IP addresses whenever possible.

Warning

Be very careful when creating rules requiring name resolution, such as hostnames and domain names. Attackers can use a variety of tricks to circumvent accurate name resolution. In addition, any disruption in DNS service would prevent even authorized users from using network services.

It is best to use IP addresses whenever possible.

15.2.1.3. Operators

At present, access control rules accept one operator, EXCEPT. It can be used in both the daemon list and the client list of a rule.

The EXCEPT operator allows specific exceptions to broader matches within the same rule.

In the following example from a hosts.allow file, all example.com hosts are allowed to connect to all services except cracker.example.com:

ALL: .example.com EXCEPT cracker.example.com

In the another example from a hosts.allow file, clients from the 192.168.0.x network can use all services except for FTP:

ALL EXCEPT vsftpd: 192.168.0.

	Note
	Organizationally, it is often easier to use `EXCEPT` operators sparingly, placing the exceptions to a rule in the other access control file. This allows other administrators to quickly scan the appropriate files to see what hosts should are allowed or denied access to services, without having to sort through the various `EXCEPT` operators.

15.2.2. Portmap and TCP Wrappers

When creating access control rules for portmap, do not use hostnames as its implementation of TCP wrappers does not support host look ups. For this reason, only use IP addresses or the keyword ALL when specifying hosts is in hosts.allow or hosts.deny.

In addition, changes to portmap access control rules may not take affect immediately.

Widely used services, such as NIS and NFS, depend on portmap to operate, so be aware of these limitations.

15.2.3. Option Fields

In addition to basic rules allowing and denying access, the Red Hat Linux implementation of TCP wrappers supports extensions to the access control language through option fields. By using option fields within hosts access rules, administrators can accomplish a variety of tasks such as altering log behavior, consolidating access control, and launching shell commands.

15.2.3.1. Logging

Option fields let administrators easily change the log facility and priority level for a rule by using the severity directive.

In the following example, connections to the SSH daemon from any host in the example.com domain are logged to the the default authpriv facility (because no facility value is specified) with a priority of emerg:

sshd : .example.com : severity emerg

It is also possible to specify a facility using the severity option. The following example logs any SSH connection attempts by hosts from the example.com domain to the local0 facility with a priority of alert:

sshd : .example.com : severity local0.alert

	Note
	In practice, this example will not work until the syslog daemon (`syslogd`) is configured to log to the `local0` facility. See the `syslog.conf` man page for information about configuring custom log facilities.

15.2.3.2. Access Control

Option fields also allow administrators to explicitly allow or deny hosts in a single rule by adding the allow or deny directive as the final option.

For instance, the following two rules allow SSH connections from client-1.example.com, but deny connections from client-2.example.com:

sshd : client-1.example.com : allow
sshd : client-2.example.com : deny

By allowing access control on a per-rule basis, the option field allows administrators to consolidate all access rules into a single file: either hosts.allow or hosts.deny. Some consider this an easier way of organizing access rules.

15.2.3.3. Shell Commands

Option fields allow access rules to launch shell commands through the following two directives:

spawn — Launches a shell command as a child process. This option directive can perform tasks like using /usr/sbin/safe_finger to get more information about the requesting client or create special log files using the echo command.
In the following example, clients attempting to access Telnet services from the example.com domain are quietly logged to a special file:
in.telnetd : .example.com \ : spawn /bin/echo `/bin/date` from %h>>/var/log/telnet.log \ : allow
twist — Replaces the requested service with the specified command. This directive is often used to set up traps for intruders (also called "honey pots"). It can also be used to send messages to connecting clients. The twist command must occur at the end of the rule line.
In the following example, clients attempting to access FTP services from the example.com domain are sent a message via the echo command:
vsftpd : .example.com \ : twist /bin/echo "421 Bad hacker, go away!"

For more information about shell command options, see the hosts_options man page.

15.2.3.4. Expansions

Expansions, when used in conjunction with the spawn and twist directives provide information about the client, server, and processes involved.

Below is a list of supported expansions:

%a — The client's IP address.
%A — The server's IP address.
%c — Supplies a variety of client information, such as the username and hostname, or the username and IP address.
%d — The daemon process name.
%h — The client's hostname (or IP address, if the hostname is unavailable).
%H — The server's hostname (or IP address, if the hostname is unavailable).
%n — The client's hostname. If unavailable, unknown is printed. If the client's hostname and host address do not match, paranoid is printed.
%N — The server's hostname. If unavailable, unknown is printed. If the server's hostname and host address do not match, paranoid is printed.
%p — The daemon process ID.
%s — Various types of server information, such as the daemon process and the host or IP address of the server.
%u — The client's username. If unavailable, unknown is printed.

The following sample rule uses an expansion in conjunction with the spawn command to identify the client host in a customized log file.

It instructs TCP wrappers that if a connection to the SSH daemon (sshd) is attempted from a host in the example.com domain, execute the echo command to log the attempt, including the client hostname (using the %h expansion), to a special file:

sshd : .example.com  \
: spawn /bin/echo `/bin/date` access denied to %h>>/var/log/sshd.log \
: deny

Similarly, expansions can be used to personalize messages back to the client. In the following example, clients attempting to access FTP services from the example.com domain are informed that they have been banned from the server:

vsftpd : .example.com \
: twist /bin/echo "421 %h has been banned from this server!"

For a full explanation of available expansions, as well as additional access control options, review see section 5 of the man pages for hosts_access (man 5 hosts_access) and the man page for hosts_options.

PAM (Pluggable authentication module)

2009-05-28T22:27:00.000-07:00

Note: this document is written in reference to Red Hat Linux 6.2+

PAM (Pluggable authentication module) is very diverse in the types of modules it provides. One could accomplish many authentication tasks using PAM. However PAM expands itself beyond typical authentication programs, as it allows an admin to employ other system-critical features such as resource limiting, su protection, and TTY restrictions. Much of PAM's features are not within the scope of this document, but for further reading you can refer to the links at the bottom of this document.

Firstly we must enable the pam_limits module, inside /etc/pam.d/login. Add the following to the end of the file:

session required /lib/security/pam_limits.so

After adding the line above, the /etc/pam.d/login file should look something like this:

#%PAM-1.0
auth required /lib/security/pam_securetty.so
auth required /lib/security/pam_stack.so service=system-auth
auth required /lib/security/pam_nologin.so
account required /lib/security/pam_stack.so service=system-auth
password required /lib/security/pam_stack.so service=system-auth
session required /lib/security/pam_stack.so service=system-auth
session optional /lib/security/pam_console.so
session required /lib/security/pam_limits.so

The limits.conf file located under the /etc/security directory can be used to control and set resource policies. limits.conf is well commented and easy to use - so do take the time to skim over its contents. It is important to set resource limits on all your users so they can't perform denial of service attacks with such things as fork bombs, amongst other things it can also stop 'stray' server processes from taking the system down with it.

It is also a good idea to separate rules for users, admins, and other (other being everything else). This is important, cause take for instance a scenario where a user fork bombs the system - it could in effect disable an administrator's ability to login to the system and take proper actions, or worse crash the server.

Below is the default policy used on a server iv configured:

# For everyone (users and other)
* hard core 0
* - maxlogins 12
* hard nproc 50
* hard rss 20000

# For group wheel (admins)
@wheel - maxlogins 5
@wheel hard nproc 80
@wheel hard rss 75000

#End of file

The first set of rules say to prohibit the creation of core files - core 0 , restrict the number of processes to 50 - nproc 50, restrict logins to 12 - maxlogins 12, and restrict memory usage to 20MB - rss 20000 for everyone except the super user. The the later rules for admins, say to restrict logins to 5 - maxlogins 5, restrict the number of processes to 80 - nproc 80, and restrict the memory usage to 75MB - rss 75000.

All the above only concerns users who have entered via the login prompt on your system. The asterisk (*) defines all users and at wheel (@wheel) defines only users in group wheel. Make sure to add your administrative users into the wheel group (this can be done in /etc/group).

Finally edit the /etc/profile file and change the following line:

ulimit -c 1000000

to read:

ulimit -S -c 1000000 > /dev/null 2<&1

This modification is used to avoid getting error messages like 'Unable to reach limit' during login. On newer editions of Red Hat Linux, the later ulimit setting is default.

Further reading is available in The Linux-PAM System Administrators' Guide located at:
http://www.kernel.org/pub/linux/libs/pam/L...M-html/pam.html

Tripwire: a very effective host intrustion detection system.

2009-05-28T22:12:00.000-07:00

A crude yet effective intrusion detection system such as Tripwire can alert systems administrators to possible intrusion attempts by periodically verifying the integrity of a server's file systems. Systems intruders will often use trojan binaries for login, su, ps, and ls, etc. to cover their tracks and keep a low profile on the system. Under normal circumstances even astute systems administrators may not observe the intrusion because the trojan binaries mimic the system binaries so well.

One tried and true method to alert systems administrators of unexpected file system alterations is to use a software package such as Tripwire to keep a database of checksums on the file sizes of critical system files. Depending on the configuration, Tripwire can notify appropriate personnel if a critical file or directory is modified or deleted.

By using a strong checksum method similar to MD5, Tripwire can identify with absolute certainty whether or not a file has been modified, unlike similar programs that use weaker algorithms such as CRC to calculate checksums.

Also, for maximum effectiveness Tripwire should be installed at the time the operating system is installed to ensure that the system does not already have any trojan binaries. Tripwire is only as reliable as the initial file system its database is based upon. If the file system has already been attacked, then Tripwire can only identify further damage to the filesystem, if that.

The Linux Open Source Edition

Recently, Tripwire, Inc. has decided to open the source for a more recent version of the Tripwire package specifically for the Linux OS. Previously, a binary only version of the software had been made available to the Linux community and another version of the software with and an older, less featured academic source license had been available to the public. The Linux open source edition includes most of the newer features of the software, such as the ability to alert specific administrators for different areas of alterations, while remaining compatible with the commercial version of the software.

Getting the Software

Binary packages are available at tripwire.org for use with the Red Hat 7.0 distribution of Linux, though the binaries work fine on similar RPM based distributions such as Mandrake. For other types of Linux distributions, Tripwire will need to be compiled from the source tarballs located on the same page. For Red Hat 7.0, the RPM binaries are also available on the second binary CD of the distribution.

Installing Tripwire

Although it isn't a difficult procedure to compile Tripwire from source, this article will be limited to describing the installation process from the binary RPM.

If Tripwire is downloaded from the website listed above, please be aware that the RPM is also tar/gzipped. Thus, to install the Tripwire RPM, issue the following commands as root:

  tar xvzf tripwire-2.3-47.i386.tar.gz

 rpm -ivh tripwire-2.3-47.i386.rpm

Once the software is installed with rpm, the installation shell script will need to be executed to finish the Tripwire installation. This is done by issuing the command:

  /etc/tripwire/twinstall.sh

as root. Note that all Tripwire associated files are kept in the /etc/tripwire directory.

Initial Tripwire Configuration

Because very few Linux installations are identical, Tripwire will need a fair amount of configuration to adequately protect the system. Configuration begins during the installation script launched above with the selection of site and local passphrases. These passphrases are the key to preventing intruders from modifying your Tripwire installation and circumventing its protection so strong passphrases are essential!

The site key is used to sign Tripwire's policy and configuration files while the local key is used for signing the database files. For enterprise wide installations, the use of multiple levels of passwords makes Tripwire more manageable by allowing a site to split administration functions across by a number of system administrators.

The installation script creates default policy and configuration files stored in /etc/tripwire as twpol.txt and twcfg.txt. These files are in cleartext and need to be removed from the system as soon as the encrypted versions are in place for obvious security reasons.

The default policy probably includes monitoring for a number of files not present on the local system, so it's important to trim these files out of policy. The following procedures will illustrate exactly how this is done.

The default policy should be installed using the command as root:

  /usr/sbin/twadmin -m P /etc/tripwire/twpol.txt

Next, generate the initial database using the following command as root:

  /usr/sbin/tripwire -m i

Note that the -m switch identifies the mode in which Tripwire is being executed, which is "i" for "initialization" in this case. Later, the "c" mode for "check" will be used. Expect the initialization to take quite a long time, even on a fast machine.

Customizing Tripwire's Configuration

Once and initial database is created, some customization is necessary to prevent the issuance of a large number of false alarms. These false alarms occur any time there is a discrepancy in the default policy and the local system's current configuration. To generate a listing of the discrepancies between the local system and the default policy, issue the following command as root:

  /usr/sbin/tripwire -m c | grep Filename >> twtest.txt

Note that this command will also take several minutes to complete. Once this listing has been generated, edit the policy file, /etc/tripwire/twpol.txt, and comment out or delete each of the filenames listed in twtest.txt.

Additionally, there are other files in the default policy that may not make sense to monitor on the local system. These include lock files (which identify that some process is in use) and pid files (which identify the process ID of some daemons). Since the files are likely to change often, if not at every system boot, they can cause Tripwire to generate false positives. To avoid such problems, comment out all of the /var/lock/subsys entries as well as the entry for /var/run.

Finalizing the Tripwire Configuration

Any time the tripwire policy file is edited, the policy needs to be reinstalled and the database will need to be recreated. As before, these tasks are accomplished by issuing the following commands as root:

  /usr/sbin/twadmin -m P /etc/tripwire/twpol.txt

 /usr/sbin/tripwire -m i

At this point it wouldn't be a bad idea to repeat the customization procedures just to ensure that none of the unnecessary files listed in twtest.txt were omitted.

It's now safe to delete the clear text versions of the Tripwire policy and configuration files, which can be performed by issuing the following command as root:

  rm /etc/tripwire/twcfg.txt /etc/tripwire/twpol.txt

If they need to be restored cleartext versions of these files can be created from the encrypted versions by issuing the command (and providing the appropriate passphrases):

  /usr/sbin/twadmin -m p > /etc/tripwire/twpol.txt

Note that unlike before, the "p" in this command is lowercase.

Finally, it is desirable to save a copy of the database at least initially and periodically if possible to read-only media such as CD-R. Having read-only copies of the database file is the only way to guarantee 100% that Tripwire's database is authentic.

Scheduling a Nightly Tripwire Analysis

Without regular checks of the filesystem, Tripwire is effectively useless, so this section will identify how to schedule Nightly Tripwire Analyses that are e-mail to the system administrator.

First, one needs to create a shell script for generating the Tripwire reports. Creating the shell script can be more useful than just placing the command in the crontab because it allows the administrator to perform a filesystem check without needing to remember the exact syntax necessary for doing so.

Create the file "runtw.sh" in the directory /usr/local/bin that has the following contents:

#!/bin/sh

/usr/sbin/tripwire -m c | mail -s "Tripwire Report from HOST" root@localhost

Of course, HOST should be changed to the hostname of the system. Don't forget to make the shell script executable by root.

Then, schedule the script to execute nightly at 1:01am by adding the line:

  1 1 * * *     /usr/local/bin/runtw.sh

to root's crontab using the command:

  crontab -e

Tripwire will now submit nightly reports to the system administrator on the status of the file system's integrity.

2009-05-28T22:10:00.000-07:00

Mailmon Installation

cd /usr/src/
wget http://www.mycutelife.net/sanju/newtickethelp/mailmon/mailmon_1-3.tar.gz
tar -xvzf mailmon_1-3.tar.gz
cd /usr/src/MailMon
cp -f /usr/sbin/sendmail /usr/sbin/mon.bkp
wget http://www.mycutelife.net/sanju/newtickethelp/mailmon/mailmon.new
sed -e s/opteron.dnsprotect.com/$hostname/g mailmon.new > mailmon.temp;
cp -f mailmon.temp /usr/sbin/sendmail
cd /usr/sbin
chown root.mailtrap sendmail
chmod 755 sendmail
chattr +i sendmail
cd /var/log
touch mailmon.log
chmod 622 mailmon.log
touch mailmon.junk
chmod 622 mailmon.junk

mysql
mysql>create database mailmon2005;
mysql>grant all privileges on mailmon2005.* to mailmon2005@localhost identified by '123dsa';
mysql>use mailmon2005;

CREATE TABLE `limits` (
`id` int(11) NOT NULL auto_increment,
`user` varchar(20) NOT NULL default '',
`speedlimit` int(11) NOT NULL default '0',
`seconds` int(11) NOT NULL default '0',
PRIMARY KEY (`id`)
) TYPE=MyISAM AUTO_INCREMENT=6 ;
INSERT INTO `limits` VALUES (6, 'cpanel', 200, 3600);
CREATE TABLE `mailmon` (
`user` varchar(20) NOT NULL default '',
`timestamp` int(10) unsigned NOT NULL default '0',
`script_name` varchar(255) NOT NULL default '',
KEY `user` (`user`,`timestamp`)
) TYPE=MyISAM;

mysql> quit;

Courtesy: Sanju Abraham

Vulnerability Scanner: Nessus

2009-05-28T21:56:00.000-07:00

If you're looking for a vulnerability scanner, chances are you've come across a number of expensive commercial products and tools with long lists of features and benefits. Unfortunately, if you're in the same situation as most of us, you simply don't have the budget to implement fancy high-priced systems. You might have considered compromising by turning to free tools like nmap. However, you probably saw these tools as a compromise, as their feature sets didn't quite match the commercial offerings.

It's time that you learn how to use Nessus! This free tool offers a surprisingly robust feature-set and is widely supported by the information security community. It doesn't take long between the discovery of a new vulnerability and the posting of an updated script for Nessus to detect it. In fact, Nessus takes advantage of the Common Vulnerabilities and Exposures (CVE) architecture that facilitates easy cross-linking between compliant security tools.

The Nessus tool works a little differently than other scanners. Rather than purporting to offer a single, all-encompassing vulnerability database that gets updated regularly, Nessus supports the Nessus Attack Scripting Language (NASL), which allows security professionals to use a simple language to describe individual attacks. Nessus administrators then simply include the NASL descriptions of all desired vulnerabilities to develop their own customized scans.

With the release of Nessus 3 in December 2005, Tenable Network Security Inc., the company behind Nessus, introduced a complete overhaul of the product. The most current version at the time of this writing, Nessus 3.2, was released in March 2008. Nessus is now available for a wide variety of platforms, including Windows, various flavors of Linux, FreeBSD, Solaris and Mac OS X. Here's an overview of the significant changes in Nessus 3:

Nessus is now closed-source. The base product is still available for free. With the introduction of Nessus 3, however, Tenable moved Nessus from an open source to a commercial licensing model. In other words, while the software itself remains free, updated vulnerability information will come with a fee, at least for enterprises (home users may download updates for free). Tenable cites the need to invest in the future of Nessus as the motivation for moving to a proprietary license scheme.
Significant speed enhancements. In benchmarking tests performed by Tenable, Nessus 3 scans systems at about twice the speed of Nessus 2. This is due to optimizations in the scan engine and a complete overhaul of NASL.
Dramatic reduction in resource requirements. Nessus 3 uses significantly less memory and CPU cycles than Nessus 2, allowing simultaneous scanning of a larger number of hosts.

Nessus uses a modular architecture consisting of centralized servers that conduct scanning and remote clients that allow for administrator interaction. You may deploy Nessus scanning servers at various points within your enterprise and control them from a single client. This allows you to effectively scan segmented networks from multiple vantage points and conduct scans of large networks that require multiple servers running simultaneously.

If you're looking for a robust, inexpensive vulnerability scanning product, definitely take Nessus out for a test drive! The tips in this tutorial will guide you along the way.

Nessus Installation on Red Hat Linux

BEFORE WE BEGIN
===============================
I understand that there are many ways to install and configure Nessus. This tutorial covers only one of them. This tutorial makes several assumptions:
1. You are competent with Windows, Linux and basic networking. If you don’t know how to use command line FTP for example, then this tutorial will be of no use to you.
2. You have 2 computers, one with a Windows and the other with Red Hat, both in good working order. It also assumes that you have at least one supported compiler such as GCC installed on your Red Hat Box.
3. This tutorial is written by me with no references or “borrowed” material. If something doesn’t work or something isn’t clear, yell at me because I am 100% responsible.

GETTING THE SOFTWARE
===============================

On your Red Hat box, from the directory of your choice, ftp to ftp.nessus.org and login anonymously. Once there, path to /pub/nessus/nessus-2.0.7/nessus-installer/ and download nessus-installer.sh

INSTALLATION OF THE NESSUS ENGINE
===============================
Now that you have all of the software, it’s time to install. Let’s begin with the Nessus engine because it requires most of the work.

1. From the directory where you downloaded nessus-installer.sh, simply type: sh nessus-installer.sh. The Nessus installation script will tell you that you need root priviledges to complete the install, press ENTER to continue if you are logged in as root already.
2. Nessus will ask where you want it installed. /usr/local is the default so just hit ENTER when you see the prompt. At this point, Nessus will tell you that it is ready to compile. Hit ENTER and sit back while it compiles. It will take a little while. When it is finished, you’ll see a screen detailing the next steps. Hit ENTER.
3. Now, at this point you have to decide if you want Nessus to start up each time you boot your box or if you just want to start it when you feel like it. To start it when you feel like it, use /usr/local/sbin/nessusd –D. If you want to start it automatically when your box boots up, add /usr/local/sbin/nessusd –D & to /etc/rc.local.
4. Now, decide how you want to handle updating the plugins. You can do it each time the box boots by adding /user/local/sbin/nessus-update-plugins & to /etc/rc.local. You can also copy the nessus-update-plugins script to /etc/cron.daily and it will go out each day and grab the updates.
5. OK, we now have to generate a certificate so go to /usr/local/sbin/ and type nessus-mkcert. This will prompt you for a bunch of information that you would see when generating any SSL certificate. Answer all the questions.
6. Now you have to add a user by running nessus-adduser from /usr/local/sbin. When run, provide a login ID of your choice. When it asks for pass or cert, hit ENTER to accept pass as the auth method. When asked for a password, provide it one. Next you will see a blurb about user rules. Simply hit Ctrl – d and Nessus will verify your input. Type in “y” and Nessus will inform you that the user has been added.

Well now all you have to do is reboot the box to launch Nessus or you need to start the deamon manually as shown in step 3.

INSTALLATION OF NESSUSWX CLIENT
===============================
OK, now all you have to do is run the installer. On the first screen, click next to continue. Next click the checkbox if you agree to the license, then hit next to continue. The next screen shows the install path, click next to continue. Select Binaries Only, then click next. The next screen names the program group, hit next to continue. It now has all the info to begin installation. Hit next and it will begin. Once this is done, look for the eyeball icon on your desktop. Launch it. It will ask about a nessusdb and all you need to do is say yes to create it.

OK, now you need to configure a session:
1) Form the mune pulldowns, select COMMUNICATIONS, then CONNECT. Enter the IP address of your Nessus server then enter the username you created on the Nessus server. You need to use password authentication and it is your choice to save the password or not. Once you do that, hit CONNECT. Accept the certificate however you like (I always do perminant because I trust the source).
2) From the menu pulldowns, select SESSION then NEW.
3) This will open a window to enter your list of target hosts. Add your hosts in here.
4) Now, each tab has tons of options so I will hit on the key ones for now. Hit the portscan tab and enter the range 1-65535.
5) Hit the plug-ins tab and check “use session specific plugin set”, then hit the select plugins button, then select either all plug-ins (bad idea for a production box that you want to scan) or Non-DOS. Click OK.
6) Now, right click on your session (green book icon) and select EXECUTE.
7) On the next pop-up hit the EXECUTE button and you should see your scan underway.

At this point, you are golden. When the scan is done you can preview it or you can generate a report. I usually select HTML output.

In conclusion, I left out *tons* of options and configs but this tutorial is only intended to get you scanning. You’ll need to look into the docs to explore all this tool has to offer.

Happy scanning!

About Mod_Security and Mod_Dosevasive

2009-05-28T21:52:00.000-07:00

What Are These Two Apache Modules and How Can They Help You?
Apache comes by default as a secure web server. However, that by no means implies that there are no methods of improving its security. On the contrary, there are two primary modules available for Apache that will increase its security strengths ten fold. They are mod_security and mod_dosevasive.

It goes without explanation that the internet is a scary, dangerous place. Particularly for web servers, the internet has tons of potential attackers just waiting to attack and cause damage. For this reason, programmers have worked hard to create defense programs and modules, two of the most useful being the Mod_security and Mod_dosevasive modules available for Apache web servers. In the unsafe world of the internet, these modules were created in order to combat hackers and other perpetrators and prevent such attacks as nuke attacks, DoS attacks, and DDoS attacks, amongst others.

Starting with Mod_dosevasive, which can be easily downloaded from Nuclear Elephant at http://www.nuclearelephant.com/projects/dosevasive/, this module allows for evasive maneuvers in the case of a DoS, DDoS, or similar attack against an Apache web server. This module is most effective when used in conjuction with a firewall or router. It can detect unusually high amounts of requests on the server on a per second basis and prevent these requests, thus evading a potential DoS or DDoS attack by having prevented the attack from consuming bandwith or disk space as it was intended to do. Mod_dosevasive is updated fairly often with improvements to prevent new forms of attacks.

Mod_security, which can also be downloaded from ModSecurity at http://www.modsecurity.org/, is a constantly updated open source protection utility for servers. It acts in a similar fashion to a firewall, although it is most effective when used in conjuction with a firewall for additional protection, by recognizing and disrupting potential known or unknown server attacks. It comes open source meaning it can be easily edited and customized. Particularly, the module can be customized with specific filtering rules for maximum efficiency.

Apache 1.3 and 2.0 Flood/DoS/DDoS Protection with mod_dosevasive (Avoiding Denial of Service Attacks)

2009-05-28T21:49:00.000-07:00

With the widespread infection of many computers with viruses, and the ever increasing number of Botnets, DoS and DDoS attacks can be quite frequent and can very easily bring a website to halt for days. This article provides a module solution for apache to help mitigate small http DoS and DDoS attacks.

Download the latest version of mod_dosevasive from: http://www.nuclearelephant.com/projects/dosevasive

The lastest version is 1.10 (http://www.nuclearelephant.com/projects/dosevasive/mod_dosevasive_1.10.tar.gz)

Untar it:

tar zxvf mod_dosevasive_1.10.tar.gz

Change into the directory:

cd mod_dosevasive

Compile mod_dosevasive apache module (Apache 2):

/usr/local/apache/bin/apxs -i -a -c mod_dosevasive20.c

or the following for apache 1.3:

/usr/local/apache/bin/apxs -i -a -c mod_dosevasive.c

Replace /usr/local/apache with your path to apache.

Edit your httpd.conf (usually located in /usr/local/apache/conf/httpd.conf):
Add:

DOSHashTableSize 3097
DOSPageCount 2
DOSSiteCount 50
DOSPageInterval 1
DOSSiteInterval 1
DOSBlockingPeriod 10
DOSEmailNotify someuser@somedomain.com
DOSSystemCommand "su - someuser -c '/sbin/... %s ...'"

- DOSHashTableSize: is the size of the table of URL and IP combined
- DOSPageCount: is the number of same page requests from the same IP during an interval that will cause that IP to be added to the block list.
- DOSSiteCount: is the number of pages requested of a site by the same IP during an interval which will cause the IP to be added to the block list.
- DOSPageInterval: is the interval that the hash table for IPs and URLs is erased (in seconds)
- DOSSiteInterval: is the intervale that the hash table of IPs is erased (in seconds)
- DOSBlockingPeriod: is the time the IP is blacked (in seconds)
- DOSEmailNotify: can be used to notify by sending an email everytime an IP is blocked
- DOSSystemCommand: is the command used to execute a command when an IP is blocked. It can be used to add a block the user from a firewall or router.
- DOSWhiteList: can be used to whitelist IPs such as 127.0.0.1

Although mod_dosevasive can be quite effective in some cases, in others it can cause more problems by blocking non-offending IPs.

Sylesh

'netstat' command MAN page

2009-05-26T23:47:00.000-07:00

netstat - Print network connections, routing tables, interface statis-
tics, masquerade connections, and multicast memberships

SYNOPSIS

netstat [address_family_options] [--tcp|-t] [--udp|-u] [--raw|-w]
[--listening|-l] [--all|-a] [--numeric|-n] [--numeric-hosts]
[--numeric-ports] [--numeric-users] [--symbolic|-N]
[--extend|-e[--extend|-e]] [--timers|-o] [--program|-p] [--verbose|-v]
[--continuous|-c]

netstat {--route|-r} [address_family_options]
[--extend|-e[--extend|-e]] [--verbose|-v] [--numeric|-n] [--numeric-
hosts] [--numeric-ports] [--numeric-users] [--continuous|-c]

netstat {--interfaces|-i} [--all|-a] [--extend|-e[--extend|-e]] [--ver-
bose|-v] [--program|-p] [--numeric|-n] [--numeric-hosts] [--numeric-
ports] [--numeric-users] [--continuous|-c]

netstat {--groups|-g} [--numeric|-n] [--numeric-hosts] [--numeric-
ports] [--numeric-users] [--continuous|-c]

netstat {--masquerade|-M} [--extend|-e] [--numeric|-n] [--numeric-
hosts] [--numeric-ports] [--numeric-users] [--continuous|-c]

netstat {--statistics|-s} [--tcp|-t] [--udp|-u] [--raw|-w]

netstat {--version|-V}

netstat {--help|-h}

address_family_options:

[--protocol={inet,unix,ipx,ax25,netrom,ddp}[,...]] [--unix|-x]
[--inet|--ip] [--ax25] [--ipx] [--netrom] [--ddp]

DESCRIPTION

Netstat prints information about the Linux networking subsystem. The
type of information printed is controlled by the first argument, as
follows:

(none)
By default, netstat displays a list of open sockets. If you don't
specify any address families, then the active sockets of all configured
address families will be printed.

--route , -r
Display the kernel routing tables.

--groups , -g
Display multicast group membership information for IPv4 and IPv6.

--interface, -i
Display a table of all network interfaces.

--masquerade , -M
Display a list of masqueraded connections.

--statistics , -s
Display summary statistics for each protocol.

OPTIONS

--verbose , -v
Tell the user what is going on by being verbose. Especially print some
useful information about unconfigured address families.

--numeric , -n
Show numerical addresses instead of trying to determine symbolic host,
port or user names.

--numeric-hosts
shows numerical host addresses but does not affect the resolution of
port or user names.

--numeric-ports
shows numerical port numbers but does not affect the resolution of host
or user names.

--numeric-users
shows numerical user IDs but does not affect the resolution of host or
port names.

--protocol=family , -A
Specifies the address families (perhaps better described as low level
protocols) for which connections are to be shown. family is a comma
(',') separated list of address family keywords like inet, unix, ipx,
ax25, netrom, and ddp. This has the same effect as using the --inet,
--unix (-x), --ipx, --ax25, --netrom, and --ddp options.

The address family inet includes raw, udp and tcp protocol sockets.

-c, --continuous
This will cause netstat to print the selected information every second
continuously.

-e, --extend
Display additional information. Use this option twice for maximum
detail.

-o, --timers
Include information related to networking timers.

-p, --program
Show the PID and name of the program to which each socket belongs.

-l, --listening
Show only listening sockets. (These are omitted by default.)

-a, --all
Show both listening and non-listening sockets. With the --interfaces
option, show interfaces that are not up

-F
Print routing information from the FIB. (This is the default.)

-C
Print routing information from the route cache. UP.

OUTPUT

Active Internet connections (TCP, UDP, raw)
Proto
The protocol (tcp, udp, raw) used by the socket.

Recv-Q
The count of bytes not copied by the user program connected to this
socket.

Send-Q
The count of bytes not acknowledged by the remote host.

Local Address
Address and port number of the local end of the socket. Unless the
--numeric (-n) option is specified, the socket address is resolved to
its canonical host name (FQDN), and the port number is translated into
the corresponding service name.

Foreign Address
Address and port number of the remote end of the socket. Analogous to
"Local Address."

State
The state of the socket. Since there are no states in raw mode and usu-
ally no states used in UDP, this column may be left blank. Normally
this can be one of several values:

ESTABLISHED
The socket has an established connection.

SYN_SENT
The socket is actively attempting to establish a connection.

SYN_RECV
A connection request has been received from the network.

FIN_WAIT1
The socket is closed, and the connection is shutting down.

FIN_WAIT2
Connection is closed, and the socket is waiting for a shutdown
from the remote end.

TIME_WAIT
The socket is waiting after close to handle packets still in the
network.

CLOSE The socket is not being used.

CLOSE_WAIT
The remote end has shut down, waiting for the socket to close.

LAST_ACK
The remote end has shut down, and the socket is closed. Waiting
for acknowledgement.

LISTEN The socket is listening for incoming connections. Such sockets
are not included in the output unless you specify the --listen-
ing (-l) or --all (-a) option.

CLOSING
Both sockets are shut down but we still don't have all our data
sent.

UNKNOWN
The state of the socket is unknown.

User
The username or the user id (UID) of the owner of the socket.

PID/Program name
Slash-separated pair of the process id (PID) and process name of the
process that owns the socket. --program causes this column to be
included. You will also need superuser privileges to see this informa-
tion on sockets you don't own. This identification information is not
yet available for IPX sockets.

Timer
(this needs to be written)

Active UNIX domain Sockets
Proto
The protocol (usually unix) used by the socket.

RefCnt
The reference count (i.e. attached processes via this socket).

Flags
The flags displayed is SO_ACCEPTON (displayed as ACC), SO_WAITDATA (W)
or SO_NOSPACE (N). SO_ACCECPTON is used on unconnected sockets if
their corresponding processes are waiting for a connect request. The
other flags are not of normal interest.

Type
There are several types of socket access:

SOCK_DGRAM
The socket is used in Datagram (connectionless) mode.

SOCK_STREAM
This is a stream (connection) socket.

SOCK_RAW
The socket is used as a raw socket.

SOCK_RDM
This one serves reliably-delivered messages.

SOCK_SEQPACKET
This is a sequential packet socket.

SOCK_PACKET
Raw interface access socket.

UNKNOWN
Who ever knows what the future will bring us - just fill in here
:-)

State
This field will contain one of the following Keywords:

FREE The socket is not allocated

LISTENING
The socket is listening for a connection request. Such sockets
are only included in the output if you specify the --listening
(-l) or --all (-a) option.

CONNECTING
The socket is about to establish a connection.

CONNECTED
The socket is connected.

DISCONNECTING
The socket is disconnecting.

(empty)
The socket is not connected to another one.

UNKNOWN
This state should never happen.

PID/Program name
Process ID (PID) and process name of the process that has the socket
open. More info available in Active Internet connections section writ-
ten above.

Path
This is the path name as which the corresponding processes attached to
the socket.

Active IPX sockets
(this needs to be done by somebody who knows it)

Active NET/ROM sockets
(this needs to be done by somebody who knows it)

Active AX.25 sockets
(this needs to be done by somebody who knows it)

'ps' command MAN page

2009-05-26T23:43:00.000-07:00

ps

Process status, information about processes running in memory. If you want a repetitive update of this status, use top.

Syntax

ps option(s)
ps [-L]

Options
-L List all the keyword options

This version of ps accepts 3 kinds of option:

-Unix98 options may be grouped and must be preceeded by a dash.
BSD options may be grouped and must not be used with a dash.
--GNU long options are preceeded by two dashes.

Options of different types may be freely mixed. The PS_PERSONALITY environment variable provides more detailed control of ps behavior.

The Options below are listed side-by-side (unless there are differences).

Simple Process Selection:
-A a select all processes (including those of other users)
-a select all with a tty except session leaders
-d select all, but omit session leaders
-e select all processes
g really all, even group leaders (does nothing w/o SunOS settings)
-N negate selection
r restrict output to running processes
T select all processes on this terminal
x select processes without controlling ttys
--deselect negate selection

Process Selection by List:

-C select by command name
-G select by RGID (supports names)
-g select by session leader OR by group name
--Group select by real group name or ID
--group select by effective group name or ID
-p p --pid select by process ID (PID)
-s --sid select by session ID
-t --tty select by terminal (tty)
-u U select by effective user ID (supports names)
-U select by RUID (supports names)
--User select by real user name or ID
--user select by effective user name or ID

-123 implied --sid
123 implied --pid

Output Format Control:

-c Different scheduler info for -l option
-f Full listing
-j j Jobs format
-l l Long format
-O O Add the information associated with the space or comma separated
list of keywords specified, after the process ID, in the default
information display.

-o o Display information associated with the space or comma separated
list of keywords specified.
--format user-defined format
s display signal format
u display user-oriented format
v display virtual memory format
X old Linux i386 register format
-y do not show flags; show rss in place of addr

Output Modifiers:
C use raw CPU time for %CPU instead of decaying average
c true command name
e show environment after the command
f ASCII-art process hierarchy (forest)
-H show process hierarchy (forest)
h do not print header lines (repeat header lines in BSD personality)
-m m show all threads
-n set namelist file
n numeric output for WCHAN and USER
N specify namelist file
O sorting order (overloaded)
S include some dead child process data (as a sum with the parent)
-w w wide output
--cols set screen width
--columns set screen width
--forest ASCII art process tree
--html HTML escaped output
--headers repeat header lines
--no-headers print no header line at all
--lines set screen height
--nul unjustified output with NULs
--null unjustified output with NULs
--rows set screen height
--sort specify sorting order
--width set screen width
--zero unjustified output with NULs

Information:
-V V print version
L list all format specifiers
--help print help message
--info print debugging info
--version print version

Obsolete:
A increase the argument space (DecUnix)
M use alternate core (try -n or N instead)
W get swap info from ... not /dev/drum (try -n or N instead)
k use /vmcore as c-dumpfile (try -n or N instead)

NOTES
The "-g" option can select by session leader OR by group name. Selection by session leader is specified by many standards, but selection by group is the logical behavior that several other operating systems use. This ps will select by session leader when the list is completely numeric (as sessions are). Group ID numbers will work only when some group names are also specified.

The "m" option should not be used. Use "-m" or "-o" with a list. ("m" displays memory info, shows threads, or sorts by memory use)

The "h" option varies between BSD personality and Linux usage (not printing the header) Regardless of the current personality, you can use the long options --headers and --no-headers

Terminals (ttys, or screens of text output) can be specified in several forms: /dev/ttyS1, ttyS1, S1. Obsolete "ps t" (your own terminal) and "ps t?" (processes without a terminal) syntax is supported, but modern options ("T","-t" with list, "x", "t" with list) should be used instead.

The BSD "O" option can act like "-O" (user-defined output format with some common fields predefined) or can be used to specify sort order. Heuristics are used to determine the behavior of this option. To ensure that the desired behavior is obtained, specify the other option (sorting or formatting) in some other way.

For sorting, BSD "O" option syntax is O[+|-]k1[,[+|-]k2[,...]] Order the process listing according to the multilevel sort specified by the sequence of short keys from SORT KEYS, k1, k2, ... The `+' is quite optional, merely re-iterating the default direction on a key. `-' reverses direction only on the key it precedes.
The O option must be the last option in a single command argument, but specifications in successive arguments are catenated.

GNU sorting syntax is --sortX[+|-]key[,[+|-]key[,...]]
Choose a multi-letter key from the SORT KEYS section. X may be any convenient separator character. To be GNU-ish use `='. The `+' is really optional since default direction is increasing numerical or lexicographic order. For example, ps jax --sort=uid,-ppid,+pid

This ps works by reading the virtual files in /proc. This ps does not need to be suid kmem or have any privileges to run. Do not give this ps any special permissions.

This ps needs access to a namelist file for proper WCHAN display. The namelist file must match the current Linux kernel exactly for correct output.

To produce the WCHAN field, ps needs to read the System.map file created when the kernel is compiled. The search path is:

$PS_SYSTEM_MAP
/boot/System.map-`uname -r`
/boot/System.map
/lib/modules/`uname -r`/System.map
/usr/src/linux/System.map
/System.map

The member used_math of task_struct is not shown, since crt0.s checks to see if math is present. This causes the math flag to be set for all processes, and so it is Programs swapped out to disk will be shown without command line arguments, and unless the c option is given, in brackets.

%CPU shows the cputime/realtime percentage. It will not add up to 100% unless you are lucky. It is time used divided by the time the process has been running.

The SIZE and RSS fields don't count the page tables and the task_struct of a proc; this is at least 12k of memory that is always resident. SIZE is the virtual size of the proc (code+data+stack).

Processes marked are dead processes (so-called"zombies") that remain because their parent has not destroyed them properly. These processes will be destroyed by init(8) if the parent process exits.

PROCESS FLAGS
ALIGNWARN 001 print alignment warning msgs
STARTING 002 being created
EXITING 004 getting shut down
PTRACED 010 set if ptrace (0) has been called
TRACESYS 020 tracing system calls
FORKNOEXEC 040 forked but didn't exec
SUPERPRIV 100 used super-user privileges
DUMPCORE 200 dumped core
SIGNALED 400 killed by a signal

PROCESS STATE CODES
D uninterruptible sleep (usually IO)
R runnable (on run queue)
S sleeping
T traced or stopped
Z a defunct ("zombie") process

For BSD formats and when the "stat" keyword is used, addi
tional letters may be displayed:
W has no resident pages
< high-priority process
N low-priority task
L has pages locked into memory (for real-time and custom IO)

ENVIRONMENT VARIABLES and PERSONALITY (posix,linux,bsd,sun)

Examples:

List every process on the system using standard syntax:
ps -e

List every process on the system using BSD syntax:
ps ax

List the top 10 CPU users.
ps -e -o pcpu -o pid -o user -o args | sort -k 1 | tail -21r

List every process except those running as root (real & effective ID)
ps -U root -u root -N

List every process with a user-defined format:
ps -eo pid,tt,user,fname,tmout,f,wchan

Odd display with AIX field descriptors:
ps -o "%u : %U : %p : %a"

Print only the process IDs of syslogd:
ps -C syslogd -o pid=

When displaying multiple fields, part of the output may be truncated, to avoid this supply a width to the arguments:

ps -e -o user:20,args.

Since ps cannot run faster than the system and is run as any other scheduled process, the information it displays can never be exact.

'.htaccess' file in detail

2009-05-26T23:35:00.000-07:00

htaccess (Hypertext Access) is the default name of Apache’s directory-level configuration file. It provides the ability to customize configuration directives defined in the main configuration file. The configuration directives need to be in .htaccess context and the user needs appropriate permissions.

Statements such as the following can be used to configure a server to send out customized documents in response to client errors such as “404: Not Found” or server errors such as “503: Service Unavailable” (see List of HTTP status codes):

ErrorDocument 404 /error-pages/not-found.html
ErrorDocument 503 /error-pages/service-unavailable.html

When setting up custom error pages, it is important to remember that these pages may be accessed from various different URLs, so the links in these error documents (including those to images, stylesheets and other documents) must be specified using URLs that are either absolute (e.g., starting with “http://”) or relative to the document root (starting with “/”). Also, the error page for “403: Forbidden” errors must be placed in a directory that is accessible to users who are denied access to other parts of the site. This is typically done by making the directory containing the error pages accessible to everyone by creating another .htaccess file in the /error-pages directory containing these lines:

Order allow,deny
Allow from all

Password protection

Make the user enter a name and password before viewing a directory.

AuthUserFile /home/newuser/www/stash/.htpasswd
AuthGroupFile /dev/null
AuthName "Protected Directory"
AuthType Basic

require user newuser

The same behavior can be applied to specific files inside a directory.

AuthUserFile /home/newuser/www/stash/.htpasswd
AuthName "Protected File"
AuthType Basic
Require valid-user

Now run this command to create a new password for the user ‘newuser’.

htpasswd /home/newuser/www/stash/.htpasswd newuser

Password unprotection

Unprotect a directory inside an otherwise protected structure:

Satisfy any

Extra secure method to force a domain to only use SSL and fix double login problem

If you really want to be sure that your server is only serving documents over an encrypted SSL channel (you wouldn’t want visitors to submit a htaccess password prompt on an unencrypted connection) then you need to use the SSLRequireSSL directive with the +StrictRequire Option turned on.

SSLOptions +StrictRequire
SSLRequireSSL
SSLRequire %{HTTP_HOST} eq "site.com" #or www.site.com
ErrorDocument 403 https://site.com

An interesting thing when using the mod_ssl instead of mod_rewrite to force SSL is that apache give mod_ssl priority ABOVE mod_rewrite so it will always require SSL. (may be able to get around first method using http://site.com:443 or https://site.com:80)

* An in-depth article about what this is doing can be found in the SSL Forum

Enable SSI

AddType text/html .shtml
AddHandler server-parsed .shtml
Options Indexes FollowSymLinks Includes

Deny users by IP address

Order allow,deny
Deny from 123.45.67.8
Deny from 123.123.7
Allow from all

This would ban anyone with an IP address of 123.45.67.8 and would also ban anyone with an IP address starting in 123.123.7: for example, 123.123.74.42 would not gain access.

Change the default directory page

DirectoryIndex homepage.html

Here, anyone visiting http://www.example.com/ would see the homepage.html page, rather than the default index.html.

Redirects

Redirect page1.html page2.html

If someone were to visit http://www.example.com/page1.html, he would be sent (with an HTTP status code of 302) to http://www.example.com/page2.html

Prevent hotlinking of images

The following .htaccess rules use mod rewrite.
From specific domains

RewriteEngine on
RewriteCond %{HTTP_REFERER} ^http://([^/]+\.)?baddomain1\.com [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://([^/]+\.)?baddomain2\.com [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://([^/]+\.)?baddomain3\.com [NC]
RewriteRule \.(gif|jpg)$ http://www.example.com/hotlink.gif [R,L]

Except from specific domains

RewriteEngine on
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http://(www\.)?example.com/.*$ [NC]
RewriteRule \.(gif|jpg)$ http://www.example.com/hotlink.gif [R,L]

Unless the image is displayed on example.com, browers would see the image hotlink.gif.

Note: Hotlink protection using .htaccess relies on the client sending the correct “Referer” value in the http GET request. Programs such as Windows Media Player send a blank referrer, so that attempts to use .htaccess to protect movie files for example are ineffective.
Standardise web address to require www with SEO-friendly 301 Redirect

If an address without the “www.” prefix is entered, this will redirect to the page with the “www.” prefix.

Options +FollowSymLinks
RewriteEngine On
RewriteBase /
RewriteCond %{HTTP_HOST} !^$ #check that HTTP_HOST field is present
RewriteCond %{HTTP_HOST} !^www\.sitename\.com$ [NC] #case-insensitive
RewriteRule ^(.*)$ http://www.sitename.com/$1 [R=301,L] #301 Redirect, very efficient

See the Ultimate htaccess File for more examples..
Directory rules

A .htaccess file controls the directory it is in, plus all subdirectories. However, by placing additional .htaccess files in the subdirectories, this can be overruled.
User permissions

The user permissions for .htaccess are controlled on server level with the AllowOverride directive which is documented in the Apache Server Documentation.

Custom PHP.ini and .htaccess rules

2009-05-15T06:59:00.000-07:00

Describes in exhaustive detail how to change configuration settings and implement a custom php.ini file for use with the Apache Web Server.

Sections:

When php run as Apache Module (mod_php)
When php run as CGI
When cgi?d php is run with wrapper (for FastCGI)

.htaccess code from Ultimate htaccess file

RUN PHP AS APACHE MODULE

AddHandler application/x-httpd-php .php .htm

RUN PHP AS CGI

AddHandler php-cgi .php .htm

CGI PHP WRAPPER FOR CUSTOM PHP.INI

AddHandler phpini-cgi .php .htm
Action phpini-cgi /cgi-bin/php5-custom-ini.cgi

FAST-CGI SETUP WITH PHP-CGI WRAPPER FOR CUSTOM PHP.INI

AddHandler fastcgi-script .fcgi
AddHandler php-cgi .php .htm
Action php-cgi /cgi-bin/php5-wrapper.fcgi

CUSTOM PHP CGI BINARY SETUP

AddHandler php-cgi .php .htm
Action php-cgi /cgi-bin/php.cgi

When php run as Apache Module (mod_php)

in root .htaccess

SetEnv PHPRC /location/todir/containing/phpinifile

When php run as CGI

Place your php.ini file in the dir of your cgi’d php, in this case /cgi-bin/

htaccess might look something like this

AddHandler php-cgi .php .htm
Action php-cgi /cgi-bin/php5.cgi

When php is run as cgi

Create a wrapper script called phpini.cgi to export the directory that contains the php.ini file as PHPRC

#!/bin/sh
export PHPRC=/home/site/askapache.com/inc
exec /user/htdocs/cgi-bin/php5.cgi

In your .htaccess or httpd.conf file

AddHandler php-cgi .php
Action php-cgi /cgi-bin/phpini.cgi

When cgi’d php is run with wrapper (for FastCGI)

You will have a shell wrapper script something like this:

#!/bin/sh
export PHP_FCGI_CHILDREN=3
exec /user/htdocs/cgi-bin/php5.cgi

Change To

#!/bin/sh
export PHP_FCGI_CHILDREN=3
exec /user/htdocs/cgi-bin/php.cgi -c /home/user/php.ini

NOTES:

Since PHP 5.1.0, it is possible to refer to existing .ini variables from within .ini files. open_basedir = ${open_basedir} ":/new/dir"
In order for PHP to read it, config file must be named php.ini
SetEnv PHPRC only works when using PHP as CGI, not when using php as an Apache Module

Default locations to look for php.ini

PHP looks for custom php.ini in this order:

In the Current working directory

Place your php.ini in the same directory as the php executable.
If php executable is here: /home/user1/htdocs/cgi-bin/
then place your php.ini file here: /home/user1/htdocs/cgi-bin/php.ini

In the path specified by the environment variable PHPRC

If you can use SetEnv in .htaccess files–> in the root .htaccess file specify the path to the directory containing php.ini SetEnv PHPRC /home/user1
If you can’t use SetEnv and you are using a wrapper shell script place this in your wrapper shell script export PHPRC=/home/user1

In the path that was defined at compile time with –with-config-file-path

The path in which the php.ini file is looked for can be overridden using the -c argument in command line mode. (cgi) /home/user1/htdocs/cgi-bin/php.cgi -c /home/user1/php.ini
With this option one can either specify a directory where to look for php.ini or you can specify a custom INI file directly (which does not need to be named php.ini),$ php -c /custom/directory/custom-file.ini my_script.php
Under Windows, the compile-time path is the Windows directory. Place php.ini in one of the directories, C:\windows or C:\winnt

php.ini is searched for in these locations in this order

NOTE: The Apache web server changes the directory to root at startup causing PHP to attempt to read php.ini from the root filesystem if it exists. If php-SAPI.ini exists (where SAPI is used SAPI, so the filename is e.g. php-cli.ini or php-apache.ini), it’s used instead of php.ini. SAPI name can be determined by php_sapi_name(). You can use also use the predefined PHP_SAPI constant instead of php_sapi_name()

Read this article: If your server is running Windows

SAPI module specific location
- PHPIniDir directive in Apache 2
- -c command line option in CGI and CLI
- php_ini parameter in NSAPI
- PHP_INI_PATH environment variable in THTTPD
The PHPRC environment variable (Before PHP 5.2.0 this was checked after the registry key mentioned below.)
HKEY_LOCAL_MACHINE\SOFTWARE\PHP\IniFilePath (Windows Registry location)
Current working directory (for CLI)
The web server’s directory (for SAPI modules)
Directory of PHP (If Windows)
Windows directory (C:\windows or C:\winnt)
–with-config-file-path compile time option

Directions for custom php.ini for Powweb Customers

Specific to Powweb, but can be used elsewhere.

SetEnv PHPRC /home/users/web/bEXAMPLE/pow.EXAMPLE

In the folder above the htdocs (your ROOT) for the domain you want a custom php.ini file for, create an htaccess file with the above content:
Then create a blank php.ini also in your ROOT directory (/home/users/web/bEXAMPLE/pow.EXAMPLE). Next copy the powweb php.ini text to your php.ini file and customize it.
You can test to make sure you are using the new php.ini by running phpinfo(); If you want multiple php.ini files, then use .htaccess files to set the PHPRC variable to the directory that the php.ini file you want to use is in.

File structure from ROOT directory

.
|-- site1.com
|   `-- htdocs
|   |   |-- cgi-bin
|   |   |   `-- dl.cgi
|   |   `-- index.html
|   |-- phpsessions
|   |-- php.ini
|   `-- .htaccess
|-- site2.org
|   `-- htdocs
|   |   |-- cgi-bin
|   |   |   `-- dl.cgi
|   |   `-- index.html
|   |-- phpsessions
|   |-- php.ini
|   `-- .htaccess
`-- site3.net
`-- htdocs
|   |-- cgi-bin
|   |   `-- dl.cgi
|   `-- index.html
|-- phpsession
|-- php.ini
`-- .htaccess

Powweb File Permissions

Remember to chmod 640 all .htaccess files, chmod 600 your php.ini files, chmod 600 your php flies, and chmod 705 your cgi scripts.. if you don’t want ftp users to be able to change the file than chmod 400.

PHP-CGI vs. MOD_PHP

What’s the difference between PHP-CGI and PHP as an Apache module?

Benefits of PHP-CGI

php-cgi is more secure. The PHP runs as your user rather than dhapache. That means you can put your database passwords in a file readable only by you and your php scripts can still access it!
php-cgi is more flexible. Because of security concerns when running PHP as an Apache module, we disabled commands with the non-CGI PHP. This will cause install problems with certain popular PHP scripts if you run PHP not as a CGI!
php-cgi is just as fast as running PHP as an Apache module, and we include more default libraries.

Caveats of PHP-CGI

If one of these is a show-stopper for you, you can easily switch to running PHP as an Apache module and not CGI, but be prepared for a bunch of potential security and ease-of-use issues! If you don’t know what any of these drawbacks mean, you’re fine just using the default setting of PHP-CGI and not worrying about anything!

Variables in the URL which are not regular ?foo=bar variables won’t work without using (mod_rewrite)
Custom php directives in .htaccess files (php_include_dir /home/user;/home/user/example_dir) won’t work.
The $_SERVER['SCRIPT_NAME'] variable will return the php.cgi binary rather than the name of your script
Persistant database connections will not work. PHP’s mysql_pconnect() function will just open a new connection because it can’t find a persistant one.

PHP’s configuration file

The configuration file (called php3.ini in PHP 3, and simply php.ini as of PHP 4) is read when PHP starts up. For the server module versions of PHP, this happens only once when the web server is started. Note: For the CGI and CLI version, php.ini is read on every invocation.

Running PHP as Apache module (mod_php)

When using PHP as an Apache module, you can also change the configuration settings using directives in Apache configuration files (e.g. httpd.conf) and .htaccess files. You will need one of these privileges:

AllowOverride Options
AllowOverride All

With PHP 4 and PHP 5, there are several Apache directives that allow you to change the PHP configuration from within the Apache configuration files.

NOTE: With PHP 3, there are Apache directives that correspond to each configuration setting in the php3.ini name, except the name is prefixed by “php3_”.

php_value name value: Sets the value of the specified directive. Can be used only with PHP_INI_ALL and PHP_INI_PERDIR type directives. To clear a previously set value use none as the value.
php_flag name on|off: Used to set a boolean configuration directive. Can be used only with PHP_INI_ALL and PHP_INI_PERDIR type directives.
php_admin_value name value: Sets the value of the specified directive. This can not be used in .htaccess files. Any directive type set with php_admin_value can not be overridden by .htaccess or virtualhost directives. To clear a previously set value use none as the value.
php_admin_flag name on|off: Used to set a boolean configuration directive. This can not be used in .htaccess files. Any directive type set with php_admin_flag can not be overridden by .htaccess or virtualhost directives.

NOTE: Don’t use php_value to set boolean values. use php_flag instead.

Change php settings in .htaccess or httpd.conf

mod_php .htaccess example

add settings to a .htaccess file with ‘php_flag’ like this:

php_flag register_globals off
php_flag magic_quotes_gpc on

In .htaccess, only true/false on/off flags can be set using php_flag. To set other values you need to use php_value, like this:

php_value upload_max_filesize 20M

PHP_INI_SYSTEM can be configured per-directory by placing it inside a per-directory block in httpd.conf

# Selectively enable APC for wildly popular directories
# apc.enabled is Off in php.ini to reduce memory use

php_flag apc.enabled On

NOTE: In order for these settings to work in your htaccess file, you will need to add “Options” to your AllowOverride specifications for the directory/webserver if it’s not already allowed.

Src: How to change configuration settings


php_value include_path ".:/home/askapache/lib/php"
php_admin_flag safe_mode on


php_value include_path ".:/home/askapache/lib/php"
php_admin_flag safe_mode on


php3_include_path ".:/home/askapache/lib/php"
php3_safe_mode on

Modify PHP configuration via Windows Registry

When running PHP on Windows, the configuration values can be modified on a per-directory basis using the Windows registry. The configuration values are stored in the registry key HKLM\SOFTWARE\PHP\Per Directory Values, in the sub-keys corresponding to the path names. For example, configuration values for the directory c:\inetpub\wwwroot would be stored in the key HKLM\SOFTWARE\PHP\Per Directory Values\c\inetpub\wwwroot. The settings for the directory would be active for any script running from this directory or any subdirectory of it. The values under the key should have the name of the PHP configuration directive and the string value. PHP constants in the values are
not parsed. However, only configuration values changeable in PHP_INI_USER can be set this way, PHP_INI_PERDIR values can not.

Methods to modify PHP configuration

Regardless of how you run PHP, you can change certain values at runtime of your scripts through ini_set().

If you are interested in a complete list of configuration settings on your system with their current values, you can execute the phpinfo() function, and review the resulting page. You can also access the values of individual configuration directives at runtime using ini_get() or get_cfg_var().

No input file specified

One of the most common reasons why you get

No input file specified

(AKA ‘the second most useful error message in the world’) is that you have set doc_root (in php.ini) to a value which is to the DocumentRoot defined in the apache configuration.

This is the same for other webservers. For example, on lighttpd, make sure the server.document-root value is the same as what is defined as doc_root in php.ini.

Sylesh H

syleshh@gmail.com

9847589760

Threads vs. Processes

2009-04-23T04:09:00.000-07:00

Threads vs. Processes

Both threads and processes are methods of parallelizing an application. However, processes are independent execution units that contain their own state information, use their own address spaces, and only interact with each other via interprocess communication mechanisms (generally managed by the operating system). Applications are typically divided into processes during the design phase, and a master process explicitly spawns sub-processes when it makes sense to logically separate significant application functionality. Processes, in other words, are an architectural construct.

By contrast, a thread is a coding construct that doesn't affect the architecture of an application. A single process might contains multiple threads; all threads within a process share the same state and same memory space, and can communicate with each other directly, because they share the same variables.

Threads typically are spawned for a short-term benefit that is usually visualized as a serial task, but which doesn't have to be performed in a linear manner (such as performing a complex mathematical computation using parallelism, or initializing a large matrix), and then are absorbed when no longer required. The scope of a thread is within a specific code module—which is why we can bolt-on threading without affecting the broader application.

Showing unlimted quota in WHM(VPS)!! or /scripts/fixauotas results 'No filesystems with quota detected '

2009-04-21T22:04:00.000-07:00

This is neither a cPanel nor a Virtuozzo issue.

You should check whether your user/group quotas are enabled or not :

e.g.

vzquota stat 101 -t

resource usage softlimit hardlimit grace
1k-blocks 14929508 51200000 51200000
inodes 222034 400000 440000

User/group quota: off,inactive
Ugids: loaded 0, total 0, limit 0
Ugid limit was exceeded: no

Enable quotas over Virtuozzo. If there are hardware or other
problems who prevent this setting do it manually:

vi /etc/sysconfig/vz-scripts/101.conf

change QUOTAUGIDLIMIT="0" to QUOTAUGIDLIMIT="3000" (QUOTAUGIDLIMIT > This is used for tracking quote for all your files in the system)

restart VPS. Done.

Login to the VPS (VEID) and run /scripts/fixquotas

Problem should be solved.

Sylesh

ffmpeg in CentOS is with yum

2009-04-15T20:41:00.001-07:00

The most easy way to install ffmpeg in CentOS is with yum.
First of all, edit /etc/yum.repos.d/CentOS-Base.repo and add those lines at bottm of file:
[dag]
name=Dag RPM Repository for Centos
baseurl=http://apt.sw.be/redhat/el$releasever/en/$basearch/dag
enabled=1

Then, run command:
yum install ffmpeg

Or, you can install from svn:

svn checkout svn://svn.mplayerhq.hu/ffmpeg/trunk ffmpeg

cd ffmpeg
./configure --help

add some parameters, like --enable-gpl, --enable-libmp3lame, etc..

make && make install

(if svn command not found, type yum install subversion)

Hope this help

traceroute

2009-04-08T15:57:00.000-07:00

When you execute a trace route command (ie traceroute yahoo.com), your machine sends out 3 UDP packets with a TTL (Time-to-Live) of 1. When those packets reach the next hop router, it will decrease the TTL to 0 and thus reject the packet. It will send an ICMP Time-to-Live Exceeded (Type 11), TTL equal 0 during transit (Code 0) back to your machine - with a source address of itself, therefore you now know the address of the first router in the path.

Next your machine will send 3 UDP packets with a TTL of 2, thus the first router that you already know passes the packets on to the next router after reducing the TTL by 1 to 1. The next router decreases the TTL to 0, thus rejecting the packet and sending the same ICMP Time-to-Live Exceeded with its address as the source back to your machine. Thus you now know the first 2 routers in the path.

This keeps going until you reach the destination. Since you are sending UDP packets with the destination address of the host you are concerned with, once it gets to the destination the UDP packet is wanting to connect to the port that you have sent as the destination port, since it is an uncommon port, it will most like be rejected with an ICMP Destination Unreachable (Type 3), Port Unreachable (Code 3). This ICMP message is sent back to your machine, which will understand this as being the last hop, therefore trace route will exit, giving you the hops between you and the destination.

The UDP packet is sent on a high port, destined to another high port. On a Linux box, these ports were not the same, although usually in the 33000. The source port stayed the same throughout the session, however the destination port was increase by one for each packet sent out.

One note, traceroute actually sends 1 UDP packet of TTL, waits for the return ICMP message, sends the second UDP packet, waits, sends the third, waits, etc, etc, etc.

If during the session, you receive * * *, this could mean that that router in the path does not return ICMP messages, it returns messages with a TTL too small to reach your machine or a router with buggy software. After a * * * within the path, trace route will still increment the TTL by 1, thus still continuing on in the path determination.

The DNS Database Files

2009-04-01T14:26:00.000-07:00

Master files included by named, like named.hosts, always have a domain associated with them, which is called the origin. This is the domain name specified with the cache and primary commands. Within a master file, you are allowed to specify domain and host names relative to this domain. A name given in a configuration file is considered absolute if it ends in a single dot, otherwise it is considered relative to the origin. The origin all by itself may be referred to using ``@''.

All data contained in a master file is split up in resource records, or RRs for short. They make up the smallest unit of information available through DNS. Each resource record has a type. A records, for instance, map a hostname to an IP-address, and a CNAME record associates an alias for a host with its official hostname.

Resource record representations in master files share a common format, which is

           [domain] [ttl] [class] type rdata

Fields are separated by spaces or tabs. An entry may be continued across several lines if an opening brace occurs before the first newline, and the last field is followed by a closing brace. Anything between a semicolon and a newline is ignored.

domain: This is the domain name to which the entry applies. If no domain name is given, the RR is assumed to apply to the domain of the previous RR.
ttl: In order to force resolvers to discard information after a certain time, each RR is associated a ``time to live'', or ttl for short. The ttl field specifies the time in seconds the information is valid after it has been retrieved from the server. It is a decimal number with at most eight digits. If no ttl value is given, it defaults to the value of the minimum field of the preceding SOA record.
class: This is an address class, like IN for IP addresses, or HS for objects in the Hesiod class. For TCP/IP networking, you have to make this IN. If no class field is given, the class of the preceding RR is assumed.
type: This describes the type of the RR. The most common types are A, SOA, PTR, and NS. The following sections describe the var- ious types of RR's.
rdata: This holds the data associated with the RR. The format of this field depends on the type of the RR. Below, it will be described for each RR separately.

The following is an incomplete list of RRs to be used in DNS master files. There are a couple more of them, which we will not explain. They are experimental, and of little use generally.

This describes a zone of authority (SOA means ``Start of Authority''). It signals that the records following the SOA RR contain authoritative information for the domain. Every master file included by a primary statement must contain an SOA record for this zone. The resource data contains the following fields:

origin

This is the canonical hostname of the primary name server for this domain. It is usually given as an absolute name.

contact

This is the email address of the person responsible for maintaining the domain, with the `@' character replaced by a dot. For instance, if the responsible person at the Virtual Brewery is janet, then this field would contain janet.vbrew.com.

serial

This is the version number of the zone file, expressed as a single decimal number. Whenever data is changed in the zone file, this number should be incremented. The serial number is used by secondary name servers to recognize when zone information has changed. To stay up to date, secondary servers request the primary server's SOA record at certain intervals, and compare the serial number to that of the cached SOA record. If the number has changed, the secondary servers transfers the whole zone database from the primary server.

refresh

This specifies the interval in seconds that the sec- ondary servers should wait between checking the SOA record of the primary server. Again, this is a deci- mal number with at most eight digits. Generally, the network topology doesn't change too often, so that this number should specify an interval of roughly a day for larger networks, and even more for smaller ones.

retry

This number determines the intervals at which a sec- ondary server should retry contacting the primary server if a request or a zone refresh fails. It must not be too low, or else a temporary failure of the server or a network problem may cause the secondary server to waste network resources. One hour, or perhaps one half hour, might be a good choice.

expire

This specifies the time in seconds after which the server should finally discard all zone data if it hasn't been able to contact the primary server. It should normally be very large. Craig Hunt ([ GETST "hunt-tcpip" ]) recommends 42 days.

minimum

This is the default ttl value for resource records that do not explicitly specify one. This requires other name servers to discard the RR after a certain amount of time. It has however nothing to do with the time after which a secondary server tries to update the zone information. minimum should be a large value, especially for LANs where the network topology almost never changes. A value of around a week or a month is probably a good choice. In the case that single RRs may change more frequently, you can still assign them different ttl's.

A

This associates an IP address with a hostname. The resource data field contains the address in dotted quad nota- tion. For each host, there must be only one A record. The hostname used in this A record is considered the official or canonical hostname. All other hostnames are aliases and must be mapped onto the canonical hostname using a CNAME record.

NS

This points to a master name server of a subordinate zone. For an explanation why one has to have NS records, see section 3.6. The resource data field contains the hostname of the name server. To resolve the hostname, an additional A record is needed, the so-called glue record which gives the name server's IP address.

CNAME

This associates an alias for a host with its canonical hostname. The canonical hostname is the one the master file provides an A record for; aliases are simply linked to that name by a CNAME record, but don't have any other records of their own.

PTR

This type of record is used to associate names in the in- addr.arpa domain with hostnames. This is used for reverse map- ping of IP addresses to hostnames. The hostname given must be the canonical hostname.

MX

This RR announces a mail exchanger for a domain. The rea- sons to have mail exchangers are discussed in section 14.4.1 in chapter 14.. The syntax of an MX record is

                     [domain] [ttl] [class] MX preference host

host names the mail exchanger for domain. Every mail exchanger has an integer preference associated with it. A mail transport agent who desires to deliver mail to domain will try all hosts who have an MX record for this domain until it succeeds. The one with the lowest preference value is tried first, then the others in order of increasing preference value.

HINFO

This record provides information on the system's hardware and software. Its syntax is

                     [domain] [ttl] [class] HINFO hardware software

The hardware field identifies the hardware used by this host. There are special conventions to specify this. A list of valid names is given in the ``Assigned Numbers'' (RFC 1340). If the field contains any blanks, it must be enclosed in double quotes. The software field names the operating sys- tem software used by the system. Again, a valid name from the ``Assigned Numbers'' RFC should be chosen.

Explaining DNS Database Files

2009-04-01T12:31:00.000-07:00

Explaining DNS Database Files

This is a typical DNS zone.domain file for the domain maxx.net. (Its name would be zone.maxx.net. It will translate from a host name to its IP address.)

;
; Addresses for the local domain

maxx.net.    IN      SOA   nova.maxx.net. tyager.nova.maxx.net. (

                          9602171        ; Serial
                         36000          ; Refresh every 10 hours
                         3600           ; Retry after 1 hour
                         360000         ; Expire after 100 hours
                         36000          ; Minimum TTL is 10 hours )

;   Define name servers
;
maxx.net.    IN      NS     nova.maxx.net.
maxx.net.    IN      A      204.251.17.241

;   Define localhost
;
localhost    IN      A      127.0.0.1

;   Set up hosts
;
maxx        IN      A       204.251.17.241
           IN      MX   5  nova.maxx.net.

maxx.net.     IN    MX   5  nova.maxx.net.
;
;   All mail for net delivered to nova
;
;*          IN     MX   10  nova.maxx.net.
www         IN     CNAME    nova.maxx.net.
ftp         IN     CNAME    nova.maxx.net.
news        IN     CNAME    nova.maxx.net.
mail        IN     CNAME    nova.maxx.net.
ns          IN     CNAME    nova.maxx.net.
loghost     IN     CNAME    nova.maxx.net.
lucy        IN     A        204.251.17.242
linux       IN     CNAME    lucy.maxx.net.
lucy        IN     MX   10  lucy.maxx.net.
messdos      IN     A        204.251.17.243
messdos      IN     MX   10  messdos.maxx.net.
pentium      IN     CNAME    messdos.maxx.net.
solaris      IN     A        204.251.17.244
solaris      IN     MX   10  solaris.maxx.net.
maxx4        IN     CNAME    solaris.maxx.net.
maxx5        IN     A        204.251.17.245
maxx5        IN     MX   10  maxx5.maxx.net.
maxx6        IN     A        204.251.17.246
maxx6        IN     MX   10  maxx6.maxx.net.

Most database file entries are known as DNS resource records. Generally, the resource records are shown in order: SOA, NS, followed by the other types, but this ordering isn't required. The data in each entry may be entered in upper, lower, or mixed case. All entries in the database file must start at the beginning of the line. Blank lines as well as any text following a semicolon is ignored.

SOA stands for Start of Authority. This acronym notifies named that operational parameters follow. The most important one is the Serial field. Every time you make a change to a database file, you must increment its serial number. Only by doing this will secondary servers know they need to reach into your system and pull out new name server data, a procedure known as a "zone transfer." Many DNS administrators use a date-time stamp for this field, like 9602171 for the first version on February 17, 1996.

First, focus on the SOA section:

maxx.net. IN SOA nova.maxx.net. tyager.maxx.maxx.net.

The "maxx.net." field tells named the domain defined by this file. The name server will automatically append it to any host name that appears in the file. The trailing dot is not a type; it keeps named from trying to tack on your domain name. Without it, the resolver would be confused by named's expansion of my domain name to "maxx.net.maxx.net."

The IN stands for the "Internet" class of data. Even though other classes exist, they aren't in common usage. The "nova.maxx.net" field is the host on which these database files reside. Finally, "tyager.nova.maxx.net" represents the e-mail address of the DNS administrator, where the first dot (between tyager and nova) would be replaced by the @ symbol to create a valid address. (The @ symbol can't be used here because it has a reserved meaning in DNS database files.)The open parenthesis at the end of the line allows you to split the SOA record across physical lines for readability:

           9602171          ; Serial

           36000            ; Refresh every 10 hours
          3600             ; Retry after 1 hour
          360000           ; Expire after 100 hours
          36000            ; Minimum TTL is 10 hours )

The "serial" field was discussed earlier.

The remaining four fields specify various time intervals (all values in seconds) used by the secondary name server:

Refresh: The time interval that must elapse between each poll of the primary by the secondary name server (here 36,000 seconds or 10 hours). If the "serial number" has been updated on the primary, the secondary assumes its data is stale and requests updated information as a "zone transfer."
Retry: The time interval used between successive connection attempts by the secondary to reach the primary name server in case the first attempt failed (here 3,600 seconds or one hour). Generally, less than the "refresh" time.
Expire: The time interval after which the secondary expires its data if it can't reach the primary name server (here 360,000 seconds or 100 hours). The secondary will refuse to service requests after this interval.
Minimum: The minimum time-to-live value, which specifies how long other servers should cache data from the name server (here 36,000 seconds or 10 hours).

There are several types of resource records, identified by the key word in field three of each record. You may present records in any order, but try to organize them for clarity. The NS (name server) record tells the hosts that query your server where the name servers for this domain can be found:

maxx.net.     IN     NS     nova.maxx.net.

You must include in this list at least one name server, that is the name of the server specified in the SOA record. You can list multiple name servers for your domain. In fact, your domain should have at least two name servers. Your Internet service provider will probably allow you to use their name server as a secondary for your domain, but it must have the trailing dots!

maxx.net     IN     A     204.251.17.241

The first A record, which resolves a fully-qualified host name to an IP address, is a special one. It defines an IP address for unqualified queries, that is, queries for the host maxx.net.

Other A records like this one:

lucy         IN     A      204.251.17.242

provide name-to-address mapping for a specific named host. The domain defined in this file (maxx.net) is appended to the host name you show in the first field.

The CNAME records create aliases for existing hosts. These examples illustrate a few common uses:

www     IN     CNAME     maxx.maxx.net.
ftp IN CNAME maxx.maxx.net.

You can give a host any alias you like, and as many aliases as you want. The host needn't answer to that name, that is, the alias doesn't need to be the host's true name as reported by hostname or uname.

The other vital type of record is MX. This tells SMTP e-mail software where to send mail for each named host:

lucy     IN     MX   10  lucy.maxx.net.

When a remote host's mail delivery program sees an e-mail address in your domain, it will query your name server for its applicable MX record or records. Every user on your LAN can receive e-mail, even if not every host is running its own e-mail software. The MX record for lucy, for instance, could easily redirect e-mail to another host on the LAN.

The number (10 in this case) in the fourth field represents a preference value. If you define multiple MX records for a host, delivery is attempted to lower-preference value hosts first. The actual value isn't important, only its relationship to other preference values.

On larger LANs it's a good idea to create backup e-mail servers. Smaller LANs can simply rely on the fact that most SMTP mailers will retry deliveries to the site for three days before returning a message to its sender.

The line, shown commented out here, would arrange to redirect e-mail for all hosts in this domain to a single machine:

;
; All mail for net delivered to nova
;
;*     IN     MX   10  nova.maxx.net.

This is a very good idea for LANs that benefit from a central e-mail repository.

Address-to-Name Mapping

Also called reverse mapping, the zone.ADDR db file allows resolvers to post queries armed with only the IP address of a host. This reverse mapping is used, for example, by Internet server software that prefers to log host names rather than less informative IP addresses.

Address-to-name mapping data will be provided for a DNS server by PTR entries in its zone.ADDR files, one for every network served by this DNS server, and its zone.LOCAL file.

Each entry will indicate the IP address in reverse order, then the host name. For example, for host littledog.maxx.net, whose IP address is 204.251.17.249, in the zone.ADDR file it's PTR entry would look like:

249.17.251.204.     IN     PTR     littledog.maxx.net.

Why is it backwards? Recall that DNS does its parsing from right to left, from most inclusive to most specific. For IP addresses, it needs to parse in the same direction. But IP addresses, from right to left, go from most specific to most inclusive. So the simple answer is to reverse the IP address in the NDS PTR records. Now DNS can parse in the same direction, and resolve in the same order — from most inclusive to most specific.

A shortcut in PTR records is often used. It looks like this:

249         IN     PTR     littledog.maxx.net.

If the dot is left off the IP address in the PTR record, DNS will complete the IP address with the IP address of the domain, specified in the file's SOA record. This is also true for A records in name-to-address mapping db files. If the dot is left off, DNS will automatically try to complete the name with the full domain name in this zone. Paying attention to the terminating dot is important.

For the zone.LOCAL file we describe the loopback address just as you would expect it, now that we know we have to reverse it. The PTR entry in the zone.LOCAL file would look like:

     1.0.0.127.     IN     PTR     localhost.

or, using the shortcut:

     1             IN     PTR     localhost.

Only one line from named.conf remains to be discussed, the "cache" entry. This is a bit of a misnomer as it doesn't have anything to do with local caching. Instead, it defines the master root domain name servers for the Internet. You can retrieve this list from ftp://nic.ddn.mil/netinfo/root-servers.txt. You will need to check this site periodically to ensure you have the latest list.

This file lists the root domain servers in human-readable format. You'll need to reformat it for consumption by named. Here's what the cache file looks like:

;          Servers from the root domain
;          ftp://nic.ddn.mil/netinfo/root-servers.txt
;
.                    99999999     IN     NS   A.ROOT-SERVERS.NET
.                    99999999     IN     NS   B.ROOT-SERVERS.NET
.                    99999999     IN     NS   C.ROOT-SERVERS.NET
.                    99999999     IN     NS   D.ROOT-SERVERS.NET
.                    99999999     IN     NS   E.ROOT-SERVERS.NET
.                    99999999     IN     NS   F.ROOT-SERVERS.NET
.                    99999999     IN     NS   G.ROOT-SERVERS.NET
.                    99999999     IN     NS   H.ROOT-SERVERS.NET
.                    99999999     IN     NS   I.ROOT-SERVERS.NET

; Root servers by address

A.ROOT-SERVERS.NET   99999999     IN   A 198.41.0.4
B.ROOT-SERVERS.NET   99999999     IN   A 128.9.0.107
C.ROOT-SERVERS.NET   99999999     IN   A 192.33.4.12
D.ROOT-SERVERS.NET   99999999     IN   A 128.8.10.90
E.ROOT-SERVERS.NET   99999999     IN   A 192.203.230.10
F.ROOT-SERVERS.NET   99999999     IN   A 192.5.5.241
G.ROOT-SERVERS.NET   99999999     IN   A 192.112.36.4
H.ROOT-SERVERS.NET   99999999     IN   A 128.63.2.53
I.ROOT-SERVERS.NET   99999999     IN   A 192.36.148.17

Here, the dot (.) refers to the root domain and the 99999999 means a very long time-to-live value. The TTL value is no longer used for caching because the data isn't discarded if it times out, but administrators generally keep it around because it does no harm.

Your site may not have access to the Internet or may have protected its connection via a firewall. Often in this type of DNS configuration, one or more machines will be designated as a root server. In this case, the cache file will contain a list of internal root servers, and not the official Internet master root domain servers.

Testing Your Name Server

Perform simple checks on your name server's health with nslookup. This utility is standard with every TCP/IP-network-aware version of UNIX. There are other similar tools available — see "List of Utilities" later in this section for details.

You can find the source code for dig at several anonymous FTP archive sites, including: ftp://ftp.wonderland.org/NetBSD/NetBSD-current/src/usr.sbin/named/dig/ for the NetBSD release. Use Archie to find other sites.

The nslookup utility can be used interactively, much like other programs, such as ftp. That is, if you invoke this program without command line arguments, it displays a prompt and waits for your command:

>server mpe3000

Default Name Server: mpe3000.cup.hp.com Address: 15.13.199.80

By default, nslookup performs queries based on host names you submit; just enter a host name after the prompt:

> romeo
Server:   mpe3000.cup.hp.com
Address:   15.13.199.80

Name:     romeo.cup.hp.com
Address:   15.13.194.242

> 15.12.194.242
Server:   mpe3000.cup.hp.com
Address:   15.13.199.80

Name:     romeo.cup.hp.com
Address:   15.12.194.242

You can check the resource records information about name server:

> set type=ns
> mpeworld
Name Server:   mpeworld.cup.hp.com
Address:   15.13.199.80

origin = dns.cup.hp.com
mail addr = dns-admin.dns.cup.hp.com
serial = 96092255
refresh = 10800 (3 hours)
retry = 3600 (1 hour) expire = 604800 (7 days)
minimum ttl = 86400 (1 day)

How SSL works??

2009-03-19T02:03:00.000-07:00

SSL is a sophistication encryption scheme that does not require the client and the server to arrange for a secret key to be exchanged between the client and server BEFORE the transaction is started. SSL uses public/private keys to provide a flexible encryption scheme that can be setup at the time of the secure transaction.
In typical encryption schemes the client and server would be required to use a secret key that has been preconfigured in the client and the server machines. In such a scheme, the client would use the secret key to encrypt the data. The server would use the same secret key to decrypt the data. Same logic applies in the server to client direction. This type of preconfigured secret keys are not suitable for Web based secure services that involve millions of users who have no prior secret key arrangement with the secure server.

SSL solves this problem by using asymmetric keys. These keys are defined in pairs of public and private keys. As the name suggests the
public key is freely available to anybody. The private key is known only to the server. The keys have two important properties:

(1) Data encrypted by the client using the pubic key can be decrypted only by the server's private key. Due to this property of the keys, the client is able to send secure data that can be understood only by the server.

(2) Data encrypted to by the server's private key can only be decrypted using the public key. This property is useful in a client level authentication of the server. If the server sends a known message (say the name of the server), the client can be sure that it is talking to the authentic server and not an imposter if it is successfully able to decrypt the message using the public key.

Note that property (1) allows us to use conventional secret keys. A secret key can be sent by the client as data that has been encrypted using the public key. This secret key can be decrypted only by the server. Once the server gets the key, the client and the server are able to communicate using this secret key.
The public/private key based encryption is used only for handshaking and secret key exchange. Once the keys have been exchanged the symmetric secret keys are used. This is done for two reasons:

(1) Public/private key based encryption techniques are computationally very expensive thus their use should be minimized.

(2) The secret key mechanism is needed for server to client communication.

Openvz - vzquota : (error) Quota on syscall for 200: Device or resource busy

2009-03-04T14:10:00.001-08:00

If you see this error when trying to start up a vps then you will need to kill off some processes.

Run this on the node to find them. Be sure to replace 200 with the correct veid.

lsof 2> /dev/null | egrep '/vz/root/200|/vz/private/200'

Kill -9 the PID's listed from the previous command.

The vps should now start up fine.

Thanks,
Sylesh

Sed Example

2009-03-04T13:53:00.001-08:00

sed -e 's/oldstuff/newstuff/g' inputFileName > outputFileName

Difference b/w Ping and Traceroute?

2009-03-01T22:00:00.000-08:00

Ping
----

Ping is a program that sends a series of packets over a network or the Internet to a specific computer in order to generate a response from that computer. The other computer responds with an acknowledgment that it received the packets. Ping was created to verify whether a specific computer on a network or the Internet exists and is connected.
Ping uses ICMP (Internet Control Message Protocol) packets. The packet from the origin computer is called an "ICMP_echo_request", and the response from the target is called an "ICMP_echo_reply". Each packet contains by default either 32 or 64 bytes of data and 8 bytes of protocol reader information, but ping can be configured at the command line to use different sized packets. You can access a list of switches and additional functions by invoking the help file for ping:

In an IP network, `ping' sends a short data burst - a single packet - and listens for a single packet in reply. Since this tests the most basic function of an IP network (delivery of single packet), it's easy to see how you can learn a lot from some `pings'.

Ping is implemented using the required ICMP Echo function, documented in RFC 792 that all hosts should implement. Of course, administrators can disable ping messages (this is rarely a good idea, unless security considerations dictate that the host should be unreachable anyway), and some implementations have (gasp) even been known not to implement all required functions. However, ping is usually a better bet than almost any other network software.

Many versions of ping are available. For the remainder of this discussion, I assume use of BSD UNIX's ping, a freely available, full-featured ping available for many UNIX systems. Most PC-based pings do not have the advanced features I describe. As always, read the manual for whatever version you use.

What Ping can tell you

* Ping places a unique sequence number on each packet it transmits, and reports which sequence numbers it receives back. Thus, you can determine if packets have been dropped, duplicated, or reordered.
* Ping checksums each packet it exchanges. You can detect some forms of damaged packets.
* Ping places a timestamp in each packet, which is echoed back and can easily be used to compute how long each packet exchange took - the Round Trip Time (RTT).
* Ping reports other ICMP messages that might otherwise get buried in the system software. It reports, for example, if a router is declaring the target host unreachable.

What Ping can not tell you

* Some routers may silently discard undeliverable packets. Others may believe a packet has been transmitted successfully when it has not been. (This is especially common over Ethernet, which does not provide link-layer acknowledgments) Therefore, ping may not always provide reasons why packets go unanswered.
* Ping can not tell you why a packet was damaged, delayed, or duplicated. It can not tell you where this happened either, although you may be able to deduce it.
* Ping can not give you a blow-by-blow description of every host that handled the packet and everything that happened at every step of the way. It is an unfortunate fact that no software can reliably provide this information for a TCP/IP network.
**********************************************************************************************
Traceroute

Traceroute is the program that shows you the route over the network between two systems, listing all the intermediate routers a connection must pass through to get to its destination. It can help you determine why your connections to a given server might be poor, and can often help you figure out where exactly the problem is. It also shows you how systems are connected to each other, letting you see how your ISP connects to the Internet as well as how the target system is connected.
Traceroute utilities work by sending packets with low time-to-live (TTL) fields. The TTL value specifies how many hops the packet is allowed before it is returned. When a packet can't reach its destination because the TTL value is too low, the last host returns the packet and identifies itself. By sending a series of packets and incrementing the TTL value with each successive packet, traceroute finds out who all the intermediary hosts are.

Sylesh

List of Different web servers on GNU Linux

2009-02-28T08:27:00.001-08:00

List of Different web servers on GNU Linux

Today we have different web servers available on the GNU/Linux platform.Here i am listing only the web servers that can be used for a mainstream web hosting environment

The web servers can be classified basically into 2

1. Process based webserver - In a process based web server each new connection is handled by a new web server process or thread. Following are some of the process based servers available on the GNU /Linux platform.A notable draw back for a process based web server is the famous C10K problem ( http://www.kegel.com/c10k.html ).That said there are many implementations of this model that are high performance in nature

* Apache HTTPD - http://httpd.apache.org/ - the most famous web server and the most widely used - supports a very large number of modules and very feature rich
* Cherokee - http://www.cherokee-project.com/ - is a fast flexible easy to configure web server - A notable feature is the administrative interface available that lessens the burden of tampering with the configuration files
* LiteSpeed - http://www.litespeedtech.com/ - Apache compatible web server ,but claimed to be of having improved perfomance

2- Asynchronus web server or non-blocking i/o webserver - Or in laymans words a web server that uses a single process to handle multiple request ,switching over to the next request when the previous one is waiting for an i/o request is to be fullfilled. This architecture is more scalable and theoretically solves the C10K problem.The famous members of this family of servers are

* Nginx - http://wiki.codemongers.com/Main - pronounced EngineX - is a powerfull light weight web server and has a cleaner configuration -It is also feature rich and widely used
* Lighttpd - http://www.lighttpd.net/ - This is another powerfull and light weight web server and has a very large number of modules and features

All the above mentioned web servers are widely used in the web hosting industry ,in dedicated ,vps and shared hosting environments

SED Tutorial

2009-02-27T01:49:00.001-08:00

SED Tutorial

* The sed utility is an "editor"
* It is also noninteractive. This means you have to insert commands to be executed on the data at the command line or in a script to be processed.
* sed accepts a series of commands and executes them on a file (or set of files).
* sed fittingly stands for stream editor.
* It can be used to change all occurrences of "SAD" to "SED" or "New York" to "Newport."
* The stream editor is ideally suited to performing repetitive edits that would take considerable time if done manually.

How sed Works

The sed utility works by sequentially reading a file, line by line, into memory. It then performs all actions specified for the line and places the line back in memory to dump to the terminal with the requested changes made. After all actions have taken place to this one line, it reads the next line of the file and repeats the process until it is finished with the file. As mentioned, the default output is to display the contents of each line on the screen. Two important factors come into play here—first, the output can be redirected to another file to save the changes; second, the original file, by default, is left unchanged. The default is for sed to read the entire file and make changes to each line within it. It can, however, be restricted to specified lines as needed.

The syntax for the utility is:

sed [options] '{command}' [filename]

In this tutorial we will walk through the most commonly used commands and options and illustrate how they work and where they would be appropriate for use.

The Substitute Command

One of the most common uses of the sed utility, and any similar editor, is to substitute one value for another. To accomplish this, the syntax for the command portion of the operation is:

's/{old value}/{new value}/'

Thus, the following illustrates how "lion" can be changed to "eagle" very simply:

$ echo The lion group will meet on Tuesday after school | sed
's/lion/eagle/'
The eagle group will meet on Tuesday after school
$

Notice that it is not necessary to specify a filename if input is being derived from the output of a preceding command—the same as is true for awk, sort, and most other LinuxUNIX command-line utility programs.

Multiple Changes

If multiple changes need to be made to the same file or line, there are three methods by which this can be accomplished. The first is to use the "-e" option, which informs the program that more than one editing command is being used. For example:

$ echo The lion group will meet on Tuesday after school | sed -e '
s/lion/eagle/' -e 's/after/before/'
The eagle group will meet on Tuesday before school
$

This is pretty much the long way of going about it, and the "-e" option is not commonly used to any great extent. A more preferable way is to separate command with semicolons:

$ echo The lion group will meet on Tuesday after school | sed '
s/lion/eagle/; s/after/before/'
The eagle group will meet on Tuesday before school
$

Notice that the semicolon must be the next character following the slash. If a space is between the two, the operation will not successfully complete and an error message will be returned. These two methods are well and good, but there is one more method that many administrators prefer. The key thing to note is that everything between the two apostrophes (' ') is interpreted as sed commands. The shell program reading in the commands will not assume you are finished entering until the second apostrophe is entered. This means that the command can be entered on multiple lines—with Linux changing the prompt from PS1 to a continuation prompt (usually ">")—until the second apostrophe is entered. As soon as it is entered, and Enter pressed, the processing will take place and the same results will be generated, as the following illustrates:

$ echo The lion group will meet on Tuesday after school | sed '
> s/lion/eagle/
> s/after/before/'
The eagle group will meet on Tuesday before school
$

Global Changes

Let's begin with a deceptively simple edit. Suppose the message that is to be changed contains more than one occurrence of the item to be changed. By default, the result can be different than what was expected, as the following illustrates:

$ echo The lion group will meet this Tuesday at the same time
as the meeting last Tuesday | sed 's/Tuesday/Thursday/'
The lion group will meet this Thursday at the same time
as the meeting last Tuesday
$

Instead of changing every occurrence of "Tuesday" for "Thursday," the sed editor moves on after finding a change and making it, without reading the whole line. The majority of sed commands function like the substitute one, meaning they all work for the first occurrence of the chosen sequence in each line. In order for every occurrence to be substituted, in the event that more than one occurrence appears in the same line, you must specify for the action to take place globally:

$ echo The lion group will meet this Tuesday at the same time
as the meeting last Tuesday | sed 's/Tuesday/Thursday/g'
The lion group will meet this Thursday at the same time
as the meeting last Thursday
$

Bear in mind that this need for globalization is true whether the sequence you are looking for consists of only one character or a phrase.

sed can also be used to change record field delimiters from one to another. For example, the following will change all tabs to spaces:

sed 's/ / /g'

where the entry between the first set of slashes is a tab, while the entry between the second set is a space. As a general rule, sed can be used to change any printable character to any other printable character. If you want to change unprintable characters to printable ones—for example, a bell to the word "bell"—sed is not the right tool for the job (but tr would be).

Sometimes, you don't want to change every occurrence that appears in a file. At times, you only want to make a change if certain conditions are met—for example, following a match of some other data. To illustrate, consider the following text file:

$ cat sample_one
one 1
two 1
three 1
one 1
two 1
two 1
three 1
$

Suppose that it would be desirable for "1" to be substituted with "2," but only after the word "two" and not throughout every line. This can be accomplished by specifying that a match is to be found before giving the substitute command:

$ sed '/two/ s/1/2/' sample_one
one 1
two 2
three 1
one 1
two 2
two 2
three 1
$

And now, to make it even more accurate:

$ sed '
> /two/ s/1/2/
> /three/ s/1/3/' sample_one
one 1
two 2
three 3
one 1
two 2
two 2
three 3
$

Bear in mind once again that the only thing changed is the display. If you look at the original file, it is the same as it always was. You must save the output to another file to create permanence. It is worth repeating that the fact that changes are not made to the original file is a true blessing in disguise—it lets you experiment with the file without causing any real harm, until you get the right commands working exactly the way you expect and want them to.

The following saves the changed output to a new file:

$ sed '
> /two/ s/1/2/
> /three/ s/1/3/' sample_one > sample_two

The output file has all the changes incorporated in it that would normally appear on the screen. It can now be viewed with head, cat, or any other similar utility.

Script Files

The sed tool allows you to create a script file containing commands that are processed from the file, rather than at the command line, and is referenced via the "-f" option. By creating a script file, you have the ability to run the same operations over and over again, and to specify far more detailed operations than what you would want to try to tackle from the command line each time.

Consider the following script file:

$ cat sedlist
/two/ s/1/2/
/three/ s/1/3/
$

It can now be used on the data file to obtain the same results we saw earlier:

$ sed -f sedlist sample_one
one 1
two 2
three 3
one 1
two 2
two 2
three 3
$

Notice that apostrophes are not used inside the source file, or from the command line when the "-f" option is invoked. Script files, also known as source files, are invaluable for operations that you intend to repeat more than once and for complicated commands where there is a possibility that you may make an error at the command line. It is far easier to edit the source file and change one character than to retype a multiple-line entry at the command line.

Restricting Lines

The default is for the editor to look at, and for editing to take place on, every line that is input to the stream editor. This can be changed by specifying restrictions preceding the command. For example, to substitute "1" with "2" only in the fifth and sixth lines of the sample file's output, the command would be:

$ sed '5,6 s/1/2/' sample_one
one 1
two 1
three 1
one 1
two 2
two 2
three 1
$

In this case, since the lines to changes were specifically specified, the substitute command was not needed. Thus you have the flexibility of choosing which lines to changes (essentially, restricting the changes) based upon matching criteria that can be either line numbers or a matched pattern.

Prohibiting the Display

The default is for sed to display on the screen (or to a file, if so redirected) every line from the original file, whether it is affected by an edit operation or not; the "-n" parameter overrides this action. "-n" overrides all printing and displays no lines whatsoever, whether they were changed by the edit or not. For example:

$ sed -n -f sedlist sample_one
$

$ sed -n -f sedlist sample_one > sample_two
$ cat sample_two
$

In the first example, nothing is displayed on the screen. In the second example, nothing is changed, and thus nothing is written to the new file—it ends up being empty. Doesn't this negate the whole purpose of the edit? Why is this useful? It is useful only because the "-n" option has the ability to be overridden by a print command (-p). To illustrate, suppose the script file were modified to now resemble the following:

$ cat sedlist
/two/ s/1/2/p
/three/ s/1/3/p
$

Then this would be the result of running it:

$ sed -n -f sedlist sample_one
two 2
three 3
two 2
two 2
three 3
$

Lines that stay the same as they were are not displayed at all. Only the lines affected by the edit are displayed. In this manner, it is possible to pull those lines only, make the changes, and place them in a separate file:

$ sed -n -f sedlist sample_one > sample_two
$

$ cat sample_two
two 2
three 3
two 2
two 2
three 3
$

Another method of utilizing this is to print only a set number of lines. For example, to print only lines two through six while making no other editing changes:

$ sed -n '2,6p' sample_one
two 1
three 1
one 1
two 1
two 1
$

All other lines are ignored, and only lines two through six are printed as output. This is something remarkable that you cannot do easily with any other utility. head will print the top of a file, and tail will print the bottom, but sed allows you to pull anything you want to from anywhere.

Deleting Lines

Substituting one value for another is far from the only function that can be performed with a stream editor. There are many more possibilities, and the second-most-used function in my opinion is delete. Delete works in the same manner as substitute, only it removes the specified lines (if you want to remove a word and not a line, don't think of deleting, but think of substituting it for nothing—s/cat//).

The syntax for the command is:

'{what to find} d'

To remove all of the lines containing "two" from the sample_one file:

$ sed '/two/ d' sample_one
one 1
three 1
one 1
three 1
$

To remove the first three lines from the display, regardless of what they are:

$ sed '1,3 d' sample_one
one 1
two 1
two 1
three 1
$

Only the remaining lines are shown, and the first three cease to exist in the display. There are several things to keep in mind with the stream editor as they relate to global expressions in general, and as they apply to deletions in particular:

# The up carat (^) signifies the beginning of a line, thus

sed '/^two/ d' sample_one

would only delete the line if "two" were the first three characters of the line.
# The dollar sign ($) represents the end of the file, or the end of a line, thus

sed '/two$/ d' sample_one

would delete the line only if "two" were the last three characters of the line.

The result of putting these two together:

sed '/^$/ d' {filename}

deletes all blank lines from a file. For example, the following substitutes "1" for "2" as well as "1" for "3" and removes any trailing lines in the file:

$ sed '/two/ s/1/2/; /three/ s/1/3/; /^$/ d' sample_one
one 1
two 1
three 1
one 1
two 2
two 2
three 1
$

A common use for this is to delete a header. The following command will delete all lines in a file, from the first line through to the first blank line:

sed '1,/^$/ d' {filename}

Appending and Inserting Text

Text can be appended to the end of a file by using sed with the "a" option. This is done in the following manner:

$ sed '$a
> This is where we stop
> the test' sample_one
one 1
two 1
three 1
one 1
two 1
two 1
three 1
This is where we stop
the test
$

Within the command, the dollar sign ($) signifies that the text is to be appended to the end of the file. The backslashes () are necessary to signify that a carriage return is coming. If they are left out, an error will result proclaiming that the command is garbled; anywhere that a carriage return is to be entered, you must use the backslash.

To append the lines into the fourth and fifth positions instead of at the end, the command becomes:

$ sed '3a
> This is where we stop
> the test' sample_one
one 1
two 1
three 1
This is where we stop
the test
one 1
two 1
two 1
three 1
$

This appends the text after the third line. As with almost any editor, you can choose to insert rather than append if you so desire. The difference between the two is that append follows the line specified, and insert starts with the line specified. When using insert instead of append, just replace the "a" with an "i," as shown below:

$ sed '3i
> This is where we stop
> the test' sample_one
one 1
two 1
This is where we stop
the test
three 1
one 1
two 1
two 1
three 1
$

The new text appears in the middle of the output, and processing resumes normally after the specified operation is carried out.

Reading and Writing Files

The ability to redirect the output has already been illustrated, but it needs to be pointed out that files can be read in and written out to simultaneously during operation of the editing commands. For example, to perform the substitution and write the lines between one and three to a file called sample_three:

$ sed '
> /two/ s/1/2/
> /three/ s/1/3/
> 1,3 w sample_three' sample_one
one 1
two 2
three 3
one 1
two 2
two 2
three 3
$

$ cat sample_three
one 1
two 2
three 3
$

Only the lines specified are written to the new file, thanks to the "1,3" specification given to the w (write) command. Regardless of those written, all lines are displayed in the default output.

The Change Command

In addition to substituting entries, it is possible to change the lines from one value to another. The thing to keep in mind is that substitute works on a character-for-character basis, whereas change functions like delete in that it affects the entire line:

$ sed '/two/ c
> We are no longer using two' sample_one
one 1
We are no longer using two
three 1
one 1
We are no longer using two
We are no longer using two
three 1
$

Working much like substitute, the change command is greater in scale—completely replacing the one entry for another, regardless of character content, or context. At the risk of overstating the obvious, when substitute was used, then only the character "1" was replaced with "2," while when using change, the entire original line was modified. In both situations, the match to look for was simply the "two."

Change All but...

With most sed commands, the functions are spelled out as to what changes are to take place. Using the exclamation mark, it is possible to have the changes take place everywhere but those specified—completely reversing the default operation.

For example, to delete all lines that contain the phrase "two," the operation is:

$ sed '/two/ d' sample_one
one 1
three 1
one 1
three 1
$

And to delete all lines except those that contain the phrase "two," the syntax becomes:

$ sed '/two/ !d' sample_one
two 1
two 1
two 1
$

If you have a file that contains a list of items and want to perform an operation on each of the items in the file, then it is important that you first do an intelligent scan of those entries and think about what you are doing. To make matters easier, you can do so by combining sed with any iteration routine (for, while, until).

As an example, assume you have a text file named "animals" with the following entries:

pig
horse
elephant
cow
dog
cat

And you want to run the following routine:

#mcd.ksh
for I in $*
do
echo Old McDonald had a $I
echo E-I, E-I-O
done

The result will be that each line is printed at the end of "Old McDonald has a." While this is correct for the majority of the entries, it is grammatically incorrect for the "elephant" entry, as the result should be "an elephant" rather than "a elephant." Using sed, you can scan the output from your shell file for such grammatical errors and correct them on the fly, by first creating a file of commands:

#sublist
/ a a/ s/ a / an /
/ a e/ s/ a / an /
/a i/ s / a / an /
/a o/ s/ a / an /
/a u/ s/ a / an /

and then executing the process as follows:

$ sh mcd.ksh 'cat animals' | sed -f sublist

Now, after the mcd script has been run, sed will scan the output for anywhere that the single letter a (space, "a," space) is followed by a vowel. If such exists, it will change the sequence to space, "an," space. This corrects the problem before it ever prints on the screen and ensures that editors everywhere sleep easier at night. The result is:

Old McDonald had a pig
E-I, E-I-O
Old McDonald had a horse
E-I, E-I-O
Old McDonald had an elephant
E-I, E-I-O
Old McDonald had a cow
E-I, E-I-O
Old McDonald had a dog
E-I, E-I-O
Old McDonald had a cat
E-I, E-I-O

Quitting Early

The default is for sed to read through an entire file and stop only when the end is reached. You can stop processing early, however, by using the quit command. Only one quit command can be specified, and processing will continue until the condition calling the quit command is satisfied.

For example, to perform substitution only on the first five lines of a file and then quit:

$ sed '
> /two/ s/1/2/
> /three/ s/1/3/
> 5q' sample_one
one 1
two 2
three 3
one 1
two 2
$

The entry preceding the quit command can be a line number, as shown, or a find/matching command like the following:

$ sed '
> /two/ s/1/2/
> /three/ s/1/3/
> /three/q' sample_one
one 1
two 2
three 3
$

You can also use the quit command to view lines beyond a standard number and add functionality that exceeds those in head. For example, the head command allows you to specify how many of the first lines of a file you want to see—the default number is ten, but any number can be used from one to ninety-nine. If you want to see the first 110 lines of a file, you cannot do so with head, but you can with sed:

sed 110q filename

Handling Problems

The main thing to keep in mind when dealing with sed is how it works. It works by reading one line in, performing all the tasks it knows to perform on that one line, and then moving on to the next line. Each line is subjected to every editing command given.

This can be troublesome if the order of your operations is not thoroughly thought out. For example, suppose you need to change all "two" entries to "three" and all "three" to "four":

$ sed '
> /two/ s/two/three/
> /three/ s/three/four/' sample_one
one 1
four 1
four 1
one 1
four 1
four 1
four 1
$

The very first "two" read was changed to "three." It then meets the criteria established for the next edit and becomes "four." The end result is not what was wanted—there are now no entries but "four" where there should be "three" and "four."

When performing such an operation, you must pay diligent attention to the manner in which the operations are specified and arrange them in an order in which one will not clobber another. For example:

$ sed '
> /three/ s/three/four/
> /two/ s/two/three/' sample_one
one 1
three 1
four 1
one 1
three 1
three 1
four 1
$

This works perfectly, since the "three" value is changed prior to "two" becoming "three."

Labels and Comments

Labels can be placed inside sed script files to make it easier to explain what is transpiring, once the files begin to grow in size. There are a variety of commands that relate to these labels, and they include:

# : The colon signifies a label name. For example:

:HERE

Labels beginning with the colon can be addressed by "b" and "t" commands.

# b {label} Works as a "goto" statement, sending processing to the label preceded by a colon. For example,

b HERE

sends processing to the line

:HERE

If no label is specified following the b, processing goes to the end of the script file.

# t {label} Branches to the label only if substitutions have been made since the last input line or execution of a "t" command. As with "b," if a label name is not given, processing moves to the end of the script file.

# # The pound sign as the first character of a line causes the entire line to be treated as a comment. Comment lines are different from labels and cannot be branched to with b or t commands.

Network File System(NFS)

2009-02-27T01:47:00.001-08:00

Network File System
Among the many different file systems that Linux supports is the Network File System, also known as NFS. NFS allows a system to share directories and files with others over a network. By using NFS, users and programs can access files on remote systems almost as if they were local files.

* There is no need for users to have separate home directories on every network machine. Home directories could be set up on the NFS server and made available throughout the network.

The server has to be running the following daemons:

* nfsd The NFS daemon which services requests from the NFS clients.
* mountd The NFS mount daemon which carries out the requests that nfsd passes on to it.
* rpcbind This daemon allows NFS clients to discover which port the NFS server is using.

# NAS uses TCP/IP and NFS/CIFS/HTTP
# NAS uses TCP/IP Networks: Ethernet, FDDI, ATM

2009-02-27T01:45:00.000-08:00

vi Editor, Learn vi
About vi editor

* vi is Found on Nearly Every Unix Computer
* vi is the standard Unix editor
* vi is Powerful and Fast
* Your terminal displays a section of the file you are editing
* vi can do anything you want
* You don't need to remove your fingers from the standard typing keys-the keys themselves give commands to vi
* vi Stays Out of Your Way
* vi has no menus
* vi commands are short

Starting vi
Open a file with vi.
Type: vi myfile.txt
If myfile.txt does not exist, a screen will appear with just a cursor at the top followed by tildes (~) in the first column.

If myfile.txt does exist, the first few line of the file will appear.

The status line at the bottom of your screen shows error messages and provides information and feedback, including the name of the file.

vi Modes

Command Mode

# Command mode is the mode you are in when you start (default mode)
# Command mode is the mode in which commands are given to move around in the file, to make changes, and to leave the file
# Commands are case sensitive: j not the same as J
# Most commands do not appear on the screen as you type them. Some commands will appear on the last line: : / ?

Insert (or Text) Mode

# The mode in which text is created. (You must press at the end of each line unless you've set wrap margin.)
# There is more than one way to get into insert mode but only one way to leave: return to command mode by pressing
# When in doubt about which mode you are in, press

Basic Cursor Movement
k Up one line
j Down one line
h Left one character
l Right one character (or use )
w Right one word
b Left one word

NOTE: Many vi commands can take a leading count (e. g., 6k, 7e).

Entering, Deleting, and Changing Text
i Enter text entry mode
x Delete a character
dd Delete a line
r Replace a character
R Overwrite text, press to end

Setting Basic Options in vi
Displaying Line Numbers
:set nu Display line numbers
:set nonu Hide line numbers

Setting Right Margin
:set wm=number Set Wrap Margin number of spaces from right edge of screen
:set wm=10 Set Wrap Margin 10 spaces from right edge of screen
:set wm=0 Turn off Wrap Margin

Exiting vi
To exit you must be in command mode-press if you are not in command mode

You must press after commands that begin with a : (colon)
ZZ Write (if there were changes), then quit
:wq Write, then quit
:q Quit (will only work if file has not been changed)
:q! Quit without saving changes to file

# Basics Summary
A Basic vi Session
To enter vi, type: vi filename
To enter insert mode, type: i
Type in the text: This is easy.
To leave insert mode and return to command mode, press:
In command mode, save changes and exit vi by typing: :wq
You are back at the Unix prompt.

INTERMEDIATE VI
More On Cursor Movement
e Move to end of current word
$ Move to end of current line
^ Move to beginning of current line
+ Move to beginning of next line
- Move to beginning of previous line
G Go to last line of the file
:n Go to line with this number (:10 goes to line 10)
d Scroll down one-half screen
u Scroll up one-half screen
f Scroll forward one full screen
b Scroll backward one full screen
) Move to the next sentence
( Move to the previous sentence
} Move to the next paragraph
{ Move to the previous paragraph
H Move to the top line of the screen
M Move to the middle line of the screen
L Move to the last line of the screen
% Move to matching bracket: ( { [ ] } )

Entering Text Mode
i Insert text before current character
a Append text after current character
I Begin text insertion at the beginning of a line
A Append text at end of a line
o Open a new line below current line
O Open a new line above current line

Commands and Objects
Format Example
operator number object c2w
number operator object 2cw

Operators
c change
d delete
y yank

Objects and Locations
w one word forward
b one word backward
e end of word
H, M, L top, middle, or bottom line on screen
), ( next sentence, previous sentence
}, { next paragraph, previous paragraph
^, $ beginning of line, end of line
/pattern/ forward to pattern

Replacing and Changing Text
r Replace only the character under the cursor. (Note: using r you remain in command mode.)
R Beginning with the character under the cursor, replace as many characters on this line as you want. (You are in overtype mode until you press
cw Beginning with the character under the cursor, change a word to whatever you type. (You are in insert mode until you press )
c$ Beginning with the character under the cursor,
C change a line to whatever you type. (You are in insert mode until you press )

Deleting Text
x Delete a character
dw Delete an alphabetic word and the following space (6dw deletes six words)
dW Delete a blank-delimited word and the following space
dd Delete a line (6dd deletes six lines)
d$ Delete all characters to the end of the line.
d} Delete all characters to the end of the paragraph.
:5,30d Delete lines 5 through 30

Deleted text goes into a temporary buffer that is replaced each time you delete (or copy) more text. The current contents of the buffer can be put back into your file.

Copying and Pasting Text
yy Copy (yank) the current line
6yy Copy (yank) six lines, beginning with the current line
yw Copy the current word
p Put the text after the cursor position
P Put the text before the cursor position
Copied text goes into a temporary buffer that is replaced each time you copy (or delete) more text. Only the current contents of the temporary buffer can be put back into your file. As a result, when you use copy (y), use the put (p) command immediately.

A yank and put procedure using colon commands:
:5,10y Copy lines 5-10

Move cursor
:put Put after cursor

Other Useful Commands
. Repeat last command
n. Repeat last command n number of times
J Join next line to current line
u Undo last single change
U Restore current line
~ Change letter's case (capital to lower and vice versa)

Buffers
Temporary Buffer
Deleted or copied text goes into a temporary unnamed buffer. The contents of the temporary buffer may be retrieved by using the p or P commands.
p Put words from temporary buffer after cursor or put lines from temporary buffer below current line
P Put words from temporary buffer before cursor or put lines from temporary buffer above current line

Lettered Buffers
There are 26 lettered buffers (a-z). Contents of a lettered buffer are saved until you copy or delete more characters into it, or until you quit your current vi session.
"ayy Copy (yank) a line into buffer a
"Ayy Appends to buffer a
"a10yy Copies 10 lines into buffer a
"a10dd Deletes 10 lines of text into buffer a
"ap Put contents of lettered buffer a below the current line

Both temporary and lettered buffers last only for the current vi session.

Copying, Deleting, or Moving Text Using Line Numbers
These commands start with a colon (:) and end with a or g shows the line number of the current line
The basic form of colon commands is

:beginning_line, ending_line command destination

where destination is the line after which you want the text placed.
:5,10 co 105 Copy lines 5-10 to the line after 105
:5,20 m $ Move lines 5-20 to end of file
:7,300 d Delete lines 7-300 (to buffer)

Searching for Text
/text Search forward (down) for text (text can include spaces and characters with special meanings.)
?text Search backward (up) for text
n Repeat last search in the same direction
N Repeat last search in the opposite direction
fchar Search forward for a charcter on current line
Fchar Search backward for a character on current line
; Repeat last character search in the same direction
% Find matching ( ), { }, or [ ]

Substitutions
The simplest way to do substitutions over a range of lines, or throughout the file, is to use the s colon command. The basic form of this command is the following:

:n1,n2s/old/new/gc
n1 is the beginning line
n2 is the ending line number
s means to substitute text matching the pattern (old) with text specified by (new)
g (global) is optional. It indicates you want to substitute all occurrences on the indicated lines. If you use g, the editor substitutes only the first occurrence on the indicated lines.
c (confirm) is optional. It indicates you want to confirm each substitution before vi completes it.

:%s/old/new/g Substitutes old with new throughout the file
:.,$s/old/new/g Substitutes old with new from the current cursor position to the end of the file
:^,.s/old/new/g Substitutes old with new from the beginning of the file to the current cursor position
:& Repeats the last substitute (:s) command

ADVANCED VI TUTORIAL

Writing to and Reading from Files
:w file Write current file to file
:w>>file Append current file to file
:5,10w file Write lines 5 through 10 to file
:5,10w>>file Append Lines 5 through 10 to file
:r file Read a copy of file into current file
:!ls See a list of files in your current directory

More About Options From Command Mode-within vi for the current file only
:set all Display all options
:set Display current settings of options
:set nooption Unset option
:set ai Set Auto Indentation during text entry
:set ic Set Ignore Case during searches
:set nu Show line Numbers
:set sm Show Matching ( or { when ) or } is entered
:set wm=10 Set Wrap Margin 10 spaces from right edge of screen

Customizing vi Sessions Options can be set four ways:
# During a vi session

* :set nu

# In a .exrc file in your home directory. Sample contents of a .exrc file

* set nu
* set ai
* set wm=10

# In a .exrc file in a subdirectory.
# By setting the EXINIT environmental variable. Example of setting the EXINIT environmental variable

* setenv EXINIT "set nu ai ic"

Order of Precedence
# If a .exrc file exists in the current directory, vi reads it when beginning a session.
# If no .exrc file exists in the current directory, vi checks the home directory for a .exrc file. If such a file exists, vi reads it when beginning a session.
# If no .exrc file is found, vi uses its defaults.
# Values set in the EXINIT environmental variable override any values set in a .exrc file.

Creating a .exrc File

At the system prompt, type: vi .exrc
Type the following commands, each on a separate line:
# set ai
# set ic
# set nu
# set wm=8 Do not leave blank lines at the beginning or end of the .exrc file.
When you are finished, type: ZZ

Abbreviations & Mapping
Abbreviations are text strings that automatically expand into larger strings during insert mode.
:ab UW University of Washington

Mapping defines a single key to execute a sequence of keystrokes when the single key is pressed in command mode. In the following example,the @ key is mapped to replace the current word with "University of Washington". The v allows you to enter the key into the command sequence.
:map @ cwUniversity of Washington v

Mapping can also be used to call commands external to vi, such as sort or fmt. In the following example, the @ sign is mapped to the sort command, so that the current paragraph (indicated by the }) will be sorted. The v allows you to enter the key into the command sequence. The second completes the map command.
:map @ !}sort v

Note: You can also put abbreviation and mapping commands in your .exrc file.

TIPS AND TRICKS

Find the line that reads
editor=
Change it to read
editor=vi
Write and quit the file. (ZZ or :wq) vi-ing More Than One File
You can edit more than one file at a time with vi.
From The Unix Shell Prompt
vi file1 file2 vi two (or more) files at the same time From Command Mode
:n Move to file2 from file1
:rew Rewind back to file1
:e! Restore original file1 file2 (start all over)
ZZ Save and quit file. (Must be done for each file.)

Moving the Left Margin
When you print a file you may want the left margin moved to the right. This leaves room for a three-hole punch.
:1,$> Move entire file 1 shift width (eight spaces) to the right
:1,$< Move entire file eight spaces to the left
:%s/^/ /g Insert any number of spaces at the beginning of each line in the entire file. Simply press the space bar the desired number of times.
:20>> Moves next 20 lines over 1 shift width.

Issuing Shell Commands From vi
You can issue a single shell command while in the vi editor. For example, to list the files in your directory (ls), follow these steps:

Press d to return to vi editing.

Double Spacing a File
Occasionally, you may want a double spaced version of your file for editing or review.
:w Write changes to your file (just in case).
:!ls List contents of your current directory on the screen. Press to return to vi. You can issue many shell commands by temporarily leaving the vi editor. From Command Mode
:w Write changes to your file.
:sh Return to the shell to enter a number of commands without leaving vi.
:w original.backup Save a backup copy of the original file
:%! sed G Double space the entire file.
:1,5! sed G Double space the lines from 1-5.
ye

Some Important GNU/Linux commands

2009-02-27T01:43:00.000-08:00

alias Create an alias
awk Find and Replace text, database sort/validate/indexbreak
cal Display a calendar
case Conditionally perform a command
cat Display the contents of a file
cd Change Directory
cfdisk Partition table manipulator for Linuxchgrp
chroot Run a command with a different root directory
cksum Print CRC checksum and byte counts
clear Clear terminal screen
cmp Compare two files
comm Compare two sorted files line by line
command Run a command - ignoring shell functions
continue Resume the next iteration of a loop
cp Copy one or more files to another
crontab Schedule a command to run at a later timec
split Split a file into context-determined pieces
cut Divide a file into several parts
date Display or change the date & time
dc Desk Calculator
dd Data Dump - Convert and copy a file
declare Declare variables and give them attributes
df Display free disk space
diff Display the differences between two files
diff3 Show differences among three files
dir Briefly list directory contents
dircolors Colour setup for `ls'
dirname Convert a full pathname to just a path
dirs Display list of remembered directories
du Estimate file space usage
echo Display message on screen
ed A line-oriented text editor (edlin)
egrep Search file(s) for lines that match an extended expression
eject Eject CD-ROM
enable Enable and disable builtin shell commands
env Display, set, or remove environment variables
eval Evaluate several commands/arguments
exec Execute a command
exit Exit the shell
expand Convert tabs to spaces
export Set an environment variable
expr Evaluate expressions
factor Print prime factors
false Do nothing, unsuccessfully
fdformat Low-level format a floppy disk
fdisk Partition table manipulator for Linux
fgrep Search file(s) for lines that match a fixed string
find Search for files that meet a desired criteria
fmt Reformat paragraph text
fold Wrap text to fit a specified width.
for Expand words, and execute commands
format Format disks or tapes
free Display memory usage
fsck Filesystem consistency check and repair.
function Define Function Macros
gawk Find and Replace text within file(s)
getopts Parse positional parameters
grep Search file(s) for lines that match a given pattern
groups Print group names a user is in
gzip Compress or decompress named file(s)
hash Remember the full pathname of a name argument
head Output the first part of file(s)
history Command History
hostname Print or set system name
id Print user and group id's
if Conditionally perform a command
import Capture an X server screen and save the image to file
info Help info
install Copy files and set attributes
join Join lines on a common field
kill Stop a process from running
less Display output one screen at a time
let Perform arithmetic on shell variables
ln Make links between files
local Create variables
locate Find files
logname Print current login name
logout Exit a login shell
lpc Line printer control program
lpr Off line print
lprint Print a file
lprintd Abort a print job
lprintq List the print queue
lprm Remove jobs from the print queue
ls List information about file(s)
m4 Macro processor
man Help manual
mkdir Create new folder(s)
mkfifo Make FIFOs (named pipes)
mknod Make block or character special files
more Display output one screen at a time
mount Mount a file system
mtools Manipulate MS-DOS files
mv Move or rename files or directories
nice Set the priority of a command or job
nl Number lines and write files
nohup Run a command immune to hangups
passwd Modify a user password
paste Merge lines of files
pathchk Check file name portability
popd Restore the previous value of the current directory
pr Convert text files for printing
printcap Printer capability database
printenv Print environment variables
printf Format and print data
ps Process status
pushd Save and then change the current directory
pwd Print Working Directory
quota Display disk usage and limits
quotacheck Scan a file system for disk usage
quotactl Set disk quotas
ram ram disk device
rcp Copy files between two machines.
read read a line from standard input
readonly Mark variables/functions as readonly
remsync Synchronize remote files via email
return Exit a shell function
rm Remove files
rmdir Remove folder(s)
rpm Remote Package Manager
rsync Remote file copy (Synchronize file trees)
screen Terminal window manager
sdiff Merge two files interactively
sed Stream Editor
select Accept keyboard input
seq Print numeric sequences
set Manipulate shell variables and functions
shift Shift positional parameters
shopt Shell Options
shutdown Shutdown or restart linux
sleep Delay for a specified time
sort Sort text files
source Run commands from a file `.'
split Split a file into fixed-size pieces
su Substitute user identity
sum Print a checksum for a file
symlink Make a new name for a file
sync Synchronize data on disk with memory
tac Concatenate and write files in reverse
tail Output the last part of files
tar Tape ARchiver
tee Redirect output to multiple files
test Evaluate a conditional expression
time Measure Program Resource Use
times User and system times
touch Change file timestamps
top List processes running on the system
traceroute Trace Route to Host
trap Run a command when a signal is set(bourne)
tr Translate, squeeze, and/or delete characters
true Do nothing, successfully
tsort Topological sort
tty Print filename of terminal on stdin
type Describe a command
ulimit Limit user resources
umask Users file creation mask
umount Unmount a device
unalias Remove an alias
uname Print system information
unexpand Convert spaces to tabs
uniq Uniquify files
units Convert units from one scale to another
unset Remove variable or function names
unshar Unpack shell archive scripts
until Execute commands (until error)
useradd Create new user account
usermod Modify user account
users List users currently logged in
uuencode Encode a binary file uudecode Decode a file created by uuencode
v Verbosely list directory contents (`ls -l -b')
vdir Verbosely list directory contents (`ls -l -b')
watch Execute/display a program periodically
wc Print byte, word, and line counts
whereis Report all known instances of a command
which Locate a program file in the user's path.
while Execute commands
who Print all usernames currently logged in
whoami Print the current user id and name (`id -un')
xargs Execute utility, passing constructed argument list(s)
yes Print a string until interrupted
.period Run commands from a file
### Comment / Remark

grep usage

2009-02-27T01:42:00.003-08:00

grep usage
------------------
Here is an example shell command that invokes GNU grep:

grep -i 'hello.*world' hello.h hello.c

This lists all lines in the files `hello.h' and `hello.c' that contain the string `hello' followed by the string `world'; this is because `.*' matches zero or more characters within a line. See section 5. Regular Expressions. The `-i' option causes grep to ignore case, causing it to match the line `Hello, world!', which it would not otherwise match. Invoking grep, for more details about how to invoke grep.

Here are some common questions and answers about grep usage.

How can I list just the names of matching files?

grep -l 'main' *.c

lists the names of all C files in the current directory whose contents mention `main'. How do I search directories recursively?

grep -r 'hello' /home/test

searches for `hello' in all files under the directory `/home/test'. For more control of which files are searched, use find, grep and xargs. For example, the following command searches only C files:

find /home/test -name '*.c' -print | xargs grep 'hello' /dev/null

This differs from the command:

grep -r 'hello' *.c

which merely looks for `hello' in all files in the current directory whose names end in `.c'. Here the `-r' is probably unnecessary, as recursion occurs only in the unlikely event that one of `.c' files is a directory. What if a pattern has a leading `-'?

grep -e '--cut here--' *

searches for all lines matching `--cut here--'. Without `-e', grep would attempt to parse `--cut here--' as a list of options. Suppose I want to search for a whole word, not a part of a word?

grep -w 'hello' *

searches only for instances of `hello' that are entire words; it does not match `Othello'. For more control, use `<' and `>' to match the start and end of words. For example:

grep 'hello>' *

searches only for words ending in `hello', so it matches the word `Othello'. How do I output context around the matching lines?

grep -C 2 'hello' *

prints two lines of context around each matching line. How do I force grep to print the name of the file?

Append `/dev/null':
grep 'eli' /etc/passwd /dev/null

gets you:
/etc/passwd:bill:TWEFWEG.IMe.:98:11:Bill Gates:/home/do/bgates:/bin/bash Why do people use strange regular expressions on ps output?

ps -ef | grep '[c]ron'

If the pattern had been written without the square brackets, it would have matched not only the ps output line for cron, but also the ps output line for grep. Note that some platforms ps limit the ouput to the width of the screen, grep does not have any limit on the length of a line except the available memory. Why does grep report "Binary file matches"?
If grep listed all matching "lines" from a binary file, it would probably generate output that is not useful, and it might even muck up your display. So GNU grep suppresses output from files that appear to be binary files. To force GNU grep to output lines even from files that appear to be binary, use the `-a' or `--binary-files=text' option. To eliminate the "Binary file matches" messages, use the `-I' or `--binary-files=without-match' option. Why doesn't `grep -lv' print nonmatching file names?
`grep -lv' lists the names of all files containing one or more lines that do not match. To list the names of all files that contain no matching lines, use the `-L' or `--files-without-match' option.
I can do OR with `|', but what about AND?

grep 'paul' /etc/motd | grep 'franc,ois'

finds all lines that contain both `paul' and `franc,ois'. How can I search in both standard input and in files?
Use the special file name `-':

cat /etc/passwd | grep 'alain' - /etc/motd

How to express palindromes in a regular expression? It can be done by using the back referecences, for example a palindrome of 4 chararcters can be written in BRE.

grep -w -e '(.)(.).21' file

It matches the word "radar" or "civic".
Guglielmo Bondioni proposed a single RE that finds all the palindromes up to 19 characters long.

egrep -e '^(.?)(.?)(.?)(.?)(.?)(.?)(.?)(.?)(.?).?987654321$' file

Note this is done by using GNU ERE extensions, it might not be portable on other greps. Why are my expressions whith the vertical bar fail?

/bin/echo "ba" | egrep '(a)1|(b)1'

The first alternate branch fails then the first group was not in the match this will make the second alternate branch fails. For example, "aaba" will match, the first group participate in the match and can be reuse in the second branch. What do grep, fgrep, egrep stand for ?
grep comes from the way line editing was done on Unix. For example, ed uses this syntax to print a list of matching lines on the screen.

global/regular expression/print
g/re/p

fgrep stands for Fixed grep, egrep Extended grep.

Linux find command

2009-02-27T01:42:00.001-08:00

Linux find command

The find command allows the Unix user to process a set of files and/or directories in a file subtree.

You can specify the following:

* where to search (pathname)
* what type of file to search for (-type: directories, data files, links)
* how to process the files (-exec: run a process against a selected file)
* the name of the file(s) (-name)
* perform logical operations on selections (-o and -a)
* Search for file with a specific name in a set of files (-name)

find . -name "rc.conf" -print

This command will search in the current directory and all sub directories for a file named rc.conf.

Note: The -print option will print out the path of any file that is found with that name. In general -print wil print out the path of any file that meets the find criteria.

How to apply a unix command to a set of file (-exec).

find . -name "rc.conf" -exec chmod o+r '{}' \;

This command will search in the current directory and all sub directories. All files named rc.conf will be processed by the chmod -o+r command. The argument '{}' inserts each found file into the chmod command line. The\; argument indicates the exec command line has ended.

The end results of this command is all rc.conf files have the other permissions set to read access (if the operator is the owner of the file).

How to apply a complex selection of files (-o and -a).

find /usr/src -not $ -name "*,v" -o -name ".*,v" $ '{}' \; -print

This command will search in the /usr/src directory and all sub directories. All files that are of the form '*,v' and '.*,v' are excluded.

Important arguments to note are:

* -not means the negation of the expression that follows
* $ means the start of a complex expression.
* $ means the end of a complex expression.
* -o means a logical or of a complex expression.

In this case the complex expression is all files like '*,v' or '.*,v'
The above example is shows how to select all file that are not part of the RCS system. This is important when you want go through a source tree and modify all the source files... but ... you don't want to affect the RCS version control files.

How to search for a string in a selection of files (-exec grep ...).

find . -exec grep "searchthisstring" '{}' \; -print

This command will search in the current directory and all sub directories. All files that contain the string will have their path printed to standard output.

If you want to just find each file then pass it on for processing use the -q grep option. This finds the first occurrance of the search string. It then signals success to find and find continues searching for more files.

find . -exec grep -q "searchthisstring" '{}' \; -print

This command is very important for process a series of files that contain a specific string. You can then process each file appropriately.

User configuration files: . (dot) files and rc files

2009-02-27T01:41:00.001-08:00

User configuration files: . (dot) files and rc files
configuration files: the first one at a "system" level, located in /etc/; and the other one, "private" to the user, that can be found in home directory.

Commonly used rc and . (dot) files

Filename Description
.bash_login Look at "man bash". Treated by bash like .bash_profileif that doesn't exist.
.bash_logout Look at "man bash".Sourced by bash login shells at exit.
.bash_profile Sourced by bash login shells after /etc/profile.
.bash_history The list of commands executed previously.
.bashrc Look at "man bash". Sourced by bash non-login interactive shells (no other files are). Non-interactive shells source nothing unless BASH_ENV or ENV are set.
.emacs Read by emacs at startup.
.forward If this contains an e-mail address, then all mail to owner of ~ will be forwarded to that e-mail address.
.fvwmrc .fvwm2rc Config files for fvwm and fvwm2 (the basic XWindow manager).
.hushlogin Look at "man login". Causes a "quiet" login (no mail notice, last login info, or MOD).
.mail.rc User init file for mail program.
.ncftp/ Directory for ncftp program; contains bookmarks, log, macros, preferences, trace. See man ncftp. The purpose of ncftp is to provide a powerful and flexible interface to the Internet standard File Transfer Protocol. It is intended to replace the stock ftp program that comes with the system.
.profile Look at "man bash". Treated by bash like ~/.bash_profile if that and .bash_login don't exist, and used by other Bourn- heritage shells too.
.pinerc Pine configuration
.muttrc Mutt configuration
.exrc Configuration of vi can be controlled by this file. Example: set ai sm ruler Writing the above line in this file makes vi set the auto-indentation, matching brackets and displaying line number and rows- columns options.
.vimrc Default "Vim" configuration file. Same as .exrc.
.gtkrc GNOME Toolkit.
.kderc KDE configuration.
.netrc Default login names and passwords for ftp.
.rhosts Used by the r-tools: rsh, rlogin, etc. Very weak security since host impersonation is easy. Must be owned by user (owner of ~/) or superuser. Lists hosts from which users may access this account. Ignored if it is a symbolic link.
.rpmrc See "man rpm". Read by rpm if /etc/rpmrc is not present.
.signature Message text that will be appended automatically to the mail sent from this account.
.twmrc Config file for twm (The Window Manager).
.xinitrc Read by X at startup (not by xinit script). Mostly starts some progs.
Example: exec /usr/sbin/startkde If the above line is present in this file, then the KDE Window Manager is started in when the startx command is issued from this account.
.xmodmaprc This file is passed to the xmodmap program, and could be named anything (.Xmodmap and .keymap.km, for example).
.xserverrc Run by xinit as the X server if it can find X to execute. ~/News/Sent-Message-IDs Default mail history file for gnus.
.Xauthority Read and written by xdm program to handle authorization. See the X, xdm, and xauth man pages.
.Xdefaults,.Xdefaults-hostname Read by X applications during startup on hostname. If the -hostname file can't be found, .Xdefaults is looked for.
.Xmodmap Points to .xmodmaprc; Red Hat had (has) .xinitrc using this name.
.Xresources Usually the name for the file passed to xrdb to load the X resources database, to avoid the need for applications to read a long .Xdefaults file. (~/.Xres has been used by some.)

What is Sticky bit?

2009-02-27T01:40:00.000-08:00

Sticky bit

The most common use of the sticky bit today is on directories, where, when set, items inside the directory can only be renamed or deleted by the item's owner, the directory's owner, or the superuser. Generally this is set on the /tmp directory to prevent ordinary users from deleting or moving other users' files.

In addition, Solaris (as of Solaris 2.5) defines special behavior when the sticky bit is set on non-executable files: those files, when accessed, will not be cached by the kernel. This is usually set on swap files to prevent access on the file from flushing more important data from the system cache. It is also used occasionally for benchmarking tests.

Examples
The sticky bit can only be set by superuser root. Using the chmod command, it can be set using its octal mode 1000 or by its symbol t (s is already used by the setuid bit). For example, to add the bit on the directory /usr/local/tmp, one would type chmod +t /usr/local/tmp. Or, to make sure that directory has standard tmp permissions, one could also type chmod 1777 /usr/local/tmp.

In Unix symbolic file system permission notation, the sticky bit is represented by the letter t in the final character-place. For instance, on Solaris 8, the /tmp directory, which by default has the sticky-bit set, shows up as:

$ ls -ld /tmp

drwxrwxrwt 4 root sys 485 Nov 10 06:01 /tmp

If the sticky-bit is set on a file or directory without the execution bit set for the others category (non-user-owner and non-group-owner), it is indicated with a capital T:

# ls -l test

-rw-r--r-- 1 root other 0 Nov 10 12:57 test

# chmod +t test; ls -l test

-rw-r--r-T 1 root other 0 Nov 10 12:57 test

Cpanel /scripts folder with explanation of every script.

2009-02-20T14:27:00.000-08:00

starts with /scripts/

adddns - Adds a DNS zone.
addfpmail - Add frontpage mail extensions to all domains without them.
addfpmail2 -Add frontpage mail extensions to all domains without them.
addnetmaskips - Add the netmask 255.255.255.0 to all IPs that have no netmask.
addnobodygrp - Adds the group nobody and activates security.
addpop - Add a Pop Account.
addservlets - Add JSP support to an account (requires tomcat).
addstatus - (Internal use never called by user).
adduser - Add a user to the system.
admin - Run WHM Lite.
apachelimits - Add rlimits to Apache
betaexim - Installs the latest version of exim.
biglogcheck - looks for logs nearing 2 gigabytes in size
bsdcryptoinstall - Installs crypto on FreeBSD.
bsdldconfig - Configures the proper lib directories in FreeBSD.
bsdpkgpingtest - Tests the connection speed for downloading FreeBSD packages.
buildbsdexpect - Install expect on FreeBSD.
buildeximconf - Rebuilds exim.conf.
buildpostgrebsd-dev - Installs postgresql on FreeBSD.
checkbadconf - Checks /usr/local/apache/conf/httpd.conf for bad users.
checkbsdgroups - Checks and repairs proftpd ownership on FreeBSD.
checkccompiler - Checks to make sure the C compiler works on your system.
checkfpkey - Checks for the FrontPage suid key
checkgd - Checks to see if GD is built.
checkinterchange - (Internal use).
checklibssl - Checks to make sure the proper libssl symlinks exist.
checkmaxclients - Checks to see if apache has reached the maximum clients allowed.
checkoldperl - Checks to see if the version of Perl on your system is old.
checkrsync - Checks to make sure rsync is up to date.
checksuexecpatch - Checks to see if mailman has been patched for suexec.
checksuspendpages - Checks to see if suspend pages are properly named.
checkup2date - Makes sure up2date is set up properly (RedHat)
checkyum - Makes sure yum is set up properly.
chkpaths - Makes sure /usr/sbin/chown has a symlink to /bin/chown
chownpublichtmls - Change ownership of all users web space to them, which is useful for converting to suexec. Files owned by nobody are deleted.
chpass - Change password.
ckillall - Allows you to kill a process (used like killall).
ckillall2 - Allows you to kill a process.
cleanbw - Cleans up old bandwidth logs.
cleandns8 - Clean up named.conf.
cleangd - Cleans up old GD installs and reinstalls GD
cleanmd5 - Fix CPAN md5 problems.
cleanmsglog - cleans exim’s msglog
cleanupmysqlprivs - Cleans up improper mySQL privileges.
compilers - Disables the usage of compilers for unprivileged users.
convert2maildir - Converts mail from mbox to maildir format and installs courier impap and pop (cpimap is removed).
courierup - Updates/Installs Courier
cpbackup - Runs backups.
distupgrade - Upgrades RedHat to the newest version (for testing only)
dnscluster - Enables DNS clustering.
dnsqueuecron - Adds a cron job to dump the DNS queue.
dnstransfer - Only if the server has a DNS master (sync with DNS master).
downgradefp - Downgrades FrontPage Extensions (to 5.0-0)
dropmysqldb - Drops a mySQL database.
easyapache - Upgrade Apache
editquota - Change a users quota.
enablechkservdwebmail - Enable service checking of webmaild.
enablefileprotect - Protects home directories if file protection is built in apache.
ensurepkg - Installs a FreeBSD package.
ensurerpm - Installs a rpm.
exim3 - Installs exim 3.
exim4 - Installs exim 4.
exim4-rh73test - Installs exim release #260. (RedHat only)
eximcron - Creates a cron job for exim_tidy_db.
eximlocalsend - Enables/Disables exim local sending.
exim_tidydb - Cleans the exim message log.
eximup - Installs/Updates exim.
fetchgd - Includes libg.so.
findhacks - Search for common Trojan Horses.
findoddrootprocesses - Lists root processes that may need to be checked out.
findphpversion - Check to see if your php version file is up to date.
findtrojans - Exhaustive Trojan Horse search.
fixallcartswithsuexec - Fixes permissions on carts when using suexec.
fixallinterchangeperm - Fixes permissions on all users’ Interchange Shopping Carts.
fixbinpath - Makes sure all bin file paths are correct.
fixbuggynamed - Updates bind to solve any problems with bugs.
fixcommonproblems - Attempt to fix the most common problems.
fixetchosts - Fixes problems with /etc/hosts
fixeverything - Fix common problems and quotas.
fixfpwml - Fix for .wml errors with frontpage.
fixheaders - Run if nothing compiles errors with .h files on compile.
fixinterchange - Reinstall interchange Perl modules.
fixinterchangeperm - fix permissions on a user’s interchange cart.
fixipsnm - Same as addnetmask ips, but Perl though.
fixlibnet - Reinstall Bundle::libnet (Perl).
fixlocalhostwithphp - Change /etc/hosts to work better with PHP 4.2.0 + MySQL.
fixmailman - Updates and restarts mailman.
fixmailmanwithsuexec -
fixmuse - Reinstalls muse.
fixmysql - Fixes problems with mySQL.
fixmysqlbsd - Fixes problesm with mySQL on FreeBSD.
fixnamed - Updates bind to handle many DNS zones (more than 512).
fixndc - Repair redhat’s broken named.conf on 7.2.
fixoldlistswithsuexec - Run after enabling suexec on the server to change the URLs that Mailman gives out to ones that don’t give a 500 internal server error.
fixperl - Symlink /usr/local/bin/perl /usr/bin/perl.
fixperlscript - Makes sure a perlscript includes all corresponding modules.
fixpop - Fix a POP account and reset password.
fixproftpdconf - Fixes problems with /usr/local/etc/proftpd.conf
fixproftpddupes - Updates proftpd.
fixquotas - Fix quotas.
fixrndc - Fixes named.conf to prevent rndc staus failed.
fixspamassassinfailedupdate - Reinstalls a failed spamassassin update.
fixsubdomainlogs - Run if subdomain logs don’t show up in cPanel.
fixsuexeccgiscripts - Fix CGI scripts that are broken after suexec installed.
fixvaliases - Fix permisions on valiases.
fixwebalizer - Repair a Webalizer that has stopped updating.
fp3 - Updates the fpexe3 patch.
fpanonuserpatch - Updates FrontPage extensions to include the anonymous user patch.
ftpcheck - Checks for FTPSSL.
ftpquaotacheck - Runs quota checking for all ftp users.
ftpup - Updates your ftp server.
fullhordereset - Resets Horde and displays the current Horde password.
futexfix - Fixes problesm with futex.
futexstartup - Starts futex.
gcc3 - Installs gcc-3.3.3
gencrt - Generate a .crt and .csr file.
grpck - Checks to see if grpck is working properly.
hdparmify - Enable dma/irq/32bit HD access, which speeds up IDE drives.
hdparmon - Turns on hdparm.
initacls - Mounts your file systems with ACL support (make sure your kernel supports ACLs)
initfpsuexec - Enable FrontPage suexec support.
initquotas - Turn on quota support on new drives.
initsslhttpd - Make sure HTTP starts with SSL.
initsuexec - Turn on suexec support if suexec is installed.
installcgipm - Installs CGI.pm
installdbi - Install Bundle::DBD::mysql.
installfpfreebsd - Installs FrontPage 5 Extensions on FreeBSD.
installfpgentoo - Installs FrontPage on Gentoo.
installgd - Builds GD.
installpkg - Installs a FreeBSD package.
installpostgres - Installs PostrgeSQL.
installrpm - Installs a rpm.
installspam - Install SpamAssassin.
installssl - Add a SSL vhost.
installzendopt - Install zend optimzer.
installzendopt-freebsd - Install zend optimizer on a freebsd machine.
isdedicatedip - Checks an ip to see if it is dedicated.
killacct - Delete an account.
killbadrpms - Security script that kills insecure RPMs from the server.
killdns - Delete a DNS zone.
killdrrootvhost - Removes the document root for a virtual host.
killndbm - Remove the broken NDBM_File module from 7.2.
killpvhost - Removes a virtual host from proftpd.conf.
killspamkeys - Removes a spam key.
killsslvhost - Removes a SSL entry for a virtual host.
killvhost - Delete a vhost.
listcheck - Checks mailing lists for issues.
listproblems - Lists common problems.
listsubdomains - List subdomains.
mailperm - Fix almost any mail permission problem.
mailscannerupdate - Updates MailScanner
mailtroubleshoot - Guided mail fix.
makecpphp - Installs php.
makesecondary - Part of DNS transfer.
manualupcp - Updates cPanel manually.
md5crypt - Encrypts a password into MD5.
mseclocal - Sets up Mandrake’s msec to allow exim to run as mailnull.
mysqladduserdb - Create a MySQL databse and user.
mysqlconnectioncheck - Attempts to connect to MySQL, restarts SQL if necessary.
mysqldeluserdb - Delete a MySQL database and user.
mysqlpasswd - Change MySQL password.
mysqlrpmpingtest - Checks your connection speed for downloading mySQL rpms.
mysqlup - Updates mySQL.
ndbmcheck - Checks to see if the nbdm module is loaded (kills in RedHat 7.2)
netftpsslpatch - Patches FTPSSL.pm.
newexim - Installs the latest version of exim.
nofsck - Make fsck always use -y
nomodattach - Removes mod_attach from httpd.conf.
nomodauthmysql -Removes mod_auth_mysql from httpd.conf.
nomodbwprotect - Removes mod_bwportect from httpd.conf.
nomodgzipconfmods - Removes mod_gzip from httpd.conf.
nomodperl - Removes mod_perl from httpd.conf.
oldaddoncgi2xaddon - Updates old addons to X addons.
park - Parks a domain.
patcheximconf - Fixes exim.conf.
perlinstaller - Installs perl.
phpini - Create a php.ini file.
pingtest - Checks your download time from cPanel mirrors.
pkgaccount-ala - backs up an Alab*nza account for transfer.
pkgacct-ciXost - backs up a ci*ost account for transfer.
pkgacct-dXm - backs up a d*m account for transfer.
pkgacct-enXim - backs up an en*im account for transfer.
pkgacct-pXa - backs up a p*a account for transfer.
proftpd128 - Installs proftpd-1.2.8.
ptycheck - Fixes permissoins on /dev/ptmx.
pwck -Verifies the integrity of system authentication information.
quickkernel - Updates your kernel.
quicksecure - Quickly kill useless services.
rebuildcpanelsslcrt - Rebuilds the cPanel SSL Certificate.
rebuildcpusers - Rebuilds /var/cpanel/users.
rebuildetcpasswd - Rebuilds /etc/passwd.
rebuildeximbsd - Rebuilds exim on FreeBSD.
rebuildhttpdconffromproftpd - Rebuild httpd.conf from the proftpd.conf file.
rebuildinterchangecfg - Used after moving a domain with Interchange to the server.
rebuildnamedconf - Restore named.conf from files in /var/named.
rebuildproftpd - Restore proftpd.conf from httpd.conf.
reinstallmailman - Reinstalls mailman.
relocatevartousr - Relocates files from /var to /usr in case of disk space issues.
remdefssl - Remove default SSL vhost.
reseteximtodefaults - Resets exim’s default settings.
resetimappasswds - Resets all imap passwords.
resetquotas - Change quotas to what they should be .
restartsrv - Restart a service.
restartsrv_apache - Restart apache.
restartsrv_bind - Restart bind.
restartsrv_clamd - Restart clamd.
restartsrv_courier - Restart courier imap.
restartsrv_cppop - Restart cppop.
restartsrv_entropychat - Restart entropy chat.
restartsrv_exim - Restart exim.
restartsrv_eximstats - Restart exim statistics.
restartsrv_ftpserver - Restart your ftp server.
restartsrv_httpd - Restart httpd.
restartsrv_imap - Restart impad.
restartsrv_inetd - Restart inetd.
restartsrv_interchange - Restart Interchange Shopping Cart.
restartsrv_melange - Restart melange chat.
restartsrv_mysql - Restart mysqld.
restartsrv_named - Restart named.
restartsrv_postgres - Restart postgresql.
restartsrv_postgresql - Restart postgresql.
restartsrv_proftpd - Restart proftpd.
restartsrv_pureftpd - Restart pure-ftpd.
restartsrv_spamd - Restart spamd.
restartsrv_sshd - Restart sshd.
restartsrv_syslogd - Restart syslogd.
restartsrv_tomcat - Restart tomcat.
restartsrv_xinetd - Restart xinetd.
restoremail - Restores a user’s mail.
reswhostmgr - Restart whostmgr.
rpmup - Upgrade redhat/mandrake errata/security.
rrdtoolinstall - Installs RRD Tool.
runstatsonce - Runs statistics (should be used from the crontab).
runweblogs - Run analog/webalizer/etc. for a user.
safeperlinstaller - Installs perl safely.
safeup2date - Runs up2date safely.
safeyum - Runs yum safely.
secureit - Remove unnecessary suid binaries.
securemysql - Attempts to secure the MySQL configuration.
securetmp - Adds securetmp to system startup.
setupfp - Install FrontPage 3 on an account.
setupfp4 - Install FrontPage 4 (2000) installer on an account.
setupfp5 - Install FrontPage 5 (2002) installer on an account.
setupfp5.nosueuxec - Install FrontPage 5 (2002) installer on an account when not using suexec.
showexelist - Shows exe processes.
simpleps - Display the process list.
smartcheck - Checks hard drive integrity.
smtpmailgdionly - Enables SMTP Mail Protection.
spamboxdisable - Disables SpamAssassin’s spambox delivery for all accounts.
suspendacct - Suspends an account.
sysup - update cPanel RPMs.
unlimitnamed - Installs the latest version of bind patched to support greater than 512 ips on the server.
unblockip - Unblocks an IP blocked by portsentry.
unsetupfp4 - Removes FrontPage 4 or 5 from an account.
unslavenamedconf - If the user accidentally sets a DNS master as local server, this will repair named.conf after the loop.
unsuspendacct - Unsuspends an account.
upcp - Updates cPanel.
updated - Updates /scripts.
updatefrontpage - Updates FrontPage
updatenow - Updates /scripts NOW.
updatephpconf - Updates PHP configuration files.
whoowns - Finds out who owns a domain.
wwwacct - Creates an account.
xaddonreport - Reports the current addon scripts installed.

Difference b/w Active and Passive Ftp

2009-02-12T23:00:00.001-08:00

This article will explain the differences between Active and Passive FTP modes. Active mode is used for servers with tight security.

Security is a major concern with any computer connected to the internet, therefore any computer connected to the internet should be protected by a Firewall. In order to connect to certain services, such as FTP, you have to allow those connections in the Firewall, on both the Client and Server side.

Although a client's computer may not have a firewall enabled, a server should always have this enabled for maximum security.In order to connect to an FTP server that has a firewall enabled, you have to connect using a specific connection mode in your FTP program.

There are different connection modes to choose from when connecting to an FTP server, typically either "Active" or "Passive" mode.

Active mode is beneficial to the FTP Server's security, while Passive mode typically requires less configuration changes on the Client's side.

What is Split DNS?

2009-01-27T11:35:00.000-08:00

Split DNS refers to using separate internal and external DNS views of your domain's network using internal and external name servers. To set up, configure your internal name servers to forward queries they can't resolve to the external name server. Under Berkeley Internet Name Domain (BIND) 4, use the "forwarders" directive. In BIND 8 systems, use the "forwarders" substatement to configure forwarding. Your external DNS records are configured to contain only a small zone file for your domain, listing things such as Web and FTP server addresses and any translated server addresses you want to publish to the world. Your internal servers hold only the DNS records for your internal networks. When internal users look up host names, the query is answered by internal DNS servers, even if the request is forwarded to an external DNS server for resolution. Internet users who look up host names in your domain are answered by external DNS servers that only know about the publicly accessible resources.
------------------------------------------------

To know more on this topic, please check the below link.
http://www.isaserver.org/tutorials/You_Need_to_Create_a_Split_DNS.html

ssh script

2009-01-24T22:28:00.000-08:00

Before executing this script, you need to know some details,

What Is Expect?

Expect is a UNIX automation and testing tool, written by Don Libes as an extension to the Tcl scripting language, for interactive applications such as telnet, ftp, passwd, fsck, rlogin, tip, ssh, and others. It uses UNIX pseudo terminals to wrap up sub-processes transparently, allowing the automation of arbitrary applications that are accessed over a terminal. With Tk, interactive applications can be wrapped in X11 GUIs. Expect has regular expression pattern matching and general program capabilities, allowing simple scripts to intelligently control programs such as telnet, ftp, and ssh, all of which lack a programming language, macros, or any other program mechanism. The result is that Expect scripts provide old tools with significant new power and flexibility.

The Script
**********
When a server keeps prompting for password at SSH attempts in spite of setting up RSA/DSA keys, this script can be used to overcome that issue. Make sure that the script has 700 permission as it will contain your password in plain text.
-------------------------------------------------------------------------
#! /usr/bin/expect

# Edit the following line - $USER@$SERVER
spawn ssh your-useid@your-server-name

# First time connection will print out some text for
# which one needs to type 'yes' to continue
# Comment these two lines after the first attempt
expect "*Are you sure you want to continue connecting*"
send "yes\r"

# Put the password here
expect "*assword*"
send "YOUR-PASSWORD\r"

# Start interacting
interact
--------------------------------------------------------------------------------------

The three commands send, expect, and spawn are the building power of Expect. The send command sends strings to a process, the expect command waits for strings from a process, and the spawn command starts a process.

The spawn Command

The spawn command starts another program. The first argument of the spawn command is the name of a program to start. The remaining arguments are passed to the program.

Thanks and Regards,
Sylesh H

Killing perl processes running.....

2009-01-23T08:18:00.000-08:00

ps auxww | grep perl | awk '{print $2}' | xargs kill -9

Checking bad blocks in Linux

2009-01-23T08:05:00.001-08:00

First find the hard disk type installed by running the command
fdisk -l

Run the following command to check the bad blocks in Hdd.
badblocks -v /dev/hda

It will display if there is any errors while checking the bad blocks..

Hope this Helps you.

Importanf files and its descriptions.

2009-01-23T07:58:00.000-08:00

* /boot/vmlinuz - the typical location and name of the Linux kernel. In the Slackware distribution, the kernel is located at /vmlinuz.

* /dev/fd0 - first floppy disk drive

* /dev/fd0H1440 - driver for the first floppy drive in high density mode. Generally, this is invoked when formatting a floppy drive for a particular density. Slackware comes with drivers that allow for formatting a 3.5" diskette with up to 1.7MB of space. Red Hat and Mandrake do not contain these device driver files by default.

* /dev/fd1 - second floppy disk drive

* /dev/hda - first IDE hard drive

* /dev/hdc - on many machines, the IDE cdrom drive. Most often, there is a symbolic link called /dev/cdrom which is just a link to the true cdrom driver file.

* /dev/null - used when you want to send output into oblivion

* /etc/aliases - file containing aliases used by sendmail and other MTAs (mail transport agents). After updating this file, it is necessary to run the newaliases utility for the changes to be passed to sendmail.

* /etc/bashrc - system-wide default functions and aliases for the bash shell

* /etc/conf.modules - aliases and options for configurable modules

* /etc/crontab - shell script to run different commands periodically (hourly, daily, weekly, monthly, etc.)

* /etc/DIR_COLORS - used to store colors for different file types when using ls command. The dircolors command uses this file when there is not a .dir_colors file in the user's home directory. Used in conjunction with the eval command (see below).

* /etc/exports - specifies hosts to which file systems can be exported using NFS. Man exports contains information on how to set up this file for remote users.

* /etc/fstab - contains information on partitions and file systems used by system to mount different partitions and devices on the directory tree

* /etc/HOSTNAME - stores the name of the host computer

* /etc/hosts - contains a list of host names and absolute IP addresses.

* /etc/hosts.allow - hosts allowed (by the tcpd daemon) to access Internet services

* /etc/hosts.deny - hosts forbidden (by the tcpd daemon) to access Internet services

* /etc/group - similar to /etc/passwd but for groups

* /etc/inetd.conf - configures the inetd daemon to tell it what TCP/IP services to provide (which daemons to load at boot time). A good start to securing a Linux box is to turn off these services unless they are necessary.

* /etc/inittab - runs different programs and processes on startup. This is typically the program which is responsible for, among other things, setting the default runlevel, running the rc.sysinit script contained in /etc/rc.d, setting up virtual login terminals, bringing down the system in an orderly fashion in response to [Ctrl][Alt][Del], running the rc script in /etc/rc.d, and running xdm for a graphical login prompt (only if the default runlevel is set for a graphical login).

* /etc/issue - pre-login message. This is often overwitten by the /etc/rc.d/rc.S script (in Slackware) or by the /etc/rc.d/rc.local script (in Mandrake and Red Hat, and perhaps other rpm-based distributions). The relevant lines should be commented out (or changed) in these scripts if a custom pre-login message is desired.

* /etc/lilo.conf - configuration file for lilo boot loader

* /etc/motd - message of the day file, printed immediately after login. This is often overwritten by /etc/rc.d/rc.S (Slackware) or /etc/rc.d/rc.local (Mandrake/Red Hat) on startup. See the remarks in connection with /etc/issue.

* /etc/mtab - shows currently mounted devices and partitions and their status

* /etc/passwd - contains passwords and other information concerning users who are registered to use the system. For obvious security reasons, this is readable only by root. It can be modified by root directly, but it is preferable to use a configuration utility such as passwd to make the changes. A corrupt /etc/passwd file can easily render a Linux box unusable.

* /etc/printcap - shows the setup of printers

* /etc/profile - sets system-wide defaults for bash shell. It is this file in Slackware that sets up the DIR_COLORS environment variable for the color ls command. Also sets up other system-wide environment variables.

* /etc/resolv.conf - contains a list of domain name servers used by the local machine

* /etc/securetty - contains a list of terminals on which root can login. For security reasons, this should not include dialup terminals.

* /etc/termcap - ASCII database defining the capabilities and characteristics of different consoles, terminals, and printers

* /etc/X11/XF86Config - X configuration file. The location in Slackware is /etc/XF86Config.

* /proc/cpuinfo - cpu information

* /proc/filesystems - prints filesystems currently in use

* /proc/interrupts - prints interrupts currently in use

* /proc/ioports - contains a list of the i/o addresses used by various devices connected to the computer

* /proc/kcore - The command ls -l /proc/kcore will give the amount of RAM on the computer. It's also possible to use the free command to get the same information (and more).

* /proc/version - prints Linux version and other info

* /var/log/messages - used by syslog daemon to store kernel boot-time messages

* /var/log/lastlog - used by system to store information about last boot

* /var/log/wtmp - contains binary data indicating login times and duration for each user on system.

How To Install CPANEL on your VPS

2009-01-20T01:20:00.000-08:00

Ok. Few Steps to setup your VPS-CPANEL:

1 - Login to your VZMC and get inside your server
2 - Create a new VPS with the Sample Ve Config call vps.cpanel
3 - Select the ips you want to use in that VPS and the dns servers.
4 - Select RedHat Enterprise Template (not minimal)
5 - Dont select any addon.You dont need it for cpanel.
6 - Select the Space / Memory / CPU . All the normal stuff of your normal VPS. Put Start on boot and the rest of the normal stuff. Rememeber to use unlimited VPs.
7 - Go to your Ev1 Member section, open a ticket with your IP / and root password and request ev1 to get your VPS register in up2date. CHECK IT IF IT IS WELL CONFIGURE!! JUST IN CASE.
8 - Go in ssh and do the following steps:
mkdir /home/cpins
cd /home/cpins
wget http://layer1.cpanel.net/latest
sh latest
This should install cpanel without asking you any questions.

If you have any problems you should check: http://www.cpanel.net/install.html

9 - Login to : https://xxx.xxx.xxx.xxx:2087 and setup your server.
If you never setup a cpanel server, you can find some usefull information here: http://www.cpanel.net/docs.htm or search ev1 forum or ask me. I will be happy to help.

Well. Hopefully for some of you was usefull and will give you something else to try/offer in your VPS server.

Btw, it needs atleast 128 MB for cpanel to work.

If you have any problems with the guide let me know.

carlos

ps: i talk to some sw-soft people and they recomend to enable second-level quota (QUOTAUGIDLIMIT), i didnt try it myself. But i will let everyone when i try it.

Filesystem attributes.

2009-01-20T00:39:00.000-08:00

As a Linux administrator, you may be called upon to set up a control system for file access. You probably already know how to set read, write, and execute permissions on files, and you will need to make extensive use of that knowledge. But, sometimes, you'll need more than just these permissions settings to get the job done. That's where filesystem attributes will come in handy. You can set different attributes on files in order to gain more control over how they are accessed.

There are two slight catches, though. You can only set file attributes on machines with hard drives that are formatted with either the ext2 or ext3 filesystems. That's not a problem for machines that are running a Red Hat-type operating system, since ext3 is your only choice with them. But, if you're setting up a machine with, say, Ubuntu Server, you'll have other filesystems to choose from. Just be sure to choose ext3 if you want to set file attributes.

Also, if you're accessing files on another computer via NFS, the attributes will still be in effect, but you won't be able to view or change the attributes.

To view file attributes, you'd use the lsattr command. Entering just the command by itself will show a list of all files in the current directory.

[sylesh@centos5 ~]$ lsattr
------------- ./mytext.txt
------------- ./Duron_backup
------------- ./iptables-L.txt
------------- ./New_error.txt
------------- ./Desktop
------------- ./moodle-2007-8-25
------------- ./test_dir
------------- ./BOINC
------------- ./ts2_client_rc2_2032.tar.bz2
------------- ./OOo_2.3.0_LinuxIntel_install_wJRE_en-US.tar.gz
------------- ./ifconfig_output.txt
------------- ./dmesg
------------- ./BOINC.tar.bz2
------------- ./ts2_client_rc2_2032
------------- ./tls_handshake_error.txt
[sylesh@centos5 ~]$ lsattr mytext.txt
------------- mytext.txt
[sylesh@centos5 ~]$
[sylesh@centos5 ~]$ chattr +A mytext.txt
[sylesh@centos5 ~]$ lsattr mytext.txt
s-S----A----- mytext.txt
[sylesh@centos5 ~]$

Of course, you'll seldom want to use the "A" attribute. If you need to turn off atime updates, you're better off mounting the filesystem with the "noatime" parameter, instead.

So far, we've performed all attribute changes with only normal user privileges, and on the user's own files. There are still two other attributes that can only be set with root privileges. Even if the file belongs to you, you'll receive an error if you try to change them with only your normal user privileges.

[sylesh@centos5 ~]$ chattr +a mytext.txt
chattr: Operation not permitted while setting flags on mytext.txt
[sylesh@centos5 ~]$

The "a" attribute will allow a file to be opened only in append mode. This will allow you to add more text or data to a file, but will not allow you to overwrite it.

[sylesh@centos5 ~]$ sudo chattr +a mytext.txt
Password:
[sylesh@centos5 ~]$ lsattr mytext.txt
s-S--a-A----- mytext.txt
[sylesh@centos5 ~]$ echo "This is a test of the a attribute." > mytext.txt
bash: mytext.txt: Operation not permitted
[sylesh@centos5 ~]$ echo "This is a test of the a attribute." >> mytext.txt
[sylesh@centos5 ~]$

The final attribute we'll cover, which also requires root privileges, is the "i" attribute. This make a file immutable. In other words, it can't be changed, renamed, or deleted. And, no links can be created to it.

[sylesh@centos5 ~]$ sudo chattr +i mytext.txt
[sylesh@centos5 ~]$ lsattr mytext.txt
s-S-ia-A----- mytext.txt
[sylesh@centos5 ~]$ rm mytext.txt
rm: remove write-protected regular file `mytext.txt'? y
rm: cannot remove `mytext.txt': Operation not permitted
[sylesh@centos5 ~]$

Finally, if you need to add or delete more than one attribute, you can combine the operations into one single command.

[sylesh@centos5 ~]$ sudo chattr -AaisS mytext.txt
[sylesh@centos5 ~]$ lsattr mytext.txt
------------- mytext.txt
[sylesh@centos5 ~]$

There are a few other attributes that we haven't covered. But they either have operational bugs, or they're attributes that are set by the system, and not by the user.

For more information, enter "man chattr" at the command-line.

Sylesh

What is a name server ?

2009-01-19T00:43:00.001-08:00

A Name Server keeps information for the translation of computer names to IP addresses (even for reverse translations). The name server takes care of a certain part from the space of names of all computers. This part is called the zone (at minimum it takes care of zone 0.0.127.in-addr.arpa). A domain or its part creates the zone. The name server can with the help of an NS type record (in its configuration) delegate administration of a subdomain to a subordinate name server. The name server is a program that performs the translation at the request of a resolver or another name server. In UNIX, the name server is materialized by the named program. Also the name BIND (Berkeley Internet Name Domain) is used for this name server.

Types of name servers differ according to the way in which they save data:

> Primary name server/primary master is the main data source for the zone. It is the authoritative server for the zone. This server acquires data about its zone from databases saved on a local disk. Names of these types of servers depend on the version of BIND they use. While only the primary name server was used for version 4.x, a primary name master is used for version 8. The administrator manually creates databases for this server. The primary server must be published as an authoritative name server for the domain in the SOA resource record, while the primary master server does not need to be published. There is only one of this type of server for each zone.

> Master name server is an authoritative server for the zone. The master server is always published as an authoritative server for the domain in NS records. The master sever is a source of data of a zone for the subordinate servers (slave/secondary servers). There can be several master servers. This type of server is used for Bind version 8 and later.

> Secondary name server/slave name server acquires data about the zone by copying the data from the primary name server (respectively from the master server) at regular time intervals. It makes no sense to edit these databases on the secondary name servers, although they are saved on the local server disk because they will be rewritten during further copying. This type of name server is also an authority for its zones, i.e., its data for the particular zone is considered irrevocable (authoritative). The name of this type of server depends again on the version of BIND it uses. For version 4, only the secondary name was used, the term slave server was used for a completely different type of server. In version 8 you can come across both names.

> Caching-only name server is neither a primary nor secondary name server (it is not an authority) for any zone. However, it uses the general characteristics of name servers, i.e., it saves data that comes through its cache. This data is called nonauthoritative. Each server is a caching server, but by the words caching, we understand that it is neither a primary nor secondary name server for any zone. (Of course, even a caching-only server is a primary name server for zone 0.0.127.in-addr.arpa, but that does not count).

> Root name server is an authoritative name server for the root domain (for the dot). Each root name server is a primary server, which differentiates it from other name servers.

> Slave name server (in BIND version 4 terminology) transmits questions for a translation to other name servers; it does not perform any iteration itself.

> Stealth name server is a secret server. This type of name server is not published anywhere. It is only known to the servers that have its IP address statically listed in their configuration. It is an authoritative server. It acquires the data for the zone with the help of a zone transfer. It can be the main server for the zone. Stealth servers can be used as a local backup if the local servers are unavailable.

Nagios in control.

2009-01-15T01:07:00.000-08:00

Nagios.

Nagios is a popular open source computer system and network monitoring application software. It watches hosts and services, alerting users when things go wrong and again when they get better.

Adding a new server.

Nagios Administration

This will help you in adding a new server to Nagios

1. Introduction

2. Adding Contacts

3. Adding Hosts

4. Adding Host Groups

5. Adding Services

6. Scheduling Host/Service Downtime

7. Acknowledging a host/services

8. Authentication Setup

In NAGIOS SERVER

contacts.cfg

Defines who gets notifications about problems with hosts and services.

define contact{
     contact_name                    sylesh
     alias                           syslogs
     service_notification_period     none ; Notification period
     host_notification_period        24x7   ; Notification period
     service_notification_options    c,r   ; Notification Options
     host_notification_options       d,r     ; Notification Options
     service_notification_commands   notify-by-email  ; Notification cmd
     service_notification_options n                              ; Notification cmd
     host_notification_commands      host-notify-by-email
     email                           syleshh@gmail.com
     }

contactgroups.cfg

Add the user to the appropriate department/server group he/she needs to be a member of

define contactgroup{
     contactgroup_name       testgroup   ; Group Name
     alias                   mycutelife   ; Alias for group
     members                 testuser  ; List of members. This user has the privilege to view the status of the
servers which is listed in the group
     }

hostgroups.cfg

This allows you to create groups to organise the hosts

define hostgroup{
     hostgroup_name  testhostgroup
     alias           syslogs
     contact_groups  testgroup
     members server1.hostname.com,server2.hostname.com,server3.hostname.com,server4.hostname.com
}

hosts.cfg

This contains all the devices you want nagios to check

#############First Server#############
define host{
     use                     generic-host            ; Name of host template to use
     host_name               server1.hostname.com
     alias                   server1
     address                 192.168.1.1
     check_command           check-host-alive
     max_check_attempts      10
     notification_interval   120
     notification_period     24x7
     notification_options    d,u,r
     }

#############Second Server#############
define host{
     use                     generic-host            ; Name of host template to use
     host_name               server2.hostname.com
     alias                   server2
     address                 192.168.1.2
     check_command           check-host-alive
     max_check_attempts      10
     notification_interval   120
     notification_period     24x7
     notification_options    d,u,r
     }

services.cfg

If we give host_name (host_name test.test.com) in place of hostgroup_name, we have to add each service entries to be monitored for each server. But if we give hostgroup_name and all hosts or servers are defined in the file hostgroups.cfg, we need only one entry for each service for all servers.

#########We can define the services to monitor#############

#########Apache#########
define service{
     use                             generic-service         ; Name of service template to use
     hostgroup_name                  testhostgroup
     service_description             Apache status
     is_volatile                     0
     check_period                    24x7
     max_check_attempts              3
     normal_check_interval           3
     retry_check_interval            1
     contact_groups                  testgroup
     notification_interval           120
     notification_period             24x7
     notification_options            w,u,c,r
     check_command                   check_http
     }

#########MTA#########
define service{
     use                             generic-service         ; Name of service template to use
     hostgroup_name                  testhostgroup
     service_description             MTA status
     is_volatile                     0
     check_period                    24x7
     max_check_attempts              3
     normal_check_interval           3
     retry_check_interval            1
     contact_groups                  testgroup
     notification_interval           120
     notification_period             24x7
     notification_options            w,u,c,r
     check_command                   check_smtp
     }

#########QMAIL Queue#########
define service{
use                             generic-service         ; Name of service template to use
     hostgroup_name                  testhostgroup
     service_description             QMAIL-Q
     is_volatile                     0
     check_period                    24x7
     max_check_attempts              3
     normal_check_interval           5
     retry_check_interval            1
     contact_groups                  testgroup
     notification_interval           120
     notification_period             24x7
     notification_options            w,u,c,r
     check_command                   check_nrpe!check_qmailq
     }

#########Exim Queue#########
define service{
      
use                             generic-service         ; Name of service template to use
    hostgroup_name                  testhostgroup
    service_description             EXIM_QUEUE
     is_volatile                     0
     check_period                    24x7
     max_check_attempts              3
     normal_check_interval           5
     retry_check_interval            1
     contact_groups                  testgroup
     notification_interval           120
     notification_period             24x7
     notification_options            w,u,c,r
     check_command                   check_nrpe!check_eximmailqueue
     }

#########FTP#########
define service{
     use                             generic-service         ; Name of service template to use
     hostgroup_name                  testhostgroup
     service_description             FTP status
     is_volatile                     0
     check_period                    24x7
     max_check_attempts              3
     normal_check_interval           3
     contact_groups                  testgroup
     retry_check_interval            1
     notification_interval           120
     notification_period             24x7
     notification_options            w,u,c,r
     check_command                   check_ftp
     }

#########Server Load#########
define service{
use                             generic-service         ; Name of service template to use
     hostgroup_name                  testhostgroup
     service_description             SYS-LOAD
     is_volatile                     0
     check_period                    24x7
     max_check_attempts              3
     normal_check_interval           5
     retry_check_interval            1
     contact_groups                  testgroup
     notification_interval           120
     notification_period             24x7
     notification_options            w,u,c,r
     check_command                   check_nrpe!check_load
     }

#########MYSQL#########
define service{
     use                             generic-service         ; Name of service template to use
     hostgroup_name                  testhostgroup
     service_description             MYSQL
     is_volatile                     0
     check_period                    24x7
     max_check_attempts              3
     normal_check_interval           3
     retry_check_interval            1
     contact_groups                  testgroup
     notification_interval           120
     notification_period             24x7
     notification_options            w,u,c,r
     check_command                   check_mysqlrc
     }

#########DNS#########
define service{
use                             generic-service         ; Name of service template to use
     hostgroup_name                  testhostgroup
     service_description             DNS
     is_volatile                     0
     check_period                    24x7
     max_check_attempts              3
     normal_check_interval           5
     retry_check_interval            1
     contact_groups                  testgroup
     notification_interval           120
     notification_period             24x7
     notification_options            w,u,c,r
     check_command                   check_dns
     }

#########SSH#########
define service{
use                             generic-service         ; Name of service template to use
     hostgroup_name                  testhostgroup
     service_description             SSH
     is_volatile                     0
     check_period                    24x7
     max_check_attempts              3
     normal_check_interval           5
     retry_check_interval            1
     contact_groups                  testgroup
     notification_interval           120
     notification_period             24x7
     notification_options            w,u,c,r
     check_command                   check_ssh
     }

#########POP#########
# Service definition
define service{
     use                             generic-service         ; Name of service template to use
     hostgroup_name                  testhostgroup
     service_description             POP3
     is_volatile                     0
     check_period                    24x7
     max_check_attempts              3
     normal_check_interval           5
     retry_check_interval            1
     contact_groups                  testgroup
     notification_interval           120
     notification_period             24x7
     notification_options            w,u,c,r
     check_command                   check_pop
     }

#########DISK_USAGE_SLASH#########
define service{
    use                             generic-service         ; Name of service template to use
    hostgroup_name                  testhostgroup
    service_description             DISK_USAGE_SLASH
    is_volatile                     0
    check_period                    24x7
    max_check_attempts              3
    normal_check_interval           5
    retry_check_interval            1
    contact_groups                  testgroup
    notification_interval           120
    notification_period             24x7
    notification_options            w,u,c,r
    check_command                   check_nrpe!check_diskslash
    }

#########DISK_USAGE_BOOT#########
define service{
     use                             generic-service         ; Name of service template to use
     hostgroup_name                  testhostgroup
     service_description             DISK_USAGE_BOOT
     is_volatile                     0
     check_period                    24x7
     max_check_attempts              3
     normal_check_interval           5
     retry_check_interval            1
     contact_groups                  testgroup
     notification_interval           120
     notification_period             24x7
     notification_options            w,u,c,r
     check_command                   check_nrpe!check_diskboot
     }

#########DISK_USAGE_BACKUP#########
define service{
     hostgroup_name                  testhostgroup
     service_description             DISK_USAGE_BACKUP
     is_volatile                     0
     check_period                    24x7
     max_check_attempts              3
     normal_check_interval           5
     retry_check_interval            1
     contact_groups                  testgroup
     notification_interval           120
     notification_period             24x7
     notification_options            w,u,c,r
     check_command                   check_nrpe!check_diskbackup
     }
#########DISK_USAGE_HOME#########
define service{
     hostgroup_name                  testhostgroup
     service_description             DISK_USAGE_HOME
     is_volatile                     0
     check_period                    24x7
     max_check_attempts              3
     normal_check_interval           5
     retry_check_interval            1
     contact_groups                  testgroup
     notification_interval           120
     notification_period             24x7
     notification_options            w,u,c,r
     check_command                   check_nrpe!check_diskhome
     }

#########DISK_USAGE_TMP#########
define service{
     hostgroup_name                  testhostgroup
     service_description             DISK_USAGE_TMP
     is_volatile                     0
     check_period                    24x7
     max_check_attempts              3
     normal_check_interval           5
     retry_check_interval            1
     contact_groups                  testgroup
     notification_interval           120
     notification_period             24x7
     notification_options            w,u,c,r
     check_command                   check_nrpe!check_disktmp
     }

#########DISK_USAGE_USR#########
define service{
     hostgroup_name                  testhostgroup
     service_description             DISK_USAGE_USR
     is_volatile                     0
     check_period                    24x7
     max_check_attempts              3
     normal_check_interval           5
     retry_check_interval            1
     contact_groups                  testgroup
     notification_interval           120
     notification_period             24x7
     notification_options            w,u,c,r
     check_command                   check_nrpe!check_diskusr
     }

#########DISK_USAGE_VAR#########
define service{
    hostgroup_name                  testhostgroup
    service_description             DISK_USAGE_VAR
    is_volatile                     0
    check_period                    24x7
    max_check_attempts              3
    normal_check_interval           5
    retry_check_interval            1
    contact_groups                  testgroup
    notification_interval           120
    notification_period             24x7
    notification_options            w,u,c,r
    check_command                   check_nrpe!check_diskvar
    }



--------------------------------------------------------------------------
Sylesh

apachectl - Apache HTTP Server Control Interface

2009-01-11T20:54:00.000-08:00

apachectl - Apache HTTP Server Control Interface

apachectl is a front end to the Apache HyperText Transfer Protocol (HTTP) server. It is designed to help the administrator control the functioning of the Apache httpd daemon.

The apachectl script can operate in two modes. First, it can act as a simple front-end to the httpd command that simply sets any necessary environment variables and then invokes httpd, passing through any command line arguments. Second, apachectl can act as a SysV init script, taking simple one-word arguments like start, restart, and stop, and translating them into appropriate signals to httpd.

If your Apache installation uses non-standard paths, you will need to edit the apachectl script to set the appropriate paths to the httpd binary. You can also specify any necessary httpd command line arguments. See the comments in the script for details.

The apachectl script returns a 0 exit value on success, and >0 if an error occurs. For more details, view the comments in the script.

Synopsis

When acting in pass-through mode, apachectl can take all the arguments available for the httpd binary.

apachectl [ httpd-argument ]

When acting in SysV init mode, apachectl takes simple, one-word commands, defined below.

apachectl command

Options

Only the SysV init-style options are defined here. Other arguments are defined on the httpd manual page.

start: Start the Apache httpd daemon. Gives an error if it is already running. This is equivalent to apachectl -k start.
stop: Stops the Apache httpd daemon. This is equivalent to apachectl -k stop.
restart: Restarts the Apache httpd daemon. If the daemon is not running, it is started. This command automatically checks the configuration files as in configtest before initiating the restart to make sure the daemon doesn't die. This is equivalent to apachectl -k restart.
fullstatus: Displays a full status report from mod_status. For this to work, you need to have mod_status enabled on your server and a text-based browser such as lynx available on your system. The URL used to access the status report can be set by editing the STATUSURL variable in the script.
status: Displays a brief status report. Similar to the fullstatus option, except that the list of requests currently being served is omitted.
graceful: Gracefully restarts the Apache httpd daemon. If the daemon is not running, it is started. This differs from a normal restart in that currently open connections are not aborted. A side effect is that old log files will not be closed immediately. This means that if used in a log rotation script, a substantial delay may be necessary to ensure that the old log files are closed before processing them. This command automatically checks the configuration files as in configtest before initiating the restart to make sure Apache doesn't die. This is equivalent to apachectl -k graceful.
configtest: Run a configuration file syntax test. It parses the configuration files and either reports Syntax Ok or detailed information about the particular syntax error. This is equivalent to apachectl -t.

The following additional option is available, but deprecated.

startssl: This is equivalent to apachectl -k start -DSSL. We recommend that you use that command explicitly, or you adjust your httpd.conf to remove the section so that SSL will always be available.

Sylesh

#Difference between ext2 and ext3??

2009-01-09T14:44:00.003-08:00

#Difference between ext2 and ext3??

Ext3 is a tiny bit slower than ext2 is, but it holds tremendous advantages.
There is really only one difference between ext2 and ext3, and that is that ext3 uses a journal to prevent filesystem corruption in the case of an unclean shutdown (ie. before the filesystem is synced to disk). That makes ext3 a bit slower than ext2 since all metadata changes are written to the journal, and then flushed to disk, but on the other hand you don't risk having the entire filesystem destroyed at power failure or if an unwitted person turns the computer off uncleanly. You don't have to check the filesystem after an unclean shutdown either.
Ext3 has three levels of journalling. Metadata (ie. internal filesystem structures) are always journalled, so that the filesystem itself is never corrupted. How ordinary data is written to the file system is controllable, though. The default option is the "ordered" mode, which causes file contents to be written to the filesystem before metadata is even committed to the journal. The highest reliable mode is called the "journal" mode, which causes file data to be committed to the journal before it is flushed to its final place, like the metadata. The least reliable mode, but rumoured to be the fastest, is called the "writeback" mode, which makes no promises at all regarding the consistency of file data. Only metadata is output reliably in writeback mode.

So as for anything else, it's mainly a matter of priority. If you don't want ultimate speed, go with ext3. If you need the highest speed that is theoratically aquirable though, then go with ext2. For that to be effective you'll probably need a really advanced hard drive controller, though.

It's very easy to convert an ext2 filesystem to ext3. Just run tune2fs -j on the device and then remount it as ext3.

------------------------------------------------------------------------------

#.what is the difference between a soft link and a hard link?

2009-01-09T14:44:00.001-08:00

#.what is the difference between a soft link and a hard link?

Unix files consist of two parts: the data part and the filename part.

The data part is associated with something called an 'inode'. The inode carries the map of where the data is, the file permissions, etc. for the data.

The filename part carries a name and an associated inode number.

More than one filename can reference the same inode number; these files are said to be 'hard linked' together.

On the other hand, there's a special file type whose data part carries a path to another file. Since it is a special file, the OS recognizes the data as a path, and redirects opens, reads, and writes so that, instead of accessing the data within the special file, they access the data in the file named by the data in the special file. This special file is called a 'soft link' or a 'symbolic link' (aka a 'symlink').
more..

Hard Links :

1. All Links have same inode number.

2.ls -l command shows all the links with the link column(Second) shows No. of links.

3. Links have actual file contents

4.Removing any link ,just reduces the link count , but doesn’t affect other links.

Soft Links(Symbolic Links) :

1.Links have different inode numbers.

2. ls -l command shows all links with second column value 1 and the link points to original file.

3. Link has the path for original file and not the contents.

4.Removing soft link doesn’t affect anything but removing original file ,the link becomes “dangling” link which points to nonexistant file.