分类: LINUX
2008-04-21 13:30:25
This
entry is for those people who have ever wondered, "Why the hell is a
simple KDE text editor taking up 25 megabytes of memory?" Many people
are led to believe that many Linux applications, especially KDE or
Gnome programs, are "bloated" based solely upon what tools like ps
report. While this may or may not be true, depending on the program, it
is not generally true -- many programs are much more memory efficient
than they seem.
The
ps tool can output various pieces of information about a process, such
as its process id, current running state, and resource utilization. Two
of the possible outputs are VSZ and RSS, which stand for "virtual set
size" and "resident set size", which are commonly used by geeks around
the world to see how much memory processes are taking up.
For example, here is the output of ps aux for KEdit on my computer:
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
dbunker 3468 0.0 2.7 25400 14452 ? S 20:19 0:00 kdeinit: kedit
According
to ps, KEdit has a virtual size of about 25 megabytes and a resident
size of about 14 megabytes (both numbers above are reported in
kilobytes). It seems that most people like to randomly choose to accept
one number or the other as representing the real memory usage of a
process. I'm not going to explain the difference between VSZ and RSS
right now but, needless to say, this is the wrong approach; neither
number is an accurate picture of what the memory cost of running KEdit
is.
Depending
on how you look at it, ps is not reporting the real memory usage of
processes. What it is really doing is showing how much real memory each
process would take up if it were the only process running.
Of course, a typical Linux machine has several dozen processes running
at any given time, which means that the VSZ and RSS numbers reported by
ps are almost definitely "wrong". In order to understand why, it is
necessary to learn how Linux handles shared libraries in programs.
Most
major programs on Linux use shared libraries to facilitate certain
functionality. For example, a KDE text editing program will use several
KDE shared libraries (to allow for interaction with other KDE
components), several X libraries (to allow it to display images and
copy and pasting), and several general system libraries (to allow it to
perform basic operations). Many of these shared libraries, especially
commonly used ones like libc, are used by many of the programs running
on a Linux system. Due to this sharing, Linux is able to use a great
trick: it will load a single copy of the shared libraries into memory
and use that one copy for every program that references it.
For
better or worse, many tools don't care very much about this very common
trick; they simply report how much memory a process uses, regardless of
whether that memory is shared with other processes as well. Two
programs could therefore use a large shared library and yet have its
size count towards both of their memory usage totals; the library is
being double-counted, which can be very misleading if you don't know
what is going on.
Unfortunately,
a perfect representation of process memory usage isn't easy to obtain.
Not only do you need to understand how the system really works, but you
need to decide how you want to deal with some hard questions. Should a
shared library that is only needed for one process be counted in that
process's memory usage? If a shared library is used my multiple
processes, should its memory usage be evenly distributed among the
different processes, or just ignored? There isn't a hard and fast rule
here; you might have different answers depending on the situation
you're facing. It's easy to see why ps doesn't try harder to report
"correct" memory usage totals, given the ambiguity.
Enough
talk; let's see what the situation is with that "huge" KEdit process.
To see what KEdit's memory looks like, we'll use the pmap program (with
the -d flag):
Address Kbytes Mode Offset Device Mapping
08048000 40 r-x-- 0000000000000000 0fe:00000 kdeinit
08052000 4 rw--- 0000000000009000 0fe:00000 kdeinit
08053000 1164 rw--- 0000000008053000 000:00000 [ anon ]
40000000 84 r-x-- 0000000000000000 0fe:00000 ld-
40015000 8 rw--- 0000000000014000 0fe:00000 ld-
40017000 4 rw--- 0000000040017000 000:00000 [ anon ]
40018000 4 r-x-- 0000000000000000 0fe:00000 kedit.so
40019000 4 rw--- 0000000000000000 0fe:00000 kedit.so
40027000 252 r-x-- 0000000000000000 0fe:00000 libkparts.so.
40066000 20 rw--- 000000000003e000 0fe:00000 libkparts.so.
4006b000 3108 r-x-- 0000000000000000 0fe:00000 libkio.so.
40374000 116 rw--- 0000000000309000 0fe:00000 libkio.so.
40391000 8 rw--- 0000000040391000 000:00000 [ anon ]
40393000 2644 r-x-- 0000000000000000 0fe:00000 libkdeui.so.
40628000 164 rw--- 0000000000295000 0fe:00000 libkdeui.so.
40651000 4 rw--- 0000000040651000 000:00000 [ anon ]
40652000 100 r-x-- 0000000000000000 0fe:00000 libkdesu.so.
4066b000 4 rw--- 0000000000019000 0fe:00000 libkdesu.so.
4067d000 4 rw--- 0000000000011000 0fe:00000 libkwalletclient.so.
4067e000 4 rw--- 000000004067e000 000:00000 [ anon ]
40898000 64 rw--- 0000000000219000 0fe:00000 libkdecore.so.
... (trimmed) ...
mapped: 25404K writeable/private: 2432K shared: 0K
I
cut out a lot of the output; the rest is similar to what is shown. Even
without the complete output, we can see some very interesting things.
One important thing to note about the output is that each shared
library is listed twice; once for its code segment and once for its
data segment. The code segments have a mode of "r-x--", while the data
is set to "rw---". The Kbytes, Mode, and Mapping columns are the only
ones we will care about, as the rest are unimportant to the discussion.
If
you go through the output, you will find that the lines with the
largest Kbytes number are usually the code segments of the included
shared libraries (the ones that start with "lib" are the shared
libraries). What is great about that is that they are the ones that can
be shared between processes. If you factor out all of the parts that
are shared between processes, you end up with the "writeable/private"
total, which is shown at the bottom of the output. This is what can be
considered the incremental cost of this process, factoring out the
shared libraries. Therefore, the cost to run this instance of KEdit
(assuming that all of the shared libraries were already loaded) is
around 2 megabytes. That is quite a different story from the 14 or 25
megabytes that ps reported.
The
moral of this story is that process memory usage on Linux is a complex
matter; you can't just run ps and know what is going on. This is
especially true when you deal with programs that create a lot of
identical children processes, like Apache. ps might report that each
Apache process uses 10 megabytes of memory, when the reality might be
that the marginal cost of each Apache process is 1 megabyte of memory.
This information becomes critial when tuning Apache's MaxClients
setting, which determines how many simultaneous requests your server
can handle (although see one of my past postings for another way of
increasing Apache's performance).