På måndag kommer jag att släppa det på LinkedIn, där jag har ganska mycket kontakter hyfsat högt upp på företagen och då hoppas jag på att få lite spridning på det så att det kan göra nytta för alla Linuxanvändare. Och givetvis så släpper jag det här lite innan det...
Ursprunget är att jag har länge tyckt att det är ganska märkligt att det inte finns något bra verktyg för att hålla koll på och administrera Swap och Cache i Linux. Det har i princip inte funnits någonting alls för detta, förutom möjligheten att på vinst och förlust ändra parametrarna. Scriptet har funktioner för både inställning och återställning, testning och en massa annat. I början av scriptet finns dokumentationen som man kan läsa antingen direkt i källkoden eller också genom att ange "--help" när man kör det.
Självfallet har man störst nytta av ett sådant här script/program på en dator med relativt litet minne (i förhållande till det man kör i den). Det är där som det finns mest att vinna, eftersom på en dator med gott om minne så funkar det ju även om man inte utnyttjar det effektivt. Men även om man inte ändrar något, så kan det vara kul att se hur det är inställt och hur det fungerar.
Prova det gärna på era system och kom med synpunkter och idéer om det är något ni tycker är fel, eller om det är något som inte funkar. Scriptet är i dagsläget testat och fungerar felfritt för Ubuntu 14.10 Utopic Unicorn, Kubuntu 14.10, Lubuntu 14.10, Linux Mint 17.1 Rebecca, openSUSE 13.2, CentOS 7.0, Fedora 21 och ElementaryOS, mer än så har jag inte hunnit med...
Kod: Markera allt
#! /bin/bash
#
# This script is written by: Magnus Ewert
# ------------------------------------------------------------------------
# -- This program is free software: you can redistribute it and/or modify
# -- it under the terms of the GNU General Public License as published by
# -- the Free Software Foundation, either version 3 of the License, or
# -- (at your option) any later version.
# --
# -- This program is distributed in the hope that it will be useful,
# -- but WITHOUT ANY WARRANTY; without even the implied warranty of
# -- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# -- GNU General Public License for more details.
# --
# -- You should have received a copy of the GNU General Public License
# -- along with this program. If not, see <http://www.gnu.org/licenses/>.
# ------------------------------------------------------------------------
#set -u # Used for easy debugging
#
#=========================
#=========================
# ADJUSTABLE VARIABLES
#=========================
#=========================
#
# --Miscellaneous values
DEFAULT_SWAPPINESS=60 # Most common value - change if your system differs
DEFAULT_CACHE_PRESSURE=100 # Most common value - change if your system differs
#
# --Default steps for "test"
DEFAULT_STEPS_SWAP="10,60,90"
DEFAULT_STEPS_CACHE="50,100,10000"
#
# --Memory sizes # These memory sizes can be changed if needed, just change first digit...
let MEM_MINI=1*1024*1024 # 1 GiB - A "mini" memory is less or equal 1 GiB
let MEM_SMALL=2*1024*1024 # 2 GiB - A "small" memory is less or equal 2 GiB (but greater than 1 GiB)
let MEM_MEDIUM=4*1024*1024 # 4 GiB - A "medium" memory is less or equal 4 GiB (but greater then 2 GiB)
let MEM_LARGE=8*1024*1024 # 8 GiB - A "large" memory is less or equal 8 GiB (but grater than 4 GiB)
let MEM_MAXI=16*1024*1024 # 16 GiB - A "maxi" memory is less or equal 16 GiB (but grater than 8 GiB)
# # An even bigger memory is referred in the script, but not categorized...
let MINI=0 # enumeration of memory sizes...
let SMALL=1 # enumeration of memory sizes...
let MEDIUM=2 # enumeration of memory sizes...
let LARGE=3 # enumeration of memory sizes...
let MAXI=4 # enumeration of memory sizes...
let EMERGENCY_SWAP_GBYTES=1 # Size of emergency swap that the script can create
# --CPU Sampling Down Factor
CPU_SAMPLING_DOWN_FACTOR=10 # Factor to use to prevent too fast CPU slow down
#
#===============================================================================================================
# DON'T TOUCH BELOW, UNLESS YOU KNOW WHAT YOU ARE DOING!
#===============================================================================================================
#
CMD=$(basename $0)
let RAM=$(free -m | grep "Mem:" | awk '{ print $2 }')
# -Not that the system seldom or never reports sizes as 512, 1024 etc. due to the base-2 and base-10 difference.
# -For example 1 MB = 10**6 bytes, while 1 MiB = 2**20 bytes. However the comparison below becomes right anyway
# -because the steps between possible memory sizes.
if [[ $RAM -le 512 ]]; then
# Actually a 512 MB system normally shows 495
RAMSIZE="mini"
elif [[ $RAM -le 1024 ]]; then
RAMSIZE="small";
elif [[ $RAM -le 4096 ]]; then
RAMSIZE="medium"
elif [[ $RAM -le 8192 ]]; then
RAMSIZE="large"
else
RAMSIZE="maxi"
fi
#
EXIT=false
if [[ -z $(which sponge 2> /dev/null) ]]; then
if [[ ! -z $(which apt-get 2> /dev/null) ]]; then
echo -n "_Moreutils is not installed, do you want to install it? [Y/n]: "
read IN
if [[ -z $IN || $IN == "y" || $IN == "Y" ]]; then
apt-get install moreutils >> /dev/null 2>&1
else
exit
fi
else
#EXIT=true
SPONGE=false
#echo ' "moreutils" must be installed'
fi
else
SPONGE=true
fi
if [[ -z $(which gawk 2> /dev/null) ]]; then
if [[ ! -z $(which apt-get 2> /dev/null) ]]; then
echo -n "_GNU awk is not installed, do you want to install it? [Y/n]: "
read IN
if [[ -z $IN || $IN == "y" || $IN == "Y" ]]; then
apt-get install gawk >> /dev/null 2>&1
fi
else
EXIT=true
echo ' "gawk" must be installed'
fi
fi
if [[ $EXIT == true ]]; then
exit
fi
#
function usage {
CMD=$(basename $0)
cat << EOD
$CMD {clear|help|set|show|temp|test|trim} {cache|swap} -p|--perm -s values|--steps=values -a|--advise
$CMD helps you to manage the system swapping and pageing.
clear - Clears the swap or the cache
Format: $CMD clear {swap | cache}
This may be used for test purposes or to clear the cache after some kind of
heavy load. It may increase performance greatly in case the swap or cache is
filled with junk that is no longer needed.
It may take quite a time to complete and to be possible to run it, there must
be enough memory available, otherwise an error is issued.
Example:
$CMD clear cache
$CMD clear swap
help - Show this
Format: $CMD {-h | --help}
Example:
$CMD --help
set - Set parameters for Swappiness or Cache pressure
Format: $CMD set {swap | cache} value [-p | --perm]
Swapping means that inactive processes are swapped out from system memory (RAM)
to a swapfile. The swapfile acts as an extension to the RAM and in that way the
system can work with a larger memory than the physical RAM. On the other hand
the access to the processes and data that is in the swapfile is a lot slower and
costs a lot more resources to access, so regarding performance it would be much
better to keep all processes in memory.
The cache is a large area in the physical memory (RAM) that is used to store
filesystem data and other frequently accessed data items. For example inodes
which is used to access files are stored there. By storing data in the cache, the
access to it becomes a lot faster than reading it from disk every time. To see
some of the data in the cache, the "slabtop -s c" command may be used.
When the system has lots of physical memory, there is no problem and both the
processes (related to swap) and the data (related to cache) can use all space it
needs. But when the memory is limited there becomes a concurrent situation between
the processes and the cache. How to trim the system for best performance depends
a lot on how the system is used, a server has a quite different needs than a
desktop and a system with a small memory must be treated totally different than
one with a large.
swap - the "swappiness"
Normally the Swappiness is set to 60, which means that the system does not
start swapping until there is only 60% of the RAM left for other things.
For a desktop it is normally better with a lower value, which makes the
processes more resident. For a server on the other hand, it mostly gives
a better throughput by increasing the value to give more room for the cache.
Basically it is so that less swappiness gives faster response due to that
the process is already in memory (and not outswapped), but on a system
with a small physical memory it may be necessary even to increase the
swappiness only to get enough room for the system to live.
cache - the "cache pressure"
The Cache Pressure is set to 100 which is default. This implies the kernel
will attempt to reclaim dentries and inodes at a “fair” rate with respect
to pagecache and swapcache reclaim. A lower value implies that the system
will keep more filesystem cache and a higher value means it will try to
reclaim the space. Generally the "100" means 100%, which shall be considered
as the normal situation where the system tries to reclaim space from the
cache and a lower level means that the system will let more of the cache
exist.
Generally it gives better performance with a larger cache, but if the memory
is limited and the system has to choose between processes (swappiness) and
cache, it -may- be better to reduce the cache, at least for a desktop.
dirty - set dirty_ratio
dirtybg - set dirty_background_ratio
These options are to be considered quite as "undocumented features", they can
be used to change dirty_ratio and dirty_background_ratio for experimental
purposes. For exmaple it is possible to test with RedHat's and Suse's default
values on a Ubuntu system and vice versa. The "default" option does not work
for these parameters, as what as is default varies between different systems.
Generally it can be seen like that the processes (swapping) and the cache competes
about the memory. This is a very important rule to have in mind, although this
is only true as long as the memory is limited.
One special situation that may occur for a system with large memory, is that the
system swaps processes out even when there is plenty of room. The setting of
swappiness is static in that way that when the memory reaches the limit, the
swapping starts, even if there is still a lot of free memory. Let us say that
the swappiness is set to the default 60, then if the processes needs 45% of the
memory and the cache needs 50%, the processes will be swapped out down to 40%
although there actually already was 5% free memory.
Example:
$CMD set swap 10 --perm # Set swap to 10 permanently
$CMD set cache 100 # Set cache pressure to 100 until next boot
$CMD set swap default # Set swap to default value ($DEFAULT_SWAPPINESS)
The value is either a numeric value or the term "default".
Note! the recommendations here are general, in real life they may differ on any
subjective system due to the specific type of load that exists.
show - Show status about swapping and pageing
Format: $CMD show {swap | cache | all} {-a | --advise}
If the command $CMD is issued without any parameters or options, then the "show all"
function is executed.
Example:
$CMD # Same as "$CMD show all"
$CMD show all
$CMD show all --advise # Same as "$CMD -a"
$CMD show cache
$CMD show swap
When showing information about the memory, the script also tries to also come with some
good recommendations about how to increase performance and make the system faster. But
although the script really tries, it can never consider all your systems specific
characteristics and loads. You shall use the recommendations as a kind of rules of
thumb and then add your own knowledge and a lot of testing!!
If the -a or --advise switch is used, the script tries to give some advises about how
to trim the system, accordingly to the amount of physical memory etc. This shall however
only be seen as an advise, as there is no analysis of the specific load on the system or
what programs and applications that are executed. So read the advises, add your own
knowledge and make up your mind about what as will be useful to try.
temp - Create a temporary swap of $EMERGENCY_SWAP_GBYTES GiB
Format: $CMD temp swap
This is basically an emergency command, to be used if the swap is full and the
computer possibly will hang without any chance for an important work to complete.
The added size is fixed to $EMERGENCY_SWAP_GBYTES which is defined in the head of
this script, but however the command may be issued more than once to create
a larger temporary swap space.
Example:
$CMD temp swap
The swap is created in /tmp, so it will be automatically deleted in the next boot.
Although the size is fixed in the header of this script, it is possible to issue the
command more than once to create more space.
test - Test performance for different values on Swappiness and Cache pressure.
Tests the time it takes to perform certain tasks. The values for Swappiness and
Cache pressure are altered to make it possible to see how that affects performance.
Format: $CMD test {swap | cache | all} {-s values | --steps=values} {-x file | --script=file}
Defult steps for cache is $DEFAULT_STEPS_CACHE and default steps for swap is $DEFAULT_STEPS_SWAP
Options:
cache $CMD test cache --steps=$DEFAULT_STEPS_CACHE --script=foo.bar
swap $CMD test swap --steps=$DEFAULT_STEPS_SWAP
all $CMD test all --steps=$DEFAULT_STEPS_CACHE:$DEFAULT_STEPS_SWAP
See the "set" option for more information about "swap" and "cache".
The steps paramater is the specific values to test for cache_pressure or swappiness. If
no values for steps are specified, the default values from the header of this script is
used. Values are specified by three comma separated numbers and when testing all the
specification for cache_pressure and swappiness are separated by a colon.
Normally the test shall continue to iterate around the best of the three values, in order
to finally find the optimal value. So if the steps in the first test are 10, 60 and 90
and 10 yields the best result, the next iteration could specify 5, 10 and 15 or 5, 10 and
20 etc.
If there is a lot of other activities in your system, you will probably have to run a lot
of tests before you get a clera picture of which setting as is the best.
It is also important to note that the values may influence each other. If you first run a
test of swappiness with the values "a,b,c" and the value "a" turns out to be best, and then
you run a test of cache pressure with the values of "d,e,f" and "e" turns out to be best.
Then the value of "e" may not at all be the best when you have changed the swappiness to
"a". The same is of course also valid in the other direction, so if you run test of the
swappiness first, and then you change swappiness to "a" before testing cache pressure, to
be sure that the test obtains the best possible value. Then it is not at all necessary that
"a" is still the best value for swappiness when you have changed the value fpr cache pressure.
The "all" test performs a cross-examination of all values to find out which combination
that gives the best performance. However it is important to remember that this is a
quite complex test. Normally you also have to run it more than once to get a good
approximation of the best value.
It is possible to specify an own script to be used in the test, but by default the
script below is used:
#! /bin/bash
if [[ -x /sbin/sysctl ]]; then
CACHE=\$(/sbin/sysctl vm.vfs_cache_pressure | awk "{ print \$3 }")
else
CACHE=\$(cat /proc/sys/vm/vfs_cache_pressure)
fi
if [[ -x /sbin/sysctl ]]; then
SWAPPINESS=\$(/sbin/sysctl vm.swappiness | awk "{ print \$3 }")
else
SWAPPINESS=\$(cat /proc/sys/vm/swappiness)
fi
echo "Cache pressure is set to $CACHE"
echo "Swappiness is set to $SWAPPINESS"
TESTFILE1=\$(mktemp)
TESTFILE2=\$(mktemp)
echo "-Step #1"
sync
echo 3 > /proc/sys/vm/drop_caches
dd if=/dev/zero of=\$TESTFILE1 count=1 bs=900M
echo "-Step #2"
find / > /dev/null 2>&1
echo "-Step #3"
cp \$TESTFILE1 \$TESTFILE2
echo -n "-Step #4"
time find / > /dev/null 2>&1
echo "-Step #5"
rm -f \$TESTFILE1 \$TESTFILE2
exit
This script tests both the swappiness and the cache pressure quite hard, but however it
is not necessarily representative for your specific load, so sometimes it might be better
to test with an own script.
If you are using an own script is is important to think of that to get a correct picture,
it has to be att quite heavy script that takes a considerable time to complete, otherwise
the result may depend too much on the current other activities in the system. The script
must also generate a type of load that is so similar as possible to the normal load in the
system, otherwise you will test the system for something completely different.
It may take quite some time to execute this command, generally the standard script takes
about 10 minutes for the test of cache or swap and about 1.5 hours for the cross-examination
of all.
trim - This command is quite unscientific and it is based mostly on "network rumors". The meaning of
this is that a sufficiently secure source has said that -"this will give better performance",
but there are no direct evidence for this.
Format: $CMD {arch | backup | cpu | desktop | mini | reset | restore | router}
Options:
arch Trim accordingly to recommendations for Arch (this may work well for
other dists too).
backup Trim accordingly to net-rumors for backup-servers or other servers with
heavy data transfers.
cpu Trim cpu by reducing the risk that it decreases CPU-frequency too fast.
desktop Standard trim for normal desktop usage. Generally based on net-rumors
and own ideas. My own experience is that this works VERY well.
desktop2 A kind of preemptive setting for desktops with small memory. This mode
causes the system to let the cache grow, while inactive processes are
swapped out. Try this mode and compare it to "desktop" if you have a
small memory.
mini Trim system for a desktop with minimal memory.
reset Reset changed parameters to their previous value. Note that this must
restore be done once for each changing and before anything else is changed.
If two "trim-functions" are done after each other, the second will
overwrite the stored values from the first.
router Same as backup.
If more than one adjustment is to be tested, the reset must be done between every one of them
to work. Otherwise the second test will overwrite the initial data from the first so that
there is nothing left of the original values.
Yet there is no general setting for "server", because there is a big difference between
servers depending on what kind of work they perform.
zram - Check or change dynamic zram (compcache).
Format: $CMD zram {-e | --edit}
Options:
If used without any options, the current size and state of zram is shown. A check
is also done to see if zram is installed and if it is not, a question is issued
about whether to install it.
-e, --edit Edit setup for zram. This is normally not necsessary as
the system automatically set up zram for best practice.
Note! that the edit-switch is not useful for all implementations. Some systems
has zram integrated into the kernel and there really is no parameters to edit.
If used without any option, the command shows info about current swap and cache (ie. "$CMD show all").
Author
Written by Magnus Ewert
Tested and works well under:
Ubuntu 14.10 Utopic Unicorn, Kubuntu 14.10, Lubuntu 14.10, Linux Mint 17.1 Rebecca,
openSUSE 13.2, CentOS 7.0, Fedora 21, ElementaryOS
EOD
}
Ha det gott och lycka till...
