2020-03-17T18:39:13

Record and report memory usage over long time

Created a Perl script to record and report memory usage of processes to discover heavy memory greedy processes.

Here is the source code, below this text block Perl Pod documentation and usage.

In the "real life" troubleshooting I need it for, it did reveal some bloating processes, but only after some time, had to increase "nice" priority for the script, because when the system get under stress, kind of everything gets slow, including the processes inspection.

Another alternative approach that I've tried was to keep htop running on the target system, record it with kazam and then using on monitoring charts could identify actually watch the interesting moments of the system behavior.


NAME

process-mem-report - record & print top N memory consumers over time

SYNOPSIS

    process-mem-report

        --help
        --runtime|t   number of minutes to run the measurement, default infinite
        --maxlen|l    maximum output line length, default 80
        --maxlines|n  number of top memory greedy processes to show, default 30
        --refresh|r   refresh rate in seconds, default 10
        --write|w     write to file
        --wstrftime|f add strftime formatted suffix to write file

DESCRIPTION

Will go through all procesed and read their memory usage using smaps and show top N. After refresh time will re-read all process together with previous N will show new top N.

If executed unprivileged, will consider processe of current user only.

EXAMPLE

    $ process-mem-report -n 10
    --------------------------------------------------
    2020-02-22T22:22:14 885.48M /usr/lib/thunderbird/thunderbird
    2020-02-22T22:22:14 653.38M firefox-esr
    2020-02-22T22:22:15 487.50M /usr/lib/firefox-esr/firefox-esr -contentproc -chil…
    2020-02-22T22:22:15 451.92M /usr/lib/chromium/chromium --show-component-extensi…
    2020-02-22T22:22:14 380.55M /usr/lib/firefox-esr/firefox-esr -contentproc -chil…
    2020-02-22T22:22:14 377.14M cinnamon --replace
    2020-02-22T22:22:15 321.80M /usr/lib/firefox-esr/firefox-esr -contentproc -chil…
    2020-02-22T22:22:14 308.17M /usr/lib/firefox-esr/firefox-esr -contentproc -chil…
    2020-02-22T22:22:15 257.77M /usr/lib/firefox-esr/firefox-esr -contentproc -chil…
    2020-02-22T22:22:15 247.15M /usr/lib/firefox-esr/firefox-esr -contentproc -chil…

    max mem used: 7.50G at: 2020-02-22T22:22:15
    since: 2020-02-22T22:22:13

    # hourly written report of top memory greedy processes
    $ sudo process-mem-report -r 5 -n 80 -w mem-top_ -f %Y%m%d_%H