zram-config to be installed by default for 14.04

Registered by amjjawad  on 2013-08-29

With the great success of enabling zram-config on the daily build of Saucy (13.10) and according to so many tests with very good feedabck and results, IMHO, we need to do the same with Lubuntu 14.04 LTS and zram-config to be enabled/installed by default as well.

Blueprint information

Status:
Not started
Approver:
None
Priority:
Essential
Drafter:
None
Direction:
Needs approval
Assignee:
Lubuntu Developers
Definition:
Discussion
Series goal:
None
Implementation:
Deferred
Milestone target:
None

Related branches

Sprints

Whiteboard

[amjjawad] due to the unexpected bugs before the release of 13.10, I set the Priority of this suggestion to 'Essential' because it is a very important feature that could cause a serious issue.

[Mélodie] Hi Phillw, hi amjjawad and Lubuntu brainstorm team.

I would like to drag attention to two points related to the configuration while using the zram module:
=> The size of the zram block device created, or block devices when there is more than one processor,

=> and the usual swappiness of the kernel, which often tends to start swapping when lots of RAM is still available.

SHORT STORY:
Patches:
For the /usr/share/initramfs-tools/hooks/compcache file here is a patch I submit for use in Lubuntu:
http://meets.free.fr/Downloads/BentoVillageProject/Configurations/System/Zram/usr/share/initramfs-tools/hooks/compcache.patch

For the /etc/init/zram-config.conf file here is a patch I submit for use in Lubuntu:
http://meets.free.fr/Downloads/BentoVillageProject/Configurations/System/Zram/etc/init/zram.config.conf.patch

For the /etc/sysctl.d directory I submit this file:
http://meets.free.fr/Downloads/BentoVillageProject/Configurations/System/etc/sysctl.d/50-local.conf

Long and detailed story: I have posted full details about why and how, and differente cases experimented, here:
http://linuxvillage.org/en/2013/10/next-lubuntu-provided-with-zram-enabled

Best regards,
Mélodie

[vanyok] I like the zRAM feature enabled by default. Also I'd think that standard swappinness value should be set to something like 20 instead of 60.

[Mélodie] Hi vanyok,
It is very easy to test several values, and tweak it as you wish. You can just add a "50-local.conf" file to "/etc/sysctl.d/" such as this one:
###############################
# Uncomment the next line if we are running a laptop
# vm.laptop_mode = 1
vm.swappiness=20
vm.vfs_cache_pressure=50

###############################

[Leszek] I would not recommend changing vm.swappiness as swapping out unused pages (memory allocated only for the initialization by an application for example) early is a good thing to keep the main memory free. The bottlneck of SWAP on HDD does not exists with ZRAM so swapping out would not make the system feel terrible slow.
As for the proposed change of zram size to 1/4 of total available memory I really had no problem yet with it using half of the memory. It could theoretically lead to a bottleneck when ram is running full and swappiness would be set to a low value because than it needs to compress half of the memory at once but honestly speaking this bottleneck is there even in the case of zram being 1/4th of the ram. (Ok it then only needs to compress 1/4th of ram which is a little bit faster). So all in all I would keep 1/2 here if no one tests both and can argue that 1/2 would be a significant performance bummer.
As for testing I would recommend to only set and experiment with one value alone to make sure if this value changed can really make a difference in terms of performance.
As for the proposed vm.swappiness=20 and vm.vfs_cache_pressure=50 it would make the usage of
ram a lot more as you lower the tendency to swapout old unused pages to swap (which means it keeps more of them in memory) and at the same time you would keep more dentry and inode caches in ram with vfs_cache_pressure=50.
With this settings I would recommend doing some file copying tests with large and small and big amount of applications and mesure the responsivness of the system before, during and after the copying process. I guess vfs_cache_pressure might make copying processes like the installation of Lubuntu use more SWAP because of the increased dentry and inode caches. Together with vm.swappiness=20 this could lead to an shorter amount of free memory in ram and maybe higher swap usage.

[Leszek] Update. A short pracitcal test I did and you might want to retry on your system and post your results with it. First of all I performed the test on a Intel Core 2 Duo with 2 cpus with default zram-config (means using half of the ram) and 1 GB physical memory available. First of all I tested

vm.swappiness=20
vm.vfs_cahce_pressure=50

The bootup was fine. It took some time to load the login manager and after that the desktop. Loading the filemanager was a little bit slow but it worked fine. Loading the browser then slowed down the system and loading a few tabs here now slowed down the system (btw. I tested chromium not firefox) until even the mouse cursor did not move for a minute. The responsiveness was bad.
Testing on the same machine with

vm.swappiness=100
vm.vfs_cache_pressure=1000

startup was as fast as before. Loading the login manager was a little bit faster. (subjective feeling)
Loading the desktop was a little bit slower I guess. Loading the filemanager was good no slow downs feels repsonsive. Loading the webbrowser took a little bit longer but the browser was more usable than before. No slowdowns on loading different webpages in tabs no stucking of the mouse cursor. The system feels overall a lot more responsive. The performance might be slower (benchmark wise) but the responsiveness of the desktop is far better than before.
Both tests were performed only with zram as swap device no hdd swapping involed.

[amjjawad] for more information, you can follow up with the thread on the mailing list: https://lists.launchpad.net/lubuntu-qa/msg03962.html

[Nio] I think zRAM works well now, so I suggest that we keep the configuration, unless there is strong evidence that it should be changed.

[Mélodie] Hello,

After a long time using zram-config successfully in custom Ubuntu Openbox versions with the 12.04 as a basis, I met recently the same annoyances than the ones detected by different people during the testing of Lubuntu Saucy, while using it at same time as some of the former kernels. I just put a bug report here about the issues I met:
https://bugs.launchpad.net/ubuntu/+source/zram-config/+bug/1247496

"zram-config issues with linux 3.2.0-55 in 12.04"

I am very much disappointed, because I had to admit that with whatever configuration, the default zram-config provided, with or without any modification of the swapping behavior the result always leaded to the same end: a failure while using the newest available kernel for this series.

For me this is a stopper, until something clear comes out and my questions are the following, for any new future Lubuntu version:
* what exactly in the kernel configurations could bring in this type of issues?
* how to prevent them before the newer kernels are out and available for updates?

The choices done for fine tuning the setup seems to me a bit like the icing on the cake.

[ppp] please delete this if not suitable for here:

1. I have somehow the impression the system is not fully aware about the memory assigned to zram. The free command for example, shows at startup almost the whole memory as free, despite half of it is given to zram devices by zram-config. This may lead to problems, on low memory situations, because not all tools may be aware of that. I cannot prove this for sure on my own, someone should verify this - I see this as critical.

2. Creating a zram device per logical cpu is too much. IMHO, one zram device for the whole swap would be enough.

3. As noted by others, using half of RAM for zram swap is too much. I would vote for 1/8th of RAM size (less than 1/4 suggested above).

[leszek]
@Melodie: Kernel 3.2.0 ships a buggy zram. 3.12 has the stable one (this was back-ported to the 13.10s kernel).
I did not experience any problems so far with the 3.12s zram module. (it's btw. officially marked as stable)
@ppp: I am no sure if this suitable here to start a discussion but short answers:
To 1. You are wrong. Its a normal swap and all apps which are aware of swap are aware of zrams swap.
To 2. Having more processors for compressing and decompressing is better performance wise
To 3. There is near to nothing benefit from lowering the ram used for zram. In contrast this might have negative impact for people with low memory which will run out of it during install for example when no harddisk swap is accessible.

- - - - - - - - - - - - - - - - - -
[amjjawad] How about we discuss this on the mailing list and just add the link of the thread later? It seems this discussion will take long time with so many input and I am not sure if it is wise enough to carry on on the whiteboard? thank you :)
- - - - - - - - - - - - - - - - - -

[gilir] I don't want to do experimentations before an LTS. Adding zram was already too much pain for 13.10. So, I'll keep zram, but only on the ISO, to reduce the risk to install broken stuff permanently. Feel free to write a documentation about all of this, and let discuss it again for 14.10

[melodie]
to leszek: the kernel 3.2.0 was just updated with a fix these last days, problem solved; For what you say to ppp I agree with you; the only detail : 50% of the ram or 25% of the ram? Well I would still stick to 25% which has been way enough on old machines as far as I can remember. (a K6 with 386MB ram once). The narrow part there has always been the CPU usage.

to gilir: having zram-config in the Live only is surely the safest way until we are sure the kernels coming after won't be affected by a new regression.

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.

Subscribers

No subscribers.