I downloaded lighttpd 1.5 (r1523) today and found mod_deflate was buggy. Bugs come from deflate.sync-flush and deflate.work-block-size.
- When deflate.sync-flush is enabled with transfer-encoding was chunked, only part of first chunk was compressed
- For chunked transfer-encoding, when deflate.work-block-size is smalled than size of one chunk, compressed output is gambled.
I removed deflate.work-block-size and deflate.sync-flush and it works for me.
Get patch here
Jan 刚发布了令人期待的LINUX AIO支持，我也来凑热闹把我写好的mod_memcompress plugin 奉献给大家;)
mod_memcompress工作方式类似mod_compress, 把浏览器的gzip,deflate请求的内容压缩后放到内存里，下次其它浏览器相同的请求直接从内存里输出结果。
mod_memcompress 的配置参数如下：
- memcompress.filetype, 设置压缩文件的类型。举例memcompress.filetype = ("text/html", "application/x-javascript", "text/css")
- memcompress.maxfilesize, memcompress压缩的最大文件大小，单位是Kbytes。缺省是4096
- memcompress.maxmemory, memcompress缓存占用最大的内存大小，单位是Mbytes。缺省是256
- memcompress.nocompress-url, PCRE正则表达式。匹配正则表达式的URI将不被处理。
- memcompress.enable, 打开memcompress的开关。缺省enable
- memcompress.lruremovenumber，当memcompress用到最大内存时，memcompress从缓存中删除的条目数量。缺省500个。
- memcompress.compression-level, 压缩级别，是1～9的数字，越大代表压缩率越高，压缩时占用cpu也越多。缺省是Z_DEFAULT_COMPRESSION
当压缩结果存到缓存里是，返回的HTTP头有X-Mem-Hit: to memcompress；当压缩结果从缓存中读取时，返回的HTTP头有X-Mem-Hit: by memcompress。方便判断缓存效果;)
在说说实现细节，挺有意思。memcompress把缓存结果信息hash到131072(2^17)的数组里，然后用数组的prev,next整型变量做lru伪双向链表，lurheader和lruend记录lru链表的头和尾。大家可能奇怪，为什么不用指针去实现双向链表呢。其实第一版的lru是用指针实现，但偶尔会出现指针指到不正确的地方导致Segment Fault。LRU指针不正确是可以在更新LUR表时遍历一次检查，但这样的开销就太大了；换成整型变量做LRU后，不再出现segment fault了。
Long Time No See! Recently I was busy with testing mod_cache, which was more sophicated than expected. Here I want to present guys with mod_deflate before mod_cache's annoncement.
mod_deflate is a lighttpd plugin to compress content on fly. It's useful when you want to reduce your bandwidth consumption to 30% or lower:) my patch is based on jakabosky's 1.4.11 patch and fixed several big bugs includes:
1) fix loop bug when content-length is bigger than work-block-size*k
2) prevent compress on buggy http 1.0 client with Accept Encoding: gzip, deflate
3) fix bug with chunk transfer encoding (for examples in mod_fastcgi+php environment)
mod_deflate don't compress uri matched deflate.nocompress-url which is a pcre regex string.
I had used mod_deflate in production environments and found no noticable bugs or memory leak. Usually mod_deflate takes less than 10% CPU and eats up less than 100M memory. Full document is here
Here I want to say big thanks to Jakabosky, his mod_deflate patch makes mod_cache on lighttpd possible.