mod_mem_cache patch for lighttpd 1.4.19
Published March 14th, 2008 in TechYou can download mod_mem_cache patch for lighttpd 1.4.19 from here.
patching steps:
tar xfz lighttpd-1.4.19.tar.gz; cd lighttpd-1.4.19
wget http://blog.quehy.com/doc/lighttpd-1.4.19.mod_mem_cache.patch;
patch -p0 < lighttpd-1.4.19.mod_mem_cache.patch
sh autogen.sh
then run "./configure --quiet; make" to compile your lighttpd
This doesn't seem to work with mod_expire. Every effort to get the Expires header to be sent failed. If at all possible, could you make it work with mod_expire?
change one line of mod_mem_cache.c
change
p->handle_physical = mod_mem_cache_uri_handler;
to
p->handle_subrequest_start = mod_mem_cache_uri_handler;
recompile and load mod_expire before mod_mem_cache at lighttpd.conf
Alright, mod_expire now works. However, mod_compress doesn't seem to work with it.
lsof -r 1 -p Lighttpd's_PID | grep "/home/USERNAME/mod_compress_cache/"
Shows file access if mod_compress is loaded before mod_mem_cache, and the files aren't sent compressed if mod_mem_cache loads before mod_compress.
mod_mem_cache doesn't cache compressed files which were created by mod_compress
Hi,
is it possible to get mod_mem_cache for the latest 1.5 version of lighty?
Thanks
I am waiting for update to date lighttpd 1.5.0 release instead of r1992
为什么我打了补丁后，有如下的错误？
routerclub:/usr/src/lighttpd-1.4.19# ./autogen.sh
./autogen.sh: running `libtoolize –copy –force'
./autogen.sh: running `aclocal'
./autogen.sh: running `autoheader'
./autogen.sh: running `automake –add-missing –copy'
doc/Makefile.am:89: `%'-style pattern rules are a GNU make extension
./autogen.sh: running `autoconf'
configure.in:70: error: possibly undefined macro: AC_DEFINE
If this token and others are legitimate, please use m4_pattern_allow.
See the Autoconf documentation.
在configure中：
checking for pkg-config... no
checking for FAM... no
./configure: line 24955: syntax error near unexpected token `FAM,'
./configure: line 24955: ` PKG_CHECK_MODULES(FAM, gamin >= 0.1.0,'
I followed the instructions but...
]# lighttpd -D -f /root/config.test.light
2008-04-16 10:10:21: (log.c.75) server started
2008-04-16 10:10:21: (server.c.931) WARNING: unknown config-key: mem-cache.enable (ignored)
2008-04-16 10:10:21: (server.c.931) WARNING: unknown config-key: mem-cache.filetypes (ignored)
2008-04-16 10:10:21: (server.c.931) WARNING: unknown config-key: mem-cache.max-memory (ignored)
2008-04-16 10:10:21: (server.c.931) WARNING: unknown config-key: mem-cache.max-file-size (ignored)
checking for pkg-config... no
checking for FAM... no
./configure: line 24955: syntax error near unexpected token `FAM,'
./configure: line 24955: ` PKG_CHECK_MODULES(FAM, gamin >= 0.1.0,'
:/
I got the same problem.
———————————————-
Sam Apr 16th, 2008 at 9:12 pm
I followed the instructions but...
]# lighttpd -D -f /root/config.test.light
2008-04-16 10:10:21: (log.c.75) server started
2008-04-16 10:10:21: (server.c.931) WARNING: unknown config-key: mem-cache.enable (ignored)
2008-04-16 10:10:21: (server.c.931) WARNING: unknown config-key: mem-cache.filetypes (ignored)
2008-04-16 10:10:21: (server.c.931) WARNING: unknown config-key: mem-cache.max-memory (ignored)
2008-04-16 10:10:21: (server.c.931) WARNING: unknown config-key: mem-cache.max-file-size (ignored)
撤销“unknown config-key”的问题，照着指南做了下来，居然忘记了在server.modules里配置，囧
# sh autogen.sh
autogen.sh: running `libtoolize –copy –force'
autogen.sh: running `aclocal'
autogen.sh: running `autoheader'
autogen.sh: running `automake –add-missing –copy'
doc/Makefile.am:89: `%'-style pattern rules are a GNU make extension
autogen.sh: running `autoconf'
configure.in:70: error: possibly undefined macro: AC_DEFINE
If this token and others are legitimate, please use m4_pattern_allow.
See the Autoconf documentation.
Hi,
I have tested Lighttpd-1.4.19 with mod_mem_cache for static file http server.(html,gif,jpg etc)
I have some questions.
Q1. how to confirm if mod_mem_cache work?
Q2.Do you have a plan to implement to cache compressed file by using mod_compress?
Q3.I understand different between mod_mem_cache and mod_cache is below.
mod_mem_cache is for static files.
mod_cache is for dynamic files using another http server(backend).
it means mod_cache is like mod_proxy.
Is it right?
best regards
Kaz.
我照着你的做，出现了下面的问题
# patch -p0
checking for FAM... no
./configure: line 24955: syntax error near unexpected token `FAM,'
./configure: line 24955: ` PKG_CHECK_MODULES(FAM, gamin >= 0.1.0,'
configure.in:70: error: possibly undefined macro: AC_DEFINE
If this token and others are legitimate, please use m4_pattern_allow.
See the Autoconf documentation.
solution:
apt-get install libfam-dev
hey, thanks for this patch.
is there a way to find out if the cache is working correctly? thanks in advance!
请问for lighttpd 1.4.20的版本啥时候发布？
刚才测试了一下，补丁可以直接打到lighttpd 1.4.20上
———–
cd lighttpd-1.4.20
patch -p0 < ../lighttpd-1.4.19.mod_mem_cache.patch
patching file ./src/http-header-glue.c
patching file ./src/Makefile.am
Hunk #1 succeeded at 126 (offset 5 lines).
patching file ./src/mod_mem_cache.c
patching file ./src/mod_staticfile.c
patching file ./src/response.h
怎样验证是否mod—cache起作用了，我正常配置了，而且也没有提示任何报错的信息。但是通过strace -p lighttp-pid观察发现还是去读硬盘上的文件。
有空麻烦回复一下，谢谢！
打好补丁后，修改mod_mem_cache.c里的
response_header_overwrite(srv, con, CONST_STR_LEN("X-Mem-Cache"), CONST_STR_LEN("by memcache"));
把前面的注释"//"去掉（有两处），重新编译安装lighttpd，重启lighttpd
之后可以在返回的http header里看是否有"X-Mem-Cache: by memcache"，有就表明memory cache生效
Hi I had a problem compiling lighttpd and mod_cache on solaris 10/08 sparc
i.e.
mod_cache.c: In function `save_chunkqueue':
mod_cache.c:1102: error: `UIO_MAXIOV' undeclared (first use in this function)
mod_cache.c:1102: error: (Each undeclared identifier is reported only once
mod_cache.c:1102: error: for each function it appears in.)
mod_cache.c:1102: warning: unused variable `chunks'
mod_cache.c: In function `check_response_iscachable':
mod_cache.c:1335: warning: implicit declaration of function `timegm'
make[2]: *** [mod_cache.lo] Error 1
make[2]: Leaving directory `/Desktop/lighttpd-1.4.19/src'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/Desktop/lighttpd-1.4.19'
make: *** [all] Error 2
so I
modified mod_cache.c
from
#ifndef UIO_MAXIOV
# ifdef __FreeBSD__
/* FreeBSD 4.7, 4.9 defined it in sys/uio.h only if _KERNEL is specified */
# define UIO_MAXIOV 1024
# endif
#endif
to
#ifndef UIO_MAXIOV
# if defined(__FreeBSD__) || defined(__APPLE__) || defined(__NetBSD__)
/* FreeBSD 4.7 defines it in sys/uio.h only if _KERNEL is specified */
# define UIO_MAXIOV 1024
# elif defined(__sgi)
/* IRIX 6.5 has sysconf(_SC_IOV_MAX) which might return 512 or bigger */
# define UIO_MAXIOV 512
# elif defined(__sun)
/* Solaris (and SunOS?) defines IOV_MAX instead */
# ifndef IOV_MAX
# define UIO_MAXIOV 16
# else
# define UIO_MAXIOV IOV_MAX
# endif
# elif defined(IOV_MAX)
# define UIO_MAXIOV IOV_MAX
# else
# error UIO_MAXIOV nor IOV_MAX are defined
# endif
#endif
I freely admit I don't know what I'm doing ( just a poor admin) but it does seem to compile now.
Can you give me some pointers/suggestions as to what correct values might be?
Cheers
from network_solaris_sendfilev.c, we can find
#ifndef UIO_MAXIOV
#define UIO_MAXIOV IOV_MAX
#endif
Adding three lines above to mod_cache.c may solve compile problem under Solaris system
您好!我安装了lighttpd 1.5.0,打了您的那个补丁,您在上面说的修改那个文件内容:
#if 0
response_header_overwrite(srv, con, CONST_STR_LEN("X-Mem-Hit"), CONST_STR_LEN("to mem-cache"));
#endif
status_counter_set(CONST_STR_LEN("mem-cache.memory-inuse(MB)"), usedmemory>>20);
status_counter_set(CONST_STR_LEN("mem-cache.cached-items"), cachenumber);
} else {
cache->ct = srv->cur_ts;
reqhit ;
#if 0
response_header_overwrite(srv, con, CONST_STR_LEN("X-Mem-Hit"), CONST_STR_LEN("by mem-cache"));
#endif
}
} else {
reqhit ;
#if 0
response_header_overwrite(srv, con, CONST_STR_LEN("X-Mem-Hit"), CONST_STR_LEN("by mem-cache"));
#endif
}
请问是直接修改去掉那几个#号吗?再重新编译吗?
那配置文件如何修改配置呢?
详细的配置说明文档在哪呢？
用了你的Mod_cache。应该是成功了有缓存文件了但是err出现了几个错误报错不知道和使用了mod_cache有关系不？
偶尔会在err——log里出现
2008-12-16 17:55:02: (network_linux_sendfile.c.143) open failed: No such file or directory
2008-12-16 17:55:02: (connections.c.685) connection closed: write failed on fd 620
前台偶尔会出现500错误。
最初以为是openfile的问题。调整到了也没用。
Lig配置文件中server.max-fds = 25000参数也不小啊
network_linux_sendfile.c.143 处
140 /* open file if not already opened */
141 if (-1 == c->file.fd) {
142 if (-1 == (c->file.fd = open(c->file.name->ptr, O_RDONLY))) {
143 log_error_write(srv, __FILE__, __LINE__, "ss", "open failed: ", strerror(errno));
144
145 return -1;
146 }
connections.c.685 处
673 /* write chunks from output_queue to network */
674 switch(network_write_chunkqueue(srv, con, con->output_queue)) {
675 case 0:
676 if (finished) {
677 connection_set_state(srv, con, CON_STATE_RESPONSE_END);
678 joblist_append(srv, con);
679 } else {
680 /* not finished yet -> WRITE */
681 con->is_writable = 1;
682 }
683 break;
684 case -1: /* error on our side */
685 log_error_write(srv, __FILE__, __LINE__, "sd",
686 "connection closed: write failed on fd", con->fd);
687 connection_set_state(srv, con, CON_STATE_ERROR);
688 joblist_append(srv, con);
689 break;
690 case -2: /* remote close */
691 connection_set_state(srv, con, CON_STATE_ERROR);
692 joblist_append(srv, con);
693 break;
694 case 1:
695 con->is_writable = 0;
696
697 /* not finished yet -> WRITE */
698 break;
699 }
700
701 return 0;
702 }
trying to compile on freebsd 7.1 and lighttpd 1.4.21 but getting compile errors:
mod_compress.c: In function 'mod_compress_physical':
mod_compress.c:763: error: too few arguments to function 'http_response_handle_cachable'
mod_compress.c:788: error: too few arguments to function 'http_response_handle_cachable'
*** Error code 1
please try new release at http://code.google.com/p/lighttpd-improved/
ok that branch compiled fine. Thanks!
FYI - i also applied the patch included in the /usr/ports for lighttpd
打上patch后，lighttpd自己做服务器时cache可以起作用，即在
http header里可以看到"X-Mem-Cache by memcache"，但是
在做为代理时，就不行了，需要改配置吗？
Thanks！
X-Mem-Cache by memcache 头是在modcache里输出的，特意表示缓存状态
如果没有被缓存，自然不输出了，节省输出流量
不好意思，没有说清楚
我现在是用lighttpd做apache的代理，lighttpd的mod_mem_cache配置如下：
mem-cache.filetypes = ("application/x-javascript", "text/css", "text/html", "text/javascript", "image/jpg", "image/jpeg", "image/gif", "image/png", "image/bmp")
mem-cache.enable = "enable"
这样也应该可以看到"X-Mem-Cache by memcache"吧？
顺便问一下，mod_cache和mod_mem_cache是什么关系啊？"X-Mem-Cache by memcache"是mod_cache还是mod_mem_cache的输出？
期待中。。。谢谢！
mod_cache 缓存后端服务器的输出到本地文件
mod_mem_cache 缓存本地文件到内存里
不好意思，是下面的问题：
patching file ./src/http-header-glue.c
Hunk #2 FAILED at 242.
1 out of 2 hunks FAILED — saving rejects to file ./src/http-header-glue.c.rej
建议用http://blog.quehy.com/archives/209.html这个帖子里的lighttpd targz，包括modcache,mod_mem_cache和mod_mem_compress 三个模块
谢谢啊！还有两个问题想请教：
1、lighttpd的mod_proxy模块是否就是一个反向代理的功能？
2、lighttpd的mod_cache有没有类似squid的日志可以看到miss或者hit呢？
mod_proxy 只有反向代理功能，而且是配置好的。
modcache 没有在lighttpd的accesslog中增加什么内容，因此没有log记录是否cache hit或miss的。
modcache输出的http 头中，缓存命中时有X-Cache: Hit头；没有X-Cache头，则说明缓存未命中
mod_cache.c: In function `save_chunkqueue':
mod_cache.c:1102: error: `UIO_MAXIOV' undeclared (first use in this function)
呈现以上出错信息后，按照你的提示对mod_cache.c添加
#ifndef UIO_MAXIOV
#define UIO_MAXIOV IOV_MAX
#endif
但是仍出现以下错误：
make[3] *** [mod_cache.la] Error 1
make[3] Leaving directory 'cygdrive/d/lighttpd-1.4.22/src'
make[2] *** [all] Error 2
make[2] Leaving directory 'cygdrive/d/lighttpd-1.4.22/src'
make[1] *** [all-recursive] Error 1
make[1] Leaving directory 'cygdrive/d/lighttpd-1.4.22'
make:*** [all] Error 2
请问是什么原因呢？我是在cygwin下根据lighttpd INSTALL来安装lighttpd的
thanks！
简单办法，自行给UIO_MAXIOV 定义一个值，比如
#define UIO_MAXIOV 4
UIO_MAXIOV 的含义是？？？ thanks！
还是出现：
make[3] *** [mod_cache.la] Error 1
make[3] Leaving directory 'cygdrive/d/lighttpd-1.4.22/src'
make[2] *** [all] Error 2
make[2] Leaving directory 'cygdrive/d/lighttpd-1.4.22/src'
make[1] *** [all-recursive] Error 1
make[1] Leaving directory 'cygdrive/d/lighttpd-1.4.22'
make:*** [all] Error 2
为什么呢？
当前安装的是lighttpd1.4.22，modcache1.8.0
估计modcache不支持cygwin，建议换成实机Linux或者在vmware里Linux测试
暂时看来是不支持cygwin，在Linux下安装可以通过。
多谢！
请教一下：
mod_cache模块中的使用memory存储内容和mod_mem_cache有什么联系和区别？？？ 谢谢
mod_cache 是把其他服务器的内容缓存到本地文件和内存。
mod_mem_cache只是缓存本地文件到内存。
当mod_cache不支持缓存内容到内存时，可以用mod_mem_cache把mod_cache缓存的文件加载到内存中
请教，类似下面这样的统计值如何查看？谢谢！
status_counter_set(srv, CONST_STR_LEN(CACHE_HIT_PERCENT), ((float)reqhit/(float)reqcount)*100);
还有一个问题，想在试验环境下对lighttpd的cache性能做一个数据统计分析，有没有什么好的工具和方法，想听听你的意见。多谢！！
加载 mod_status，设置 status.statistics-url = "/lighttpd_status"。然后访问http://ip/lighttpd_status 可看到modcache 的缓存命中率
mod_mem_cache中根据对象当前被点击否进行lru排队，在空间不够的情况下清除部分对象，但是check_probation_lru(srv, p, hash)中好像又用到对象被点击的历史记录，能否解释一下这个矛盾点？？？谢谢
check_probation_lru调用的目的是防止只有一次访问的文件也被加载到内存里。也就是说需要访问过几次的文件才会被加载到内存中，这样可以提高内存缓存的效率