squid过滤特殊网站
Squid中文权威指南
译者序: 本人在工作中维护着数台Squid服务器,多次参阅Duane Wessels(他也是Squid的创始人)的这本书,原书名是"Squid: The Definitive Guide",由O'Reilly出版。我在业余时间把它翻译成中文,希望对中文Squid用户有所帮助。对普通的单位上网用户,Squid可充当代理服务器;而对Sina,NetEase这样的大型站点,Squid又充当WEB加速器。这两个角色它都扮演得异常优秀。窗外繁星点点,开源的世界亦如这星空般美丽,而Squid是其中耀眼的一颗星。 目录 第1章介绍....................................................................................................................................2 1.1 Web缓存.............................................................................................................................2 1.2 Squid的简明历史...............................................................................................................2 1.3 硬件和操作系统要求........................................................................................................2 1.4 squid是开源的....................................................................................................................3 1.5 Squid的Web主页................................................................................................................3 1.6 获取帮助............................................................................................................................3 1.6.1 FAQ..........................................................................................................................3 1.6.2 邮件列表.................................................................................................................3 1.6.3 职业支持.................................................................................................................4 1.7 启动Squid..........................................................................................................................4 第2章获取Squid..........................................................................................................................2 2.1版本和发布.........................................................................................................................2 2.2使用源代码.........................................................................................................................2 2.3预编译的二进制文件.........................................................................................................3 2.4 匿名CVS............................................................................................................................3 2.5 devel.squid-cache.org..........................................................................................................4 第3章编译和安装.........................................................................................................................2 3.1安装之前.............................................................................................................................2 3.2解开源代码包.....................................................................................................................2 3.3调整内核.............................................................................................................................2 3.3.1文件描述符..............................................................................................................3 3.3.2Mbuf Clusters............................................................................................................4 3.3.3临时端口范围..........................................................................................................5 3.4 Configure脚本....................................................................................................................5 3.4.1 configure选项..........................................................................................................6 3.4.2运行configure........................................................................................................12 3.5编译..................................................................................................................................12 3.6安装..................................................................................................................................14 3.7打补丁...............................................................................................................................16 3.8重运行configure...............................................................................................................17 第4章快速配置向导.....................................................................................................................2 4.1 squid.conf语法....................................................................................................................2 4.2 User ID...............................................................................................................................3 4.3端口号................................................................................................................................4 4.4日志文件路径.....................................................................................................................4 4.5访问控制.............................................................................................................................5 4.6可见主机名.........................................................................................................................5 4.7 管理联系信息....................................................................................................................6 4.8 下一步...............................................................................................................................6 第5章运行Squid..........................................................................................................................2 5.1 squid命令行选项................................................................................................................2 5.2 对配置文件查错................................................................................................................3 5.3 初始化cache目录...............................................................................................................4 5.4 在终端窗口里测试squid...................................................................................................4 5.5 将squid作为服务进程运行...............................................................................................5 5.5.1 squid_start脚本........................................................................................................6 5.6 启动脚本............................................................................................................................6 5.6.1 /etc/rc.local...............................................................................................................6 5.6.2 init.d和rc.d...............................................................................................................6 5.6.3 /etc/inittab.................................................................................................................7 5.7 chroot环境..........................................................................................................................7 5.8 停止squid..........................................................................................................................8 5.9 重配置运行中的squid进程...............................................................................................9 5.10 滚动日志文件..................................................................................................................9 控制元素............................................................................................................2 6.1.1 一些基本的ACL类型.....................................................................................2 6.1.2 ACL类型..........................................................................................................6 6.1.3 外部ACL.......................................................................................................18 6.1.4 处理长ACL列表...........................................................................................19 6.1.5 Squid如何匹配访问控制元素.......................................................................20 6.2 访问控制规则..........................................................................................................21 6.2.1 访问规则语法...............................................................................................23 6.2.2 Squid如何匹配访问规则...............................................................................23 6.2.3 访问列表风格...............................................................................................24 6.2.4 延时检查.......................................................................................................25 6.2.5 减缓和加速规则检查...................................................................................26 6.3 常见用法..................................................................................................................26 6.3.1 仅仅允许本地客户.......................................................................................27 6.3.2 阻止恶意客户...............................................................................................27 6.3.3 内容过滤.......................................................................................................27 6.3.4 在工作时间的受限使用...............................................................................28 6.3.5 阻止squid与非HTTP服务器会话................................................................28 6.3.6 授予某些用户特殊的访问...........................................................................29 6.3.7 阻止邻近cache的滥用..................................................................................30 6.3.8 使用IP地址拒绝请求...................................................................................31 6.3.9 http_reply_access示例...................................................................................31 6.3.10 阻止对本地站点的cache命中....................................................................31 6.4 测试访问控制..........................................................................................................32 盘缓存基础.............................................................................................................................2 7.1 cache_dir指令.....................................................................................................................2 7.1.1 参数:Scheme........................................................................................................2 7.1.2 参数:Directory.....................................................................................................2 7.1.3 参数:Size..............................................................................................................3 7.1.4 参数:L1和L2.......................................................................................................4 7.1.5 参数:Options........................................................................................................5 7.2 磁盘空间基准....................................................................................................................6 7.3 对象大小限制....................................................................................................................6 7.4 分配对象到缓存目录........................................................................................................7 7.5 置换策略............................................................................................................................7 7.6 删除缓存对象....................................................................................................................8 7.6.1 删除个别对象.........................................................................................................8 7.6.2 删除一组对象.........................................................................................................9 7.6.3 删除所有对象.......................................................................................................10 7.7 refresh_pattern..................................................................................................................10 ...........................................................................................................5 8.4.2 aufs发行...................................................................................................................5 8.4.3 监视aufs操作..........................................................................................................6 8.5 diskd存储机制....................................................................................................................6 8.5.1 diskd如何工作.........................................................................................................7 8.5.2 编译和配置diskd....................................................................................................7 8.5.3 监视diskd..............................................................................................................10 8.6 coss存储机制....................................................................................................................10 8.6.1 coss如何工作.........................................................................................................11 8.6.2 编译和配置coss....................................................................................................11 8.6.3 coss发行.................................................................................................................12 8.7 null存储机制....................................................................................................................13 8.8 哪种最适合我?..............................................................................................................13 作?.....................................................................................................................2 9.2为何要(或不要)拦截?.................................................................................................5 9.3 网络设备............................................................................................................................7 9.3.1 内置Squid...............................................................................................................7 9.3.2 四层交换.................................................................................................................7 9.3.3 Cisco策略路由.......................................................................................................14 9.3.4 Web Cache Coordination协议................................................................................16 9.4 操作系统配置..................................................................................................................18 9.4.1 Linux......................................................................................................................18 9.4.2 FreeBSD.................................................................................................................21 9.4.3 OpenBSD...............................................................................................................22 9.4.4 在NetBSD和其他系统上的IPFilter.....................................................................23 9.5 配置Squid........................................................................................................................24 9.5.1 配置WCCPv1.......................................................................................................24 9.6 调试问题..........................................................................................................................25 第10章与其他Squid会话.............................................................................................................2 10.1 某些术语..........................................................................................................................2 10.2 为何要(或不要)使用堆叠?......................................................................................2 10.3 配置Squid与邻居通信....................................................................................................3 10.3.1 cache_peer选项......................................................................................................4 10.3.2 邻居状态...............................................................................................................6 10.3.3 改变关系...............................................................................................................7 10.4 对邻居的请求限制..........................................................................................................7 10.4.1 cache_peer_access..................................................................................................8 10.4.2 cache_peer_domain................................................................................................8 10.4.3 never_direct............................................................................................................9 10.4.4 always_direct.........................................................................................................9 10.4.5 hierarchy_stoplist.................................................................................................10 10.4.6 nonhierarchical_direct..........................................................................................10 10.4.7 prefer_direct.........................................................................................................10 10.5 网络度量数据库(netdb)...........................................................................................11 10.6 Internet Cache协议(ICP)...........................................................................................13 10.6.1 成为ICP服务器...................................................................................................13 10.6.2 成为ICP客户......................................................................................................16 10.6.3 广播ICP..............................................................................................................18 10.7 Cache摘要(Cache Digest)...............................................................................................20 10.7.1 配置squid的cache摘要.......................................................................................20 10.8 超文本cache协议(HTCP)........................................................................................21 10.8.1 配置Squid使用HTCP.........................................................................................22 10.9 Cache数组路由协议(CARP)..........................................................................................23 10.9.1 配置Squid使用CARP.........................................................................................24 10.10 归纳所有......................................................................................................................25 10.10.1 步骤1:直接决定选项....................................................................................25 10.10.2 步骤2:邻居选择协议....................................................................................25 10.10.3 步骤2a:ICP/HTCP应答处理...........................................................................26 10.10.4 步骤3:从父cache选择...................................................................................27 10.10.5 重试...................................................................................................................28 10.11 该怎么做?..................................................................................................................28 10.11.1 通过另外的代理发送所有请求?...................................................................28 10.11.2 通过另外的代理发送所有请求,除非它down了?......................................28 10.11.3 确认squid对某些请求,不使用邻居cache吗?.............................................29 10.11.4 通过父cache发送某些请求来绕过本地过滤器?..........................................29 向器...........................................................................................................................2 11.1 重定向器接口..................................................................................................................2 11.1.1 处理包含空格的URI............................................................................................3 11.1.2 产生HTTP重定向消息.........................................................................................4 11.2 重定向器示例..................................................................................................................4 11.3 重定向器池......................................................................................................................7 11.4 配置Squid........................................................................................................................8 11.4.1 redirect_program....................................................................................................8 11.4.2 redirect_children.....................................................................................................8 11.4.3 redirect_rewrites_host_header................................................................................9 11.4.4 redirector_access....................................................................................................9 11.4.5 redirector_bypass.................................................................................................10 11.5 流行的重定向器............................................................................................................10 11.5.1 Squirm..................................................................................................................10 11.5.2 Jesred....................................................................................................................10 11.5.3 squidGuard...........................................................................................................11 11.5.4 AdZapper..............................................................................................................11 HTTP基本验证.................................................................................................................3 12.2.1 NCSA.....................................................................................................................4 12.2.2 LDAP.....................................................................................................................5 12.2.3 MSNT.....................................................................................................................5 12.2.4 Multi-domain-NTLM.............................................................................................6 12.2.5 PAM.......................................................................................................................6 12.2.6 SASL......................................................................................................................7 12.2.7 SMB.......................................................................................................................7 12.2.8 YP...........................................................................................................................7 12.2.9 getpwnam...............................................................................................................8 12.2.10 winbind.................................................................................................................8 12.2.11 基本验证API......................................................................................................8 12.3 HTTP摘要验证.................................................................................................................9 12.3.1 password...............................................................................................................10 12.3.2 摘要验证API......................................................................................................11 12.4 Microsoft NTLM验证.....................................................................................................12 12.4.1 SMB.....................................................................................................................13 12.4.2 winbind.................................................................................................................13 12.4.3 NTLM验证API....................................................................................................13 12.5 外部ACL........................................................................................................................14 12.5.1 ip_user..................................................................................................................15 12.5.2 ldap_group...........................................................................................................15 12.5.3 unix_group...........................................................................................................16 12.5.4 wbinfo_group.......................................................................................................16 12.5.5 winbind_group.....................................................................................................16 12.5.6 编写自己的外部ACL辅助器.............................................................................17 第13章日志文件...........................................................................................................................2 13.1 cache.log...........................................................................................................................2 13.1.1 debug级别..............................................................................................................3 13.1.2 转发cache.log消息到系统日志............................................................................4 13.1.3 dump cache.log消息到终端...................................................................................4 13.2 access.log..........................................................................................................................4 13.2.1 access.log结果编码...............................................................................................9 13.2.2 HTTP响应状态码................................................................................................11 13.2.3 access.log对端编码.............................................................................................13 13.2.4 影响access.log的配置指令................................................................................15 13.2.5 access.log分析工具.............................................................................................19 13.3 store.log...........................................................................................................................19 13.3.1 转换文件号到路径名.........................................................................................23 13.4 referer.log........................................................................................................................24 13.5 useragent.log...................................................................................................................25 13.6 swap.state........................................................................................................................27 13.7 轮转日志........................................................................................................................29 13.8 隐私和安全....................................................................................................................30 第14章监视Squid.........................................................................................................................2 14.1 cache.log告警...................................................................................................................2 14.2 Cache管理器.....................................................................................................................3 14.3 使用SNMP.......................................................................................................................4 Squid 中 第15章服务加速模式...................................................................................................................2 15.1 概述.................................................................................................................................2 15.2 配置Squid........................................................................................................................3 15.2.1 http_port.................................................................................................................3 15.5.2 https_port...............................................................................................................3 15.2.3 httpd_accel_host.....................................................................................................4 15.2.4 httpd_accel_port.....................................................................................................5 15.2.5 httpd_accel_uses_host_header...............................................................................5 15.2.6 httpd_accel_single_host.........................................................................................6 15.2.7 httpd_accel_with_proxy.........................................................................................7 15.3 令人疑惑之处..................................................................................................................7 15.3.1 一个主机,一个主机名.......................................................................................7 15.3.2 一个主机,多个主机名.......................................................................................7 15.3.3 多个主机,一个主机名.......................................................................................8 15.3.4 多个主机,多个主机名.......................................................................................8 15.4 访问控制..........................................................................................................................9 15.5 内容协商........................................................................................................................10 15.6 补充...............................................................................................................................11 15.6.1 日志.....................................................................................................................11 15.6.2 忽略Reload.........................................................................................................11 15.6.3 不可cache的内容................................................................................................12 15.6.4 错误.....................................................................................................................12 15.6.5 刷新目标.............................................................................................................12 15.6.6 邻居.....................................................................................................................12 "Failed to make swap directory"............................................................................2 16.1.2 "Address already in use"........................................................................................2 16.1.3 "Could not determine fully qualified hostname"....................................................2 16.1.4 "DNS name lookup tests failed".............................................................................3 16.1.5 "Illegal character in hostname"..............................................................................3 16.1.6 "Running out of filedescriptors".............................................................................4 16.1.7 "icmpRecv: Connection refused"...........................................................................4 16.1.8 在运行一段时间后,Squid变慢了......................................................................4 16.1.9 调试访问控制.......................................................................................................5 16.2 通过cache.log进行调试...................................................................................................5 16.3 Coredump,断点,和堆栈跟踪....................................................................................11 16.3.1 不能找到core文件?..........................................................................................14 16.4 重现问题........................................................................................................................15 16.5 报告Bug.........................................................................................................................17 译后序...........................................................................................................................................19