class UrlManager(): '''url管理器''' def __init__(self): self.new_urls=set() self.old_urls=set() def add_new_url(self,url): if url is None or len(url)==0: return if url in self.new_urls or url in self.old_urls: return self.add_new_url(url) def add_new_urls(self,urls): if urls is None or len(urls)==0: return for url in urls: self.add_new_url(url) def get_url(self): if self.has_new_url(): url=self.new_urls.pop(); #pop方法从集合里移除1个元素并返回 self.old_urls.add(url) #把这个url标记为已爬取 else: return None; def has_new_url(self): return len(self.new_urls)>0 #列表里为空即爬取完毕 if __name__=="__main__": url_manager=UrlManager() url_manager.add_new_url("url1") url_manager.add_new_url(["url1","url2"]) print(url_manager.new_urls,url_manager.old_urls) print("#"*30) 哪里报错
时间: 2024-04-17 08:25:39 浏览: 148
urlmanager:URL Scheme为基础的NavigationController,让ViewController实现松耦合,不依赖
这段代码中有一个潜在的错误。在方法`add_new_url`中,你调用了`self.add_new_url(url)`,实际上应该调用`self.new_urls.add(url)`来将新的URL添加到`new_urls`集合中。代码应该修改为:
```python
def add_new_url(self, url):
if url is None or len(url) == 0:
return
if url in self.new_urls or url in self.old_urls:
return
self.new_urls.add(url)
```
这样,新的URL才会被正确地添加到`new_urls`集合中。
阅读全文