iWatch网络爬虫：在线数据收集与隐私实践分析

4星 · 超过85%的资源需积分: 10 2 浏览量更新于2024-09-19 3 收藏 386KB PDF 举报

"这篇文档介绍了iWatch网络爬虫，这是一个用于追踪网站数据收集和隐私实践的工具。通过iWatch，可以编目和分析在线数据行为以及与隐私相关的指标和技术。其目的是实现对趋势、法规对实践的影响以及地理和社会差异的新型分析。文中展示了使用该工具分析15个月间隔内的两组数据的初步发现，样本覆盖了超过24,000个域名、47个国家的240,000多页内容。" 网络爬虫是一种自动化程序，它按照预设的目标去抓取互联网上的网页，并从中提取所需信息。聚焦爬虫是爬虫的一种类型，与通用爬虫不同，它不追求广泛覆盖整个网络，而是专注于获取特定主题或领域的网页。这种爬虫通常用于特定目的的数据采集，例如市场研究、学术研究或者特定行业的信息监控。 iWatch网络爬虫则是专门设计来追踪和分析网站的数据收集策略和隐私保护措施。它能够帮助研究人员或观察者了解网站如何处理用户的个人信息，以及网站是否遵循相关的隐私政策和技术。通过iWatch，可以收集大量的网页数据，然后使用分析工具来揭示关于数据实践和隐私保护的模式和趋势。在文中提到的初步研究中，研究人员使用iWatch爬虫收集了来自24,000多个域名的超过240,000个页面，这些数据跨越了47个国家。这些大量数据使得研究人员能够进行深入的跨地区和跨文化的比较，揭示不同国家和地区在数据收集和隐私保护上的差异。此外，他们还能分析时间间隔内这些做法的变化，这可能涉及到法律法规的影响，例如GDPR（欧洲通用数据保护条例）等。通过这样的分析，iWatch爬虫不仅提供了对当前在线数据实践的洞察，还可能揭示了政策变化如何影响网站的行为。这对于政策制定者、企业、隐私倡导者以及任何关心个人数据安全的人来说都是非常有价值的。同时，这也为未来的趋势预测提供了基础，有助于我们更好地理解和应对网络环境中的隐私挑战。网络爬虫，特别是像iWatch这样的专业工具，是理解和研究互联网上数据收集和隐私实践的关键手段。它们使得大数据分析不再局限于简单的网页抓取，而是扩展到了对网络行为和隐私保护的深度剖析，这对于推动互联网安全和隐私保护的进步具有重要意义。

[10] in a system sharing many methodological similarities to our

iWatch.

A number of other tools independent of P3P have also been

developed over the years, including filtering and privacy

protecting proxy servers, popup-blockers, cookie blockers and

analysis tools, anti-phishing tools, etc. Given that many of these

functions have subsequently been absorbed by the latest

generation of web-browsers, their numbers and user base is

unknown today.

Regardless of the underlying technology, HCI researchers have

been examining the issue of how to improve the usability and

usefulness of such systems, an early shortcoming of many.

Classic papers and studies include [37, 38]. This research showed

that a secure system would fail unless these security measures

were made usable. In recent years we have seen excellent papers

on why phishing attacks work [14, 13], and how our tools and

warning tend to go unheeded, regardless of the information

presented [39]. While excellent results, it is obvious more work

still needs to be done in this area as there are far more studies of

why things fail than how to succeed.

Our approach of harvesting and examining large amounts of data

via the use of a web-crawler has been employed by other security

and privacy researchers. Recently, this approach has produced

interesting results in the identification of malware and spyware

disseminating websites [30, 32]. In these studies, researchers were

able to scan and classify a large enough sample to convincingly

argue about the state of the Internet as a whole.

3. Definitions

Before diving into the meat of our study, it is important to define

certain terms in order to avoid misunderstandings or ambiguity.

Our definitions should most often match generally accepted

definitions, but may in some cases have a rather more narrow

definition, chosen for practical considerations.

In this paper, domain, web server, and website are terms which

are used interchangeably. While in the real-world, a given domain

can host many distinct sites, we differentiate between sites based

solely on domain-names. A distinct domain-name in our study

identifies a distinct domain. Our classification of domains was

very simplistic. We did not attempt to identify synonymous

domain names (

www.theregister.co.uk is not recognized as a

synonym for

www.theregister.com), or sub-domains

(

news.bbc.co.uk is not identified as a sub-domain of

www.bbc.co.uk). The first is a hard problem and requires either a

set of records from domain registrars, or a lot of hand-tuning. The

second, though technically simple to implement, would cause

problems with hosting services and smaller or related web-sites,

which may lack unique second-level domain names.

We will also use the terms 1

party and 3

party frequently. In

this context a 1

party typically refers to the domain or website

which served the page, and a 3

party is any other

domain/website which either receives information about the

transaction, or supplies information or resources used by the

requested page. Examples are 3

party cookies, webbugs, and

banner ads.

In this paper we will talk about technologies such as P3P policies,

webbugs, cookies, popups, and banners. P3P stands for the

Platform for Privacy Preferences, and is a standard for specifying

privacy policies in a machine-readable XML format [8]. There are

two types of P3P policies, the compact policy (CP) and the full

policy. The P3P compact policy is a keyword abbreviated P3P

policy, offering less detail and nuance, but often used by browsers

to filter cookies. P3P and P3P policy will be terms that are used

interchangeably in this paper.

The P3P protocol specifies 3 ways of publishing a P3P policy; in

the HTTP header (can either be a compact policy, or a link to a

full policy), in the HTML document as a link tag, or in a well-

known location on the server. Because of some quirks of the way

web servers implement the serving of P3P policies (see discussion

in methodology), our current version of iWatch only finds

policies posted in the HTTP header or the body of the document,

it does not search the known locations. In order to fetch these

remaining policies without bringing the crawler to a halt we

delegate this task to a standalone program.

Privacy Seals are, in this paper, a combination of different

certificates or trustmarks issued by TRUSTe and BBBOnline

(BBBPrivacy and BBBReliability seals). These seals certify that

the site discloses or follows a minimum set of privacy protection

and security practices. While different seals or certificates are

enforced by different agencies, have different meanings, and offer

different enforcement mechanisms and guarantees, they are all

meant to calm potential users concerns. Given the relatively low

usage numbers, the different seal programs are grouped together

for most of our analysis.

Webbugs, also known as web-beacons or pixel tag, are a

collection of techniques aimed to tag and collect information from

web and email users without their knowledge. In a web page,

webbugs are typically used to track users navigating a given site,

and have become quite ubiquitous. Webbugs technically can be

implemented through a number of different techniques, but are

most commonly associated with a 1x1 pixel transparent gif,

invisible to the user. Webbugs are often used to augment the

tracking available with cookies, and are most troubling when set

by third parties, usually without user knowledge or consent. In

iWatch we group a number of tracking techniques under the label

of webbugs, but only when these are set and used by 3

parties.

We do not classify banner ads or 3

party cookies as webbugs,

but rather track these separately.

Much has been written about cookies, and so a discussion of how

they work and their potential threats to user privacy is omitted

here. We will just mention that in this work we do track the three

main categories of cookies separately, session cookies, defined as

cookies set by the first party and expiring with the browsing

session, 1

party cookies, set by the 1

party and set to persist,

and 3

party cookies, which are set for any domain other than the

party.

Unsolicited popups, or just popups for short, refers to the much

hated technique of opening new browser windows, typically for

the purpose of advertising. Affiliated techniques include the pop-

under (popups which try to hide themselves). They present a

potential danger to end-users as they often serve up content for

third parties, enabling these to track users much like webbugs.

Popups have stopped being as big a focus in recent years as

blocking tools and techniques have become ubiquitous and

effective.

Web banners, or banners for short, do not present a privacy risk in

and of themselves, unless served by a third party. In this case,

they serve much the same function as a webbug, though at least

remaining visible to the user. Banners in our study are identified

by their size (these are the standardized sizes set by the Internet

剩余11页未读，继续阅读

ycfps

粉丝: 0
资源: 1

iWatch网络爬虫：在线数据收集与隐私实践分析

Python网络爬虫介绍 从基础入门到进阶知识学习 含真实案例分析和程序源代码 共62页.pdf

网络爬虫简介

UofT课程数据抓取利器：course-crawlers网络爬虫介绍

网络爬虫详细介绍

网络爬虫设计原理介绍

网络爬虫的相关介绍

爬虫技术-网络爬虫概念介绍.pptx

信息检索 第2讲 网络爬虫技术.zip_爬虫_网络爬虫_网络爬虫PPT

网络爬虫：我对网络爬虫的介绍，然后将其演变为自然语言处理

网络爬虫基本原理介绍.md

最新资源

Python网络爬虫介绍从基础入门到进阶知识学习含真实案例分析和程序源代码共62页.pdf

信息检索第2讲网络爬虫技术.zip_爬虫_网络爬虫_网络爬虫PPT