ROS：机架式光存储系统，实现长期数据保存的在线访问

需积分: 5 148 浏览量更新于2024-07-10 收藏 3.74MB PDF 举报

"ROS是一种机架规模的光存储系统，旨在解决大数据时代长期数据保存和高效访问的需求。系统在42U机架内集成大量光盘，实现PB级别的存储容量，并支持在线访问。为了提高存储密度和简化机械结构，ROS采用了可旋转滚轮和分离式机械臂设计。该系统采用分层存储策略，结合固态硬盘、硬盘和光盘的优势，通过固态硬盘和硬盘缓存来掩盖机械操作的延迟。同时，设计了一种名为OLFS的光学库文件系统，该系统基于FUSE并进行了优化，以提供POSIX接口，使用户能透明地操作分层存储，减少了不必要的用户/内核上下文切换，提高了性能。通过评估，ROS在10GbE网络环境下，能接近饱和地提供Samba的吞吐量，文件写入延迟约为53ms，读取延迟约为15ms，展示了良好的内联可访问性。此外，ROS还能有效地隐藏和虚拟化内部复杂操作，便于在数据中心部署。" ROS系统是针对数字数据快速增长和长期保存需求而设计的新型存储解决方案。传统的HDD阵列和磁带库在成本和访问性方面存在局限，而ROS通过优化光盘存储，克服了这些局限。在42U的标准机架中，ROS集成了数千张光盘，提供了PB级的存储容量，这远超单个光盘的存储能力。同时，ROS的创新之处在于其机械设计，使用可旋转滚轮和独立的机械臂，提高了光盘的存储密度，降低了复杂性。为了实现高效的数据访问，ROS采用了一个分层存储架构，其中固态硬盘（SSD）和硬盘（HDD）作为高速缓存，用于处理频繁访问的数据，而光盘则用于长期存储。这种分层策略有助于隐藏机械操作的延迟，使得用户可以近乎实时地访问数据。OLFS（光学库文件系统）作为ROS的核心组件，不仅负责调度机械操作，还提供了符合POSIX标准的用户接口，让用户能够在不知道底层存储细节的情况下，如同操作常规文件系统一样使用ROS。OLFS通过优化FUSE框架，减少了上下文切换，提升了系统性能。性能评估显示，ROS在NAS模式下，通过Samba和FUSE堆叠，能充分利用10GbE网络的带宽，为外部用户提供接近饱和的吞吐量。在这样的设置下，ROS的写入和读取延迟分别保持在53毫秒和15毫秒左右，这证明了其优秀的内联可访问性。ROS还能够有效地将内部的复杂操作隐藏和虚拟化，简化了数据中心的部署和管理。 ROS是一种创新的光存储系统，通过结合光盘的长期保存优势、分层存储的效率以及优化的文件系统，为大数据时代的长期数据保存和在线访问提供了一个经济且高效的解决方案。

ROS: A Rack-based Optical Storage System with Inline Accessibility 28:5

target discs. Specically, the test optical discs are placed in a room with temperature varying from

−40

◦

Cto60

◦

C, in a chamber with highly corrosive gas or in seawater. The test results manifested

that the archive-level optical discs can reliably store data in very tough environment and have an

average 50-year rated lifetime under normal environment. Besides the long life cycle, optical discs

have good compatibility. The rst generation compact discs (CDs) manufactured 30 years ago is

still understandable using the current generation of optical drives. The Blu-ray disc technology

inherits the 120mm physical feature from CDs with higher storage density. Additionally, optical

discs are resilient against disasters [40], such as ood and electromagnetic pulse. The optical discs

were the only digital media that survived the hurricane Katrina [42]. Considering that optical

discs are simply made up of a piece of plastics plus multiple thin layers of coating lms at several

ums, the material and manufacturing costs of optical discs are relatively low. Once the amount of

produced discs exceeds a threshold where the investment of production lines for optical discs can

be suciently amortized, the cost per GB for discs will surely be competitive with hard disks and

tapes. Current media cost per GB of 25GB discs has become close to that of tapes. Hologram discs

[14] with 2TB capacity have been realized and demonstrated, although their drives are planning to

be productized in two years. In the foreseeable future, 5D optical discs are poised to oer hundreds

of TB capacity [20, 34].

Assuming the use of tapes, hard disks, SSDs, and Blu-ray discs to build a datacenter with a

capacity of 1PB for 100 years, Preeti Gupta and her group construct an analytical model to cal-

culate the TCO of such a system [21]. In her experimental setup, the lifetime of optical discs are

15 years, while the lifetime of HDDs are only 5 years, so the HDD-based datacenter needs more

data migration cost and media repurchase cost than the optical-based datacenter. As mentioned

above, the preservation of tapes requires strict environment and regular rewinding operations,

thus the tape-based datacenter needs more operational cost than the optical-disc-based datacen-

ter. The simulation result shows that the TCO of an optical-disc-based datacenter is $250K/PB,

about 1/3 of an HDD-based datacenter, 1/2 of a tape-based datacenter. If the lifecycle of the optical

disc extend to 50 years in the model, then the cost of storage media replacement can be further

reduced. Consequently, industry and academia are beginning to consider store long-term data on

optical discs as an alternative approach.

Despite these potential advantages, the current capacity and performance of Blu-ray discs are far

lower than those of hard disks. Even though 300GB Blu-ray discs are now becoming increasingly

popular, their capacity are still an order of magnitude smaller than the 8TB hard disks. Additionally,

the basic reference speed of a Blu-ray disc is at most 4.49MB/s, which is dened as 1×. Current

standard reference speeds for 25GB and 100GB discs are, respectively, 6X and 4X [27]. The maximal

speed obtained in our experiments are 12× for 25GB discs and 6× for 100GB discs, which are far

lower than the speed of hard disks of almost 150MB/s.

Most recordable optical discs are write-once-read-multiple (WORM) while re-writable (RW)

discs can re-write with relatively low burning speed (2×) and limited erase cycle (at most 1,000).

Optical drives at high voltage burn a series of physical grooves on the unused smooth surface of

discs. It is best to steadily and sequentially burn data once into discs to ensure high quality and

low risk of errors. An entire prepared disc image is burned into a disc once, referred to as the

write-all-once mode. Optical drives also support the Pseudo Over-Write mechanism where the

drive can write multiple data tracks into a disc, with each track representing an independent disc

image. An optical drive rst takes tens of seconds to format a predened metadata area and writes

data. When over-write happens, the previously burned area cannot be used and the drive needs

to format a new metadata area and then writes data. This mechanism causes capacity loss and

performance degradation and thus is not recommended.

ACM Transactions on Storage, Vol. 14, No. 3, Article 28. Publication date: November 2018.

28:6 W. Yan et al.

2.4 Optical Library

To overcome the limitations of individual optical discs, optical libraries have been proposed to

increase system-level capacity by leveraging the combined capacity of a large number of discs

and improve performance by deploying multiple optical drives to access discs concurrently. The

rst generation optical jukebox only contained up to hundreds of discs in relevant physical slots

and accessed by 1–6 drives. Their mechanical appliances can deliver discs between drives and

slots. Current large-scale optical libraries, such as Panasonic LB-DH8 [10], Facebook Blu-ray cold

storage system [25], and Sony Everspan [38], further deploy thousands of discs and tens of drives.

These discs are grouped into a series of disc arrays as a logical block volume with an internal RAID

redundant mechanism. The disc arrays can solely mount/unmount to a host via the SCSI interface.

LB-DH8s are 6U racked-devices. Everspan has a controller rack unit consisting of drives and up to

13 additional expansion rack units with discs.

However, these optical libraries are more like tape devices in logical rather than storage nodes

deployed in current cloud datacenters, making them dicult to be integrated into datacenters

seamlessly. Current tape and optical libraries generally rely on a dedicated backup system running

on a front host to manage all data on media in an o-line mode. The typical backup process can

involve datasets collection, catalog creation or update, compression or deduplication, transforming

datasets into a specic format suitable for tapes, then copying these formatted data into tapes.

The restore process essentially reverses the above procedure. Besides, raw data on tapes are not

readable for applications without interpretation of the backup system. To address this problem,

IBM developed LTFS [12] that allows its les on tapes to be directly accessed by user applications

through a standard POSIX. However, LTFS is only based on single tape like UDF for discs.

2.5 Motivation

Optical discs have exhibited their advantages in long-term data preservation. However, their dis-

advantage in capacity and performance will persist for some time to come. An optical library is

expected to maximize disc placement density within a limited physical space but provide a reli-

able and simple mechanical structure. Furthermore, an optical library is also designed to eectively

manage discs and their storing of data, as well as to hide disc boundary, internal specic access

behaviors and mechanical delays.

Additionally, for long-term data preservation, considering that hardware, software, and me-

chanical components are not likely to have the same lifetime as discs and parts of discs might be

lost, data on survived discs should be, by and large, self-descriptive, independently accessible, and

understandable.

More importantly, in the era of big data, long-term preserved data are required to be conve-

niently and directly searched and retrieved through programmable interfaces by big-data analyt-

ics applications without the aforementioned extra backup/archival system intervention. Therefore,

optical libraries should provide a persistent online view of their data so that the data can be shared

by external clients using standard storage interfaces that can be easily integrated and scaled in

cloud datacenters. Furthermore, as storage nodes in current datacenters, optical libraries should

have in-place computing capacity to process their data locally.

In summary, optical libraries as storage nodes used in cloud datacenters should be reliable, self-

manageable, scalable, low cost, easy to use, and capable of local processing of data. To this end, we

design and implement a Rack-based Optical library System (ROS) with a sophisticated, eective

mechanical-electrical integrated system with a full visualized le view and tiered storage consoli-

dating SSDs, hard disks, and discs. More importantly, ROS is designed as a rack-level storage node

not only for cloud data centers but also for enterprise data centers to preserve and process data

ACM Transactions on Storage, Vol. 14, No. 3, Article 28. Publication date: November 2018.

剩余25页未读，继续阅读

weixin_38693506

粉丝: 5
资源: 966

ROS：机架式光存储系统，实现长期数据保存的在线访问

基于ros noetic的ros包录制功能包

visp_ros:一篮子基于ViSP库的通用ros节点

ros：基于Python的机器人操作系统（ROS）

ROS:所有ROS代码的UDMRT存储库

darknet_ros：YOLO ROS：ROS的实时对象检测

coordination_oru_ros:基于corrosion_oru并具有ROS支持的多机器人协调程序包

ROS:基于旋转的开放集

royale-ros:基于Royale SDK的基于Pmd的ToF相机的ROS软件包

KORAL-ROS:ROS管道，用于基于GPU的特征检测，描述和匹配

docker-make-ros:基于ROS的项目中Docker + Make工作流的示例存储库

最新资源