SAS技术解构物联网：数据流、边缘计算与预测分析

需积分: 9 133 浏览量更新于2024-07-18 1 收藏 4.41MB PDF 举报

"SAS-the-internet-of-things" 物联网（IoT）是一个复杂且不断发展的领域，涉及从汽车到工厂再到农场的各种设备通过互联网进行数据传输和接收。尽管预测到2020年物联网设备可能达到数十亿，但确切的数量和可能出现的新智能设备类型仍充满不确定性。关键在于理解讨论物联网时的上下文或参考框架。 SAS公司在其技术应用中，针对客户的物联网需求提供了解决方案，包括处理流数据、边缘计算、预测性分析等多个方面。这本书摘自SAS全球用户组会议的论文，展示了SAS如何利用其技术应对物联网带来的挑战。书中提到的具体资源包括Tamara Dull于2017年编写的《The Internet of Things with SAS®: Special Collection》，该书详细阐述了SAS在物联网领域的应用。物联网的核心是设备间的互联和数据交换，SAS的技术强调了对这些实时数据的管理和分析能力。流数据处理允许SAS实时解析和响应大量涌入的数据，这对于实时监控和决策至关重要。边缘计算则将分析能力推送到数据产生的源头，减少了延迟并减轻了云端的处理压力。预测性分析是SAS在物联网领域的一大亮点，它通过机器学习和人工智能算法，不仅能够识别模式，还能预测未来趋势，为企业决策提供依据。例如，在制造业中，预测性维护可以预测设备故障，减少停机时间；在农业中，通过分析传感器数据，可以优化灌溉策略和作物管理。此外，SAS提供的解决方案还涵盖了数据安全和隐私保护，这是物联网领域的重要考虑因素。随着数据量的增加，保护个人信息和商业敏感信息变得愈发重要。SAS的工具可以帮助企业确保数据在收集、传输和分析过程中的安全。书中的论文摘录提供了丰富的实例，展示了SAS如何帮助不同行业的客户应对物联网带来的机遇和挑战。同时，读者可以通过访问SAS官方网站获取更多的用户组会议和全球论坛的论文，以深入了解SAS在物联网领域的最新进展和资源。总结来说，SAS在物联网领域的贡献在于提供了一套全面的工具和服务，包括流数据处理、边缘计算、预测性分析等，旨在帮助企业和组织从物联网数据中提取价值，并确保数据的安全。通过这些技术，SAS助力客户应对物联网时代的复杂性和不确定性，推动业务创新和增长。

• Predictive analytics identifies future likelihoods of events that have not yet happened

• Prescriptive analytics provides the instructions for event actions

SAS Event Stream Processing provides procedural windows to include both descriptive and predictive

algorithms defined in SAS DATA step, SAS DS2, and other languages. As with traditional analysis, these

models are built in SAS

High-Performance Data Mining, SAS

Factory Miner, SAS

Contextual Analysis,

SAS

Forecast Server, and any other SAS

product or solution that generates SAS DATA step or SAS

DS2 code. For this broad selection of algorithms, the models are built upon an event history that has

been stored. As with traditional analysis, models are built, tested, and validated. The resulting model

code, however, is included into the SAS Event Stream Processing continuous query as a pre-defined,

procedural calculation. As part of the continuous query, the model scores individual events are they are

ingested. In other words, the analytics are performed on live data streams, synonymous with the term

‘streaming analytics’.

Taking advantage of machine learning and deep learning techniques, SAS Event Stream Processing

includes a growing suite of methods to build and score event data solely based on streaming data, and

without out-of-stream model development and event history. In this case, algorithms such as K-Means

clustering, are both defined and applied to events in motion, learning with new events. This exciting field

of new techniques further expands the streaming analytics methods available to streaming data.

STREAMING DECISIONS

The focus of this paper is enabling prescriptive analytics in stream, a term we describe as ‘streaming

decisions’. Streaming decisions define the instructions for real-time actions based on live, streaming

events. They are of particular importance to actions taken by object in the IoT. They combine descriptive

and predictive algorithms, with the business rules that trigger when the models are relevant to the current

streaming event data scenarios. In other words, they are the instructions needed by an IoT object to take

the right action, something of core importance to the adoption and successful proliferation of autonomous

IoT activity.

As we distribute analytics further out to the edges of the IoT there is a classification that provides some

guiding principles that direct when one type of analysis, and the corresponding actions, is more applicable

than another. The following types of analytics are described on the IoTHub (2016):

• Edge Analytics is the analysis at the same device from which it is streaming

• In Stream Analytics is the analysis that occurs as data streams from one device to another, or from

multiple sensors to an aggregation point

• At Rest Analytics is the analysis processed after the event fact has passed, based on saved

historical event data and/or other stored information.

In general, the closer to the edge, the less event data there is to analyze. At the edge, there is just that

one object/sensor/device, with its limited supply of data. As mentioned, data quality issues are present at

the edge, and events can be aggregated in windows of time to correct and filter out the irrelevant noise

from the signal of interest. Analytical calculations are more limited due to the data restrictions, and

prescriptive analytics (say, instructions emanating from another object) are limited to real-time actions that

can be performed in isolation – like commands to turn up or down, turn on or off.

As more objects are related to each other in-stream, at aggregation points, the data are richer (emanating

from several sources) and correspondingly there are more data quality issues. The contextual

understanding of the scenario is also richer (with more event data over time, space, and so on) and, in

turn, more complex patterns of interest can be identified. Streaming decisions can thus be made that

relate to more objects, even becoming a series of inter-connected actions.

As typical of any analysis, the decision to apply different types of models is dependent on the data as well

as the business problem the analysis solves. More often than not, IoT analytic solutions require

multiphase analytics, that is models defined in the traditional, stored data paradigm, and scoring for new

analytical insight, as well as in-stream model derivation/calculation, and analytics applied to the edge.

SAS

does this. With the same integrated code, and with over one-hundred and fifty (at last count)

adapters and connectors linking streaming data, SAS Event Stream Processing is used to define the

complete continuous query that can be as simple or complex as the business problem itself. Moreover,

built into SAS Event Stream processing is the ability to automatically issue alerts and notifications for real-

time situational awareness and understanding of event status.

When we consider the IoT we are describing, an analytically driven network of objects that communicate

with each other. When we automate actions between objects, especially when there is no human

intervention, the risks associated with rogue actions, as well as the technical debt that accumulates from

both machine learning algorithms (Sculley et al., 2015) and from any unmanaged advanced analytics

environments, will outweigh the advantages. As such, the IoT demands a governed, reliable and secure

environment for streaming analytics and associated prescribed analytic actions. SAS® Decision Manager

is a prescriptive analytics solution. With fully traceable workflows, versioning and audit trails to assure

command control over streaming analytics for real-time, reliable, and accurate IoT applications.

BUILDING STREAMING DECISIONS WITH SAS®

Decision Construction

When designing, building, and testing decisions it’s best to begin with a description of what is included

within a decision. For the discussion within this paper, we consider the types of decision-making that

organizations use, that is strategic and tactical decisions, and the ways that organizations leverage

analytical models and their output to make decisions that meet businesses goals.

Strategic and Tactical Decisions

Organizations are required to address decisions at both the strategic and the tactical levels since both are

required for a business to run effectively. Strategic decisions typically represent the less common

decisions that an organization makes such as creating new product lines or expanding into new territories

or merging with another firm. These decisions, while important, are not typically made on a frequent basis

and therefore businesses can take the time and effort needed to create specific processes that aren’t

required to be repeatable nor require automation.

Tactical decisions, on the other hand, and which can include operational decisions, are made frequently

and often (often thousands of decisions), made in a single day or even in minutes or seconds. Loan

underwriting, fleet maintenance operations, point of sale operations, fraud detection, and remediation are

examples of the decisions that process high volumes of rapidly moving information. Tactical decisions like

these are numerous, require short timeframes, high rates of data ingestion, as well as automation and

analytics, and of course ways to prescribe an appropriate action based on the analytic model output.

Analytics, an important element for tactical decision making, has become pervasive within organizations

due to a couple of factors. First, the accessibility of analytics has increased due to the rise of tools that

assist and guide users through the analytical process to suggest relevant algorithms based on the

available data. This data-driven, guided approach includes better visualizations to identify patterns in the

data and recommendations, as to which is the best model to use. Second, analytics is being applied to a

wider set of problems, ranging across industries from retail to manufacturing, to health care and drug

development. Organizations have come to recognize that the application of analytics can help with a vast

array of problems. The emergence of the data scientist and the citizen data scientist represents the wider

number of users that are using the new and powerful analytical tools, applying them in a variety of ways

to solve difficult and complex business problems within a single business.

SCALING TO DATA STREAMS

Prescribing Action from Analytics

Analytics has become pervasive in business, and can result in large volumes of analytical output, which

applied to a business process in order to generate a decision. The output is usually a score, a numerical

representation of the likelihood of an outcome. But how can businesses translate that numerical output

into a decision that is repeatable, automated, and scalable?

Some scoring models deliver probabilities of an event occurring based on analysis of historical trends and

events. This output is derived using, in many cases, IF-THEN-ELSE (conditional) logic applied to models

is part of the probability determination, but that score doesn’t define the action to be taken. For example,

a typical credit score for a customer applying for a loan doesn’t specify if the loan officer (or underwriting

system) should approve or deny the loan, what rate to offer the customer, or what product to cross-

sell/upsell. Comparatively, the actions of a decision would provide the prescriptive instruction suggesting

approve over deny, defining the appropriate rate, or that the customer would likely be interested personal

insurance with that loan. These prescriptive instructions combine the output of the analytical model along

with business rules, suggesting, or defining the action that is needed. Such prescriptive action drives the

numerous tactical and operational decisions for a business, and be scalable to address the high-

frequency data inherent to business operations.

Decisions in Action

Streaming data, as we have seen, represents data from sources such as customers web clicks, call data

records, fleet vehicle GPS, point-of-sale systems, and now more commonly, sensors from corporate

assets, such as machines on manufacturing floor, or sensors in an electric power grid.

Data from these systems has historically been extracted, transformed or cleansed from the source, and

then loaded to data warehouses for storage and later to analyzed. But, as described above, soon, if not

already – organizations will conclude that they can’t afford to store it all, and it certainly can’t afford the

lag times of analyzing it after data has been stored. In business operations, the value of data diminishes

the longer we wait to use it, so we need new ways to analyze it sooner – closer to where it originates, and

that means we need new ways to tap into the value of data streams.

Tapping into data, while it’s still in-motion in data streams, before it’s stored empowers actions to be

applied sooner, before its value diminishes and before missed an opportunity or prevented a threat. The

diminishing value of data can be seen in Figure 1, depicts the relationship between our ability to ingest,

and analyze the data for an action and the value of making that decision sooner rather than later.

Figure 1. Decision Decay and Diminishing Value

DRIVING INTERNET OF THINGS (IOT) ACTIONS

In this nascent hyper-connected world of IoT, data is being generated rapidly and businesses want

effective approaches to leverage their analytical resources to not only analyze and gain insights from the

rising tide of data but also take action from it - to obtain the most, real-time value.

IDC research have found that only 0.5% of the data being generated through the IoT is being analyzed to

derive value (see Figure 2). This means that only 0.5% of the data from “things” was being analyzed at

that time, leaving a rich set of opportunities to understand and take action untapped. And while this

research is from 2012, and more organizations have begun to examine IoT data for deriving business

value, the vast majority of organizations have not yet used IoT data for business operations. This report

also points out the amount of untagged data (often associated with unstructured text data) that would be

more useful if it was tagged and analysis-ready. However, with the aforementioned traditional technique

of storing first and then analyzing, tagging content for use if often prohibitive because of the

consequential large storage costs – even if we assume that all potentially useful unstructured text could

be stored (Gantz and Reinsel, 2012).

Figure 2. IDC: The Untapped Big Data Gap (Gantz and Reinsel, 2012)

With increasing numbers of objects, sensors and devices joining the connected network of the IoT, the

big data gap is growing. Processing data at the necessary scale and speed associated with streaming

data generated from the ‘things’ requires new architectures can analyze and make decisions on the

streaming, in-motion data, filtering out the irrelevant from any downstream activity – including data

storage. As such, there is a growing necessity to move analytics, the associated operational decisions,

and the corresponding actions closer to the data - and in some cases, right into the data streams, near

the point of data generation.

SAS

Event Stream Processing and SAS

Decision Manager together enable organizations to analyze

data while it’s still in motion and also apply prescriptive actions sooner, deriving maximum value from live

events and before streaming data value diminishes. The communication between prescriptive instruction

from SAS Decision Manager and the real-time analytical determination embedded within SAS Event

Stream Processing is achieved using SAS

Micro Analytic Services

. SAS Micro Analytic Services

provides the ability to quickly execute decisions based on the results of in-stream scoring.

SAS Micro Analytic Services are included within the SAS Decision Manager offering.

SAS Decision Manager and SAS Event Stream Processing

SAS Decision Manager is being used to automate tactical decision making by prescribing actions to take

through the design, development, testing, and publishing of decision flows. SAS Decision Manager

supports the Business Analyst, Data Scientist, Data Miner, and Statistician, collaboratively developing

tactical decision actions from building decision flows that combine the analytical models with necessary

business rules that drive operational business processes.

These same decision flows, authored from SAS Decision Manager and published into SAS Micro Analytic

Services, can be can be executed within streaming data by inclusion in the event data flow activity

defined in SAS

Event Stream Processing Studio.

CASE STUDY: PRESCRIPTIVE ACTION FOR TRANSPORTATION

The following real-world case study illustrates the power of streaming analytics calculated close to event

data origination for real-time prescriptive actions. We describe a simple yet powerful approach with SAS

Event Stream Processing to build the streaming model and analyze streaming data, with SAS Decision

Manager acting on the real-time events using SAS Micro Analytics Services.

BUSINESS PROBLEM

A fleet management company wants to minimize the time vehicles are out of service to reduce costs,

minimize lost revenue, and maximize uptime. The trucks have sensors that transmit data that monitoring

location, vibration, rpm, temperature, speed, steering angle, pressures, and so on. The company needs to

proactively prioritize maintenance before a vehicle unexpectedly is out-of-service. The somewhat obvious

and immediate business benefit for analyzing vehicle sensors in the transportation industry is to maximize

assets efficiently.

Streaming Analytics for IoT Actions: Two Aspects Driving One Outcome

The company has collected data on its fleet and processing it offline, identifying problems, and then

generating notices sent to maintenance locations, to try to improve the maintenance scheduling and

overall efficiency of vehicle assets. This approach, however, is typically expensive, given the vehicle is

often inoperable, taken off the road before parts are available for the necessary maintenance, mechanics

can fit the additional work their schedules. This can result in unplanned downtime, overtime servicing

costs, expensive non-scheduled parts delivery, and more.

In this scenario, the company wants to use real-time data they’re collecting from vehicles to identify

issues sooner and increase the time available to address maintenance proactively. In fact, the truck

sensors can be analyzed in an onboard SAS Event Stream Processing engine, reducing the latency from

the time the data is collected to the time the issue is identified and addressed.

On-board processing of streaming data is one solution to better predict the likelihood of a vehicle issue.

An even better solution is to drive a prescriptive action that instructs where to route the vehicle to, notifies

suppliers of necessary parts for on-time delivery, and identifies mechanic schedules aligned to a service

stop that minimizes deleterious impact of the fleet’s transportation of goods to its customers.

Together, the in-stream analysis along with the prescriptive actions enable alerts to be generated in-real

time to all stakeholders, empowering them to take the right action, minimizing costs, maximizing revenue

and drastically reducing unplanned down times for the fleet.

SAS

DECISION SPECIFICATION

Analytics can be found at the core of any quality decision making process. In this case, we want to build a

predictive model that will help us decide when trucks are likely to experience a failure that requires

downtime and maintenance. We need a history of truck sensor readings and a history of scheduled and

unscheduled maintenance events. We can then use the sensor readings to predict the unscheduled

events and use those predictions to make better decisions. To build the history data, we have

accumulated data from a small fleet of trucks equipped with sensors. These trucks were operated for a

period of time in different locations and conditions and their sensor readings stored in on board memory.

The saved readings were later downloaded to a database. In addition, we have data on the maintenance

剩余85页未读，继续阅读

sinat_41581062

粉丝: 21
资源: 174

SAS技术解构物联网：数据流、边缘计算与预测分析

utl-is-the-file-a-sas-v5-transport-cport-or-is-the-tranport-file-corrupted:该文件是sas v5传输，cport还是传输文件已损坏

matlab导入excel代码-Sorting-columns-of-a-table-in-place-in-a-SAS-dataset:将表

超越SAS - Beyond the Basics Book-FINAL（英文版）

The-little-SAS-book(中文版)

sas入门书----the little sas book

The-little-SAS-book(中文版).pdf

CARBOGRES-SAS-

SAS--Programming-Projects:我在CalPoly-SLO完成的SAS项目

SAS-IOM-WORD

machine-learning-with-sas-special-collection

最新资源