Write a Cloud computing summary

Write a brief summary of the important concepts and your thoughts use two files below

Chapter 7. Cloud Applications
Contents
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
Cloud application development.
Architectural styles for cloud applications.
Coordination of multiple activities.
Workflow patterns.
Coordination based on a state machine model.
MapReduce programming model.
A case study: the GrepTheWeb application.
Hadoop, Yarn, and Tez
SQL on Hadoop: Pig, Hive, and Impala
Clouds for science and engineering.
High performance computing on a cloud.
Legacy applications on a cloud.
Social computing, digital content, and cloud computing.
Dan C. Marinescu
Cloud Computing Second Edition – Chapter 7
2
7.1 Cloud application development
◼
◼
Cloud computing changed the views of users and providers of service
on how to compute more efficiently and at a lower cost.
Most of the challenges faced by application developers and service
providers either disappeared, or are significantly diminished.
 Cloud application developers enjoy the advantages of a just-in-time
infrastructure, they are free to design an application without being
concerned where the application will run.
 Cloud elasticity allows applications to seamlessly absorb additional
workload. Cloud users also benefit from the speedup due to
parallelization.
 Cloud computing is beneficial for the providers of computing cycles as it
typically leads to a more efficient resource utilization.
 A significant percentage of the typical workloads are dominated by
frameworks such as MapReduce and that multiple frameworks must share
the large computer clusters populating the cloud infrastructure.
Dan C. Marinescu
Cloud Computing Second Edition – Chapter 7
3
Remaining challenges for cloud applications
◼
◼
◼
◼
Performance isolation – nearly impossible to reach in a real system,
especially when the system is heavily loaded.
Reliability – major concern; server failures expected when a large
number of servers cooperate for the computations.
Cloud infrastructure exhibits latency and bandwidth fluctuations
which affect the application performance.
Performance considerations limit the amount of data logging; the
ability to identify the source of unexpected results and errors is
helped by frequent logging.
Dan C. Marinescu
Cloud Computing Second Edition – Chapter 7
4
Suitable and less suitable cloud applications
◼
Ideal applications for cloud computing:
Web services.
◼ Database services.
◼ Transaction-based service. The resource requirements of
transaction-oriented services benefit from an elastic environment
where resources are available when needed and where one pays
only for the resources it consumes.
◼
◼
Applications unlikely to perform well on a cloud:
Applications with a complex workflow and multiple dependencies,
as is often the case in high-performance computing.
◼ Applications which require intensive communication among
concurrent instances.
◼ When the workload cannot be arbitrarily partitioned.
◼
Dan C. Marinescu
Cloud Computing Second Edition – Chapter 7
5
Existing and new cloud application opportunities
◼
Three broad categories of existing applications:
Processing pipelines.
◼ Batch processing systems.
◼ Web applications.
◼
◼
Potentially new applications
Batch processing for decision support systems and business
analytics.
◼ Mobile interactive applications which process large volumes of
data from different types of sensors.
◼ Science and engineering could greatly benefit from cloud
computing as many applications in these areas are computeintensive and data-intensive.
◼
Dan C. Marinescu
Cloud Computing Second Edition – Chapter 7
6
Processing pipelines
◼
◼
◼
Indexing large datasets created by web crawler engines.
Data mining – searching large collections of records to locate items
of interests.
Image processing .
Image conversion, e.g., enlarge an image or create thumbnails.
◼ Compress or encrypt images.
◼
◼
◼
Video transcoding from one video format to another, e.g., from AVI
to MPEG.
Document processing.
Convert large collections of documents from one format to
another, e.g., from Word to PDF.
◼ Encrypt documents.
◼ Use Optical Character Recognition to produce digital images of
documents.
◼
Dan C. Marinescu
Cloud Computing Second Edition – Chapter 7
7
Batch processing applications
◼
◼
◼
◼
◼
Generation of daily, weekly, monthly, and annual activity reports
for retail, manufacturing, other economical sectors.
Processing, aggregation, and summaries of daily transactions
for financial institutions, insurance companies, and healthcare
organizations.
Processing billing and payroll records.
Management of the software development, e.g., nightly updates
of software repositories.
Automatic testing and verification of software and hardware
systems.
Dan C. Marinescu
Cloud Computing Second Edition – Chapter 7
8
Web access
◼
◼
Sites for online commerce.
Sites with a periodic or temporary presence.
Conferences or other events.
◼ Active during a particular season (e.g., the Holidays Season) or
income tax reporting.
◼
◼
◼
Sites for promotional activities.
Sites that “sleep” during the night and auto-scale during the
day.
Dan C. Marinescu
Cloud Computing Second Edition – Chapter 7
9
7.2 Architectural styles for cloud applications
◼
◼
◼
◼
◼
Based on the client-server paradigm.
Stateless servers – view a client request as an independent
transaction and respond to it; the client is not required to first
establish a connection to the server.
Often clients and servers communicate using Remote Procedure
Calls (RPCs).
Simple Object Access Protocol (SOAP) – application protocol for
web applications; message format based on the XML. Uses TCP
or UDP transport protocols.
Representational State Transfer (REST) – software architecture
for distributed hypermedia systems. Supports client
communication with stateless servers, it is platform independent,
language independent, supports data caching, and can be used in
the presence of firewalls.
Dan C. Marinescu
Cloud Computing Second Edition – Chapter 7
10
7.3 Coordination of multiple activities
◼
◼
◼
Many applications require completion of multiple interdependent tasks.
Workflow → description of a complex activity involving a set of tasks.
Task → unit of work to be performed on the cloud and it is
characterized by several attributes:
 Name – a string of characters uniquely identifying the task.
 Description – a natural language description of the task.
 Actions – an action is a modification of the environment caused by the
execution of the task.
 Preconditions – boolean expressions that must be true before the action(s)
of the task can take place.
 Postconditions – boolean expressions that must be true after the action(s)
of the task do take place.
 Attributes – provide indications of the type and quantity of resources
necessary for the execution of the task, the actors in charge of the tasks,
the security requirements, whether the task is reversible or not, and other
task characteristics.
 Exceptions – provide information on how to handle abnormal events.
Dan C. Marinescu
Cloud Computing Second Edition – Chapter 7
11
Workflow tasks
◼
◼
◼
Composite task → a structure describing a subset of tasks and the
order of their execution.
Primitive task → is one that cannot be decomposed into simpler
tasks.
Routing task → a special-purpose task connecting two tasks in a
workflow description. Could trigger the sequential, concurrent, or
iterative execution. Several types of routing tasks exist:
 Fork routing task → triggers execution of several successor tasks.
 Join routing task → waits for completion of its predecessor tasks.
Dan C. Marinescu
Cloud Computing Second Edition – Chapter 7
12
7.4 Workflows and workflow patterns
◼
Basic Concepts
 Process description – structure describing the tasks to be executed
and the order of their execution. Resembles a flowchart.
 Case – an instance of a process description.
 State of a case at time t – defined in terms of tasks already completed
at that time.
 Events – cause transitions between states.
 The life cycle of a workflow – creation, definition, verification, and
enactment; similar to the life cycle of a traditional program (creation,
compilation, and execution).
◼
Similarity of workflows and programs → both describe activities
consisting of a set of steps.
Process description → program.
 Case → process.
 Case state → process state.

Dan C. Marinescu
Cloud Computing Second Edition – Chapter 7
13
Dynamic Workflows
Component
Database
Static Workflows
Workflow
Description
Language
User
Static Programs
Dynamic Programs
Programming
Language
Component
Libraries
User
Automatic
Programming
Planning
Engine
Workflow
Database
Workflow
Description
Computer
Program
Verification
Engine
Compiler
Workflow
Description
Object
Code
Program
Libraries
Case Activation Record
Unanticipated Exception
Handling
(a) Workflow
Dan C. Marinescu
Enactment
Engine
Data
Processor
Running
the Process
Run-Time Program
Modification Requests
(b) Program
Cloud Computing Second Edition – Chapter 7
14
Desirable properties of workflows.
◼
◼
Safety → nothing “bad” ever happens.
Liveness → something “good” will eventually happen.
Dan C. Marinescu
Cloud Computing Second Edition – Chapter 7
15
Dan C. Marinescu
Cloud Computing Second Edition – Chapter 7
16
Basic workflow patterns
◼
Workflow patterns – the temporal relationship among the tasks of a
process
a.
Sequence – several tasks have to be scheduled one after the
completion of the other.
AND split – both tasks B and C are activated when task A terminates.
Synchronization – task C can only start after tasks A and B terminate.
XOR split – after completion of task A, either B or C can be activated.
XOR merge – task C is enabled when either A or B terminate.
OR split – after completion of task A one could activate either B, C, or
both.
b.
c.
d.
e.
f.
Dan C. Marinescu
Cloud Computing Second Edition – Chapter 7
17
More workflow patterns
a.
b.
c.
d.
Multiple Merge – once task A terminates, B and C execute
concurrently; when the first of them, say B, terminates, then D is
activated; then, when C terminates, D is activated again.
Discriminator – wait for a number of incoming branches to
complete before activating the subsequent activity; then wait for
the remaining branches to finish without taking any action until all
of them have terminated. Next, resets itself.
N out of M join – barrier synchronization. Assuming that M tasks
run concurrently, N (N