Managing Data in Motion : Data Integration Best Practice Techniques and Technologies.

Format:

Book

Author/Creator:

Reeve, April.

Series:

The Morgan Kaufmann Series on Business Intelligence Series

Language:

English

Subjects (All):

Data integration (Computer science).

Physical Description:

1 online resource (203 pages)

Edition:

1st ed.

Place of Publication:

San Diego : Elsevier Science & Technology, 2013.

Contents:

Front Cover

Managing Data in Motion

Copyright page

Dedication

Contents

Foreword

Acknowledgements

Biography

Introduction

What this book is about and why it's necessary

What the reader will learn

Who should read this book

Senior Business and Information Technology Managers

Enterprise Data, Application, and Technical Architects

Data-Related Program and Project Managers

Data analysts, data modelers, database practitioners, and data integration programmers

Data management students

How this book is organized

Part 1: Introduction to data integration

Chapter 1: What is data integration?

Chapter 2: The importance of data integration

Chapter 3: Types and complexity of data integration

Chapter 4: The process of data integration development

Part 2: Batch data integration

Chapter 5: Introduction to batch data integration

Chapter 6: Extract, transformation, and load

Chapter 7: Data warehousing

Chapter 8: Data conversion

Chapter 9: Data archiving

Chapter 10: Batch data integration architecture and metadata

Part 3: Real-time data integration

Chapter 11: Introduction to real-time data integration

Chapter 12: Data integration patterns

Chapter 13: Core real-time data integration technologies

Chapter 14: Data integration modeling

Chapter 15: Master data management

Chapter 16: Data warehousing with real-time updates

Chapter 17: Real-time data integration architecture and metadata

Part 4: Big data integration

Chapter 18: Introduction to big data integration

Chapter 19: Cloud architecture and data integration

Chapter 20: Data virtualization

Chapter 21: Big data integration

Chapter 22: Conclusion to managing data in motion

1 Introduction to Data Integration

1 The Importance of Data Integration

The natural complexity of data interfaces.

The rise of purchased vendor packages

Key enablement of big data and virtualization

2 What Is Data Integration?

Data in motion

Integrating into a common format-transforming data

Migrating data from one system to another

Moving data around the organization

Pulling information from unstructured data

Moving process to data

3 Types and Complexity of Data Integration

The differences and similarities in managing data in motion and persistent data

Batch data integration

Real-time data integration

Big data integration

Data virtualization

4 The Process of Data Integration Development

The data integration development life cycle

Inclusion of business knowledge and expertise

2 Batch Data Integration

5 Introduction to Batch Data Integration

What is batch data integration?

Batch data integration life cycle

6 Extract, Transform, and Load

What is ETL?

Profiling

Extract

Staging

Access layers

Transform

Simple mapping

Lookups

Aggregation and normalization

Calculation

Load

7 Data Warehousing

What is data warehousing?

Layers in an enterprise data warehouse architecture

Operational application layer

External data

Data staging areas coming into a data warehouse

Data warehouse data structure

Staging from data warehouse to data mart or business intelligence

Business Intelligence Layer

Types of data to load in a data warehouse

Master data in a data warehouse

Balance and snapshot data in a data warehouse

Transactional data in a data warehouse

Events

Reconciliation

8 Data Conversion

What is data conversion?

Data conversion life cycle

Data conversion analysis

Best practice data loading

Improving source data quality

Mapping to target

Configuration data

Testing and dependencies

Private data

Proving.

Environments

9 Data Archiving

What is data archiving?

Selecting data to archive

Can the archived data be retrieved?

Conforming data structures in the archiving environment

Flexible data structures

10 Batch Data Integration Architecture and Metadata

What is batch data integration architecture?

Profiling tool

Modeling tool

Metadata repository

Data movement

Transformation

Scheduling

3 Real Time Data Integration

11 Introduction to Real-Time Data Integration

Why real-time data integration?

Why two sets of technologies?

12 Data Integration Patterns

Interaction patterns

Loose coupling

Hub and spoke

Synchronous and asynchronous interaction

Request and reply

Publish and subscribe

Two-phase commit

Integrating interaction types

13 Core Real-Time Data Integration Technologies

Confusing terminology

Enterprise service bus (ESB)

Service-oriented architecture (SOA)

Extensible markup language (XML)

Data replication and change data capture

Enterprise application integration (EAI)

Enterprise information integration (EII)

14 Data Integration Modeling

Canonical modeling

Message modeling

15 Master Data Management

Introduction to master data management

Reasons for a master data management solution

Purchased packages and master data

Reference data

Masters and slaves

Master data management functionality

Types of master data management solutions-registry and data hub

16 Data Warehousing with Real-Time Updates

Corporate information factory

Operational data store

Master data moving to the data warehouse

17 Real-Time Data Integration Architecture and Metadata

What is real-time data integration metadata?

Modeling

Metadata repository.

Enterprise service bus-data transformation and orchestration

Technical mediation

Business content

Data movement and middleware

External interaction

4 Big, Cloud, Virtual Data

18 Introduction to Big Data Integration

Data integration and unstructured data

Big data, cloud data, and data virtualization

19 Cloud Architecture and Data Integration

Why is data integration important in the cloud?

Public cloud

Cloud security

Cloud latency

Cloud redundancy

20 Data Virtualization

A technology whose time has come

Business uses of data virtualization

Business intelligence solutions

Integrating different types of data

Quickly add or prototype adding data to a data warehouse

Present physically disparate data together

Leverage various data and models triggering transactions

Data virtualization architecture

Sources and adapters

Mappings and models and views

Transformation and presentation

21 Big Data Integration

What is big data?

Big data dimension-volume

Massive parallel processing-moving process to data

Hadoop and MapReduce

Integrating with external data

Visualization

Big data dimension-variety

Types of data

Big data dimension-velocity

Streaming data

Sensor and GPS data

Social media data

Traditional big data use cases

More big data use cases

Health care

Logistics

National security

Leveraging the power of big data-real-time decision support

Triggering action

Speed of data retrieval from memory versus disk

From data analytics to models, from streaming data to decisions

Big data architecture

Operational systems and data sources

Intermediate data hubs

Business intelligence tools

Structured business intelligence

Search business intelligence.

Hadoop and MapReduce business intelligence

Data virtualization server

Batch and real-time data integration tools

Analytic sandbox

Risk response systems/recommendation engines

22 Conclusion to Managing Data in Motion

Data integration architecture

Why data integration architecture?

Data integration life cycle and expertise

Security and privacy

Data integration engines

Operational continuity

ETL engine

Enterprise service bus

Data integration hubs

Master data

Data warehouse and operational data store

Enterprise content management

Data archive

Metadata management

Data discovery

Data profiling

Data modeling

Data flow modeling

The end

References

Index.

Notes:

Description based on publisher supplied metadata and other sources.

Other Format:

Print version: Reeve, April Managing Data in Motion

ISBN:

9780123977915

OCLC:

831118849