Parallel implementations of backpropagation neural networks on transputers : a study of training set parallelism / editors, P. Saratchandran, N. Sundararajan, Shou King Foo.

Format:

Book

Contributor:

Saratchandran, P.

Sundararajan, N.

Foo, Shou King.

Series:

Progress in neural processing ; 3.

Progress in neural processing ; 3

Language:

English

Subjects (All):

Parallel processing (Electronic computers).

Neural networks (Computer science).

Back propagation (Artificial intelligence).

Transputers.

Physical Description:

1 online resource (222 p.)

Place of Publication:

Singapore ; River Edge, N.J. : World Scientific, c1996.

Language Note:

English

Summary:

This book presents a systematic approach to parallel implementation of feedforward neural networks on an array of transputers. The emphasis is on backpropagation learning and training set parallelism. Using systematic analysis, a theoretical model has been developed for the parallel implementation. The model is used to find the optimal mapping to minimize the training time for large backpropagation neural networks. The model has been validated experimentally on several well known benchmark problems. Use of genetic algorithms for optimizing the performance of the parallel implementations is des

Contents:

Preface; Purpose and Goals; Overview; Acknowledgements; Contents; Chapter 1 Introduction; 1.1 Multilayer Feedforward Neural Networks; 1.2 The Basic BP Algorithm; 1.3 Parallelism in the BP algorithm; 1.3.1 Network-based Parallelism; 1.3.2 Training-Set Parallelism; 1.4 Some Parallel Implementations; Chapter 2 Transputer Topologies for Parallel Implementation; 2.1 The Transputer; 2.2 Topologies; 2.2.1 Ring; 2.2.2 Mesh; 2.2.3 Torus; 2.2.4 Hypercube; 2.3 Topology Chosen in this study; 2.4 Software Used; 2.4.1 Creating Threads; 2.4.2 Interprocess Communication; 2.4.3 Semaphores; 2.4.4 Configuration

2.5 Performance Metrics and Benchmark Problems2.5.1 Encoder; 2.5.2 NETTALK; 2.5.3 Sonar; Chapter 3 Development of a Theoretical Model for Training Set Parallelism in a Homogeneous Array of Transputers; 3.1 Time Components of Parallel Transputer Implementation; 3.2 Timing Aspects of Parallelizing the Backpropagation Algorithm; 3.3 Time Components for the Parallelized Backpropagation Algorithm; 3.3.1 Communication part [Tcomm]; 3.3.2 Computation part [Tcomp]; 3.4 Validation of the Tepoch Model; Chapter 4 Equal Distribution of Patterns Amongst a Homogeneous Array of Transputers

4.1 Analytical Model for Time per Epoch4.2 Validation of the Model for Equal Distribution; 4.2.1 Speedup; 4.3 Optimal Number of Transputers Needed for the Case of Equal Distribution; 4.4 Cost Benefits Analysis of Adding Additional Processors; Chapter 5 Optimization Model for Unequal Distribution of Patterns in a Homogeneous Array of Transputers; 5.1 Constraints for Optimization; 5.1.1 Synchronized inter process communication; 5.1.2 Processor waiting times must be non-negative; 5.1.3 Memory limits in the processors; 5.2 Optimal Pattern Distribution

5.3 Validation of the Pattern Optimization Model5.4 Experimental Results for Benchmark problems; 5.4.1 An overview of the experiments conducted; 5.4.2 Is equal distribution optimal?; 5.4.3 Examples when the total number of training patterns is not an integer multiple of the number of processors present; 5.5 Locating Surplus Processors and to Find Out the Optimal Number of Processors Needed to Obtain Minimum Time Per Epoch; Chapter 6 Optimization Model for Unequal Distribution of Patterns in a Heterogeneous Array of Transputers; 6.1 Experimental Results for Benchmark Problems

6.2 Statistical Verification of the Optimal Epoch Time6.3 Discussion; 6.3.1 Worthiness of finding the optimal solution; Chapter 7 Pattern Allocation Schemes Using Genetic Algorithm; 7.1 Optimization Algorithm and Computational Complexity; 7.2 Solution Time for Optimal Pattern Distribution; 7.3 Sub-optimal Method: Heuristic Distribution; 7.4 Genetic Algorithm For Pattern Allocation; 7.4.1 Obtaining the Initial Population; 7.4.2 Calculating the Fitness of Each Chromosome; 7.4.3 Selecting Chromosomes for the Intermediate Population; 7.4.4 Crossover Operations; 7.4.5 Mutating the Chromosomes

7.4.6 Stopping Criterion

Notes:

Description based upon print version of record.

Includes bibliographical references (p. 189-199) and index.

ISBN:

9789812814968

9812814965