Pick, Click, Flick!: The Story of Interaction Techniques/ Brad A. Myers

Format:

Book

Author/Creator:

Myers, Brad A., author.

Series:

ACM books - Collection 3 ; #57.

ACM books, 2374-6777 ; #57

Language:

English

Subjects (All):

Pick, Click, Flick!(Computer Science).

Genre:

Electronic books.

Physical Description:

1 PDF (lxiv, 742pages) LuaTEX

Edition:

First Edition

Place of Publication:

[New York, NY, USA] : Association for Computing Machinery; [2024].

System Details:

Mode of access: World Wide Web

System requirements: Adobe Acrobat Reader

Contents:

List of Figures

List of Tables

Acknowledgments

Preface

Overview:

Who is this book for:

Use in classes:

2 Introduction 1

1.1 Importance of the Topic

1.2 What Are Interaction Techniques?

1.3 Formal Definitions

1.3.1 User and Designer

1.3.2 User Interface and Usability

1.3.3 Pointing Device

1.3.4 Input and Output

1.3.5 Graphical User Interface

1.3.6 Human-Computer Interaction

1.3.7 States and State Diagrams

1.3.8 Interaction Technique

1.3.9 Other Names

1.4 What Is Not an Interaction Technique?

1.5 Why Am I Writing this Book?

1.6 Value of Interaction Techniques

1.7 Need for New Interaction Techniques

1.8 Why Are Interaction Techniques Hard to Design?

1.8.1 Many Needs

1.8.2 Details of the Behavior

1.8.3 Abort, Cancel, and Undo

1.8.4 The Look

1.8.5 Localization and Internationalization

1.9 Organization of the Book

3 History of Desktop Devices

2.1 Introduction

2.2 Computer Hardware

2.2.1 Mainframes

2.2.2 Personal Workstations

2.2.3 Personal Computers

2.2.4 Other Devices

2.3 Display Technologies

2.4 Important Systems

2.4.1 Sketchpad, 1963

2.4.2 NLS, 1968

2.4.3 Xerox Alto, 1970s

2.4.3.1 Smalltalk, 1976

2.4.3.2 Bravo Text Editor, 1974

2.4.3.3 Various Drawing and Painting Programs, 1970s

2.4.3.4 Xerox Programming Environments, 1980-1981

2.4.4 VisiCalc, 1977

2.4.5 Spatial Data Management System, 1978

2.4.6 Xerox Star, 1981

2.4.7 Apple Lisa (1983) and Macintosh (1984)

2.4.8 IBM PCs (1981) and Microsoft Windows (1985)

2.4.9 Apple HyperCard (1987)

2.4.10 Alternate Reality Kit, 1986

2.4.11 NeXT, 1985-1997

2.4.12 SunWindows, NeWS, Andrew, X, OpenLook, and Motif

2.4.13 World Wide Web

2.5 Brief Overview of Human-Computer Interaction Technology

2.5.1 Interaction and Application Styles

2.5.1.1 Command Line Interfaces

2.5.1.2 Form Fill-in

2.5.1.3 Direct Manipulation of Graphical Objects

2.5.1.4 Window Managers

2.5.1.5 Drawing and Painting Programs

2.5.1.6 Text (Document) Editing

2.5.1.7 Spreadsheets

2.5.1.8 Hypertext and the World Wide Web

2.5.1.9 Computer-aided Design

2.5.1.10 Video Games

2.5.1.11 Gesture Recognition

2.5.1.12 Multimedia

2.5.1.13 3D

2.5.1.14 Virtual Reality and Augmented Reality

2.5.1.15 Computer-supported Cooperative Work

2.5.1.16 Natural Language and Speech

2.5.2 Implementation Techniques

2.5.2.1 User Interface Management Systems and Toolkits

2.5.2.2 Interface Builders

2.5.2.3 Component Architectures

2.6 Chapter Conclusion

4 History of Mobile Devices

3.1 Introduction

3.2 Definitions

3.2.1 Mobile Devices and Handheld Devices

3.2.2 Touchscreens and Touchpads

3.2.3 Palm-size Devices, Tablets, and Wearables

3.2.4 Pen Computing

3.2.5 Gestures

3.2.6 Handwriting Recognition and Hand Printing Recognition

3.2.7 Personal Digital Assistants

3.2.8 Mobile Phones, Feature Phones, and Smartphones

3.3 Calculators

3.4 Ubiquitous Computing

3.5 Pen Operating Systems

3.6 GRiDPad

3.7 Microsoft Pen Windows

3.8 Apple Newton

3.9 General Magic's "Magic Cap" OS

3.10 Early Wireless Phones

3.11 Early Smartphones

3.12 Wireless Networks

3.13 Palm

3.14 Windows CE, PocketPC, etc.

3.15 RIM Blackberry

3.16 Windows TabletPC

3.17 iPod

3.18 iPhone

3.19 Android

3.20 iPad

3.21 Chapter Conclusion

5 Pointing Devices

4.1 Introduction

4.2 Cursor and Selection

4.3 Mouse Buttons, Press, Hold, Click, Tap, and Drag-Definitions

4.4 Lightpen

4.5 Arrow Keys

4.6 Mouse

4.7 Joysticks and Trackballs

4.8 IBM TrackPoint

4.9 Graphics Tablets

4.10 Touchscreens

4.11 Remote Interaction

4.12 Dimensions for Pointing Devices

4.12.1 Direct versus Indirect

4.12.2 Absolute versus Relative

4.12.3 How Many States Are Supported?

4.13 Other Issues

4.14 Testing for Speed and Accuracy

4.14.1 Fitts' Law

4.15 Research About Pointing

4.16 Summary of the Considerations for Pointing

4.17 Chapter Conclusion

6 Scrolling and One-Dimensional Input

5.1 Introduction

5.2 Auto-Scroll

5.3 Knobs

5.4 Physical One-Dimensional Sliders

5.5 Keys and Physical Buttons

5.6 Scrollbars

5.6.1 Bravo's Scrollbar, 1974

5.6.2 Smalltalk's Scrollbar, 1977

5.6.3 Interlisp-D, Symbolics Lisp Machine, XDE Scrollbars, and Andrew, 1980-1984

5.6.4 Xerox Star (1981) and Viewpoint (1985)

5.6.5 Apple Lisa (1983) and Macintosh (1984) Scrollbars

5.6.6 Windows, OpenLook, and NeXT Scrollbars

5.6.7 Mobile Device Scrollbars

5.6.8 iPhone Scrollbars

5.6.9 Variations on Scrollbars

5.6.9.1 Showing Other Information in the Bar

5.6.9.2 Auto-Hide

5.6.9.3 Infinite Scrolling

5.7 Using a "Hand" to Scroll

5.8 Rate-Controlled Scrolling

5.9 Gestures for Scrolling

5.9.1 Flick-to-Scroll

5.9.2 One- and Two-Finger Scrolling

5.10 Sliders

5.11 Research on Scrolling and Sliders

5.11.1 Evaluating Scrolling Speed

5.11.2 Research on Scrolling and Sliders

5.12 Summary of the Considerations for Scrolling

5.13 Chapter Summary

7 Percent-done Progress Indicators

6.1 Introduction

6.2 History

6.3 Implementation

6.4 Variations

6.5 Design Considerations and Conclusions

8 Menus

7.1 Definitions

7.2 Different Categories of Menus

7.3 Dimensions

7.3.1 How Many Options Can Be Selected?

7.3.2 How Many and What Kind of Options Are There?

7.3.3 Fixed or Popup?

7.3.4 Invalid?

7.3.5 Cancellable?

7.3.6 Submenus

7.3.7 Graphic Design

7.3.8 Default Value?

7.3.9 Layout

7.3.9.1 Pie Menus and Marking Menus

7.3.10 Where the Menu Is Displayed

7.3.11 Behaviors

7.3.11.1 How Many States

7.3.11.2 Moving from One Item to Another

7.3.11.3 Pointer Operation

7.3.11.4 Keyboard Operation

7.4 Examples of Historical Menus

7.5 Research on How to Organize Menu Contents

7.6 Summary of Design Considerations for Menus

7.7 Chapter Conclusions

9 Text Entry and Editing for Computers and Handhelds

8.1 Introduction

8.2 Text Entry for Desktops

8.2.1 Typewriters

8.2.2 Chinese Text Entry

8.2.2.1 The Chinese Language

8.2.2.2 Chinese Keyboard Typing Methods

8.2.2.3 Chinese Text Input Interaction Techniques

8.2.3 Chorded Text Entry

8.2.4 Other Text Entry for Desktops

8.3 Text Entry for Mobile Devices

8.3.1 Hand Printing and Handwriting

8.3.2 Gestural Alphabets

8.3.3 Keypads

8.3.3.1 Multi-tap

8.3.3.2 T9

8.3.4 Small Physical Keyboards

8.3.5 Portable Chording Keyboard

8.3.6 Virtual Keyboards

8.3.6.1 Research on Virtual Keyboard Layouts

8.3.6.2 Gestures Over Virtual Keyboards

8.3.7 Text Entry on Wearables and Watches

8.4 Auto-Fill, Auto-Correction, and Autocomplete

8.4.1 Auto-Fill

8.4.2 Spell Checking and Auto-Correction on Regular Computers

8.4.3 Autocomplete on Regular Computers

8.4.4 Auto-Correction and Autocompletion on Mobile Devices

8.5 Measuring Text Entry Speed and Accuracy

8.5.1 Speed

8.5.2 Learning

8.5.3 Accuracy

8.5.4 Confounds from Auto-Correct and Autocomplete

8.5.5 Target Text for Testing

8.5.6 Summary of Speed and Accuracies of Popular Methods

8.6 Text Editing and Formatting

8.6.1 Brief History of Text Editing and Formatting

8.6.2 Editing Systems for Code

8.6.3 Text Editing Techniques

8.6.3.1 Moded versus Modeless Editing

8.6.3.2 Positioning the Text Cursor and Selecting Text on Desktops

8.6.3.3 Positioning the Text Cursor and Selecting Text on Mobile Devices

8.6.3.4 Operations for Editing

8.6.4 Multi-User Text Editing

8.7 Text Entry Widgets

8.8 Typeahead

8.9 Summary of Considerations for Designing New Text Entry Methods

8.10 Chapter Conclusions

10 Forms, Property Sheets, and Dialog Boxes

9.1 Introduction

9.2 Xerox Star Property Sheets

9.3 Apple's Dialog Boxes

9.4 Tabbed Dialog Boxes

9.5 Toolbars and Ribbons

9.6 Typical Built-In Dialog Boxes

9.7 Web Pages as Forms

9.8 Dialog Boxes on Mobile Devices

9.9 Research About Dialog Boxes

9.10 Summary of Design Considerations for Dialog Boxes

9.11 Chapter Conclusions

11 Creating, Selecting, and Manipulating Objects

10.1 Introduction

10.2 Syntax of Operations

10.3 Graphical Object Hierarchies

10.4 Creating New Objects and Painting

10.5 Selection of Graphics

10.5.1 Selecting One or Multiple Objects

10.5.2 Clicking on Objects

10.5.3 Selecting in a Region

10.5.4 Feedback for Selecting Objects

10.5.5 Selecting Pixels for Painting Programs

10.6 Specifying Position, Size, and Shape

10.6.1 Coordinate Systems

10.6.2 Global Parameters-Feedback and Aborting

10.6.3 Size and Position for Bounding Boxes

10.6.4 Size and Position for Lines, Curves, Polylines, and Freehand Scribbles

10.6.5 Grids

10.6.6 Snapping

10.7 Specifying Colors

10.8 Specifying Other Graphical Properties

10.9 Specifying Animations and Behaviors

10.10 Selection in Spreadsheets

10.11 Selections in Linear and Hierarchical Lists

10.12 Selection in Consumer Electronics

10.13 Brief History of Graphical Editing

10.14 Research in Selection and Editing Objects

10.15 Summary of Considerations for Selecting and Creating Objects

10.16 Chapter Conclusion

12 Desktop Metaphors, Window Managers, and Icons

11.1 Introduction

11.2 Definitions

11.3 A Brief History of Window Managers

11.4 Input Focus

11.5 Overlapping Windows versus Tiled versus Tabbed versus Single Window

11.6 Icons

11.7 Task Bar and Dock

11.8 Decorations for Individual Windows

11.9 Interaction Techniques for Windows

11.10 Collections of Windows

11.11 Alternatives

11.11.1 Zoomable User Interface

11.11.1.1 SketchPad

11.11.1.2 Spatial Data Management System

11.11.1.3 PAD, PAD++, and Jazz

11.11.2 Timeline Systems

11.11.3 The Open Data Model

11.11.4 Microsoft Bob

11.11.5 Other Organizations

11.12 Summary of Considerations for Desktops, Windows, and Icons

11.13 Chapter Conclusion

13 Interactions in 3D, Virtual and Augmented Reality, and Games

12.1 Introduction

12.2 Definitions

12.3 Why Are 3D-IxTs Difficult

12.4 Input Devices for 3D IxTs

12.4.1 1D and 2D Input Devices for 3D

12.4.2 Early 3D Position Sensors and HMDs

12.4.3 Early Tracking of the User's Fingers-Gloves

12.4.4 Tracking a Wand

12.4.5 Camera-Based Techniques

12.4.6 Input Techniques with Haptic Feedback

12.4.7 Other Sensors

12.5 3D Interaction Techniques

12.5.1 IxTs for Games

12.5.2 IxTs Using the Mouse

12.5.3 IxTs Using 3D Inputs

12.5.4 3D Widgets

12.6 Summary of Considerations for 3D Interactions

12.7 Chapter Conclusion

14 Ubiquitous Computing, Physical Widgets, Sensors, Wearables, and Internet of Things (IoT) 13.1 Introduction

13.2 Definitions

13.3 Turtles to Robots

13.4 Physical Objects

13.5 Wearables

13.6 Handheld Projectors

13.7 IoT

13.8 Summary of Considerations

13.9 Chapter Conclusions

15 Gesture Recognition

14.1 Introduction and Definitions

14.2 Advantages and Disadvantages of Gestures

14.2.1 Advantages

14.2.2 Disadvantages

14.3 Applications for Gestures

14.3.1 Gestures for Text Entry

14.3.1.1 Recognizing Sign Language Gestures

14.3.2 Gestures for Proofreading and Editing Text

14.3.3 Gestures for Object Entry and Editing

14.3.4 Gestures for Other Commands

14.4 Gesture Recognition Algorithms

14.5 Evaluating and Improving the Learnability and Accuracy of Gestures

14.6 Funny

14.7 Summary of Considerations for Gesture Recognition

14.8 Chapter Conclusions

16 Undo, Redo, Repeat, Cancel, Command History, and Macro Recording

15.1 Introduction

15.2 Definitions

15.3 History of Undo/Redo

15.4 Cancel or Abort

15.5 Unit of Undo

15.6 What Can Be Undone

15.7 Redo/Repeat

15.8 Undo/Redo Models

15.8.1 Single-level Undo

15.8.2 Multi-level Undo

15.8.2.1 Linear Model

15.8.2.2 Emacs Model

15.8.2.3 Tree Model

15.8.3 Selective Undo

15.8.3.1 Selective Undo by Region

15.8.3.2 Script Model

15.8.3.3 Inverse Model

15.9 The Back Button

15.10 Multi-user Undo

15.11 Scope of Undo

15.12 Implementing Undo and the Command Object Pattern

15.13 Macro Recording and Programming by Example

15.14 Summary of Considerations for Undo/Redo

15.15 Chapter Conclusions

17 Interaction Techniques for Help

16.1 Introduction

16.2 Status Bar

16.3 Tooltips

16.4 Autocomplete as Help

16.5 Help Modes

16.5.1 Ask About a Screen Item

16.5.2 Stencils

16.6 Considerations for Help and Chapter Conclusions

18 Interaction Techniques for People with Disabilities

17.1 Introduction

17.2 Related Concepts and Definitions

17.3 Kinds and Causes of Disabilities

17.3.1 Muscular Problems

17.3.2 Visual Impairment and Blindness

17.3.3 Deafness and Hearing Loss

17.4 Interaction Techniques Related to Disabilities

17.4.1 Adaptive Input Devices

17.4.2 Selecting Items by Scanning

17.4.3 Text Entry

17.4.4 Augmentative and Alternative Communication Devices

17.4.5 Brain-Computer Interfaces

17.5 Screen Readers

17.6 Adaptations Built into PC and Macintosh

17.7 Adaptations Built into Smartphones and Tablets

17.8 Web, App, and Consumer Electronics Accessibility

17.9 Summary of Consideration for Interaction Techniques for People with Disabilities

17.10 Chapter Conclusions

19 Intelligent Interfaces: Artificial Intelligence in Interaction Techniques, Speech Recognition, and Conversational Agents

18.1 Introduction

18.2 Definitions

18.3 Speech and Natural Language Interfaces

18.3.1 Kinds of Dialogs and What Can Be Said

18.3.2 How Users Provide Input

18.3.3 Feedback, Verifying, and Correcting

18.3.4 System Actions

18.4 Multi-Modal Interaction Techniques

18.5 Embodied Intelligent Agents

18.6 Algorithms and Machine Learning for Content Selection

18.7 Other "Intelligent" Interfaces

18.8 Challenges, Risks, and Controversy About Intelligent Interfaces

18.9 Summary of Considerations for Intelligent Interfaces

18.10 Chapter Conclusions

20 Formal and Informal Models, Metaphors, and Patterns for the Design of Interaction Techniques

19.1 Introduction

19.2 "Model"

19.3 Metaphors and UI Design Patterns for UIs

19.4 State Transition Diagrams

19.5 Input Device Models

19.5.1 Foley and Wallace (1974) and Chan (1984)

19.5.2 Buxton's Classification (1983)

19.5.3 Card, Mackinlay, Robertson Model (1990)

19.6 Pragmatic, Lexical, Syntactic, Semantic, and Conceptual Level Model

19.7 Human Performance Models

19.7.1 Keystroke-level Model

19.7.2 Goals, Operators, Methods, and Selection Rules

19.8 Other Models

19.9 Considerations for Models

19.10 Chapter Conclusions

21 Tools and Models for Using and Implementing Interaction Techniques

20.1 Introduction

20.2 Why Hard to Implement

20.3 Definitions

20.4 Why Tools Are Important

20.5 Toolkits and Frameworks

20.5.1 Look and Feel

20.5.2 Frameworks

20.5.3 Toolkit and Framework Class Hierarchies

20.5.4 Research Toolkits and Frameworks

20.6 Geometry and Layout Management

20.6.1 Absolute Layout

20.6.2 General Constraints

20.6.3 Layout Managers

20.6.3.1 Special Values for Coordinates

20.6.3.2 Special Kinds of Containers

20.6.3.3 Stretchy Spacers

20.7 Resource Files and Internationalization

20.8 Interactive and Graphical Tools for Building UIs

20.9 Interactive Prototyping Tools

20.9.1 Kinds of Prototyping Tools

20.9.2 Examples of Prototyping Tools

20.9.3 Research on Prototyping Tools

20.10 Automatic Creation from Models

20.11 Tools for 3D, AR/VR/XR Interactions

20.12 Models and Patterns for the Implementation

20.12.1 Early Models

20.12.2 Event Model

20.12.3 Listener Pattern

20.12.4 Interactors Model

20.12.5 Imaging Model

20.12.5.1 Retained Object Model

20.12.6 Model-View-Controller

20.12.6.1 Model-View

20.12.7 Document Model

20.13 Considerations for the Implementation

20.14 Chapter Conclusions

22 21 Methods for Designing and Evaluating Interaction Techniques

21.1 Introduction

21.2 Methods for Needs Finding

21.3 Methods for Design

21.3.1 Graphic Design

21.3.2 Prototyping and Implementing

21.4 Methods for Evaluation

21.4.1 Usability Evaluation

21.4.2 Controlled Experiments

21.5 Considerations and Chapter Conclusions

23 Legal Issues and Patents

22.1 Introduction

22.2 Copyrights

22.3 Trademarks

22.4 Trade Secrets

22.5 Patents

22.5.1 Patent Controversies

22.6 Considerations for Intellectual Property and Chapter Conclusions

24 Interview: Larry Tesler

23.1 Short Bio of Larry Tesler

23.2 User Studies-Stanford Card Stunt Program

23.3 Blank Screen Study

23.4 RCG User Study

23.5 Cut and Paste

23.6 Modeless Software and the Gypsy Text Editor-The Shift Away from Moded Software

23.7 The Gypsy Text Editor

23.8 Other Problems with Modes

23.9 Interaction Techniques at Apple

25 Interview: David Canfield Smith

24.1 Short Bio of David Canfield Smith

24.2 The Story of Icons

24.3 Pygmalion

24.4 Xerox Star

24.5 Techniques for Reducing System Commands

24.5.1 Universal Commands

24.5.2 Creation by Copying

24.5.3 Property Sheets

24.6 Other Star Designs

26 Interview: Bill Buxton

25.1 Short Bio of Bill Buxton

25.2 Interacting with Two Hands

25.3 Adjusting Values

25.4 Future

27 Interview: Norm Cox

26.1 Short Bio of Norm Cox

26.2 Visual Icon Design

26.3 Legal Consulting

26.4 Interaction Design for Scrolling

26.5 Mnemonic Gestures for Interacting with a Device

28 Interview: Dan Bricklin

27.1 Short Bio of Dan Bricklin

27.2 VisiCalc

27.3 Dan Bricklin's Demo

27.4 Chapter Conclusions

29 Interview: Bill Atkinson

28.1 Short Bio of Bill Atkinson

28.2 Initial Designs of the Lisa

28.3 HyperCard

28.4 General Magic

30 Interview: Bruce Horn

29.1 Short Bio of Bruce Horn

29.2 PARC and Smalltalk

29.3 Apple Macintosh

31 Interview: Rob Haitani

30.1 Short Bio of Rob Haitani

30.2 Casio Zoomer

30.3 Palm Pilot

32 Interview: Tandy Trower

31.1 Short Bio of Tandy Trower

31.2 Precursors to Microsoft Windows

31.3 Microsoft Windows 1.0, 2.0, and 3.0

31.4 Graphic Design

31.5 Microsoft's User Interface Design Group and Visual Basic

31.6 Windows NT and Windows 95

31.7 Microsoft Bob and the Windows Assistant

31.8 Other Microsoft Products

33 Interview: Loren Brichter

32.1 Short Bio of Loren Brichter

32.2 Pull-to-Refresh

32.3 Other Spring-Action Interactions

32.4 Pull-to-Refresh Patent

34 Interview: Don Hopkins

33.1 Short Bio of Don Hopkins

33.2 Definitions and Key Properties of Pie Menus

33.3 History of Pie Menus

35 Interview: Ted Selker

34.1 Short Bio of Ted Selker

34.2 Origins of the Pointing Stick

34.3 Refining the TrackPoint

36 Interview: Shumin Zhai

35.1 Short Bio of Shumin Zhai

35.2 Effective Text Entry Requirements

35.3 Origins of Shape Writing

35.4 ShapeWriter

35.5 Algorithms

35.6 Pixel Squeeze-to-Assist in Google Pixel 2 Phone

37 Interview: Gregg Vanderheiden

36.1 Short Bio of Gregg Vanderheiden

36.2 Personal History

36.3 Digital Affinity

36.4 Morphic

36.5 Accessibility Today and Tomorrow

36.6 Info-bot

36.7 Chapter Conclusions

38 Interview: Chris Harrison

37.1 Short Bio of Chris Harrison

37.2 Context

37.3 Using the Body for Input/Output

37.4 Increasing the Richness of Touch Interfaces

39 Interview: Julia Schwarz

38.1 Short Bio of Julia Schwarz

38.2 Devices

38.3 HoloLens 2 Interaction Model

38.4 HoloLens 2 Prototypes and Interactions

38.5 Mixed Reality Toolkit

38.6 Chapter Conclusions

40 Future of Interaction Techniques

39.1 IxT Class Projects

39.1.1 New Smartphone Interaction Techniques

39.1.2 New Smartwatch Interaction Techniques

39.1.3 AR, VR, and 3D Interaction Techniques

39.1.4 Wikipedia Entries

39.1.5 Comparison and Testing of Historical and Current Interaction Techniques

39.1.6 New Testing Methods

39.1.7 Parts of This Book

39.2 Other Important Future Work

39.2.1 UIs for Conversational Agents

39.2.2 Better Text Entry

39.2.3 IxTs for 3D and Virtual and Augmented Reality

39.2.4 Two-Handed Interfaces

39.2.5 Brain Interfaces

39.2.6 Other GUI IxTs

39.2.7 Open Data Model

41 Conclusions

Bibliography

Index

Author's Biography

Other Format:

Print version:

ISBN:

3617448

9798400709500

9798400709487

Access Restriction:

Restricted for use by site license.