343 -

• | Table of Contents
• | Examples
DB2® Universal Database for OS/390® Version 7.1 Certification Guide
By Richard Yevich, Susan Lawson
Publisher | : Prentice Hall PTR
Pub Date | : August 13, 2001
ISBN | : 0-13-065076-5
Pages | : 672

IBM's DB2 Universal Database Version 7.1 for OS/390 delivers unparalleled performance, scalability, and reliability in today's enterprise e-business environments! Now, there's a complete, authoritative guide to deploying, managing, and developing applications with DB2 UDB V7.1 in both OS/390 and z/OS environments—and preparing for IBM's updated DB2 UDB V7.1 for OS/390 certification exam!

Direct from IBM, this comprehensive day-to-day reference for running DB2 UDB in OS/390 and z/OS environments is also the only book with the in-depth knowledge professionals need in order to pass IBM's latest DB2 for OS/390DBA certification exam. Full of expert tips, techniques, and guidelines, it covers virtually every aspect of working with DB2 UDB in enterprise server: administration, connectivity, development, and much more.

IBM Gold Consultants Richard Yevich and Susan Lawson present start-to-finish coverage of DB2's latest and most powerful enhancements, tools, and utilities, showing how to make the most of DB2 running on both OS/390 and next-generation z/OS platforms. Coverage includes:

  • An overview of the DB2 UDB product family

  • DB2 in OS/390 and z/OS environments: address spaces, attachments, interfaces, commands, and utilities

  • Comprehensive security coverage, including access controls and auditing techniques

  • SQL in depth: DB2 data structures, database objects, basic database design, and advanced coding

  • DB2 UDB administration: maintaining data, backup/recovery, and more

  • Performance monitoring and optimization: access paths, indexes, parallelism, buffer pool tuning, and subsystem tuning

  • Data sharing: architecture and implementation in a Parallel Sysplex environment

  • Detailed application development coverage: database extenders, cursors, stored procedures, triggers, object-relational extensions, locking, and concurrency

Whether you're deploying DB2 UDB V7.1 in an OS/390 environment, managing DB2 UDB V7.1, preparing for DB2 UDB V7.1 certification, or all three, DB2 Universal Database Version 7.1 for OS/390 Certification Guide will be your single most valuable resource.


001 -

• | Table of Contents
• | Examples
DB2® Universal Database for OS/390® Version 7.1 Certification Guide
By Richard Yevich, Susan Lawson
Publisher | : Prentice Hall PTR
Pub Date | : August 13, 2001
ISBN | : 0-13-065076-5
Pages | : 672

Copyright
Preface
Part I. Introduction to DB2 UDB for OS/390
 | Chapter 1. Product Overview
 |  | DB2 and E-Business
 |  | DB2 Universal Database
 |  | DB2 Middleware and Connectivity
 |  | DB2 Application Development
 |  | DB2 Administration
 |  | Data Management Tools
 |  | Summary
 |  | Additional Product Resources
 | Chapter 2. Environment
 |  | OS/390 and z/OS
 |  | DB2 for OS/390 and z/OS
 |  | Security
 |  | Parallel Sysplex Support
 |  | Interfaces
 |  | Installation and Migration Overview
 |  | DSNZPARMS
 |  | Commands
 |  | DB2 Utilities
 |  | Catalog and Directory
 |  | Distributed Data
 |  | Subsystem Pools
 |  | Summary
 |  | Additional DB2 Environment Resources
 | Chapter 3. Controlling Data Access
 |  | DB2 Subsystem Access
 |  | Data Set Protection
 |  | Access to DB2 Objects
 |  | Auditing
 |  | Summary
 |  | Additional Resources
Part II. Using SQL
 | Chapter 4. Database Objects
 |  | Understanding Data Structures
 |  | Managing Database Objects
 |  | Database Design and Implementation
 |  | Summary
 |  | Additional Resources
 | Chapter 5. Manipulating Database Objects
 |  | Data Retrieval
 |  | Data Modification
 |  | View Classification
 |  | Summary
 |  | Additional Resources
 | Chapter 6. Advanced SQL
 |  | Subqueries
 |  | Unions
 |  | Nested Table Expressions
 |  | Joins
 |  | CASE Expressions
 |  | Row Expressions
 |  | OLAP Features
 |  | Summary
 |  | Additional Resources
Part III. DB2 UDB for OS/390 and z/OS Administration
 | Chapter 7. Maintaining Data
 |  | Data Movement
 |  | Data Maintenance
 |  | Data Statistics
 |  | Data Maintenance Process
 |  | Standalone Utilities
 |  | Resolving Restrictive and Advisory States
 |  | Summary
 |  | Additional Resources
 | Chapter 8. Database Recovery
 |  | Database Recovery Concepts
 |  | Logging
 |  | Image Copies
 |  | Establishing a Point of Consistency
 |  | Recovery Concepts
 |  | Backup and Recovery of the DB2 Catalog and Directory
 |  | Disaster Recovery
 |  | LOB Recovery
 |  | Tracker Site Recovery
 |  | Set Log Suspend/Resume
 |  | Checkpoint Interval
 |  | Minimizing Outages
 |  | Data Sharing Recovery
 |  | Summary
 |  | Additional Recovery Resources
 | Chapter 9. Data Sharing
 |  | Data Sharing Benefits
 |  | Data Sharing Components
 |  | Shared Data
 |  | To Share or Not to Share
 |  | Maintaining Data Integrity
 |  | Performance
 |  | Processing Costs
 |  | Movement to Data Sharing
 |  | Workload Management and Affinity Processing
 |  | Distributed Processing
 |  | Sysplex Query Parallelism
 |  | Recovery Considerations
 |  | Summary
 |  | Additional Data Sharing Resources
 | Chapter 10. Performance Monitoring and Tuning
 |  | Elements of Performance
 |  | Tuning Guidelines
 |  | Access Paths and Optimization
 |  | Designing Indexes for Performance
 |  | Query Parallelism
 |  | Database Monitoring
 |  | Tracing Problems in DB2
 |  | Using DISPLAY Commands
 |  | Buffer Pool and Subsystem Tuning
 |  | Summary
Part IV. Developing Applications
 | Chapter 11. Application Development Overview
 |  | DB2 Application Development Environment
 |  | Delimiting SQL in a Program
 |  | Declaring Table and View Definitions
 |  | Host Variables and Host Structures
 |  | SQL Execution Validation
 |  | Using Cursors
 |  | Using Dynamic SQL
 |  | Precompile and Bind
 |  | Application Program Features
 |  | Stored Procedure Builder
 |  | Summary
 |  | Additional Resources
 | Chapter 12. Development Considerations
 |  | SQL Procedure Language
 |  | Extenders
 |  | DB2 ODBC Programming
 |  | DB2 and Java
 |  | Stored Procedure Builder
 |  | Summary
 |  | Additional Resources
 | Chapter 13. Advanced Functions
 |  | Stored Procedures
 |  | Triggers
 |  | Object-Relational Extensions
 |  | Summary
 |  | Additional Resources
 | Chapter 14. Locking and Concurrency
 |  | Locking Data
 |  | Overriding or Avoiding Locks
 |  | System Parameters
 |  | Claims and Drains
 |  | Locking Issues and Problems
 |  | Summary
 |  | Additional Resources
Appendix A. Appendix
Appendix B. DB2 UDB for OS/390 and z/OS Certification
 | Certification Test Objectives
Bibliography
 | DB2 Universal Database for OS/390
 | DB2 Administration Tool
 | DB2 Buffer Pool Tool
 | DB2 DataPropagator
 | Net.Data for OS/390: The following books are available at http://www.ibm.com/software/net.data/library.html:
 | DB2 PM for OS/390
 | Query Management Facility
 | Ada/370
 | APL2
 | AS/400
 | BASIC
 | BookManager READ/MVS
 | C/370
 | Character Data Representation Architecture
 | CICS/ESA
 | CICS/MVS
 | CICS Transaction Server for OS/390
 | IBM C/C++ for MVS/ESA
 | IBM COBOL
 | Cooperative Development Environment
 | Data Extract (DXT)
 | DataPropagator NonRelational
 | Data Facility Data Set Services
 | Database Design
 | DataHub
 | DB2 Connect
 | DB2 Universal Database (DB2 UDB)
 | Device Support Facilities
 | DFSMS/MVS
 | DFSORT
 | Distributed Relational Database Data
 | Domain Name System
 | Education
 | Enterprise System/9000 and Enterprise System/3090
 | High-Level Assembler
 | Parallel Sysplex Library
 | sICSF/MVS
 | ISPF
 | Language Environment
 | National Language Support
 | NetView
 | ODBC
 | OS/390
 | PL/I for MVS &VM
 | OS PL/I
 | Prolog
 | RAMAC
 | Remote Recovery Data Facility
 | Storage Management
 | MVS/ESA Storage Management Library
 | System/370 and System/390
 | System Network Architecture (SNA)
 | TCP/IP
 | VS COBOL II
 | VS FORTRAN
 | VTAM

Top


002 - Copyright

Team-Fly
Table of Contents

Copyright

Editorial/production supervision: Kathleen M. Caren

Acquisitions editor: Michael Meehan

Marketing manager: Debby van Dijk

Manufacturing manager: Maura Zaldivar

Editorial assistant: Linda Ramagnano

Cover design director: Jerry Votta

Cover designer: Bruce Kenselaar

Art director: Gail Cocker-Boguzs

Interior design: Meg Van Arsdale

© 2002 International Business Machines Corp.

Published by Prentice Hall PTR

Prentice-Hall, Inc.

Upper Saddle River, NJ 07458

Prentice Hall books are widely used by corporations and government agencies for training, marketing, and resale.

The publisher offers discounts on this book when ordered in bulk quantities. For more information, contact: Corporate Sales Department, Phone: 800-382-3419; Fax: 201-236-7141; E-mail: corpsales@prenhall.com; or write: Prentice Hall PTR, Corp. Sales Dept., One Lake Street, Upper Saddle River, NJ 07458.

All products or services mentioned in this book are the trademarks or service marks of their respective companies or organizations.

All rights reserved. No part of this book may be reproduced, in any form or by any means, without permission in writing from the publisher.

Printed in the United States of America

10 9 8 7 6 5 4 3 2 1

Pearson Education LTD.

Pearson Education Australia PTY., Limited

Pearson Education Singapore, Pte. Ltd.

Pearson Education North Asia Ltd.

Pearson Education Canada, Ltd.

Pearson Educación de Mexico, S.A. de C.V.

Pearson Education—Japan

Pearson Education Malaysia, Pte. Ltd.

Pearson Education, Upper Saddle River, New Jersey

Dedication

We would like to dedicate this book to many people. In particular, our IBM and technical expert reviewers: Klaas Brant—KBCE, Dan Luksetich—SoftAlt Inc., Lisa Dougherty—MBNA, Terry Purcell—YL&A, Mary Petras—IBM, Bart Steegmans—IBM, Susan Visser—IBM, and many others at IBM.

We would also like to thank our staff at YL&A for their continued help and support during many of our efforts: Jim Bishop, Susan Ward, Susan McCormick, Lisa Camacho, Kevin Davis, Dianna Lewis, Jorge Rivera, Dan Underwood, Joe Ubelhor, Tom Ubelhor, Jeremy Peck, John Maenpaa, Terry Purcell, Dave Mazmanian and, of course, YL&A principal—Alex Lawson.

This book is also dedicated in memory of Roger Scott.

    Team-Fly


003 - Preface

Preface

This book is a complete guide to IBM's relational database server, known as DB2 Universal Database Version 7.1 for OS/390 and z/OS. In order to understand the DB2 family and all of the options you have for implementation, you need to be familiar with the fundamentals of all platforms. This book covers DB2 UDB for OS/390 and z/OS; for details on the other platforms, you will need to refer to the other certification guides. Perhaps you are planning to become certified, or you would simply like to understand the powerful new DB2 UDB database servers from IBM and the capabilities of DB2 for OS/390 and z/OS. Those interested in becoming an IBM Certified Professional will want to review the sample questions at the end of this book.

The book is divided into four parts:

  • Part 1—Introduction to DB2 UDB for OS/390 and z/OS (Chapters 1–3).

    The family of DB2 products, the DB2 environment, and access controls are covered in Chapters 1, 2, and 3.

  • Part 2—Using SQL (Chapters 4–6).

    The Structured Query Language (SQL) and database objects are discussed in Chapters 4, 5, and 6.

  • Part 3—DB2 UDB Database Administration for OS/390 and z/OS (Chapters 7–10).

    DB2 UDB for OS/390 and z/OS common administration tasks and recovery issues are discussed in Chapters 7 and 8. Chapter 9 covers the architecture and implementation of data sharing in a parallel sysplex environment. Database monitoring and performance considerations are discussed in Chapter 10.

  • Part 4—Developing Applications with DB2 (Chapters 11–14).

    An introduction to application development for DBAs is given in Chapter 11. Chapter 12 deals with some of the additional considerations for application development. Many of the advanced functions and object relational extensions are covered in Chapter 13. Chapter 14 covers issues surrounding locking and concurrency.

This book can be used as a self-study guide to help you prepare for the DB2 Universal Database V7.1 for OS/390 and z/OS certification exam or simply as a guide to DB2 Universal Database V7.1 for OS/390 and z/OS. For information about the DB2 UDB Family, refer to the other certification books.


004 - Part I: Introduction to DB2 UDB for OS/390

Part I: Introduction to DB2 UDB for OS/390

1 Product Overview

2 Environment

3 Controlling Data Access


005 - Chapter 1. Product Overview

Part I. Introduction to DB2 UDB for OS/390

Chapter 1. Product Overview

  • DB2 Universal Database

  • DB2 Middleware and Connectivity

  • DB2 Application Development

  • DB2 Administration

  • Data Management Tools

In this chapter, you will be introduced to the DB2 Universal Database (DB2 UDB) family of products for S/390, zSeries, UNIX and Intel platforms. DB2 has the ability to store all kinds of electronic information. This includes traditional relational data as well as structured and unstructured binary information, documents and text in many languages, graphics, images, multimedia (audio and video), information specific to business needs such as engineering drawings, maps, insurance claims forms, numerical control streams, or any type of electronic information. This chapter illustrates ways to access data in a DB2 database using interfaces provided with the products. A description of each of the DB2 products is provided to illustrate some of the features and functions.

The DB2 database is an important part of IBM's e-business software portfolio. The e-business Application Framework provides an open blueprint on how to build e-business applications. Popular IBM e-business tools include Visual Age for Java for developing Java programs or components and Tivioli software for distributed systems management. As for application server software, IBM offers several types of servers depending on the business requirement, from Message Queuing (MQ) software to Java-based transaction processing with Websphere Application Server.

The most popular IBM software servers are its database servers, specifically the DB2 Family. Figure 1.1 shows IBM's application framework for e-business.

Figure 1.1. The E-Business Application Framework

The DB2 family executes on pervasive devices, Intel, UNIX, AS/400, and mainframe platforms (OS/390, z/OS, and Linux). Supported operating environments include: OS/2, Windows 95/98/2000/NT, Linux, AIX, HP-UX, Sun Solaris, NUMA-Q, OS/400, VSE/VM, OS/390, and z/OS. The DB2 code base is optimized for each platform to ensure maximum performance. The SQL API is common to all platforms, which allows applications written on one platform to access data on any platform. Internally, DB2 on the OS/400, VM/VSE, and OS/390 differ from DB2 on the UNIX and Intel platforms, but it is the common SQL API that enables applications to work together. The DB2 code base on Intel and UNIX platforms are identical.

DB2 provides seamless database connectivity using the most popular network communications protocols, including NetBIOS, TCP/IP, IPX/SPX, Named Pipes, and APPC.


006 - DB2 and E-Business

Chapter 1. Product Overview

DB2 and E-Business

As a core component of IBM's e-business cycle, DB2 is a catalyst for delivering applications that transform a company's operations. Transform is the process that converts a business to an e-business; common applications in this area include: electronic commerce, Enterprise Resource Planning (ERP), Customer Relationship Management (CRM), and Supply Chain Management (SCM). Build is the process of exploiting the integrated Java and multimedia features of DB2. Run is the part of the e-business cycle that ensures performance and scalability; this is especially important with the new Internet-based companies. Finally, with respect to leveraging the data assets, DB2 offers a variety of business intelligence tools to enable end-users to make more effective business decisions. Figure 1.2 Shows a diagram of the e-business cycle.

Figure 1.2. The E-Business Cycle

In leveraging information in an e-business environment, IBM's Enterprise Information Portal (EIP) provides a secure foundation for a single point of access to diverse information, business processes, and expertise. Today's high demand for complete and correlated information requires portal access not only to structured transactional and warehouse data, but also to a broader range of content, including XML, HTML, host computer-generated output, images, and audio/video. The IBM Enterprise Information Portal offers access to business data from sources such as spreadsheets, document libraries, company literature, databases, data warehouses, and unstructured information from Web pages. The information can also be searched using parametric or contextual search technologies, with results aggregated across multiple sources and relevant information presented in a context tailored to the user's needs. Figure 1.3 shows a picture of IBMs Enterprise Information Portal.

Figure 1.3. IBM's Enterprise Information Portal

    Team-Fly
    Top


007 - DB2 Universal Database

Chapter 1. Product Overview

DB2 Universal Database

The DB2 Universal Database family consists of many different platforms that can coexist in a distributed environment:

  • z/OS or OS/390

    • This offering is the largest of the DB2 family, often used as an enterprise server handling some of the largest applications in the world. The System/390 is IBM's largest hardware platform.

  • Enterprise Edition

    • This offering is often used to build e-business applications and to support large departmental applications. It offers the most connectivity options and can share data with third-party databases and DB2 on heterogeneous platforms.

  • Workgroup Edition

    • This offering is often used in smaller departmental applications or for applications that do not need access to remote databases on the OS/400, VM/VSE, or OS/390 platform.

  • Enterprise Extended Edition

    • This offering is used most often to support very large databases. Popular applications include supporting large data warehouses. By providing intra- and interquery parallelism, databases can scale to multiple terabytes. DB2 UDB EEE can exploit clusters or massively parallel hardware architectures.

  • Personal Edition

    • This full-function database offering is for standalone use and will not accept remote database requests. This offering is available on Windows, OS/2, and Linux.

  • Satellite Edition

    • This offering is for single users and has a smaller footprint than Personal Edition. It will not accept remote database requests. This offering is available on the Windows platform.

  • Everyplace

    • This is a mobile computing offering that gives mobile workers access to DB2 data sources in the enterprise through handheld devices such as personal digital assistants (PDAs) and handheld personal computers (HPCs).

Figure 1.4 shows the different DB2 platforms.

Figure 1.4. DB2 Universal Database

DB2 Everyplace

DB2 Everyplace (or DB2E) is a tiny "fingerprint" database of about 100K. It is designed for low-cost, low-power, small form-factor devices such as personal digital assistants (PDAs), handheld personal computers (HPCs), or embedded devices. DB2 Everyplace runs on devices that use the Palm Computing® Platform, Windows CE, the EPOC operating system, and the QNX Neutrino. DB2 Everyplace provides a local data store on the mobile or embedded device for storing relational data from elsewhere in the enterprise. Relational data can be synchronized to the handheld device from other DB2 data sources such as DB2 Universal Database for UNIX, OS/2 and Windows NT, DB2 for OS/390, and DB2 for AS/400. DB2 Everyplace with IBM Mobile Connect will also synchronize data from other ODBC-compliant data sources such as Oracle and Microsoft.

The DB2 Everyplace Sync Server mobilizes professionals with e-business information anywhere, any time. It extends the power of DB2 to a wide range of handheld devices, such as those running the Palm Pilot.

Figure 1.5 shows a picture of DB2 Everyplace.

Figure 1.5. DB2 Everyplace

DB2 Satellite Edition

DB2 Satellite Edition forms part of a DB2 solution to support systems that operate disconnected from the corporate system for the majority of the time and connect occasionally to the corporation's central database to exchange data.

DB2 Satellite Edition is a full-function, high-performance DB2 database especially designed for occasionally connected workers using systems running Windows operating systems. The satellite system itself does not require the user to manage the database. This means the end-user is free to focus on business results—the user does not even need to know a database is installed.

The application areas that can benefit from this solution include contract management, insurance application automation, securities marketing, and automobile insurance claims processing. In addition, large-scale branch office deployments are being used to provide automation to franchise stores, insurance agents, and regional offices of large corporations. In these kinds of environments, the applications are custom-built, purchased, or a combination.

The satellite environment administration model minimizes the cost of administering a large number of systems by administering collections of systems running the same application and database in groups. Scripts stored at a central administration control point are used to accomplish administrative tasks on the systems in a group.

This model also allows for centralized problem determination.

Replication administration uses the same model. Replication subscription definitions are stored in a central control server as scripts. When executed at the satellite system, the scripts set up each satellite for replication with a corporate data store. DB2 Satellite Edition's small footprint and application compatibility with the rest of the DB2 family make it an ideal candidate for the delivery of distributed applications on systems that will occasionally connect with corporate data stores to exchange data.

This product is available only on the Windows platform. Figure 1.6 shows the DB2 Satellite environment.

Figure 1.6. DB2 Satellite Environment

DB2 Personal Edition

DB2 Personal Edition (or DB2 PE) is a full-function database that enables a single user to create databases on the workstation on which it was installed. It can be used as a remote client to a DB2 server, since it also contains the DB2 client components.

The DB2 Personal Edition product is often used by end-users requiring access to local and remote DB2 databases or developers prototyping applications that will be accessing other DB2 databases.

Figure 1.7 shows an example of a DB2 Personal Edition installation. In this example, the user can access a local database on his or her desktop machine and access remote databases located on the database server. From the desktop, the user can make changes to the database throughout the day and replicate those changes as a client to the remote database on the DB2 server.

Figure 1.7. DB2 Personal Edition

DB2 Personal Edition includes graphical tools (all of the other products include these tools as well via the DB2 Administration Client component) that enable a user to administer, tune for performance, access remote DB2 servers, process SQL queries, and manage other servers from a single workstation. This product is available on Windows, OS/2, and Linux.

DB2 Workgroup Edition

DB2 Workgroup Edition (or DB2 WE) is designed for use in a LAN environment. It provides support for both remote and local clients. A server with DB2 Workgroup Edition installed can be connected to a network and participate in a distributed environment, as shown in Figure 1.8.

Figure 1.8. DB2 Workgroup Edition with Remote Clients

In Figure 1.8, App1 and App2 are local database applications. Remote clients can also execute App1 and App2 if the necessary setup has been performed. A DB2 application does not contain any specific information regarding the physical location of the database. DB2 client applications communicate with DB2 Workgroup Edition using a supported distributed protocol. Depending on the client and server operating systems involved, DB2 Workgroup Edition supports the TCP/IP, Net-BIOS, IPX/SPX, Named Pipes, and APPC protocols.

DB2 Workgroup Edition includes DB2 Extenders which provide the ability to manipulate data outside of conventional rows and columns, and Net.Data. Net.Data is a product that allows you to build Internet-ready applications that store data in DB2 databases. DB2 Workgroup is available on the Windows, OS/2, AIX, Solaris, HP-UX, and Linux platforms.

DB2 Enterprise Edition

DB2 Enterprise Edition (or DB2 EE) is fully Web enabled, scalable from single processors to symmetric multiprocessors and to massively parallel clusters; it supports unstructured data such as image, audio, video, text, spatial, and XML with its object relational capabilities.

It includes all the functions found in DB2 WE. Applications for DB2 Enterprise Edition can scale upward and execute on massively parallel clusters or can scale downward with applications executing on single-user database systems.

DB2 Enterprise Edition is available on the Windows, OS/2, Linux, and UNIX platforms.

DB2 Enterprise-Extended Edition

DB2 Enterprise-Extended Edition (or DB2 EEE) is the scalability option that enables DB2 to partition data across clusters or massively parallel computers. It includes all the functionality found in DB2 EE and adds to it. To the end-user or application developer, the database appears to be on a single computer. All SQL statements are processed in parallel, thus increasing the execution speed for any given query. DB2 Enterprise-Extended Edition is available on the Windows, AIX, Solaris, HP-UX, and NUMA-Q platforms. Figure 1.9 shows how a query is processed in parallel across partitions with DB2 EEE.

Figure 1.9. DB2 Processing a Query in Parallel

DB2 for OS/390 and z/OS

DB2 for OS/390 and z/OS is a relational database management system that is the foundation of many e-business, business intelligence, and mission-critical systems. It is the primary focus of this certification guide. DB2 for OS/390 and z/OS is the largest of the DB2 family, often serving as an enterprise server handling many of the largest applications in the world. The environment provided by System/390 and either OS/390 or z/OS is IBM's largest and most powerful, providing the most scalable and available platform.


008 - DB2 Middleware and Connectivity

Chapter 1. Product Overview

DB2 Middleware and Connectivity

DB2 is a very open database and provides a variety of options for connecting to DB2 and non-DB2 databases.

  • DB2 Clients

    • Client code is required on workstations for remote users to access a DB2 database.

  • DB2 Connect

    • This product provides support for applications executing on UNIX and Intel platforms to transparently access DB2 databases on the OS/400, VM/VSE, OS/390 and Linux on S/390 environments. Note that DB2 Connect is not required to access DB2 for any of the UNIX or Intel platforms.

  • DB2 Net.Data

    • This provides the ability for a Web browser to access relational or non-relational data. This middleware provides an easy way to embed database calls into a macro language.

  • DB2 DataJoiner

    • This product allows DB2 clients to access, join, and update tables from heterogeneous databases, such as Sybase, Informix, and Microsoft SQL Server.

  • IBM Classic Connect

    • This product is a separately orderable component of DataJoiner that provides read-only access to non-relational data stored in IMS databases and VSAM data sets on MVS.

  • DB2 Relational Connect

    • This product allows DB2 clients to access and join tables from DB2 and Oracle databases.

  • WebSphere Application Server

    • This Application Server is shipped with DB2 Enterprise Edition and allows developers to use Java as a platform in a transaction processing environment.

DB2 Universal Database Clients

DB2 supports a variety of clients. These clients include:

  • Windows (2000, NT, 98, 95 and 3.x)

  • OS/2

  • AIX

  • HP-UX

  • SGI-IRIX

  • LINUX

  • Solaris Operating Environment

  • Web Browsers

DB2 Connect

The DB2 Connect product allows clients to access data stored on data servers that implement the Distributed Relational Database Architecture (DRDA). The target database server for a DB2 Connect installation is known as DRDA Application Server.

Note

The most commonly accessed DRDA application server is DB2 for OS/390 and z/OS.

DB2 Connect supports both the TCP/IP and APPC DRDA communication protocols. The protocol supported depends on the DRDA application server being connected to and the version of the host software being run. For instance, a DB2 Connect server acting as a DRDA Application Requester to a host DRDA Application Server can connect to DB2 for OS/390 at version 5.1 and higher. Any of the supported network protocols can be used for a DB2 client to establish a connection to the DB2 Connect Server.

Some of the major capabilities provided by DB2 Connect include:

  • Supports for ODBCS, OLE DB, CLI, JDBC, and SQLJ applications

  • Distributed Join access across all DB2 databases

  • Distributed Join with Oracle (via Relational Connect)

  • Connection Pooling

  • S/390 Sysplex Exploitation for failover and load balancing

Some of the common uses of DB2 Connect are:

  • Web-enabling DB2 OS/390 by providing browsers direct access

  • Leveraging Microsoft application written using ADO, ODBC, or OLE DB can transparently access DB2 on mainframe platforms

  • Offloading mainframe development cycles

The database application must request the data from a DRDA Application Server through a DRDA Application Requester.

Note

The DB2 Connect product provides the DRDA Application Requestor functionality.

The DRDA Application Server accessed using DB2 Connect could be any DB2 Server on OS/390, VM, VSE, or OS/400. If TCP/IP is the protocol of choice, then the following are prerequisites:

  • DB2 for OS/390. Only version DB2 5.1 or higher supports TCP/IP in a DRDA environment.

  • DB2 for OS/400. Only OS/400 version 4.3 or higher supports TCP/IP.

  • DB2 Server for VSE/VM. Only version DB2 6.1 or higher for VM supports TCP/IP.

DB2 Connect is available as a server (Enterprise Edition) and a single-user package (Personal Edition). The DB2 Connect Enterprise Edition product provides the ability for multiple clients to access host data. A DB2 Connect server routes the ability for multiple clients to access host data. A DB2 Connect server routs each database request from the DB2 clients to the appropriate DRDA Application Server. The remote client communicates with the DB2 Connect server using any of the supported communication protocols. DB2 Connect Personal Edition is available on the following platforms: Windows, OS/2, and Linux. It provides access to the host database from the system where it is installed.

Figure 1.10 shows the DRDA application flow.

Figure 1.10. DRDA Application Flow

DB2 Net.Data

Net.Data provides high-performance Web applications with robust application development function. Net.Data exploits Web server interfaces (APIs), providing higher performance than common gateway interface (CGI) applications. Net.Data supports client-side processing as well as server-side processing with languages such as Java, REXX, Perl, and C++. Net.Data provides conditional logic and a rich macro language, support for Java, JavaScripts, and XML.

In a Net.Data environment, users connected to the Internet or an Intranet can access database applications. Users can either select automated queries or define new ones that retrieve specified information directly from a variety of data sources. The results are returned to the Web browser in HTML format. Net.Data not only allows access to DB2 data, but also offers native access to Oracle and Sybase, flat files, and other data sources that support ODBC.

Figure 1.11 shows some of the databases that can be accessed from the Internet using Net.Data.

Figure 1.11. DB2 Net.Data

DB2 DataJoiner

DB2 DataJoiner provides applications to both DB2 and multi-vendor relational databases such as Microsoft, Sybase, and Informix. It acts as middleware for access to heterogeneous data sources. With a single SQL statement, an application can transparently access, join, and update data located across multiple data sources. The access can be made without needing to know the location of the data or the specifics of the SQL dialects for the data source.

DataJoiner can run on Windows NT, Windows 2000, AIX, Solaris, or HP-UX.

Figure 1.12 shows some of the functionality of DataJoiner.

Figure 1.12. DB2 DataJoiner

DB2 Relational Connect

DB2 Relational Connect provides applications access to both DB2 and Oracle. This is particularly useful if there is a heterogeneous database environment where DB2 and Oracle coexist.

DB2 Relational Connect is targeted for environments where DB2 and Oracle need to share data.

Figure 1.13 shows an example of using DB2 Relational Connect to access both DB2 for OS/390 and Oracle in the same query.

Figure 1.13. DB2 Relational Connect

IBM WebSphere Application Server

IBM WebSphere Application Server is built on an open Java-based platform that enables applications to leverage existing application resources and access various databases, including DB2. It is part of the WebSphere family of products that help you develop and manage high-performance Web sites, and to integrate those Web sites with new or existing non-Web business systems. Some of the capabilities of WebSphere Application Server include use of:

  • Java servlets

  • Java Server Pages

  • XML

These allow you to quickly transform static Web sites into vital sources of dynamic Web content. Enterprise Java Beans (EJB) can also be used for implementing EJB components that incorporate business logic. WebSphere Application Server is packaged with DB2.


009 - DB2 Application Development

Chapter 1. Product Overview

DB2 Application Development

DB2 offers a rich application development environment that allows the developer to build databases supporting requirements from e-business and business intelligence applications. Many of these tools are integrated with the database; the major tools will be reviewed.

  • Stored Procedure Builder

    • Enables the creation, testing, and debugging of stored procedures on local and remote DB2 servers.

  • DB2 Relational Extender

    • Enables the SQL API to access unstructured data types, including text, image, audio, video, and XML.

  • DB2 OLAP Server Starter Kit

    • Provides the ability to build OLAP cubes using DB2 as the relational data store; a higher-end version, DB2 OLAP Server, is available when there are more users.

  • DB2 Data Warehouse Center

    • Provides the ability to build data marts/warehouses by automating the processes involved in managing, refreshing, moving, and transforming data, including the ability to define star schema model.

  • DB2 Data Warehouse Manager

    • Provides all the capabilities of the Data Warehouse Center, but includes support for more data sources and includes the Information Catalog, Query Patroller, and QMF for Windows.

  • QMF–Query Management Facility

    • Provides an integrated, powerful, and reliable query and reporting tool set for DB2 data. There is also an optional QMF for Windows.

DB2 Stored Procedure Builder

The DB2 Stored Procedure Builder is a graphical Windows application that runs on Windows, AIX, and Solaris that supports the rapid development of DB2 stored procedures written in either Java or SQL Procedure Language (the SQL Procedural Language is part of the SQL3 standard). It can work as a Microsoft Visual Studio plug-in, or with Microsoft Visual Basic, IBM Visual Age for Java, or as a standalone. The Stored Procedure Builder (SPB) supports the entire DB2 family from a single development environment. It has facilities for debugging and deploying stored procedures.

Figure 1.14 shows the debugging facility in the Stored Procedure Builder.

Figure 1.14. DB2 Stored Procedure Builder

DB2 OLAP Server

The DB2 OLAP server can be used for developing analytic applications in order to give your organization an edge on the competition. The DB2 OLAP server presents the data in business dimensions such as category of data or time period, allowing you to ask questions intuitively. The DB2 OLAP server helps you to do fast, intuitive, multidimensional analysis.

There are many advantages offered by the DB2 OLAP Server because the OLAP server integrates the popular Hyperion Essbase OLAP product with the e-business-ready IBM Universal Database family, including DB2 for OS/390. The DB2 OLAP server allows you to analyze data on OS/390 using popular Web and desktop tools such as Microsoft Excel, Lotus 1-2-3, Hyperion Wired for OLAP, Cognos PowerPlay, Brio products and Business Objects. It also works with Web-ready tools to support full-function access to the analytical applications. These tools, such as Wired for OLAP (a product from Hyperion), allow you to deliver applications on the Web to different users. The OLAP Server also provides concurrent read/write support for "what-if" analysis for tasks such as forecasting and budgeting. OLAP Server combines the flexibility of a relational database management system and the performance of a multidimensional store. Figure 1.15 shows an example of a multi-diminsional cube.

Figure 1.15. Multidimensional Cube

The DB2 OLAP Server runs on:

  • OS/390

  • AIX

  • HP-UX

  • OS/400

  • Solaris Operating Environment

  • Windows

There are many applications and tools that support DB2 OLAP Server available in the marketplace, or you can create your own application.

DB2 Relational Extenders

DB2 Relational Extenders offer the ability to manipulate data outside of conventional rows and columns to include the manipulation of these types of data: text, image, audio, video, and XML. The DB2 Relational Extenders encapsulate the attributes, structure, and behavior of these unstructured data types and store this information in DB2. From the developer's perspective, the DB2 Relational Extenders appear as seamless extensions to the database and enable the development of the multimedia-based applications. The following DB2 Relational Extenders are provided by IBM:

  • Text Extender

  • Image Extender

  • Audio Extender

  • Video Extender

  • XML Extender

  • Net.Search Extender (Solaris, NT only)

  • Spatial Extender

The purpose of the DB2 Relational Extenders is to provide for the management of unstructured data through the SQL API. By preserving the current investment in relational applications and, by leveraging existing skills, new nontraditional applications can be introduced. This open environment enables developers and independent software vendors to develop and introduce their own extenders as extensions to DB2. Figure 1.16 shows an example of using Relational Extenders to access unstructured data.

Figure 1.16. DB2 Relational Extenders

The XML extender is provided with DB2 and allows you to store eXtensible Markup Language (XML) documents as a new column datatype. You also have the ability to decompose and store XML in its component parts as columns in multiple tables. In either case, indexes can be defined over the element or attribute of an XML document for fast retrieval. Furthermore, text search and section search can be enabled on the XML column or its decomposed part via DB2 Text Extender. You can also formulate an XML document from existing DB2 tables for data interchange in business-to-business environments. Figure 1.17 shows an example of using XML extenders.

Figure 1.17. XML Extender

QMF (Query Management Facility)

QMF—Query Management Facility—is an optional tightly integrated, powerful, and reliable query and reporting tool set for DB2. With QMF you can execute queries, format reports, and build procedures to perform multiple activities. QMF stores the queries, forms, and procedures in its own database so you can reuse them.

QMF for Windows

QMF for Windows is an easy-to-use query and reporting tool for publishing reports either locally or on the Internet. It also integrates with other Windows desktop tools. It can be used to access DB2 on the OS/390 via DB2 Connect. With QMF for Windows end-users can easily do the following:

  • Build queries and reports easily using its graphical interface.

  • Integrate query results with desktop tools such as spreadsheets and personal databases.

  • Rapidly build data access and update applications.

  • Exploit DB2 performance and all of its SQL capabilities.

Figure 1.18 shows a wiindow in QMF for Windows where an end user could supply table names, column names and search conditions in order to build an SQL query.

Figure 1.18. QMF for Windows

DB2 Warehouse Center and Warehouse Manager

The DB2 Warehouse Center and Warehouse Manager are tools that provide a way to access, manage, build, and control data warehouses built in DB2. The DB2 Warehouse Center provides you with agents that will manage the flow of data between the sources and targets in the warehouse. One very nice feature is that it provides the means for cleansing data, generating keys, and pivoting data by means of Java stored procedures and user-defined functions. There are also extensive reporting capabilities. An unlimited license for QMF for Windows is included in the product.

These warehouse functions are managed through a component called the Data Warehouse Center. The Data Warehouse Center is itself incorporated into the Control Center. The Data Warehouse Center includes functions to define data sources, manage data movement and transformation, populate the metadata repository, and create generic schema models, including the logical star model for the DB2 OLAP Server. Figure 1.19 shows an example of using the Data Warehouse Center to construct a process model.

Figure 1.19. Data Warehouse Center

A set of warehouse transformers is included for data cleansing, generating key columns and period tables, and inverting and pivoting data. A wide selection of statistical transformers performs the following functions: basic statistics, subtotals, rollups, cube calculations, moving averages, regression, correlation analysis, and other functions. In all, there are over 100 built-in transformers. You can also build user-defined functions to create your own transformers.

There is an important family side benefit with the DB2 Warehouse Manager. It can be used for either OS/390 to OS/390 point-to-point data movement or point-to-point to any member of the DB2 family. With federated data access provided by DB2 DataJoiner integration, the warehouse center can be used to manage the process of moving data from Oracle to DB2 as well.

When this product is purchased with DB2 for OS/390, it not only includes the workstation GUI control center, but also a full set of procedures for managing the warehouse and data natively on OS/390. The replication controls with Warehouse Center will also allow for heterogeneous data sourcing into a warehouse on any of our possible target platforms.

With DB2 Warehouse Manager for OS/390, the QMF family includes QMF for Windows, QMF for OS/390, and QMF High Performance Option (HPO), for query and management of your OS/390 warehouse.


010 - DB2 Administration

Chapter 1. Product Overview

DB2 Administration

DB2 has several tools available to help with administration of the subsystem environment and database objects. Some of these tools are optional and Windows based.

  • DB2 Installer

    • Provides a graphical user interface for installing DB2 from a workstation.

  • Control Center

    • Provides a single point of entry for controlling the entire DB2 family. Can use the Control Center to display data base objects and their relationships to each other.

  • Visual Explain

    • Gives the user a graphical tool to analyze the access paths that DB2 chooses for your SQL queries or statements.

  • Index Advisor

    • Assists in choosing an optimal set of indexes for table data.

  • DB2 Estimator

    • Provides a facility for estimating the performance of applications.

DB2 Installer

The DB2 Installer provides a graphical user interface for installing DB2 for OS/390 and z/OS from a workstation. An element of the DB2 Management Tools Package, DB2 Installer provides an alternative to the ISPF installation panels and CLISTs currently used on OS/390 systems.

The DB2 Installer application illustrates the overall installation process and keeps a graphical record of how each subsystem is defined. You can use DB2 Installer to increase your productivity. It has a "command center" from which you can install, migrate, or update DB2 for OS/390. The graphical interface follows an easy-to-read map through the entire installation process, including SMP/E, fallback, and sample jobs. In addition, it provides a graphical record of completed and uncompleted tasks by subsystem.

The DB2 Installer helps you in the following:

  • Installation

    • Prompts the user for the appropriate fields necessary to customize DB2 for OS/390 installation jobs. These installation jobs are executed from your workstation or they can be uploaded to the host for execution.

  • Migration

    • Generates DB2 for OS/390 migration jobs and makes you aware of parameters you might want to customize to take advantage of new functions. It also generates an optional set of jobs that check for catalog consistency and other migration considerations.

  • Update

    • Allows you to modify the current parameters and saves each updated set of parameters in a unique file.

There is also data sharing support so you can use DB2 Installer to enable, add, or migrate data sharing members. It also comes with a full help facility.

Figure 1.20 shows some of the GUI screens in the DB2 Installer product.

Figure 1.20. DB2 Installer

Control Center

The Control Center supports the entire DB2 family including OS/390. The Control Center can be used as a single point of entry for controlling the entire DB2 family. You can use the Control Center to display database objects and their relationships to each other. The Control Center has a GUI that allows you to easily manage local and remote servers from a single workstation. Figure 1.21 shows the hierarchy of DB2 objects as viewed through Control Center.

Figure 1.21. DB2 Control Center

The Control Center also has support for DB2 functions through the following components:

  • Menu Bar

    • Used to access Control Center functions and online help.

  • Tool bar

    • Used to access the other administration tools.

  • Objects Pane

    • This is shown on the left side of the Control Center window. It contains all the objects that can be managed from the Control Center as well as their relationship to each other.

  • Contents Pane

    • This is found on the right side of the Control Center window and contains the objects that belong or correspond to the object selected on the Objects Pane.

  • Contents Pane Toolbar

    • These icons are used to tailor the view of the objects and information in the Contents Pane. These functions can also be selected in the View Menu.

You can also launch other tools such as DB2 Warehouse Manager, DB2 Performance Monitor, DB2 Visual Explain, Command Center, Script Center, Stored Procedure Builder, and more.

Some of the types of support items in the Control Center added for the OS/390 include:

  • Utility Support

    • Support for utility wildcarding gives the ability to execute utilities against a list of objects matching a specified pattern of matching characters. There is also support for restarting utilities from the last committed phase (phase) or the last committed point (current). This is available only for utilities originally started in the Control Center.

  • Dataset Management

    • Ability to manage your datasets via the Control Center. This allows you to retrieve dataset lists, and, depending on the type of dataset, you can show, rename, and delete members. Datasets can be dynamically allocated for utilities through the use of dataset templates used to automatically generate datasets based upon a defined set of criteria.

  • DDL Generation

    • A feature used to recreate database objects and dependent objects for the DB2 catalog. The output then can be saved to a dataset (maybe for input to SPUFI) or to a workstation file.

  • Integrated SQL Assist

    • Assists in helping build standalone SQL statements or SQL statements in triggers or views. The SQL statement will then be inserted into the text area of the dialog after it is created.

Wizards

Wizards are help screens that guide you through several different activities such as creating objects. Each Wizard had detailed information available to help perform these tasks. These Wizards are integrated into the administration tools and assist in completing various tasks such as Add Database. These Wizards are invoked from the Control Center and they extract information and ask questions about the tasks being performed. They then run any necessary calculations to help determine appropriate settings.

Visual Explain

DB2 Visual Explain lets you graphically analyze the access paths that DB2 chooses for your SQL queries or statements. The graph of the access path is displayed on an IBM OS/2 or Microsoft Windows NT workstation, eliminating the need to interpret the plan table output manually. Figure 1.22 shows an example of this.

Figure 1.22. Visual Explain

This tool offers suggestions for improving the performance of your SQL queries or statements. You can change an SQL statement and dynamically explain it to see if the access path is improved by the change. You can also use it to browse the current values of the subsystem parameters.

Visual Explain issues Distributed Relational Database Architecture (DRDA) queries through a DB2 client on the workstation (via DB2 Connect) to get the information it needs.

DB2 Visual Explain helps database administrators and application developers to:

  • See the access path for a given SQL statement, graphically.

  • View statement cost in milliseconds and service units.

  • Tune SQL statements for better performance.

  • View the current values for subsystem parameters.

  • View catalog statistics for table and indexes.

  • Generate custom reports.

Index Advisor

The Index Advisor is a tool to assist in choosing an optimal set of indexes for table data. This tool helps to determine the best indexes for a problem query or for a particular set of queries. It is helpful for testing an index on a workload without having to create the index.

This process is performed by creating a modeling database for DB2 for OS/390 and z/OS on a DB2 for UNIX, Windows, OS/2 workstation using configuration parameters to mimic DB2 for OS/390. One way to invoke this tool is through the Control Center.

This tool works on a set of dynamic SQL statements which have to be processed over a given period of time, and it handles SELECT, INSERT, UPDATE, and DELETE statements. In addition, it uses statistics from the system and user input to evaluate strategies for given workloads. It also scans the catalog tables holding SQL from bound applications and can make determinations and suggestions.

DB2 Estimator

DB2 Estimator is an easy-to-use, standalone tool for estimating the performance of DB2 applications. It can be run on PCs that support Microsoft Windows 3.1, 95, 98, NT, 2000, or OS/2.

It supports simple table sizing to a detailed performance analysis of an entire DB2 application. DB2 Estimator saves time and lowers costs by allowing you to investigate the impact of new or modified applications to your production system before you implement them. DB2 Estimator helps answer questions about topics such as:

  • Elapsed time for an SQL statement that fetches N rows

  • Processor resource used during an N-way join

  • Impact of adding and dropping an index from a table

  • System support ability for an anticipated increase in workload

  • Effects of doubling processor resource on transaction response time

  • Storage required for a new table and its indexes

  • Index support for predicates

  • Effect of doubling table size on performance

  • Ability of application to complete within the batch window

  • Execution time of utility job

  • Effects of data compression

  • Effects of data sharing

  • Effects of triggers

DB2 Estimator lets you change your definitions easily to help you evaluate alternative designs. By comparing the cost and performance of these designs, you can determine which design provides the optimum cost/performance ratio before you invest valuable time and resources coding your application or creating a real database. It helps you build more efficient databases by building a model of your system, based on your data knowledge, with reasonable access paths for your SQL statements. If actual DB2 information is available through DB2 Explain and/or DB2 Performance Monitor, you can tune the model by overriding the model assumptions. You can also import table, index, and SQL statements from your DB2 database to DB2 Estimator, and then modify them to suit your modeling requirements.


011 - Data Management Tools

Chapter 1. Product Overview

Data Management Tools

IBM has made a commitment to delivering a quality set of tools as viable alternatives to third-party tools. These tools are offered in the Data Management set of tools and are designed to perform at the level that today's systems require. The tools are divided into four categories: Database Administration, Database Performance Management, Database Recovery and Replication, and Database Application Management.

  • DB2 Administration Tool

    • Provides a comprehensive set of functions to efficiently and effectively manage DB2 environments.

  • DB2 Forms

    • Provides drag-and-drop solution for quickly building robust, user-friendly e-business front-ends to any DB2 database.

  • DB2 High Performance Unload

    • Offers sequential reading and the capability to access your DB2 data at top speed.

  • DB2 Log Analysis

    • Provides immediate pinpoint focus to recovering data changes in real time during normal operations.

  • DB2 Object Restore

    • Allows for the restoration of dropped objects and all related dependencies automatically.

  • DB2 Buffer Pool Tool

    • Helps performance analysts evaluate tuning alternatives and achieve optimal database performance with minimal effort.

  • DB2 Performance Monitor

    • Helps you get optimum performance from your database management system and provides useful support for resolving critical performance issues.

  • DB2 Query Monitor

    • Provides the ability to monitor queries and to quickly pinpoint and resolve problems without going through large amounts of output.

  • DB2 SQL Performance Analyzer

    • Gives you the ability to estimate the costs of database queries prior to execution.

  • DB2 DataPropagator

    • This product provides replication capabilities for DB2 databases and is integrated in DB2 on the UNIX and Intel platforms.

  • DB2 Recovery Manager

    • Significantly speeds up and simplifies the access to and recovery of databases for productive use throughout an enterprise.

  • DB2 Row Archive Manager

    • Provides a complete solution for selecting, archiving, managing, and retrieving aged data.

  • DB2 Bind Manager

    • Helps reduce the time and cost of connecting new application programs to a database.

  • DB2 Web Query Tool

    • Lets end users access, create, share, and execute SQL queries through their browsers against multiple IBM DB2 databases.

Each of these tools is further described in the sections that follow.

Database Administration

DB2 Administration Tool for OS/390

IBM DB2 Administration Tool for OS/390¨ Version can help you derive the maximum performance and results from your DB2 system, dramatically reducing the effort required to support DB2. This tool provides a comprehensive set of functions that help manage DB2 environments efficiently and effectively. It is an interactive, intuitive, and easy-to-use tool that provides a cost-effective solution for a full range of database administration functions such as:

  • Display and interpret objects in the DB2 catalog.

  • Execute dynamic SQL statistics.

  • Issue DB2 commands against objects.

  • Manage SQLIDs.

  • Perform complex performance and space utilization queries.

  • Reverse engineer objects in the DB2 catalog.

  • Display and cancel threads.

  • Manage the CDB and RLIMITS tables.

  • Manage your DB2 environment effectively.

It also has Alter and Migrate capabilities that allow you to change name and owner of tables, modify the definitions of table columns, change the sequence of columns in a table, drop columns, insert new columns, create a separate DB2 test system, move a test system into a production system, and consolidate two separate database systems into one.

It integrates with other DB2 utilities and simplifies creation of DB2 utility jobs. It also has catalog update functions and has facilities for issuing any DB2 command, issuing GRANT and REVOKE on the objects, performing BIND, REBIND, and FREE operations on selected plans and packages.

DB2 Forms for OS/390

DB2 Forms for OS/390 is IBM's drag-and-drop solution for quickly building robust, user-friendly e-business front ends to any DB2 database. It provides a single development environment that enables users to simultaneously build both Java-based and 32-bit Windows-based applications for Web sites and intranets from existing code. The server-based application code in DB2 Forms allows a virtually unlimited number of users to open applications. Its centralized management provides excellent concurrency and control over application versions and user permissions. This tool makes it easier than ever to rapidly build, edit, and share front-end applications for data warehouses over the Internet.

DB2 Forms provides additional support through enhanced data editing and referential capabilities, full-screen table editing and new applications. It is delivered as four modules:

  • DB2 Forms Developer

    • Provides a rapid "drag-and-drop" development environment for creating custom forms that end users can access from virtually any environment or location. Applications can be made available only to their creators or to all users, and made fully visible within the Web page, or made available as a link in a Web page.

  • DB2 Forms Console

    • Lets you set up DB2 servers to centrally store Java- and Windows- based DB2 Forms applications for access by end-users.

  • DB2 Forms User

    • Is the Windows client run-time that provides users with access to Windows-based DB2 Forms applications. This function allows users to select and launch any number of applications from the catalog of custom forms without having to install additional code on their workstations.

  • DB2 Forms Java Player

    • Can be installed on an HTTP server to bring Java-based DB2 Forms applications to Web browsers.

DB2 High Performance Unload for OS/390

DB2 High Performance Unload offers sequential reading and the capability to access your DB2 data at top speed. It can scan a tablespace and create output files in the format in which you need them. All you have to do is select the criteria. You can elect from many types of conversions, giving your output the desired appearance such as delimited file for export to another platform or DSNTIAUL compatible.

This tool provides the following benefits:

  • Increased speed for unload functions

  • High-speed unloads of DB2 data using native VSAM

  • Ability to unload from image copies as well as active tables

  • Multiple output files during a single unload with minimal additional cost

  • Multiple output formats, including the opportunity to tailor your own

  • Comprehensive and powerful SELECT statement

  • More efficient unloads of DB2 data

  • Help with batch windows constraints

  • Ability to unload multiple tables each to a separate file or a single table many times

Large scan performance can become critical when several unloads have to read the same tablespace concurrently. Performance symptoms can be caused by multiple reads of the same DB2 pages. Sometimes there are channel conflicts. IBM DB2 High Performance Unload can help solve these problems.

You can code as many select statements as you want for any tables belonging to the same tablespace. Different output files can be created during the same unload process at very little additional cost. Multiple tables can be unloaded at once, each to its own file, as long as all the tables belong to the same tablespace. You can also unload the same table many times with different SELECT statements, using only one tablespace scan. You can also run this utility against incremental image copies or any full image copy as well as against active tables.

DB2 Log Analysis for OS/390

The DB2 Log Analysis tool provides you many benefits in the area of recovery and availability. It allows you to locate, undo, and redo changes with ease and quickly isolate accidental or undesired changes to your database tables. You can then selectively reverse these changes from an easy-to-use, menu-driven interface. The tool also allows you to selectively redo specific transactions after large-scale reversals of data changes, and locate changes made to one database and copy those changes to another database. These types of activities can be performed while the DB2 systems continue to run as normal. This helps to minimize downtime from conventional recovery sessions and maximize uptime for e-business availability.

DB2 Log Analysis Tool provides a simple, menu-driven interface that lets you quickly specify the scope and magnitude of data changes you want to view—by dates, users, tables, and much more. Valuable time and system resources are further saved by support of data compression in DB2 for OS/390 and by the elimination of the need for DATA CAPTURE CHANGES and associated administrative, I/O, and storage overhead. Once you've spotted changes that must be reversed, they're easily undone with the tool. It generates the necessary SQL which is sent to a file for execution by any method preferred by the database administrator.

The tool has a comprehensive set of reports. Some of the data change reporting features include:

  • Set filtering criteria for reporting data changes by authid, job name, plan name, table, database, tablespace, or data ranges.

  • Produce summary reports from which drill-down detailed reports can be obtained.

  • View filtered summary of desired date range, tables, and more.

  • View detailed report showing all data change events including original state, current state, INSERT, UPDATE, or DELETE executed.

  • Save filtered reports as JCL to be run at will.

DB2 Object Restore for OS/390

IBM DB2 Object Restore for OS/390 can restore dropped objects and all related dependencies automatically, even if they no longer exist in the DB2 catalog. This can include dropped databases, tablespaces, tables, indexes, table and column authorizations, and data.

This tool eliminates the need for a duplicate shadow copy of the catalog to recover objects. The result is that DB2 Object Restore saves DASD space. With this tool you can feel secure in cleaning up your DB2 system because you can restore discarded DB2 objects, if necessary.

Some highlights to DB2 Object Restore include:

  • DDL generation, which recreates the dropped objects

  • Versioning repository to generate the DDL and to run analyses on the repositories to map old DBID, PSID, and OBIDs to new IDs for recreated objects

  • Move/copy object feature, which allows you to move the objects from one DB2 subsystem to another

  • Restore object hierarchy, which rebuilds automatically the structure of dependent objects

Database Performance Management

DB2 Buffer Pool Tool for OS/390

The IBM DB2 Buffer Pool Tool is designed to help performance analysts evaluate tuning alternatives and achieve optimal database performance with minimal effort. While DB2 online performance monitors can tell you when something is wrong, it can often be difficult and time-consuming to determine the most appropriate remedy.

This optional tool can take the guesswork out of buffer pool tuning, which is one of the most critical factors in improving DB2 performance. The tool provides you with comprehensive statistical data and easy-to-read reports and graphs for all pools and objects. It uses the IFI interface and its own data collection facility to avoid the costly overhead associated with other tools and easily and efficiently processes a meaningful data collection interval.

By using this tool you have the capability to shorten problem analysis time, optimize memory usage, save CPU cycles, improve response times, and reduce the cost of DB2 workloads.

It also provides unique and highly detailed statistical analysis. It shows you exactly how the pool and objects are used. The extensive statistics include:

  • System and application hit ratios

  • Get page activity and I/O rates

  • Random access activity

  • Prefetch (including dynamic and list) access activity

  • Read and write I/O activity

  • Average and maximum I/O elapsed times

  • Random page hiper pool retrieval

  • Average page residency times

  • Average pages/write

This tool can help you clearly see which objects perform poorly, which ones perform well, and the impact each has on its current pool. It will also help you determine which objects will benefit the most from changes.

DB2 Performance Monitor for OS/390

DB2 Performance Monitor is designed to help you get optimum performance from your database management system and provides useful support for resolving critical performance issues. It gives you the ability to analyze, control, and tune the performance of DB2 and DB2 applications. It also includes a real-time online monitor, a wide variety of reports for in-depth analysis and an EXPLAIN feature to analyze and optimize SQL statements. It can be executed using the ISPF panels or on a Windows workstation.

DB2 Performance Monitor can help you monitor:

  • DB2 and DB2 applications online and by Batch reports alerted immediately when problems occur

  • Usage of workstations which allow parallel monitoring of different DB2 subsystems from one point of control.

  • The status of a DB2 subsystem and its applications while they are currently active, or investigate events and performance problems that happened in the past

  • Individual data sharing members or entire data sharing groups in one or batch modes

  • Applications running in a query parallelism environment on a Parallel Sysplex cluster, even if the parallel tasks are executed on different processors

This tool also allows you to analyze performance problems online or through a vast set of detailed reports and obtain tuning recommendations. It also has the ability to run EXPLAINs, recognize periodic/event exceptions and take appropriate actions by means of a user exit, restrict user authorization to specific threads, recognize trends and anticipate potential bottlenecks, and save DB2 trace data.

DB2 Query Monitor for OS/390

The DB2 Query Monitor for OS/390 lets you quickly pinpoint the kinds of activity that matter to you. It provides both real-time and historical views into query activity throughout your DB2 subsystems. It also provides extensive choice in determining what data is gathered during monitoring, when it is gathered, and about whom. When the thresholds you specify are crossed, activity is lagged for rapid drill-down and, if needed, termination. At intervals also under your control, real-time activity is saved from memory to disk and, if desired, from disk to database. So, even while your world of DB2 subsystems is completely open to your view (both as current activity and histories), you incur the lowest system monitoring overhead.

The DB2 Query Monitor also allows you to:

  • Navigate database activity with ease.

  • Identify trouble in real time before DB2 resources are wasted.

  • Locate problem queries during execution.

  • Identify users and locations issuing problem queries.

  • Set thresholds that automate early warnings and corrective action.

  • Terminate problem queries.

  • Fix processes that are hurting DB2 availability.

This tool has the ability to have multiple monitoring agents, each capable of watching up to 64 DB2 subsystems, to track the activity you specify. You can customize it also via user-defined profiles that highlight real-time events of interest. You also have the ability to set thresholds by CPU time, elapsed time, get page requests, rows returned, delay time, number of SQL calls, lock failures, lock escalations, invalid SQLCODES, or application abends.

DB2 SQL Performance Analyzer for OS/390

The DB2 SQL Performance Analyzer makes it easier to reduce the escalating costs of database queries by estimating their cost prior to execution. Many times applications fail to perform according to needs, particularly at high transaction volumes. Frequently, the problem is that database queries run too long, causing the batch production window to shrink, which frustrates users and adds to costs.

This tool can perform data modeling during the design phase of an application, determine which SQL statements in major production are the most resource-intensive, and provide a tool that allows efficient queries to be developed and saved. It also can be used to analyze existing SQL statements that have demonstrated performance problems.

It can also approximate how long queries will take before database resources are unnecessarily consumed and even estimate the cost of a query before it is delivered. Estimates are given in familiar units, such as CPU time, I/O count, elapsed time, or monetary cost. This allows you to perform "what-if" scenarios to determine what performance can be obtained now and in the future with various database design alternatives and production volumes.

Database Recovery and Replication Management

DB2 DataPropagator for OS/390

DB2 DataPropagator offers numerous features for a comprehensive replication solution such as:

  • A highly efficient architecture that automatically captures and asynchronously propagates data changes to the DB2 family of databases.

  • A high-performance log-based change-capture component, which captures all DB2 updates without application changes.

  • Powerful data transformation using standard SQL, including multi-table joins and stored procedures.

The IBM replication solution enables full-refresh and change propagation across a hetereogeneous database environment. DB2 DataPropagator supports a wide variety of databases including DB2, Oracle, Microsoft, Sybase, Informix, IMS, Lotus Notes, and flat files. Replication is also a core technology that enables mobile users to keep their data synchronized with corporate data residing on a DB2 server.

Some of the highlights for this product include:

  • Replicating information across your enterprise.

  • Capturing data changes against a source database automatically and propagates the changes to a target database.

  • Replicating across MVS, OS/390, z/OS, VM, VSE, AS/400, OS/2, AIX, HP-UX, Sun's Solaris Operating Environment, Linux for S/390, NUMA-Q, and Microsoft Windows platforms.

  • Uniting your distributed, heterogeneous databases into an integrated database environment.

The IBM DB2 DataPropagator can help you improve data availability and operational database performance. By replicating data between your central database and regional transactional databases, you can make business data available to the regional databases for prompt transaction processing. It also has sophisticated data transformation capabilities, maximizing the value of your data warehouse and business intelligence systems. It will allow you to derive data using arithmetic (Boolean or any valid SQL expression), aggregate data to produce sums or averages using SQL column functions, convert data by translating encoded fields to descriptive fields, and consolidate data through joins or unions.

DB2 Recovery Manager for OS/390

The IBM DB2 Recovery Manager function significantly speeds and simplifies the access to and recovery of databases for productive use throughout an enterprise. It enables the versatile database recovery functions in e-business transaction environments. It can be used to recover a DB2 or IMS application individually and coordinate the recovery of both DB2 and IMS data to a common point, cutting the time and cost of data recovery and availability. This ability eliminates the error-prone complexity of managing different logs, utilities, and processes to do recovery from both databases.

DB2 Recovery Manager works with either DB2 or IMS or with both. It uses image copies for both databases and establishes synchronization points in each called a Virtual Image Copy (VIC) to which recovery may be done. After an application is run in which both DB2 and IMS are accessed, DB2 Recovery Manager is invoked to establish a VIC. It also establishes a quiesce point and notes where the point is located on the respective logs. During recovery, the user specifies the need to recover to this VIC, and DB2 Recovery Manager applies the appropriate image copies and causes the log to be applied up to this point.

You also have the option to automate the recovery of resources. This tool will generate the Job Control Language (JCL), locate the proper image copies, and control the execution of jobs for either DB2 or IMS.

DB2 Row Archive Manager for OS/390

DB2 Row Archive Manager is a complete solution for selecting, archiving, managing, and retrieving aged data. By applying appropriate judgment on the need to access your aged data, you can use DB2 Row Archive Manager to reduce the required DASD storage, improve your DB2 performance, and reduce the cost of your overall DB2 environment. This can be achieved by archiving aged DB2 data and doing this archiving at a table row or column level to reduce storage requirements. Performance improvements can be seen by removing aged data, which reduces time spent on table scans and decreases index size. This would also reduce the amount of data processed and time consumed by costly maintenance activities such as reorganizations, image copies, and recoveries.

The DB2 RAM administrator implements a set of rules to determine what data is eligible to be archived. There are no programs to write. Only the selected rows are archived at any specified time. DB2 RAM also allows you to determine which columns of a row should be archived. Referentially linked rows from related tables or systems of related tables can be archived as a unit. This archived data can be selectively retrieved on demand. The Row Archive Manager catalog is used to locate and retrieve the needed data, usually running as an overnight batch job.

This tool also includes additional utilities such as REMOVE, which can discard erroneous or obsolete archive specifications from the system. Storage management functions are also performed with DB2 RAM utilities to effectively manage the physical storage used by the archived table spaces.

Database Application Management

DB2 Bind Manager for OS/390

The DB2 Bind Manager helps reduce the time and cost of connecting new application programs to a database. It reduces costs by avoiding unnecessary binding steps between application programs and the database. It allows application developers to safely bypass the DB2 bind process for source code changes that do not alter the existing SQL structure.

This tool analyzes each precompile to determine whether the SQL structure has changed. If it has not, then a bind is not needed and will not be executed. This can result in major cost savings in change management. Since DB2 Bind Manager detects only production application changes requiring a bind, it frees database administrators from doing unnecessary binds and allows them to concentrate only on changes that affect the SQL structure.

Advanced checking functions DB2 Bind Manager offers optional checking functions that further simplify the bind process via the Path Checker function which gives developers and database administrators the tool they need to avoid performance problems created by unexpected access path changes as programs are migrated or rebound. Another function, DBRM Checker, enables consistency checking for an existing DBRMLIB and a DB2 subsystem. DBRM Checker will identify DBRMs by plan that have consistency tokens inconsistent with those in the DB2 catalog tables. DBRM Checker determines which of the packages needs binding.

DB2 Web Query Tool for OS/390

The DB2 Web Query Tool for OS/390 lets end users access, create, share, and execute SQL queries through their browsers against multiple IBM DB2 databases. They can save query results in multiple formats, including XML, and integrate query results with popular PC applications such as spreadsheets.

This tool will allow for maximize DB2 availability for e-business by providing global data access to any user without new software or plug-ins on client machines. It has tightly controlled data access with support for existing DB2 security and can be made available to multiple DB2 database servers, safely and simultaneously, regardless of size, hardware, operating system or location.

It sets a new standard for business responsiveness. Now anyone in your organization can take robust data access for granted, virtually anywhere and at any time. This responsiveness means more opportunity and more velocity for every business decision. With the trusted architecture of DB2 behind it, DB2 Web Query Tool enables pervasive connectivity over the Internet to every desktop, for the novice or the expert. Query operations available include: ability to query any DB2 database server made available by administrator, create ad hoc queries, store queries for access by others, browse stored queries by name or description, execute any DB2-specific SQL, view query results in browser or download results, download results as .TXT or .CSV file for use in spreadsheets or other tools, export results as XML, and import table data from XML files.


012 - Summary

Chapter 1. Product Overview

Summary

IBM provides a very comprehensive set of tools in many different categories to support and use DB2. Many of these tools are included with the base product itself and others are available for optional purchase.


013 - Additional Product Resources

Chapter 1. Product Overview

Additional Product Resources

www-4.ibm.com/software/data/db2imstools


014 - Chapter 2. Environment

Part I. Introduction to DB2 UDB for OS/390

Chapter 2. Environment

  • OS/390 and z/OS

  • Address Spaces

  • Configuration Parameters

  • Commands

  • Utilities

  • Catalog and Directory

  • Distributed Data

DB2 operates as a formal subsystem of OS/390 and z/OS. DB2 utilities run in the batch environment, and applications that access DB2 resources can run in the batch, TSO, IMS, or CICS environments. IBM provides attachment facilities to connect DB2 to each of these environments.

In this chapter the subsystem architecture that supports DB2 and the attachments available.

Although not covered in detail, the installation process of the DB2 subsystem and the subsystem parameters known as DSNZPARMS will be discussed. We will also take a look at how to execute the utilities and various commands.

DB2 also has the ability to access other DB2 subsystems and other relational databases. We will take a look at the methods for doing this.


015 - OS/390 and z/OS

Chapter 2. Environment

OS/390 and z/OS

The IBM System/390 family of enterprises servers use OS/390 and z/OS as the operating system software. The core of this operating system is the base control program MVS. Open standard application server functions which support scalable, secure e-business applications are integrated with the MVS base.

The OS/390 and z/OS platforms can provide scalable, secure processing for different type of workloads in a high-performance environment such as stock trading. The applications can be batch, OLTP, or DSS. The major strengths of this environment include: reliability, manageability, scalability, flexibility, availability, and security.


016 - DB2 for OS/390 and z/OS

Chapter 2. Environment

DB2 for OS/390 and z/OS

The DB2 database management system operates as a formal subsystem of the OS/390 and z/OS operating system. DB2 processes can be executed in several different address space regions within OS/390 and z/OS, such as IMS or CICS.

Address Spaces

The address spaces used in the operation of DB2 include:

  • System Services Address Space (SSAS)

  • Database Services Address Space (DSAS)

  • Internal Resource Lock Manager (IRLM)

  • Distributed Database Facility (DDF)

  • Stored Procedure Address Space (SPAS)

  • Allied Address Spaces

Figure 2.1 shows the relationship between the address spaces used by DB2.

Figure 2.1. Relationship Between DB2 Users and DB2 Related Address Spaces

IRLM–Internal Resource Lock Manager

Each DB2 subsystem will have its own IRLM. The IRLM controls access to the application data and is used by DB2 as the lock manager to ensure the integrity of the data.

SSAS–System Services Address Space

This is the DSN1MSTR address space, the system services address space. The following subcomponents execute in the system services address space:

  • General command processor

  • Subsystem support

  • Agent services manager

  • Storage manager

  • Message generator

  • Initialization procedures

  • Instrumentation facilities

  • System parameter manager

  • Recovery manager

  • Recovery log manager

  • Group manager

  • Distributed transaction manager

DBM1–Database Services Address Space

The DSNDBM1 address space is a critical address because it manages most of the activities in DB2. This is the database services address space. The following subcomponents execute in the database services address space:

  • Service controller

  • Data manager

  • LOB manager (LOBM)

  • Data space manager

  • Relational data system (RDS)

  • Stored procedures manager

  • Utilities (work with associated code in an allied address space)

  • Buffer manager

DDF–Distributed Data Facility Services Address Space

The DDF is what allows client applications running in an environment which supports DRDA (Distributed Relational Database Architecture) to access DB2 data. It also allows one DB2 subsystem to access data on another DB2 subsystem. Other relational database servers can be accessed as long as they support DRDA. TCP/IP and SNA are the supported network protocols.

DDF (Distributed Data Facility) allows for up to 150,000 distributed concurrent treads to be attached to a single DB2 subsystem at a time. The following resource managers execute in the distributed data facility services address space:

  • Data communications resource manager (DCRM)

  • Distributed transaction manager (DTM)

  • Distributed relational data system manager (DRDS)

  • Distributed data interchange services (DDIS)

SPAS–Stored Procedure Address Space

This DB2-established address space (DSN1SPAS) provides an isolated environment in which to execute stored procedures. WLM-managed SPAS can also be used.

Note

It is recommended to use WLM (Workload Manager) address spaces for stored procedures.

Note

UDFs (user-defined functions) require Workload Manager address spaces.

Allied Address Spaces

These are the address spaces which communicate with DB2. The stored procedures address spaces, discussed above, actually operate as allied address spaces. The following subcomponents execute in allied address spaces:

  • CICS attachment facility

  • TSO attachment facility

  • IMS attachment facility

  • Call attachment facility

  • Recoverable Resource Manager Services attachment facility

  • Utilities (work with associated code in database services address space)

  • Standalone utilities

  • Subsystem support

  • Message generator (standalone only)

Many of these address spaces will be discussed in the following sections.

Address Space Priority

There are some general recommendations for how to establish the OS/390 address space dispatching priorities. This would be for the IRLM address space to be placed above IMS, CICS, and DB2, with DB2's MSTR and DBM1 address spaces placed above CICS. The following is a recommended priority listing:

  • MVS Monitor with IRLM capabilities

  • IRLM

  • DB2 performance monitors

  • DBM1

  • MSTR

  • CICS

DB2 locks will not get set and released without causing excessive wait time, if the IRLM address space is not at the top of the dispatching priorities (see Chapter 14–Locking and Concurrency for information on DB2 Locks). A warning message will be issued if the IRLM is not above DBM1 at DB2 startup. It is important to pay attention to the dispatching recommendations of the vendor of the various performance monitors. Each has one or more address spaces and always recommends specific dispatching priorities. If a performance monitor is being used that can analyze problems in the IRLM, then dispatching an IRLM higher than a DB2 and/or MVS monitor could be a problem. If a performance monitor can't get dispatched ahead of the IRLM, you will not be able to find or analyze the problem.

Note

Under OS/390, the DPRTY parameter is no longer valid, and all priorities have to be set using the MVS parameters in either the IPS/ICS table or the program priority table. If the IPS/ICS table is used to define relative priorities, then the priorities can be changed dynamically.

Attachments

Attachments facilities provide the interface between DB2 other environments. These facilities include CICS, IMS, TSO, CAF, and RRS. We will take a brief look at each.

CICS–Customer Information Control System

CICS is an application server that provides online transaction management for applications. In order to connect to DB2 from this environment you can use the CICS attachment facility. This attachment facility allows access to DB2 data from the CICS environment. A CICS application can access both DB2 and CICS data and in the event of a failure, CICS coordinates the recovery between DB2 and CICS.

Once the DB2 subsystem has been started, DB2 can be operated from a CICS terminal. DB2 and CICS can be started independently of each other. The connection can also be made to be automatic. Figure 2.2 shows the relationship between DB2 and CICS.

Figure 2.2. Relationship of DB2 to CICS

IMS–Information Management Systems

IMS is a database computing system that includes a hierarchical database manager, transaction manager, and middleware products for access to the IMS databases and transactions. In order to connect to DB2, you can use the IMS attachment facility. This facility receives and interprets requests for access to DB2 data via exit routines in the IMS subsystems. IMS can connect automatically to DB2 without operator intervention. You can make DB2 calls from IMS applications by using embedded SQL statements. DB2 also provides the database services for IMS dependent regions with the IMS attachment facility. In and IMS batch environment, there is support for DL/1 batch jobs to have access to both IMS (DL/1) and DB2 data.

IMS will also coordinate recoveries between DB2 and IMS in the event of a failure. You also have the option to include DB2 in the IMS Extended Recovery Facility (XRF) recovery scenario. Figure 2.3 shows a picture of the relationship between IMS and DB2.

Figure 2.3. Relationship Between DB2 and IMS

TSO–Time Sharing Option

TSO allows for interactive time sharing capabilities from remote terminals. In order to connect to DB2, you can use the TSO CAF (Call Attachment Facility) and the RRS (Resource Recovery Services).

Using TSO, you can bind application plans and packages and execute several online functions of DB2. Application programs and databases can be created, modified, and maintained via the TSO attach. You can run in either the foreground or batch when accessing DB2.

Access to two different command processors is allowed when using TSO:

  • DSN command processor

  • DB2 Interactive (DB2I)

The DSN command processor runs as a TSO command processor using the attachment facility. It provides an alternative method for executing programs in a TSO environment that accesses DB2. This processor can be invoked from the foreground by issuing a TSO command or from batch by invoking the TMP (terminal monitor program) from an MVS batch job passing the commands in the SYSTSIN data set to TMP. When DSN is running and DB2 is up, you can issue DB2 or DSN commands (covered later in this chapter).

The DB2I is comprised of ISPF panels that allow for an interactive connection to DB2 and invokes the DSN command processor. With this processor you can invoke utilities, issue commands, and run SQL statements. DB2I is discussed in more detail later in this chapter.

The TSO attachment facility is used by the majority of TSO applications.

Figure 2.4 shows the relationship between DB2 and TSO.

Figure 2.4. Relationship Between DB2 and TSO

CAF–Call Attach Facility

The call attach facility is a DB2 facility used for application programs that run under TSO or MVS batch. The CAF is an alternative to the DSN command processor and provides greater control over the execution environment.

RRSAF–Recoverable Resource Services Attachment Facility

RRSAF is a DB2 subcomponent that uses OS/390 Transaction Management and Recoverable Resource Manager Services to coordinate resource commitment between DB2 and all other resource managers that also use OS/390 RRS in an OS/390 subsystem.


017 - Security

Chapter 2. Environment

Security

One way of adding additional security to your DB2 subsystem is using RACF–Resource Access Control Facility. RACF is a component of the SecureWay Security Server for OS/390. There are also other equivalent security packages on the market. Use of RACF will prevent unauthorized users having access to the system. It can be used to protect many different resources in DB2 and to check the identity of DB2 users.

For more information on security options, refer to Chapter 3, Controlling Access.


018 - Parallel Sysplex Support

Chapter 2. Environment

Parallel Sysplex Support

DB2 also has the ability to run in a Parallel Sysplex environment. Refer to Chapter 9, Data Sharing, for more details.


019 - Interfaces

Chapter 2. Environment

Interfaces

There are a variety of ways to interface with DB2 such as with programs using TSO, CAF, CICS, etc. Two of the most common interfaces provided with the DB2 product are DB2I and SPUFI.

DB2I (DB2 Interactive)

The interactive program DB2I can be used to run application programs and perform many DB2 operations by entering values on panels. DB2I runs under TSO using ISPF (Interactive System Productivity Facility) services. To use it, follow the local procedures for logging on to TSO, and enter ISPF. Each operation is controlled by entering the parameters that describe it on the panels provided. DB2 also provides help panels to:

  • Explain how to use each operation

  • Provide the syntax for and examples of DSN subcommands,DB2 operator commands, and DB2 utility control statements

Using DB2I the following tasks can be performed:

  • Execute SQL statements using SPUFI

  • Perform a DCLGEN

  • Prepare a DB2 program

  • Invoke the DB2 Precompiler

  • Execute BIND/REBIND/FREE command

  • RUN an SQL program

  • Issue DB2 commands

  • Invoke DB2 utilities

  • Change DB2I defaults

An example of the DB2I menu is shown in Figure 2.5.

Figure 2.5. DB2I Primary Option Menu

Figure 2.6. shows the DB2I defaults. These defaults can be changed for each individual user's DB2I panel. Some parameters that can be changed include: DB2 subsystem on which to run the requests, application language of choice, number or rows returned to the session, and what string delimiter to use.

Figure 2.6. DB2I Defaults Screen

SPUFI

SPUFI (Sequential Processing Using File Input) is an option on the DB2I menu that allows you to execute SQL statements interactively with DB2. Several options on the main menu in SPUFI are as follows:

  • Input data set name:

    • This is the data set that contains one or more SQL statements that are to be executed. It must exist before SPUFI can be used.

  • Output data set name:

    • This is the data set that will receive the output from the SQL statement that is executed. It does not have to exist before execution.

  • Change Defaults:

    • Allows for changing control values and characteristics of the output data set and format of SPUFI settings.

  • Edit Input:

    • Allows for editing of the SQL statements.

  • Execute:

    • Indicates to execute the SQL statement in the input file.

  • Autocommit:

    • Tells DB2 whether or not to make the changes to the data permanent.

  • Browse Output:

    • Allows for browsing of the query output.

  • Connect Location:

    • Specifies the name of the application server to submit the queries to.

SPUFI allows you to have multiple queries in one input data set member for execution. The queries can be stacked by using a ";" between the statements. If there is no explicit COMMIT statement, then all of the SQL statements are considered a unit of work.

After the SQL has been executed, SPUFI will display both the SQLCODE and the SQLSTATE. Shown in Figure 2.7 is the main screen for SPUFI.

Figure 2.7. SPUFI Panel

    Team-Fly
    Top


020 - Installation and Migration Overview

Chapter 2. Environment

Installation and Migration Overview

  • Installation is the process of preparing DB2 to operate as an OS/390 or z/OS subsystem.

  • Migration is the process of upgrading from a release of DB2 to a more current release.

Whether you are installing or migrating, there are several steps you must perform that are the same. This book will not cover the details of each migration step; if you require more information, you can refer to the DB2 Installation Manual. Before you begin installing or migrating, plan the amount of direct access storage and virtual storage you need. Planning and coordinating with other DB2 subsystems are essential if you plan to install the distributed data facility (DDF). For more information, see Section 2 (Volume 1) of DB2 Administration Guide. Review what values are needed for the parameters on the installation and migration panels.

By planning in advance, filling in the parameters becomes an easier task. The DB2 Installer is a workstation tool that is delivered as an element of the Management Tools Package. It enhances your productivity whether you are installing DB2 for OS/390 for the first time or you are an experienced installer. DB2 Installer runs on your OS/2 or Windows NT workstation. The DB2 Installer graphical interface illustrates the overall installation process and keeps a graphical record of how each subsystem is defined. You can define a basic DB2 subsystem quickly, or customize every installation option.

DB2 provides a set of tools that automate the process of installing or migrating. These tools include most of the job control language (JCL) needed to install and migrate the product. This JCL constitutes the installation and migration jobs. Each of these jobs helps you perform a task when installing or migrating.

It also includes the installation CLIST (command list) to help tailor the installation and migration jobs. This CLIST is also called the migration CLIST or simply the CLIST. It contains the code necessary to tailor the jobs to your needs and a series of ISPF panels that you can use to pass information to the CLIST. With the Interactive Systems Productivity Facility (ISPF) and Interactive Systems Productivity Facility/Program Development Facility (ISPF/ PDF), you can use a series of ISPF panels to pass parameter values to the CLIST. The CLIST uses these values to tailor the installation and migration jobs.

DB2 also provides a set of sample programs and procedures that help you determine if DB2 is functioning correctly. Due to the fact that it is distributed as object code, DB2 requires few assemblies. An assembly to specify DB2 initialization parameters must be performed, but this requires only a few seconds.

DB2 allows you to specify many subsystem characteristics during DB2 operation, thereby providing the ability to defer decisions about DB2 characteristics until the install or migration. Some of these decisions include: authorizing users, defining databases and tables, and tuning DB2.

DB2 allows for updating of most of the install options without requiring you to reinstall or re-migrate. Defaults can be accepted for certain options and, after acquiring experience with DB2, you can tailor them to the needs of a particular environment.

High Level Overview

Whether installing or migrating DB2, the following procedures need to be performed:

  • If using distributed data, install VTAM and, optionally, TCP/IP.

  • If planning implementation in a data sharing parallel sysplex environment, there are additional considerations to be examined for the parallel sysplex installation before DB2 can be installed or migrated to data sharing.

  • Load the DB2 libraries (do the SMP/E steps).

  • There are additional jobs that need to be executed if you plan to use DB2's Call Level Interface (CLI) or DB2 for OS/390 Java Edition. (Refer to the IBM DB2 ODBC Guide and Reference or the IBM DB2 for OS/390 Java Edition.)

  • Customize the installation or migration jobs.

  • Install or migrate DB2.

  • Connect the DB2 attachment facilities.

  • Prepare DB2 for use.

  • Verify installation or migration.


021 - DSNZPARMS

Chapter 2. Environment

DSNZPARMS

The DB2 DSNZPARMS are the parameters that you supply values for at installation time, either via the install panels or DB2 Installer. After installation, these DSNZPARMS can be changed without having to go back through the installation process. You can do this by changing the member containing the DSMZPARMS, reassembling it and bouncing the DB2 subsystem for them to take effect.

Online DSNZPARMS

DB2 also allows for about 60 of the most popular DSNZPARMS to be dynamically changed without having to stop and restart the DB2 subsystem, thus avoiding an unwanted outage.

The advantage to changing ZPARMs dynamically is to be able to tailor parameters to the current workload. For example, an approved change to the EDM Pool size can be implemented sooner without a DB2 outage. It may also be desirable to change not only buffer pool size but also EDM pool size and checkpoint frequency for overnight batch processing.

The DSNZPARM member is changed dynamically in its entirety by activating a different DSNZPARM member. The–SET SYSPARM command is used from an OS/390 console, a DSN session under TSO, a DB2I panel, a CICS or IMS terminal, or via an application or product using the Instrumentation Facility. The issuer must have SYSADM, SYSCTRL, or SYSOPR authority (for more on these authorities, refer to Chapter 3, Controlling Access). The following forms of the SET SYSPARM statement can be used for controlling dynamic DSNZPARM settings:

-SET SYSPARM LOAD (modname)
Loads the named parameter module
Default is DSNZPARM

-SET SYSPARM RELOAD
Last named subsystem parameter module is loaded into storage
-SET SYSPARM STARTUP
Loads the initial parameters from DB2 startup

Table 2.1 shows all the DSNZPARMs, a description of each, the allowable values, and whether or not they can be changed on line.

Table 2.1. DSNZPARM Descriptions and Values

Parameter

Description

Values

Update Online

ABEXP

EXPLAIN Processing

YES, NO

Yes

ABIND

Auto BIND

YES, NO, COEXIST

Yes

AGCCSID

ASCII Coded Char Set (graphic)

0-65533

—

ALCUNIT

Allocation Units

BLK, TRK, CYL

Yes

ALL/dbname

Start Names

ALL, space names

—

AMCCSID

ASCII Coded Char Set (mixed)

0-65533

—

APPENSCH

—

ARCPFX1

Copy 1 Prefix

1-34 Char

Yes

ARCPFX2

Copy 2 Prefix

1-34 Char

Yes

ARCRETN

Retention Period

0-9999

Yes

ARCWRTC

WTOR Route Code

1-16

Yes

ARCWTOR

Write to Operator

NO, YES

Yes

ARC2FRST

Read Copy2 Archive

NO, YES

Yes

ASCCSID

ASCII Coded Char Set (single-byte)

0-65533

—

ASSIST

Assistant

YES, NO

Yes

AUDITST

Audit Trace

NO, YES

Yes

AUTH

Use Protection

YES, NO

Yes

AUTHCACH

Plan Auth Cache

0-4096

Yes

BACKODUR

Backout Duration

0-255

Yes

BINDNV

Bind New Package

BINDADD, BIND

Yes

BLKSIZE

Block Size

8192-28672

Yes

BMPTOUT

IMS BMP Timeout

1-254

Yes

CACHEDYN

Cache Dynamic SQL

NO, YES

Yes

CACHEPAC

Package Auth Cache

0-2M

Yes

CACHERAC

Routine Auth Cache

0-2M

Yes

CATALOG

Catalog Alias

1-8 Char

Yes

CDSSRDEF

Current Degree

1, ANY

Yes

CHARSET

—

CHKFREQ

Checkpoint Frequency

200-16M rec (50K), or 1-60 minutes

Yes

CHGDC

DROP Support

1, 2, 3

Yes

CMTSTAT

DDF Threads

ACTIVE, INACTIVE

Yes

COMPACT

Compact Data

NO, YES

Yes

COMPAT

—

CONDBAT

Max Remote Connected

0-25000

Yes

CONTSTOR

Contract Thread Stg

NO, YES

Yes

COORDNTR

Coordinator

NO, YES

Yes

CTHREAD

Max Users

1-2000

Yes

DBACRVW

DBADM can create view for other authid

YES, NO

Yes

DBCHK

Early Detection of data/index errors

YES, NO

—

DBPROTCL

Database Protocol

DRDA, PRIVATE

Yes

DATE

Date Format

NO, AUTO,COMMAND

—

DATELEN

Local Date Length

—

DDF

DDF Startup Option

Yes

DEALLCT

Deallocate Period

0-1439 min, 0-59 sec, NOLIMIT

Yes

DECARTH

Decimal Arithmetic

—

DECDIV3

Minimum Divide Scale

NO, YES

—

DECIMAL

Decimal Point

—

DEFLANG

Language Default

—

DEFLTID

Unknown Authid

IBMUSER, or authid

Yes

DELIM

String Delimiter

—

DESCSTAT

Describe for Static

NO, YES

Yes

DLDFREQ

Level ID Update Frequency

0-32767

Yes

DLITOUT

DL/I Batch Timeout

1-254

Yes

DSHARE

Data Sharing

YES, NO, blank

No

DSMAX

Dataset Maximum

1-32767

Yes

DSQLDELI

Dist SQL Str Delimiter

—

DSSTIME

Dataset Stats Time

1-1440

Yes

DYNRULES

Use for Dynamicrules

—

EDMBFIT

Algorithm for Free Chain Search

YES, NO

—

EDMDSPAC

EDMPOOL Data Space Size

1-2097152

Yes

EDMPOOL

EDMPOOL Storage Size

1k –2097152K

Yes

EDPROP

DROP Support

1, 2, 3

Yes

ENSCHEME

Def Encoding Scheme

EBCDIC, ASCII

—

EXTRAREQ

Extra Blocks Req

0-100

Yes

EVALUNC

Pred Evaluation with UR and RS

YES, NO

Yes

EXTRASRV

Extra Blocks Srv

0-100

Yes

EXTSEC

Extended Security

NO, YES

Yes

GCCSID

EBCDIC Coded Char Set–graphic byte

0-65533

—

GRPNAME

Group Name

1-8 Char

Yes

HOPAUTH

Auth at Hop Site

BOTH, RUNNER

Yes

IDBACK

Max Batch Connect

1-2000

Yes

IDFORE

Max TSO Connect

1-2000

Yes

IDTHTOIN

Idle Thread Timeout

0-9999

Yes

IDXBPOOL

Default Buffer Pool for User Indexes

BP0-BPx

Yes

IMMEDWRI

Immediate Write

NO, YES, PH1

Yes

IRLMAUT

Auto Start

YES, NO

Yes

IRLMPRC

Proc Name

IRLMPROC, IRLM Proc Name

Yes

IRLMRWT

Resource Timeout

1-3600

Yes

IRLMSID

Subsystem Name

IRLM, IRLM Name

Yes

IRLMSWT

Time to Autostart

1-3600

Yes

LBACKOUT

AUTO, YES, NO

Yes

LC_CTYPE

Locale LC_CTYPE

—

LEMAX

Maximum LE Tokens

0-50, 20

Yes

LOBVALA

User LOB Value Storage

1-2097152

Yes

LOBVALS

User LOB Value Storage

1-510002

Yes

LOGAPSTG

Log Apply Storage

1-100M

Yes

MAXARCH

Recording Max

10-1000

Yes

MAXDBAT

Max Remote Active

0-1999

Yes

MAXKEEPD

Maximum Kept Dyn Stmts

0-65535

Yes

MAXRBLK

RID Pool Size

0, 16K-1000000K

Yes

MAXRTU

Read Tape Units

1-99

Yes

MAXTYPE1

Max Type 1 Inactive

0-MAX REMOTE CON

Yes

MCCSID

EBCDIC Coded Char Set–mixed byte

0-65533

—

MEMBNAME

Member Name

1-8 Char

Yes

MINRBLK

No. of ridlists for each RID map

l, n

—

MINSTOR

Thread Management

YES, NO

Yes

MIXED

Mixed Data

—

MON

Monitor Trace

NO, YES

Yes

MONSIZE

Monitor Size

8K to 1M

Yes

NPGTHRSH

Use of Index after table growth

0, -1, n

—

NUMLKTS

Locks Per Tablespace

0-50000

Yes

NUMLKUS

Locks Per User

0-100000

Yes

OPTHINTS

Optimization Hints

NO, YES

Yes

OUTBUFF

Output Buffer

40K-400mb, 40mb

Yes

PARAMDEG

Degree of Parallelism

0-no upper limit

—

PARTKEYU

Allow Partitioning keys to be updated

YES, NO or same

—

PCLOSEN

RO Switch Checkpoints

1-32767

Yes

PCLOSET

RO Switch Time

1-32767

Yes

POOLINAC

Pool Thread Timeout

0-9999

Yes

PRIQTY

Primary Quantity

Blank, Calc'd

Yes

PROTECT

NO, YES

Yes

PTASKROL

YES, NO

Yes

QUIESCE

Quiesce Period

0-999

Yes

RECALL

Recall Database

YES, NO

Yes

RECALLD

Recall Delay

0-32767

Yes

RELCURHL

Release Locks

YES, NO

Yes

RESTART/DEFER

Restart or Defer

RESTART, DEFER

—

RESYNC

Resync Interval

1, 2–99

Yes

RETLWAIT

Retained Lock Timeout

0-254

Yes

RETVLCFK

Varchar from Index

NO, YES

Yes

RGFCOLID

Registration Owner

1-8 Char

Yes

RGFDBNAM

Registration Database

1-8 Char

Yes

RGFDEDPL

Control All Applications

NO, YES

Yes

RGFDEFLT

Unregistered DDL Default

NO, YES

Yes

RGFESCP

ART/ORT Escape Character

Non-alphanumeric char

Yes

RGFFULLQ

Require Full Names

YES, NO

Yes

RGFINSTL

Install DD Control Supt

NO, YES

Yes

RGFNMORT

OBJT Registration Table

1-17 Char

Yes

RGFNMPRT

APPL Registration Table

1-17 Char

Yes

RLF

RLF Auto Start

NO, YES

Yes

RLFAUTH

Resource Authid

SYSIBM, or authid

Yes

RLFERR

RLST Access Error

NOLIMIT, NORUN

Yes

RLFERRD

RLST Access Error

NOLIMIT, NORUN

Yes

RLFTBL

RLST Name Suffix

01, 2 alphanumber char

Yes

ROUTCDE

WTO Route Codes

1, 1-14 Route codes

Yes

RRULOCK

U Lock for RR/RS

NO, YES

Yes

SCCSID

EBCDIC Coded Char Set–Single Byte

0-65533

—

SECQTY

Secondary Qty

Blank, clist calc'd

Yes

SEQCACH

Sequential Cache

BYPASS, SEQ

Yes

SEQPRES

Utility Cache Option

NO, YES

Yes

SITETYP

Site Type

LOCALSITE, RECOVERYSITE

Yes

SMFACCT

SMF Accounting

1, NO, YES (1), list (1-5, 7, 8), *

Yes

SMFSTAT

SMF Statistics

YES (1, 3, 4), NO, list (1-5)

Yes

SMSDCFL

SMS Data Class for file tablespace

Blank, 1-8 Char.

—

SMSDCIX

SMS Data Class for index tablespace

Blank, 1-8 Char.

—

SPRMEDX

Yes

SPRMLTD

—

SQLDELI

SQL String Delimiter

—

SRTPOOL

Sort Pool Size

240K-64000K

Yes

SSID

Subsystem Name

—

STARJOIN

Enabling Star join

Disable, enable, 1, 2-32768

—

STATHIST

Collect Historical Statistics

Space, None, All, Access Path

Yes

STATROLL

Runstats aggregates partition level statistics

YES, NO

Yes

STATIME

Statistics Time

1-1440 min

Yes

STDSQL

Std SQL Language

—

STORMXAB

Max Abend Count

0-225

Yes

STORPROC

DB2 Proc Name

1-8 Char

Yes

STORTIME

Timeout Value

5-1800

Yes

SUPERRS

Suppress Logree recording during soft errors

YES, NO

Yes

SYNCVAL

Yes

SYSADM

System Admin 1

SYSADM or authid

Yes

SYSADM2

System Admin 2

SYSADM or authid

Yes

SYSOPR1

System Operator 1

SYSOPR or authid

Yes

SYSOPR2

System Operator 2

SYSOPR or authid

Yes

TBSBPOOL

Default Buffer Pool for User Data

BP0–BPx

Yes

TCPALVER

TCP/IP Already Verified

NO, YES

Yes

TCPKPALV

TCP/IP Keep Alive

ENABLE, DISABLE, 1-65524

Yes

TIME

Time Format

—

TIMELEN

Local Time Length

—

TRACSTR

Trace Auto Start

NO, YES (1-3), list (1-9)

Yes

TRACTBL

Trace Size

4K to 396K

Yes

TRKRSITE

Remote Tracker Site Usage

NO, YES

Yes

TSTAMP

Timestamp Archives

NO, YES, EXT

Yes

TWOACTV

Number of Active Copies

2, 1

Yes

TWOARCH

Number of Archive Copies

2, 1

Yes

TWOBSDS

Number of BSDSs

YES, NO

—

UGCCSID

—

UMCCSID

—

UNIT

Device Type 1

TAPE or any device

Yes

UNIT2

Device Type 2

Device type or Unit Name

Yes

URCHKTH

UR Check Frequency

0-255

Yes

URLGWTH

Yes

USCCSID

—

UTIMOUT

Utility Timeout

1-254

Yes

WLMENV

WLM Environment

Valid name, 1-18 char

Yes

WRTHRSH

Write Threshold

1-256

—

XLKUPDLT

X Lock for Searched U/D

NO, YES

Yes


022 - Commands

Chapter 2. Environment

Commands

Commands in the DB2 Environment are divided into 6 different categories:

  • The DSN command and its subcommands

  • DB2 commands

  • IMS commands

  • CICS attachment facility commands

  • MVS IRLM commands

  • TSO CLISTs

We will take a look at the DSN Commands and the DB2 Commands. The other commands will not be covered.

DSN Commands

DSN is the DB2 command processor and executes as a TSO command processor. All of its subcommands, except SPUFI, run under DSN in either the foreground or background, and all, except END, also run under DB2 Interactive (DB2I). SPUFI runs only in the foreground under ISPF. Table 2.2 shows the DSN Commands.

Table 2.2. DSN Commands

DSN Command (or subcommand)

Function

BIND

Builds an application package or plan

DB2 Command

Executes a DB2 command

DCLGEN (DECLARATIONS GENERATOR)

Produces declarations for tables or views

DSN

Starts a DSN session

END

Ends the DSN session

FREE

Deletes an application package or plan

REBIND

Updates an application package or plan

REBIND TRIGGER PACKAGE

Updates an application trigger package

RUN

Executes an application program

SPUFI

Executes the SQL Processor using File Input (SPUFI)

*

Comment

DB2 Commands

The command START DB2 can be issued only from an OS/390 or z/OS console. All other DB2 commands can be issued from:

  • An MVS console or MVS application program

  • A DSN session

  • A DB2I panel

  • An IMS terminal

  • A CICS terminal

An application program, using the DB2 instrumentation facility interface (IFI), may issue DB2 commands. DB2 commands issued from an MVS console are not associated with any secondary authorization IDs.

Table 2.3 shows a list of the DB2 Commands.

Table 2.3. DB2 Commands

Command

Description

-ALTER BUFFERPOOL

Alters attributes for the buffer pools

-ALTER GROUPBUFFERPOOL

Alters attributes for the group buffer pools

-ALTER UTILITY

Alters parameter values of the REORG utility

-ARCHIVE LOG

Enables a site to close a current active log and open the next available log data set

-CANCEL THREAD

Cancels processing for specific local or distributed threads

-DISPLAY ARCHIVE

Displays information about archive log processing

-DISPLAY BUFFERPOOL

Displays information about the buffer pools

-DISPLAY DATABASE

Displays status information about DB2 databases

-DISPLAY FUNCTION SPECIFIC

Displays statistics about external user-defined functions

-DISPLAY GROUP

Displays information about the data sharing group to which a DB2 subsystem belongs

-DISPLAY GROUPBUFFERPOOL

Displays status information about DB2 group buffer pools

-DISPLAY LOCATION

Displays status information about distributed threads

-DISPLAY LOG

Displays log information and status of the offload task

-DISPLAY PROCEDURE

Displays status information about stored procedures

-DISPLAY RLIMIT

Displays status information about the resource limit facility (governor)

-DISPLAY THREAD

Displays information about DB2 threads

-DISPLAY TRACE

Displays information about DB2 traces

-DISPLAY UTILITY

Displays status information about a DB2 utility

-MODIFY TRACE

Changes the IFCIDs (trace events) associated with a particular active trace

-RECOVER BSDS

Reestablishes dual bootstrap data sets

-RECOVER INDOUBT

Recovers threads left indoubt

-RECOVER POSTPONED

Completes back-out processing for units of recovery left incomplete during an earlier restart

-RESET GENERICLU

Purges information stored by VTAM in the coupling facility

-RESET INDOUBT

Purges information displayed in the indoubt thread report generated by the -DISPLAY THREAD command

-SET ARCHIVE

Controls the allocation of tape units and the deallocation time of the tape units for archive log processing

-SET LOG

Modifies the checkpoint frequency

-START DATABASE

Makes the specified database available for use

-START DB2

Initializes the DB2 subsystem (can be issued only from an MVS console)

-START DDF

Starts the distributed data facility

-START FUNCTION SPECIFIC

Activates an external function that is stopped

-START PROCEDURE

Activates the definition of stopped or cached stored procedures

-START RLIMIT

Starts the resource limit facility (governor)

-START TRACE

Initiates DB2 trace activity

-STOP DATABASE

Makes specified databases unavailable for applications

-STOP DB2

Stops the DB2 subsystem

-STOP DDF

Stops the distributed data facility

-STOP FUNCTION SPECIFIC

Stops the acceptance of SQL statements for specified functions

-STOP PROCEDURE

Stops the acceptance of SQL CALL statements for stored procedures

-STOP RLIMIT

Stops the resource limit facility (governor)

-STOP TRACE

Stops trace activity

-TERM UTILITY

Terminates execution of a utility

A DSN9022I message indicates the normal end of DB2 command processing; DSN9023I indicates the abnormal end of DB2 command processing.


023 - DB2 Utilities

Chapter 2. Environment

DB2 Utilities

DB2 utilities can be viewed in two general categories: those that execute within the subsystem (on line) and those that are standalone which run outside the subsystem (off line). The following list shows the activities performed by the utilities and where they are discussed in detail in this book:

  • Loading data (Chapter 7–Maintaining Data)

  • Reorganizing data (Chapter 7–Maintaining Data)

  • Recovery data and indexes (Chapter 8–Database Recovery)

  • Rebuilding indexes (Chapter 8–Database Recovery)

  • Gathering statistics (Chapter 7–Maintaining Data)

  • Quiescing data (Chapter 8–Database Recovery)

  • Repairing data (Chapter 7–Maintaining Data)

Executing Utilities

The most common way to execute the DB2 utilities is the creation of JCL with the appropriate control cards. The utilities must be executed on the subsystem where the objects reside (however, this is different in a data sharing environment; see Chapter 9, Data Sharing, for more information). There are also other options for utility execution and we will take a brief look at each.

DB2I

The DB2I has a panel that can be used to generate and run utilities. This option would not require a great deal of JCL knowledge and the jobs can be saved for future execution and editing. Figure 2.8 is an example of the DB2I panel for DB2 Utilities.

Figure 2.8. DB2 Utilities Panel

DSNU Command

A DB2 online utility can be executed by invoking the DSNU CLIST command under TSO. The CLIST command generates the JCL data set required to execute the DSNUPROC procedure and execute online utilities as batch jobs. When you use the CLIST command, you need not be concerned with details of the JCL data set.

The CLIST command will create a job to perform one utility operation only. The CLIST command can be issued for each utility operation necessary, and then the issuer can edit and merge the outputs into one job or step.

Control Center

Utility execution is also supported in the DB2 Control Center. There is also support for utility wildcarding, which provides the ability to execute utilities against a list of objects matching a specified pattern of matching characters. A utility procedure could be created that would allow for the ability to run a mixture of several utilities against several objects with one command—making maintenance much easier for the DBA. There is also support for restarting utilities from the last committed phase or the last committed point. This will be available only for utilities originally started in the Control Center. The restart will be accessible through the Display Utility dialog.

Support is also available for utility IDs. In the Tools Settings notebook there is an option to create a utility ID template using a variety of variables such as USERID and UTILNAME. The utility ID can be edited to make it more meaningful before execution of the utility. Figure 2.9 shows an example of how to invoke a utility against a DB2 object using control center.

Figure 2.9. DB2 Control Center

DSNUTILS

DSNUTILS is a DB2 supplied stored procedure that allows for execution of utilities from a local or remote application via SQL CALL. The client application calls in DSNUTILS stored procedure with appropriate parameters. DSNUTILS then analyzes the parameters to create a SYSIN stream and allocate all necessary data sets. After the data sets are allocated, DSNUTILS calls DSNUTILB, which then executes the appropriate utility. The utility statements are then processed and DSNUTILS retrieves the data (execution results) in the SYSPRINT file and puts it in the SYSIBM.SYSPRINT temporary table, then opens a cursor on the table and passes control back to the client application. The client application then fetches all rows from the result set. An example of this is shown in Figure 2.10.

Figure 2.10. DSNUTILS Stored Procedure Execution

For more information on stored procedures, refer to Chapter 13, Advanced Functions.

Utility Templates

Some DB2 utilities produce data sets as a by-product or as an end result of utility execution. These data sets are referenced on utility control statements by a set of DD name keywords and are specified in detail on the corresponding JCL DD cards.

These DD cards must be coded for each utility, as required, and maintained over time as the structure of data changes. Database administrators establish data set policies, refer to those policies on utility control statements, and let DB2 utilities administer those policies at execution time. Many DB2 utilities will accept a template construct in lieu of DD cards to dynamically allocate utility data sets.

Templates contain:

  • The data set naming convention

  • DF/SMS parameters

  • DASD or TAPE allocation parameters

A TEMPLATE can be specified in the SYSIN data set, preceding the utility control statement that references it, or in one or more TEMPLATE data sets. The TEMPLATE data set DD name is specified on the OPTIONS utility control statement by TEMPLATEDD (ddname) and applies to all subsequent utility control statements until the end of input or until DB2 encounters a new OPTIONS TEMPLATEDD (ddname) specification. The default TEMPLATE data set DD name is SYSTEMPL.

TEMPLATE data sets may contain only TEMPLATE utility control statements. Any TEMPLATE defined within SYSIN overrides another TEMPLATE definition of the same name found in a TEMPLATE data set.

With this functionality, database administrators can standardize data set allocation and the utility control statements that refer to those data sets, reducing the need to customize and alter utility job streams.

Note

Templates cannot be used with the REPAIR utility.

The TEMPLATE specification can be used for both DASD and TAPE data set allocation, including support for data set stacking on tape and GDG base definition. TEMPLATE syntax also allows user-specified DASD SPACE parameters. If the SPACE keyword is not specified, the size of the data set will be estimated based on formulas that vary by utility and by data set.

The TEMPLATE statement required for a COPY example might look something like this:

TEMPLATE tmp1
DSNAME(DB2.&TS..D&JDATE..COPY&ICTYPE.&LOCREM.&PRIBAK.)
VOLUMES(vol1,vol2,vol3)
TEMPLATE tmp2
DSNAME(DB2.&TS..D&JDATE..COPY&ICTYPE.&LOCREM.&PRIBAK.)
VOLUMES(vol4,vol5,vol6)
LISTDEF payroll INCLUDE TABLESPACE CERTTS.*
INCLUDE INDEXSPACE CERTTS.*IX
EXCLUDE TABLESPACE CERTTS.TEMP*
EXCLUDE INDEXSPACE CERTTS.TMPIX*
COPY LIST payroll ...COPYDDN(tmp1,tmp1)RECOVERYDDN(tmp2,tmp2)

Database administrators can check their utility control statements without actual execution using the PREVIEW function. In PREVIEW mode, DB2 expands all TEMPLATE data set names appearing in the SYSIN DD, as well as any from the TEMPLATE DD that are referenced on a utility control statement. DB2 then prints the information to the SYSPRINT data set halts execution. You can specify PREVIEW in one of two ways, either as a JCL PARM or on the OPTIONS PREVIEW utility control statement.

For more information on templates and the DB2 Utilities, refer to the IBM DB2 UDB for OS/390 Utility Guide and Reference.


024 - Catalog and Directory

Chapter 2. Environment

Catalog and Directory

The DB2 catalog and directory act as central repositories for all information about objects, authorizations, and communications regarding the support and operations of DB2.

The catalog is composed of several DB2 tables and can be accessed via SQL. The catalog contains details about DB2 objects obtained from the DDL (Data Definition Language) when an object is created or altered, or from DCL (Data Control Language) when an authorization is granted on an object or group of objects. The DB2 catalog will also contain information about communications with other DB2 and non-DB2 databases through the use of the Communications Database (CDB), which contains information about VTAM and TCP/IP addresses.

Table 2.4 shows a list of the DB2 catalog tables and the types of information in each.

Note

Descriptions of all of the tables in the DB2 catalog tables can be found in detail in Appendix D of the IBM DB2 UDB for OS/390 SQL Reference Manual.

DB2 Directory

The DB2 directory is used to store information regarding the operation and housekeeping of the DB2 environment, and it cannot be accessed by using SQL, as the DB2 catalog can be accessed. It contains information required to start DB2, and there are activities and utilities in the DB2 environment that actually do the updating and deleting of table entries in the DB2 directory. The DB2 directory contains five tables, and a description of each is given in Table 2.5.

Table 2.4. DB2 Catalog Tables

Table Name (SYSIBM.table)

Information Contents

IPNAMES

Defines the remote DRDA servers DB2 can access using TCP/IP. Rows in this table can be inserted, updated, and deleted.

LOCATIONS

Contains a row for every accessible remote server. The row associates a LOCATION name with the TCP/IP or SNA network attributes for the remote server. Requesters are not defined in this table. Rows in this table can be inserted, updated, and deleted.

LULIST

Allows multiple LU names to be specified for a given LOCATION. Insert rows into this table when you want to define a remote DB2 data sharing group. The same value for LUNAME column cannot appear in both the SYSIBM.LUNAMES table and the SYSIBM.LULIST table. Rows in this table can be inserted, updated, and deleted.

LUMODES

Each row of the table provides VTAM with conversation limits for a specific combination of LUNAME and MODENAME. The table is accessed only during the initial conversation limit negotiation between DB2 and a remote LU. This negotiation is called change-number-of-sessions (CNOS) processing. Rows in this table can be inserted, updated, and deleted.

MODESELECT

Associates a mode name with any conversation created to support an outgoing SQL request. Each row represents one or more combinations of LUNAME, authorization ID, and application plan name. Rows in this table can be inserted, updated, and deleted.

SYSAUXRELS

Contains one row for each auxiliary table created for a LOB column. A base table space that is partitioned must have one auxiliary table for each partition of each LOB column.

SYSCHECKDEP

Contains one row for each reference to a column in a table check constraint.

SYSCHECKS

Contains one row for each table check constraint.

SYSCHECKS2

Contains one row for each table check constraint.

SYSCOLAUTH

Records the UPDATE or REFERENCES privileges that are held by users on individual columns of a table or view.

SYSCOLDIST

Contains one or more rows for the first key column of an index key. Rows in this table can be inserted, updated, and deleted.

SYSCOLDIST_HIST

Contains rows from SYSCOLDIST. Whenever rows are added or changed in SYSCOLDIST, the rows are also written to the new history table. Rows in this table can be inserted, updated, and deleted.

SYSCOLDISTSTATS

Contains zero or more rows per partition for the first key column of a partitioning index. Rows are inserted when RUNSTATS scans index partitions of the partitioning index. No row is inserted if the index is a non-partitioning index. Rows in this table can be inserted, updated, and deleted.

SYSCOLSTATS

Contains partition statistics for selected columns. For each column, a row exists for each partition in the table. Rows are inserted when RUNSTATS collects either indexed column statistics or non-indexed column statistics for a partitioned table space. No row is inserted if the table space is non-partitioned. Rows in this table can be inserted, updated, and deleted

SYSCOLUMNS

Contains one row for every column of each table and view.

SYSCOLUMNS_HIST

Contains rows from SYSCOLUMNS. Whenever rows are added or changed in SYSCOLUMNS, the rows are also written to the new history table. Rows in this table can be inserted, updated, and deleted.

SYSCONSTDEP

Records dependencies on check constraints or user-defined defaults for a column.

SYSCOPY

Contains information needed for recovery.

SYSDATABASE

Contains one row for each database, except for database DSNDB01.

SYSDATATYPES

Contains one row for each distinct type defined to the system.

SYSDBAUTH

Records the privileges that are held by users over databases.

SYSDUMMY1

Contains one row. The table is used for SQL statements in which a table reference is required, but the contents of the table are not important.

SYSFIELDS

Contains one row for every column that has a field procedure.

SYSFOREIGNKEYS

Contains one row for every column of every foreign key.

SYSINDEXES

Contains one row for every index.

SYSINDEXES_HIST

Contains rows from SYSINDEXES. Whenever rows are added or changed in SYSINDEXES, the rows are also written to the new history table. Rows in this table can be inserted, updated, and deleted.

SYSINDEXPART

Contains one row for each non-partitioning index and one row for each partition of a partitioning index.

SYSINDEXPART_HIST

Contains rows from SYSINDEXPART. Whenever rows are added or changed in SYSINDEXPART, the rows are also written to the new history table. Rows in this table can be inserted, updated, and deleted.

SYSINDEXSTATS

Contains one row for each partition of a partitioning index. Rows in this table can be inserted, updated, and deleted.

SYSINDEXSTATS_HIST

Contains rows from SYSINDEXSTATS. Whenever rows are added or changed in SYSINDEXSTATS, the rows are also written to the new history table. Rows in this table can be inserted, updated, and deleted.

SYSJARCLASS_SOURCE

Auxiliary table for SYSIBM.SYSCONTENTS.

SYSJARCONTENTS

Contains Java class source for installed jar.

SYSJARDATA_SOURCE

Auxiliary table for SYSIBM.SYSOBJECTS.

SYSJAROBJECTS

Contains binary large object representing the installed jar.

SYSJAVAOPTS

Contains build options used during INSTALL_JAR.

SYSKEYCOLUSE

Contains a row for every column in a unique constraint (primary key or unique key) from the SYSIBM.SYSTABCONST table.

SYSKEYS

Contains one row for each column of an index key.

SYSLOBSTATS

Contains one row for each LOB table space.

SYSLOBSTATS_HIST

Contains rows from SYSLOBSTATS. Whenever rows are added or changed in SYSLOBSTATS, the rows are also written to the new history table. Rows in this table can be inserted, updated, and deleted.

SYSPACKAGE

Contains a row for every package.

SYSPACKAUTH

Records the privileges that are held by users over packages.

SYSPACKLIST

Contains one or more rows for every local application plan bound with a package list. Each row represents a unique entry in the plan's package list.

SYSPACKSTMT

Contains one or more rows for each statement in a package.

SYSPARMS

Contains a row for each parameter of a routine or multiple rows for table parameters (one for each column of the table).

SYSPKSYSTEM

Contains zero or more rows for every package. Each row for a given package represents one or more connections to an environment in which the package could be executed.

SYSPLAN

Contains one row for each application plan.

SYSPLANAUTH

Records the privileges that are held by users over application plans.

SYSPLANDEP

Records the dependencies of plans on tables, views, aliases, synonyms, table spaces, indexes, functions, and stored procedures.

SYSPLSYSTEM

Contains zero or more rows for every plan. Each row for a given plan represents one or more connections to an environment in which the plan could be used.

SYSPROCEDURES

In releases of DB2 for OS/390 prior to Version 6, users were required to use the SYSPROCEDURES catalog table to define stored procedures to DB2. In Version 6 and Version 7, the SYSROUTINES catalog table contains information about stored procedures. When the versions were installed, the rows in SYSPROCEDURES that had non-blank values for AUTHID and LUNAME were copied, with appropriate formatting, to SYSROUTINES. Although Version 6 and Version 7 of DB2 for OS/390 do not use SYSPROCEDURES, SYSPROCEDURES is available for fallback to V5.

SYSRELS

Contains one row for every referential constraint.

SYSRESAUTH

Records CREATE IN and PACKADM ON privileges for collections; USAGE privileges for distinct types; and USE privileges for buffer pools, storage groups, and table spaces.

SYSROUTINEAUTH

Records the privileges that are held by users on routines.(A routine can be a user-defined function, cast function, or stored procedure.)

SYSROUTINES

Contains a row for every routine.(A routine can be a user-defined function, cast function, or stored procedure.)

SYSROUTINES_OPTS

Contains a row for each generated routine, such as one created by the DB2 Stored Procedure Builder tool, that records the build options for the routine. Rows in this table can be inserted, updated, and deleted.

SYSSCHEMAAUTH

Contains one or more rows for each user that is granted a privilege on a particular schema in the database.

SYSSEQUENCES

Contains one row for each identity column.

SYSSEQUENCESDEP

Records the dependencies of identity columns on tables.

SYSSTMT

Contains one or more rows for each SQL statement of each DBRM.

SYSSTOGROUP

Contains one row for each storage group.

SYSSTRINGS

Contains information about character conversion. Each row describes a conversion from one coded character set to another.

SYSSYNONYMS

Contains one row for each synonym of a table or view.

SYSTABAUTH

Records the privileges that users hold on tables and views.

SYSTABCONST

Contains one row for each unique constraint (primary key or unique key) created in DB2 for OS/390 Version 7 or later.

SYSTABLEPART

Contains one row for each non-partitioned table space and one row for each partition of a partitioned table space.

SYSTABLEPART_HIST

Contains rows from SYSTABLEPART. Whenever rows are added or changed in SYSTABLEPART, the rows are also written to the new history table. Rows in this table can be inserted, updated, and deleted.

SYSTABLES

Contains one row for each table, view, or alias.

SYSTABLES_HIST

Contains rows from SYSTABLES. Whenever rows are added or changed in SYSTABLES, the rows are also written to the new history table. Rows in this table can be inserted, updated, and deleted.

SYSTABLESPACE

Contains one row for each table space.

SYSTABSTATS

Contains one row for each partition of a partitioned table space. Rows in this table can be inserted, updated, and deleted.

SYSTABSTATS_HIST

Whenever rows are added or changed in SYSTABSTATS, the rows are also written to the new history table. Rows in this table can be inserted, updated, and deleted.

SYSTRIGGERS

Contains one row for each trigger.

SYSUSERAUTH

Records the system privileges that are held by users.

SYSVIEWDEP

Records the dependencies of views on tables, functions, and other views.

SYSVIEWS

Contains one or more rows for each view.

SYSVOLUMES

Contains one row for each volume of each storage group.

USERNAMES

Each row in the table is used to carry out one of the following operations:

  • Outbound ID translation

  • Inbound ID translation and "come from "checking

Rows in this table can be inserted, updated, and deleted.

Table 2.5. DB2 Directory Tables and Descritptions

Directory Table

Information Contents

SPT01

Referred to as the skeleton package table (SKPT), this table contains information about the access paths and the internal form of the SQL for a package at bind time. Entries are made into this table during bind time (BIND PACKAGE), and entries are deleted when a package is freed (FREE PACKAGE). This table is loaded into memory at execution time (along with the SPT02 table described next).

SCT02

Referred to as the skeleton cursor table (SKCT), this table contains information regarding access paths and the internal form of the SQL for an application plan. Entries in this table are made when a plan is bound (BIND PLAN), and deleted when a plan is freed (FREE PLAN). This table is also loaded into memory at execution time.

DBD01

Information about DBDs (Database Descriptors), which are internal control blocks, is kept in this table. Each database in DB2 has one DBD for its objects (tablespaces, indexes, tables, RI constraints, and check constraints). Updates to this table are made when a database is created or updated. This information is accessed by DB2 in place of continually using the DB2 catalog. This allows for faster, more efficient access to this information. The information in the DBD01 directory table is also contained in the DB2 catalog.

SYSLGRNX

Referred to as the log range table, this table contains information from the DB2 logs regarding the RBA (Relative Byte Address) range for updates. This allows DB2 to efficiently find the RBAs needed from the DB2 logs for recovery purposes. A row is inserted every time a tablespace or partition is opened or updated, and it is updated when the object is closed.

SYSUTILX

This system utilities table stores information about the execution of DB2 utilities, including the status and the steps during execution. This information is used when a utility needs to be restarted. Information in this table is added when a utility is started, and the entry is removed when the execution has ended.


025 - Distributed Data

Chapter 2. Environment

Distributed Data

Using the distributed data facility (DDF) of DB2 will allow for access to data held by other data management systems, or to make your DB2 data accessible to other systems. A DB2 application program can use SQL to access data at database management systems (DBMSs) other than the DB2 at which the application's plan is bound. This DB2 is known as the local DB2. The local DB2 and the other DBMSs are called application servers. Any application server other than the local DB2 is considered a remote server and access to its data is a distributed operation. DB2 provides two methods of accessing data at remote application servers: DRDA and DB2 private protocol access.

For application servers that support the two-phase commit process, both methods allow for updating data at several remote locations within the same unit of work.

The location name of the DB2 subsystem is defined during DB2 installation. The communications database (CDB) records the location name and the network address of a remote DBMS. The tables in the CDB are part of the DB2 catalog.

DRDA–Distributed Relational Database Architecture

With DRDA (the recommended method), the application connects to a server at another location and executes packages that have been previously bound at that server. The application uses a CONNECT statement, a three-part name, or an alias (if bound with DBPROTOCOL (DRDA))to access the server.

Queries can originate from any system or application that issues SQL statements as an application requester in the formats required by DRDA.

DRDA (Distributed Relational Database Architecture) access supports the execution of dynamic SQL statements and SQL statements that satisfy all of the following conditions:

  • The static statements appear in a package bound to an accessible server.

  • The statements are executed using that package.

  • The objects involved in the execution of the statements are at the server where the package is bound. If the server is a DB2 subsystem, three-part names and aliases can be used to refer to another DB2 server.

DRDA access can be used in application programs by coding explicit CONNECT statements or by coding three-part names and specifying the DBPROTOCOL(DRDA) bind option. For more on bind options, refer to Chapter 12.

DRDA access is based on a set of protocols known as Distributed Relational Database Architecture (DRDA). These protocols are documented by the Open Group Technical Standard in DRDA Volume 1: Distributed Relational Database Architecture (DRDA). DRDA communication conventions are invisible to DB2 applications, and allow a DB2 to bind and rebind packages at other servers and to execute the statements in those packages.

For two-phase commit using SNA connections, DB2 supports both presumed abort and presumed nothing protocols that are defined by DRDA. If you are using TCP/IP, DB2 uses the sync point manager defined in the documentation for DRDA Level 3.

DB2 Private Protocol

With private protocol, the application must use an alias or three-part name to direct the SQL statement to a given location. Private protocol works only between application requesters and servers that are both DB2 for OS/390 subsystems.

A statement is executed using DB2 private protocol access if it refers to objects that are not at the current server and is implicitly or explicitly bound with DBPROTOCOL(PRIVATE). The current server is the DBMS to which an application is actively connected. DB2 private protocol access uses DB2 private connections. The statements that can be executed are SQL INSERT, UPDATE, and DELETE, and SELECT statements with their associated SQL OPEN, FETCH, and CLOSE statements.

In a program running under DB2, a three-part name or an alias can refer to a table or view at another DB2.The location name identifies the other DB2 to the DB2 application server. A three-part name consists of a location, an authorization ID, and an object name. For example, the name NYSERVER .DB2USER1.TEST refers to a table named DB2USER1.TEST at the server whose location name is NYSERVER.

Alias names have the same allowable forms as table or view names. The name can refer to a table or view at the current server or to a table or view elsewhere. For more on aliases, refer to Chapter 5.

Note

Private protocol does not support many distributed functions, such as TCP/IP or stored procedures. The newer data types, such as LOB or user-defined types, are also not supported by private protocol. It is not the recommended method to use and is no longer being enhanced.

Communications Protocols

DDF uses TCP/IP or SNA to communicate with other systems. Figure 2.11 shows the connectivity options you have with DB2 's DDF.

Figure 2.11. Connectivity Options

Setting up a network for use by database management systems requires knowledge of both database management and communications. Thus, you must put together a team of people with those skills to plan and implement the network.

TCP/IP

Transmission Control Protocol/Internet Protocol (TCP/IP) is a standard communication protocol for network communications. Previous versions of DB2 supported TCP/IP requesters, although additional software and configuration were required. Native TCP/IP eliminates these requirements, allowing gateway-less connectivity to DB2 for systems running UNIX System Services.

SNA

System Network Architecture is the description of the logical structure, formats, protocols, and operational sequences for transmitting information through and controlling the configuration and operation of the networks. It is one of the two main network architectures used for network communications to the enterprise servers.

VTAM

DB2 also uses VTAM for communicating with remote databases. This is done by assigning two names for the local DB2 subsystem: a location name and a logical unit name (LU name). A location name distinguishes a specific database management system in a network, so applications use this name to direct requests to the local DB2 subsystem. Other systems use different terms for a location name. For example, DB2 Connect calls this the target database name. DB2 uses the DRDA term, RDBNAM, to refer to non-DB2 relational database names.

Communications Database (CDB)

The DB2 catalog includes the communications database (CDB), which contains several tables that hold information about connections with remote systems. These tables include:

  • SYSIBM.LOCATIONS

  • SYSIBM.LUNAMES

  • SYSIBM.IPNAMES

  • SYSIBM.MODESELECT

  • SYSIBM.USERNAMES

  • SYSIBM.LULIST

  • SYSIBM.LUMODES

Some of these tables must be populated before data can be requested from remote systems. If this DB2 system services only data requests, the CDB does not have to be populated; the default values can be used.

When sending a request, DB2 uses the LINKNAME column of the SYSIBM.LOCATIONS catalog table to determine which protocol to use.

  • To receive VTAM requests, an LUNAME must be selected in installation panel DSNTIPR.

  • To receive TCP/IP requests, a DRDA port and a resynchronization port must be selected in installation panel DSNTIP5. TCP/IP uses the server's port number to pass network requests to the correct DB2 subsystem. If the value in the LINKNAME column is found in the SYSIBM.IPNAMES table, TCP/IP is used for DRDA connections. If the value is found in SYSIBM.LUNAMES table, SNA is used.

  • If the same name is in both SYSIBM.LUNAMES and SYSIBM .IPNAMES,TCP/IP is used to connect to the location.

Note

A requester cannot connect to a given location using both SNA and TCP/IP protocols. For example, if the SYSIBM.LOCATIONS specifies a LINKNAME of LU1, and if LU1 is defined in both the SYSIBM.IPNAMES and SYSIBM .LUNAMES tables, TCP/IP is the only protocol used to connect to LU1 from this requester for DRDA connections. For private protocol connections, the SNA protocols are used. If private protocol connections are being used, the SYSIBM.LUNAMES table must be defined for the remote location's LUNAME.


026 - Subsystem Pools

Chapter 2. Environment

Subsystem Pools

In the DBM1 address space there are many objects critical to the operation of DB2. Here we will take a brief look at buffer pools, the RID pool, EDM pool and the Sort pool.

Buffer Pools

Buffer pools are database objects used to cache database data pages in memory. If an object's data page is placed in a buffer pool, physical I/O access to disks will be avoided. Buffer pools can be assigned to cache only a particular table space's data. This is assigned within the table space definition.

Buffer pools are areas of virtual storage that temporarily store pages of table spaces or indexes. When a program accesses a row of a table, DB2 places the page containing that row in a buffer. When a program changes a row of a table, DB2 must write the data in the buffer back to disk (eventually), normally either at a checkpoint or a write threshold. The write thresholds are either a vertical threshold at the page set level or a horizontal threshold at the buffer pool level. Storage for buffer pools is backed by memory in the DBM1 address space and this can be extended to hiper space (normally expanded memory), or buffer pools can exist in a data space.

There are up to 80 virtual buffer pools available. This allows for:

  • 50 4K page buffer pools (BP0-BP49)

  • 10 32K page buffer pools (BP32K-BP32K9)

  • 10 8K page buffer pools (BP8K0-9)

  • 10 16K page buffer pools (BP16K0-9)

80 hiper pools are also available for use.

Virtual Pools

Virtual buffer pools are those that take their space in the DBM1 address space or in a data space (discussed later) . As mentioned previously, there can be up to 80 buffer pools defined. It is very easy to create a buffer pool; it is done via an –ALTER BUFFERPOOL command. An example below shows how to create a 3000-page buffer pool with a sequential threshold of 50 percent (50% of the buffer pool can be used for sequentially processed pages and 50% will be used for randomly processed pages). For more information on tuning these parameters, refer to Chapter 10.

-ALTER BUFFERPOOL (BP3) VPSIZE (3000) VPSEQT(50)

Each buffer pool can have a different size and different parameter settings. These parameters control such activities as writing changed pages to disk. Changes to the buffer pool sizes and thresholds are also done via the –ALTER BUFFERPOOL command.

Data Spaces

Buffer pools can also be defined in data spaces. Data spaces are basically buffer pools defined in a separate area of memory. They are defined as data-only spaces and no program code is allowed to run in these areas. The resources to back data spaces are about the same as those used for virtual storage: a combination of central storage, auxiliary storage slots and expanded storage frames. Low use pages will still be paged out. There can be paging activity between the paging device and central storage.

Virtual buffer pools take space from DBM1 whereas data spaces take advantage of a data space in MVS. Data spaces do still require a small amount of memory in DBM1 (each buffer requires 128 bytes), but by using data spaces we are freeing up memory in DBM1 for other uses. With data spaces we are allowed up to 32GB (for 4k pages) of storage. More space is available in a data spaces if you have larger page sizes (for example, 32K) because there is still a limit of 8 million pages total (regardless of size). A buffer pool is defined to use a data space by specifying VPTYPE(DATASPACE). The default is VPTYPE(PRIMARY). Below is an example of creating buffer pool BP1 in a data space.

-ALTER BUFFERPOOL (BP1) VPSIZE (2000) VPTYPE(DATASPACE)
Hiper Pools

Hiper pools allow for additional caching of data to keep it around longer for application usage. Usage of a hiper pool requires ADMF (Asynchronous Data Movement Facility) and up to a 2GB ESA (Extended Storage Area).

Note

There is no support for ADMF in 2/Architecture; i. e., ADMF is discontinued on Freeway and above and is not required for hiper pool use.

Data movement by the system service ADMSERV is performed in 4K blocks. Sometimes a page may be moved between the virtual buffer pool and the hiper pool using an MVPG command where only one page would be moved at a time. ADMF is used at other times and can move up to 125 pages in one ADMF request. Only clean pages are stored in the hiper pools, those that are falling off the virtual buffer pool available queues. Hiper pools can be backed by auxiliary storage, meaning paging to disk.

MVS ESA hiper spaces should be used for hiper pools in order to improve the buffer hit ratio (referenced pages found in the buffer pool). Hiper pools should be considered when paging is high in the virtual buffer pool and there is a need to increase the number of pages available to the application and no additional central storage is available. Hiper pools should also be considered when a larger area for random processing is required and the virtual pool is mostly sequential.

Performance can be improved by using hiper pools when there is a low hit ratio in the virtual pool due to pages that are needed by the application being written out to disk before they can be referenced.

Note

A buffer pool defined in a data space cannot have a hiper pool associated with it.

EDM Pool

The EDM pool (Environmental Descriptor Manager) contains many internal objects, including the following:

  • DBDs– database descriptors

  • SKCTs– skeleton cursor tables

  • CTs– cursor tables (copies of the SKCTs)

  • SKPTs– skeleton package tables

  • PTs– package tables (copies of the SKPTs)

  • Authorization cache block for each plan

    • Except those with CACHESIZE set to 0

  • Skeletons of dynamic SQL for CACHE DYNAMIC SQL

If the pool is too small, then you will see increased I/O activity in the following DB2 table spaces, which support the DB2 directory:

DSNDB01.DBD01

DSNDB01.SPT01

DSNDB01.SCT02

Our main goal for the EDM pool is to limit the I/O against the directory and catalog. If the pool is too small, then you will also see increased response times due to the loading of the SKCTs, SKPTs, and DBDs, and re-preparing the dynamic SQL statements because they could not remain cached.

By correctly sizing the EDM pool, unnecessary I/Os can be avoided from accumulating for a transaction. If a SKCT, SKPT or DBD has to be reloaded into the EDM pool, this is additional I/O. This can happen if the pool pages are stolen because the EDM pool is too small. Pages in the pool are maintained on an LRU queue and the least recently used pages get stolen, if required.

RID Pool

The RID (Row Identifier) pool is used for storing and sorting RIDs for operations such as:

  • List Prefetch

  • Multiple Index Access

  • Hybrid Joins

  • Enforcing unique keys while updating multiple rows

The RID pool is looked at by the optimizer for prefetch and RID use. The full use of RID POOL is possible for any single user at run-time. Run-time can result in a table space scan if not enough space is available in the RID pool. For example, if you want to retrieve 10,000 rows from 100,000,000-row table and there is no RID pool available, then a scan of 100,000,000 rows would occur, at any time and without external notification. The optimizer assumes physical I/O will be less with a large pool.

The size is set with an installation parameter (16 KB to 1000 MB). The RID pool is created at startup time, but no space is allocated until RID storage is actually needed. It is then allocated above the 16MB line in 16KB blocks as needed, until the maximum size you specified on installation panel DSNTIPC is reached.

Sort Pool

DB2 allocates at startup a sort pool in the private area of the DBM1 address space. DB2 uses a special sorting technique called a tournament sort. During the sorting processes, it is not uncommon for this algorithm to produce logical work files called runs, which are intermediate sets of ordered data. If the sort pool is large enough, then the sort completes in that area. More often than not, the sort cannot complete in the sort pool and the runs are moved into the DSNDB07 work files that are used to perform sorts. These runs are later merged to complete the sort. When DSNDB07 is used for holding the pages that make up the sort runs, you could experience performance degradation if the pages get externalized to the physical work files since they will have to be read back in later in order to complete the sort.

The sort pool size is also defined by a DSNZPARM and is currently 240 KB to 64 MB with a 1MB default.


027 - Summary

Chapter 2. Environment

Summary

In this chapter, we have examined most of the components that make up the DB2 environment. The operating systems for DB2 on the S/390 and z-series enterprise servers are based on the core operating system of MVS. The current operating systems are OS/390 and z/OS. Within this structure, there are several address spaces that make up the environment, which really is a horizontal memory expansion, with each address spaces allocated certain functions. The main components of DB2, and all its managers are in the DBAS and SSAS. The IRLM communicates with DB2 and provides the internal lock manager.

Across the environment there are security services, as a total umbrella to provide many levels of security. The highest level is the component called SecureWay Security Server, which houses such components as RACF.

DB2I and SPUFI are only two of several interfaces to DB2, but provide for most of the direct interface to DB2 for DBAs and developers. SPUFI is used as a batch program, and DB2I as a real-time interface.

The installation process, performed from TSO or the GUI Windows installer, is primarily focused with picking options for the many DSNZPARMS, the configuration parameters for DB2. There are currently over 60 parameters that can be modified online, without taking an outage.

There are six different categories of commands, and many utilities in the DB2 environment. The most used are the DSN commands and the DB2 commands.

The DB2 Catalog and Directory are the control repository for the environment. There is a large number of catalog tables that are primarily populated by SQL DDL and DCL, for every object creation and modification.

We examined briefly the distributed networks architecture of SNA and TCP/IP, and looked at how DRDA provides the services on top of those layers.

Other subsystems objects such as the buffer pools, the EDM pool, the RID pool, and the SORT pool were also discussed in terms of how they are used by DB2. We will revisit these objects in Chapter 10 when we look at Performance Monitoring and Tuning.


028 - Additional DB2 Environment Resources

Chapter 2. Environment

Additional DB2 Environment Resources

  • IBM DB2 UDB for OS/390 Installation Guide

  • IBM DB2 UDB for OS/390 Utility Guide and Reference

  • IBM DB2 UDB for OS/390 Administration Guide

  • IBM DB2 UDB for OS/390 SQL Guide and Reference

  • Open Group Technical Standard in DRDA Volume 1: Distributed Relational Database Architecture (DRDA)

  • DRDA info www.opengroup.com

  • SNA LU 6.2 Peer Protocols Reference


029 - Chapter 3. Controlling Data Access

Part I. Introduction to DB2 UDB for OS/390

Chapter 3. Controlling Data Access

  • Subsystem Access

  • Data Set Protection

  • Authorization IDs

  • Authorizations and Privileges

  • Auditing

Security is an important consideration whenever data is stored in a relational database management system. In this chapter, we discuss controlling data access using many different methods. Access to data within DB2 is controlled at many levels, including subsystem, database objects, and application plan/packages. The topics of authentication of user IDs and passwords are discussed. We configure groups of typical database users such as database administrators, system administrators, transactional processing personnel, and decision support users. Each of these database user types may require different access privileges. We will also discuss how to perform auditing, allowing you to monitor access and manipulation of data.

There are several routes from a process to DB2 data, with controls on every route, as shown in Figure 3.1.

Figure 3.1. DB2 Data Access Control

    Team-Fly
    Top


030 - DB2 Subsystem Access

Chapter 3. Controlling Data Access

DB2 Subsystem Access

Whether or not a process can gain access to a specific DB2 subsystem can be controlled outside of DB2. A common procedure is to grant access only through RACF or some similar security system. Profiles for access to DB2 from various environments, and DB2 address spaces, are defined as resources to RACF. Each request to access DB2 is associated with an ID. RACF checks that the ID is authorized for DB2 resources and permits, or does not permit, access to DB2. The RACF system provides several advantages of its own. For example, it can:

  • Identify and verify the identifier associated with a process.

  • Connect those identifiers to RACF group names.

  • Log and report unauthorized attempts to access protected resources.

Local DB2 Access

A local DB2 user is subject to several checks even before reaching DB2. For example, if you are running DB2 under TSO and using the TSO logon ID as the DB2 primary authorization ID, then that ID was verified with a password when the user logs on. When the user gains access to DB2, a user-written or IBM-supplied exit routine connected to DB2 can check the authorization ID further, change it, and associate it with secondary IDs (see authorization IDs in the next section). In providing these services, DB2 can use the services of an external security system again.

Remote Access

A remote user is also subject to several checks before reaching your DB2 subsystem. You can use RACF or an equivalent third party security product.

RACF can:

  • Verify an identifier associated with a remote attachment request and check it with a password.

  • Generate PassTickets on the sending side. PassTickets can be used instead of passwords. A PassTicket lets a user gain access to a host system without sending the RACF password across the network.

Note

DB2 's communications database (CDB) does allow some control of authentication in that you can cause IDs to be translated before sending them to the remote system. For more information about accessing DB2 and the CDB, refer to Chapter 2.

IMS and CICS Security

Access to DB2 can also be controlled from within IMS or CICS.

  • IMS terminal security lets you limit the entry of a transaction code to a particular logical terminal (LTERM) or group of LTERMs in the system. To protect a particular program, you can authorize a transaction code to be entered only from any terminal on a list of LTERMs. Alternatively, you can associate each LTERM with a list of the transaction codes that a user can enter from that LTERM. IMS then passes the validated LTERM name to DB2 as the initial primary authorization ID.

  • CICS transaction code security works with RACF to control the transactions and programs that can access DB2. Within DB2, you can use the ENABLE and DISABLE options of the bind operation to limit access to specific CICS subsystems.

Kerberos Security

Kerberos security is a network security technology developed at the Massachusetts Institute of Technology. DB2 for OS/390 can use Kerberos security services to authenticate remote users. With Kerberos security services, remote end-users access DB2 for OS/390 when they issue their Kerberos name and password. This same name and password is used for access thoughout the network, so a separate MVS password to access DB2 for OS/390 is not necessary.

The Kerberos security technology does not require passwords to flow in readable text, making it secure even in client/server environments. This flexibility is possible because Kerberos uses an authentication technology that uses encrypted tickets that contain authentication information for the end-user.

DB2 for OS/390 support for Kerberos security requires OS/390 SecureWay Security Server Network Authentication and Privacy Service and the OS/390 SecureWay Security Server (formerly known as RACF), or the functional equivalent. The Network Authentication and Privacy Service provides Kerberos support and relies on a security product, such as RACF, to provide registry support. The OS/390 SecureWay Security Server allows administrators who are already familiar with RACF commands and RACF ISPF panels to define Kerberos configuration and principal information.

Note

You can use only Kerberos security if you have RACF.


031 - Data Set Protection

Chapter 3. Controlling Data Access

Data Set Protection

The underlying data in a DB2 subsystem is contained in VSAM linear data sets. Those data sets might be accessed directly without going through DB2 at all. If the data is sensitive, you want to control that route.

If RACF or a similar security system is being used to control access to DB2, the simplest means of controlling data set access outside of DB2 is to use RACF for that purpose also. That means defining RACF profiles for data sets and permitting access to them for certain DB2 IDs.

If the data is very sensitive, you may want to consider encrypting it, for protection against unauthorized access to data sets and backup copies outside DB2. You can use DB2 edit procedures or field procedures to encrypt data, and those routines can use the Integrated Cryptographic Service Facility (ICSF) of OS/390 or z/OS. Data compression is not a substitute for encryption. In some cases, the compression method does not actually shorten the data, and then the data is left uncompressed and readable. If you both encrypt and compress data, compress it first to obtain the maximum compression, and then encrypt the result. When retrieving, take the steps in reverse order: Decrypt the data first, and then decompress the result.


032 - Access to DB2 Objects

Chapter 3. Controlling Data Access

Access to DB2 Objects

An individual process can be represented by a primary authorization identifier (ID), possibly one or more secondary IDs, and an SQL ID. The use of IDs is affected by the security and network systems, and by the DB2 connections made.

DB2 controls access to objects by assigning privileges and authorities to either primary or secondary IDs. Ownership of an object carries with it a set of related privileges over the object. An ID can own an object it creates, or it can create an object to be owned by another ID. There are separate controls for creation and ownership. We will take a look at how privileges, authorities, and ownership work together to provide security for access to DB2 objects.

Executing a plan or package exercises implicitly all the privileges that the owner needed when binding it. Hence, granting the privilege to execute can provide a finely detailed set of privileges and can eliminate the need to grant other privileges separately.

Note

You can use RACF access control to supplement or replace the DB2 GRANT and REVOKE.

Explicit Privileges and Authorities

One way of controlling access within DB2 is by granting, not granting, or revoking explicit privileges and authorities to authorization IDs. A privilege allows a specific operation, sometimes on a specific object. An explicit privilege has a name and is held as the result of an SQL GRANT or REVOKE statement. The authorities group these privileges together. Both privileges and authorities are held by authorization IDs. These will be covered in detail in this chapter.

Note

When installing or updating DB2, the entire system of control within DB2 can be disabled, by setting USE PROTECTION to NO DB2. If protection in DB2 is disabled, then any user who gains access can do anything, but no GRANT or REVOKE statements are allowed.

Authorization IDs

Every process that connects to or signs on to DB2 is represented by a set of one or more DB2 short identifiers called authorization IDs. Authorization IDs can be assigned to a process by default procedures or by user-written exit routines.

Primary Authorization ID

A primary authorization ID is assigned to every process. There will only be one primary authorization ID for a process and it is the ID is normally used to uniquely identify a process.

Secondary Authorization ID

A secondary authorization ID is optional and this ID can hold additional privileges. Secondary authorizations are often used for groups, such as RACF groups, and a primary authorization ID can be associated with multiple secondary authorization IDs.

Current SQLID

Either the primary ID or the secondary ID can be the current SQL ID at any given time.

Furthermore, one ID (either primary or secondary) is designated as the current SQL IDs. You can change the value of the SQL ID during your session. If DB2EXPT is your primary or one of your secondary authorization IDs, you can make it your current SQL ID by issuing the SQL statement:

SET CURRENT SQLID='DB2EXPT';

An ID with SYSADM authority (covered in this section) can set the current SQL ID to any string of up to 8 bytes, whether or not it is an authorization ID or associated with the process that is running.

Explicit Privileges

There are several explicit privileges that can be granted to a primary or secondary authorization ID, allowing that ID the privilege to perform a particular task. When some privileges are granted, they also provide and inherited authority (e.g., if CREATEDBA is granted to an ID, then that ID will become DBADM over the database it creates). The privileges are grouped into several categories.

  • Plans

  • Packages

  • Collections

  • Databases

  • Systems

  • Usage

  • Schemas

  • Distinct types

  • Routines

Table 3.1 provides a list of the available privileges that can be granted to a primary or secondary authorization ID and the type of usage associated with the privilege.

Note

There are also additional privileges for statements, commands, and utility jobs.

Table 3.1. Privileges and Usage

Privilege

Provides the Following Usage:

Table

ALTER

ALTER TABLE, to change the table definition

DELETE

DELETE, to delete rows

INDEX

CREATE INDEX, to create an index on the table

INSERT

INSERT, to insert rows

REFERENCES

ALTER or CREATE TABLE, to add or remove a referential constraint referring to the named table or to a list of columns in the table

SELECT

SELECT, to retrieve data from the table

TRIGGER

CREATE TRIGGER, to define a trigger on a table

GRANT ALL

SQL statements of all table privileges

Plan

BIND

BIND, REBIND, and FREE PLAN, to bind or free the plan

EXECUTE

RUN, to use the plan when running the application

Package

BIND

The BIND, REBIND, and FREE PACKAGE subcommands, and the DROP PACKAGE statement, to bind or free the package, and, depending on the installation option BIND NEW PACKAGE, to bind a new version of a package

COPY

The COPY option of BIND PACKAGE, to copy a package

EXECUTE

Inclusion of the package in the PKLIST option of BIND PLAN

GRANT ALL

All package privileges

Collection

CREATE IN

Naming the collection in the BIND PACKAGE subcommand

Database

CREATETAB

The CREATE TABLE statement, to create tables in the database

CREATETS

The CREATE TABLESPACE statement, to create table spaces in the database

DISPLAYDB

The DISPLAY DATABASE command, to display the database status

DROP

The DROP and ALTER DATABASE statements, to drop or alter the database

IMAGCOPY

The QUIESCE, COPY, and MERGECOPY utilities, to prepare for, make, and merge copies of table spaces in the database; the MODIFY utility, to remove records of copies

LOAD

The LOAD utility, to load tables in the database

RECOVERDB

The RECOVER, REBUILD INDEX, and REPORT utilities, to recover objects in the database and report their recovery status

REORG

The REORG utility, to reorganize objects in the database

REPAIR

The REPAIR and DIAGNOSE utilities (except REPAIR DBD and DIAGNOSE WAIT) to generate diagnostic information about, and repair data in, objects in the database

STARTDB

The START DATABASE command, to start the database

STATS

The RUNSTATS and CHECK utilities, to gather statistics and check indexes and referential constraints for objects in the database

STOPDB

The STOP DATABASE command, to stop the database

Subsystem

ARCHIVE

The ARCHIVE LOG command, to archive the current active log, the DISPLAY ARCHIVE command, to give information about input archive logs, the SET LOG command, to modify the checkpoint frequency specified during installation, and the SET ARCHIVE command, to control allocation and deallocation of tape units for archive processing

BINDADD

The BIND subcommand with the ADD option, to create new plans and packages

BINDAGENT

The BIND, REBIND, and FREE subcommands, and the DROP PACKAGE statement, to bind, rebind, or free a plan or package, or copy a package, on behalf of the grantor. The BINDAGENT privilege is intended for separation of function, not for added security. A bind agent with the EXECUTE privilege might be able to gain all the authority of the grantor of BINDAGENT.

BSDS

The RECOVER BSDS command, to recover the bootstrap data set

CREATEALIAS

The CREATE ALIAS statement, to create an alias for a table or view name

CREATEDBA

The CREATE DATABASE statement, to create a database and have DBADM authority over it

CREATEDBC

The CREATE DATABASE statement, to create a database and have DBCTRL authority over it

CREATEESG

The CREATE STOGROUP statement, to create a storage group

CREATETMTAB

The CREATE GLOBAL TEMPORARY TABLE statement, to define a created temporary table

DISPLAY

The DISPLAY ARCHIVE, DISPLAY BUFFERPOOL, DISPLAY DATABASE, DISPLAY LOCATION, DISPLAY LOG, DISPLAY THREAD, and DISPLAY TRACE commands, to display system information

MONITOR1

Receive trace data that is not potentially sensitive

MONITOR2

Receive all trace data

RECOVER

The RECOVER INDOUBT command, to recover threads

STOPALL

The STOP DB2 command, to stop DB2

STOSPACE

The STOSPACE utility, to obtain data about space usage

TRACE

The START TRACE, STOP TRACE, and MODIFY TRACE commands, to control tracing

Use

USE OF BUFFERPOOL

A buffer pool

USE OF STOGROUP

A storage group

USE OF TABLESPACE

A table space

Schema

CREATEIN

Create distinct types, user-defined functions, triggers, and stored procedures in the designated schemas

ALTERIN

Alter user-defined functions or stored procedures, or specify a comment for distinct types, user-defined functions, triggers, and stored procedures in the designated schemas

DROPIN

Drop distinct types, user-defined functions, triggers, and stored procedures in the designated schemas

Distinct Type

USAGE ON DISTINCT TYPE

A distinct type

Routine

EXECUTE ON FUNCTION

A user-defined function

EXECUTE ON PROCEDURE

A stored procedure

GRANTing and REVOKEing Privileges

The privileges in Table 3.1 must be GRANTed to an authorization ID. The GRANT and REVOKE statement are part of the SQL language known as DCL (Data Control Language). We will take a look at a couple of examples of GRANTing and REVOKEing privileges.

In order to grant DB2EXPT the ability to select data from a particular table, we must execute the following SQL statement:

GRANT SELECT ON DB2CERT.TEST_TAKEN TO DB2EXPT

To grant DB2EXPT the ability to BIND packages to the CERTCL collection, we would execute the following statement:

GRANT BIND ON PACKAGE CERTCL.* TO DB2EXPT

To grant everyone the ability to select, update, insert, or delete data from a particular table, we must execute the following SQL statement:

GRANT ALL ON DB2CERT.TEST_TAKEN TO PUBLIC

Note

The keyword PUBLIC allows for any user to have the privilege granted.

To take away everyone's delete data from a particular table, we must execute the following SQL statement:

REVOKE DELETE ON DB2CERT.TEST_TAKEN FROM PUBLIC

Revoking a privilege from a user can also cause that privilege to be revoked from other users. This is called a cascade revoke.

Related and Inherited Privileges

DB2 has defined sets of related privileges that are identified by administrative authorities. We will take a look at these authorities in the next section. This grouping make it easier to administrate authority because instead of having to grant several individual privileges to an ID, one can simple grant the administrative authority—which includes all applicable privileges.

There are also privileges that are inherited with object ownership.

Authorities

An administrative authority is a set of privileges, often covering a related set of objects. Authorities often include privileges that are not explicit, have no name, and cannot be specifically granted, for example, the ability to terminate any utility job, which is included in the SYSOPR authority.

Table 3.2. Authorities and Privileges

Authorization

Description of Capabilities

Privileges

Installation SYSADM

Assigned during DB2 installation. This ID has all the privileges of SYSADM, plus:

  • Authority is not recorded in the DB2 catalog. The catalog need not be available to check installation SYSADM authority. (The authority outside the catalog is crucial: If the catalog table space SYSDBAUT is stopped, for example, DB2 cannot check the authority to start it again.Only an installation SYSADM can start it.)

  • No ID can revoke this authority; it can be removed only by changing the module that contains the subsystem initialization parameters (typically DSNZPARM).

  • Those IDs can also:

    • Run the CATMAINT utility

    • Access DB2 when the subsystem is started with ACCESS (MAINT)

    • Start databases DSNDB01 and DSNDB06 when those are stopped or in restricted status

    • Run the DIAGNOSE utility with the WAIT statement

    • Start and stop the database containing the ART and ORT

All privileges of all the authorities

SYSCTRL

Almost complete control of the DB2 subsystem but cannot access user data directly, unless granted the privilege to do so. Designed for administering a system containing sensitive data, SYSCTRL can:

  • Act as installation SYSOPR (when the catalog is available) or DBCTRL over any database

  • Run any allowable utility on any database

  • Issue a COMMENT ON, LABEL ON, or LOCK TABLE statement for any table

  • Create a view for itself or others on any catalog table

  • Create tables and aliases for itself or others

  • Bind a new plan or package, naming any ID as the owner

  • Without additional privileges, it cannot

  • Execute DML statements on user tables or views

  • Run plans or packages

  • Set the current SQL ID to a value that is not one of its primary or secondary Ids

  • Start or stop the database containing the ART and ORT

  • Act fully as SYSADM or as DBADM over any database

  • Access DB2 when the sub-system is started with ACCESS(MAINT)

NOTE: SYSCTRL authority is intended for separation of function, not for added security.

System privileges:

  • BINDADD

  • BINDAGENT

  • CREATESG

  • BSDS

  • CREATETMTAB

  • CREATEALIAS

  • CREATEDBA

  • MONITOR1

  • MONITOR2

  • STOSPACE

  • Privileges on all tables:

    • ALTER INDEX

    • REFERENCES

    • TRIGGER

  • Privileges on catalog tables:

    • SELECT

    • UPDATE

    • INSERT

    • DELETE

  • Privileges on all plans:

    • BIND

  • Privileges on all packages:

    • BIND

    • COPY

  • Privileges on all collections:

    • CREATE IN

  • Privileges on all schemas:

    • CREATE IN

    • DROPIN

    • ALTERIN

    • Use privileges on:

    • BUFFERPOOL

    • TABLESPACE

    • STOGROUP

SYSADM

Includes SYSCTRL, plus access to all data. SYSADM can:

  • Use all the privileges of DBADM over any database

  • Use EXECUTE and BIND on any plan or package, COPY on any package

  • Use privileges over views that are owned by others

  • Set the current SQL ID to any valid value, whether it is currently a primary or secondary authorization ID

  • Create and drop synonyms and views for others on any table

  • Use any valid value for OWNER in BIND or REBIND

  • Drop database DSNDB07

  • Grant any of the privileges listed above to others

  • Holders of SYSADM authority can also drop or alter any DB2 object, except system databases, issue a COMMENT ON or LABEL ON statement for any table or view, and terminate any utility job, but SYSADM cannot specifically grant those privileges.

All privileges held by SYSCTRL and DBADM

Plan privileges:

  • EXECUTE

Package privileges:

  • BIND

  • COPY

  • FREE

Routine privileges :

  • EXECUTE

Distinct type privileges:

  • USAGE

SYSOPR

  • Ability to issue most DB2 commands except ARCHIVE LOG, START DATABASE, STOP DATABASE, and RECOVER BSDS

  • Ability to terminate any utility job

  • Ability to execute the DSN1SDMP utility

Privileges:

  • DISPLAY

  • STOPALL

  • RECOVER

  • TRACE

Privileges on routines:

  • START DISPLAY

  • STOP

Installation SYSOPR

Assigned during DB2 installation. Privileges in addition to SYSOPR:

  • Authority is not recorded in the DB2 catalog. The catalog need not be available to check installation SYSOPR authority.

  • No ID can revoke the authority; it can be removed only by changing the module that contains the subsystem initialization parameters (typically DSNZPARM).

  • Access DB2 when the sub-system is started with ACCESS(MAINT).

  • Run all allowable utilities on the directory and catalog databases (DSNDB01 and DSNDB06).

  • Run the REPAIR utility with the DBD statement.

  • Start and stop the database containing the application registration table (ART), and object registration table (ORT).

  • Issue dynamic SQL statements that are not controlled by the DB2 governor. Issue a START DATABASE command to recover objects that have LPL entries or group buffer pool recovery-pending status. These IDs cannot change the access mode.

All privileges held by SYSOPR

System privileges:

  • ARCHIVE STARTDB (Cannot change access mode)

PACKADM

Has all package privileges on all packages in specific collections, or on all collections, plus the CREATE IN privilege on those collections. If the installation option BIND NEW PACKAGE is BIND, PACKADM also has the privilege to add new packages or new versions of existing packages.

Privileges on a collection:

  • CREATE IN

Privileges on all packages in the collection:

  • EXECUTE

  • BIND

  • COPY

DBMAINT

Granted for a specific database, in which the ID can create certain objects, run certain utilities, and issue certain commands. Can use the TERM UTILITY command to terminate all utilities except DIAGNOSE, REPORT, and STOSPACE on the database.

Privileges on one database:

  • CREATETAB

  • STARTDB

  • CREATETS

  • STATS

  • DISPLAYDB

  • STOPDB

  • IMAGCOPY

DBCTRL

In addition to DBMAINT privileges, it can run utilities that can change the data.

All privileges held by DBMAINT on a database

Privileges on one database:

  • DROP LOAD

  • RECOVERDB REORG

  • REPAIR

DBADM

In addition to the privileges held by DBCTRL over a specific database, DBADM has privileges to access any of its tables through SQL statements. Can also drop and alter any table space, table, or index in the database and issue a COMMENT ON, LABEL ON, or LOCK TABLE statement for any table. If the value of field DBADM CREATE VIEW on installation panel DSNTIPP was set to YES during DB2 installation, a user with DBADM authority can create a view for another user ID on any table or combination of tables and views in a database.

All privileges held by DBCTRL on a database

Privileges on tables and views in one database:

  • ALTER

  • INSERT

  • DELETE

  • SELECT

  • INDEX

  • UPDATE

  • REFERENCES

  • TRIGGER

GRANTing and REVOKEing Authorities

The authorities in Table 3.2 must be GRANTed to an authorization ID. Just as with privileges, we can do this via the GRANT and REVOKE statements. We will take a look at a couple of examples of GRANTing and REVOKEing authorities.

In order to grant DB2EXPT DBADM authority on a the DB2CERT database, the following statement must be issued:

GRANT DBADM ON DB2CERT TO DB2EXPT

In order to remove PACKADM authority from DB2EXPT, the following statement must be issued:

REVOKE PACKADM FROM DB2EXPT
WITH GRANT OPTION

If an authority is GRANTed with the WITH GRANT option, the holder can GRANT the privileges contained in that authority to others.

In order to grant DB2EXPT DBADM authority on the DB2CERT database and allow DB2EXPT to give this authority to others, the following statement must be issued:

GRANT DBADM ON DB2CERT TO DB2EXPT WITH GRANT OPTION

Note

If the DBADM authority is ever revoked from DB2EXPT, then any ID that has been granted DBADM from this ID will also be automatically revoked.

Ownership

There are implicit privileges included with ownership of an object. When you create DB2 objects (except for plans and packages) by issuing SQL CREATE statements in which you name the object, you establish ownership. The creator implicitly holds certain privileges over the created object.

Note

The privileges inherent in the ownership of an object cannot be revoked.

Unqualified Objects

If an object name is unqualified, how ownership of the object is established depends on the type of object. Ownership of tables, views, indexes, aliases, and synonyms with unqualified names is established differently from ownership of user-defined functions, stored procedures, distinct types, and triggers with unqualified names. This section describes how ownership is established for each group of objects.

If the name of a table, view, index, alias, or synonym is unqualified, you establish the object's ownership in these ways:

  • If the CREATE statement was issued dynamically (via SPUFI, QMF), the owner of the created object is the current SQL ID of the issuer. That ID must have the privileges that are needed to create the object.

  • If the CREATE statement was issued statically (via execution of a plan or package that contains it), the ownership of the created object depends on the option used for the bind operation. The plan or package can be bound with the QUALIFIER option, the OWNER option, or both.

    • With the QUALIFIER option only, the QUALIFIER is the owner of the object. The QUALIFIER option allows the binder to name a qualifier to use for all unqualified names of tables, views, indexes, aliases, or synonyms that appear in the plan or package.

    • With the OWNER option only, the OWNER is the owner of the object.

    • With both the QUALIFIER option and the OWNER option, the QUALIFIER is the owner of the object.

    • If neither option is specified, the binder of the plan or package is implicitly the object owner.

Note

The plan or package owner must have all required privileges on the objects designated by the qualified names.

Explicitly establishing the ownership a user-defined function, stored procedure, distinct type, or trigger can be accomplished in the following ways:

  • If the CREATE statement is issued dynamically, the owner of the created object is the current SQL ID of the issuer. That ID must have the privileges that are needed to create the object.

  • If the CREATE statement is issued statically, by running a plan or package that contains it, the owner of the object is the plan or package owner. The OWNER bind option can be used to explicitly name the object owner. If the OWNER bind option is not used, the binder of the package or plan is implicitly the object owner.

The following section will describe how the implicit qualifier is determined for a user-defined function, stored procedure, distinct type, or trigger if they are unqualified:

Qualified Objects

If an object name is qualified, how ownership of the object is established depends, again, on the type of object.

For tables, views, indexes, aliases, or synonyms created with a qualified name, the qualifier becomes the owner of the object, subject to these restrictions for specifying the qualifier:

  • If you issue the CREATE statement dynamically, and have no administrative authority, the qualifier must be your primary ID or one of your secondary IDs.

  • If the CREATE statement is issued statically, and the owner of the plan or package that contains the statement has no administrative authority, the qualifier can be the only owner. However, if the owner has at least DBCTRL authority, the plan or package can use any qualifier for a table or an index, and if the owner has SYSADM or SYSCTRL authority, it can also use any qualifier for a view or alias.

Note

If your current SQL ID has at least DBCTRL authority, you can use any qualifier for a table or index, and if it has SYSADM or SYSCTRL authority, you can also use any qualifier for a view or alias. If the current SQL ID has at least DBCTRL authority, the qualifier ID does not need any privileges. Otherwise, the SQL ID must have any additional privileges that are needed to create the object. If the current SQL ID does not have at least DBCTRL authority, the qualifier ID must hold all the necessary privileges.

Schema Name

If you create a distinct type, user-defined function, stored procedure, or trigger with a qualified name, the qualifier will be the schema name. It identifies the schema to which the object belongs. You can think of all objects that are qualified by the same schema name as a group of related objects. Unlike other objects, however, this qualifier does not identify the owner of the object. You establish ownership of a distinct type, user-defined function, stored procedure, or trigger in these ways:

  • If the CREATE statement is issued dynamically, the owner of the created object is your current SQL ID. That ID must have the privileges that are needed to create the object.

  • If the CREATE statement is issued statically, by running a plan or package that contains it, the owner of the object is the plan or package owner. The OWNER bind option can be used to explicitly name the object owner. If the OWNER bind option is not used, the binder of the package or plan is the implicit object owner.

For more information on schemas refer to Chapter 13, Advanced Functions.

Privileges of Ownership by Object

Table 3.3 shows the privileges that inherited with ownership of an object.

Plan or Package Ownership

An application plan or a package can take many actions on many tables, all of them requiring one or more privileges. The owner of the plan or package must hold every required privilege. Another ID can execute the plan or package if it has only the EXECUTE privilege. In that way, another ID can exercise all the privileges that are used in validating the plan or package, but only within the restrictions that are imposed by the SQL statements in the original program.

The executing ID can use some of the owner's privileges, within limits. If the privileges are revoked from the owner, the plan or the package is in validated. It must be rebound, and the new owner must have the required privileges.

The BIND and REBIND subcommands create or change an application plan or a package. On either subcommand, use the OWNER option to name the owner of the resulting plan or package. Keep these points in mind when naming an owner:

Table 3.3. Implicit Privileges of Ownership

Object Type

Implicit Privileges of Ownership

Storage group

  • ALTER or DROP the storage group

  • Name it in the USING clause of a CREATE INDEX or CREATE TABLESPACE statement

Database

DBCTRL or DBADM authority over the database, depending on the privilege (CREATEDBC or CREATEDBA) that is used to create it

Table space

  • ALTER or DROP the table space

  • Name it in the IN clause of a CREATE TABLE statement

Table

  • ALTER or DROP the table or any indexes on it

  • Use LOCK TABLE, COMMENT ON, or LABEL on it

  • CREATE an index or view on the table

  • SELECT or UPDATE any row or column

  • INSERT or DELETE any row

  • Use of the LOAD utility for the table

  • Define referential constraints on any table or set of columns

  • CREATE a trigger on the table

Index

ALTER or DROP the index

View

  • DROP, COMMENT ON, or LABEL the view, or SELECT any row or column

  • UPDATE any row or column

  • INSERT or DELETE any row (if the view is not read-only)

Synonym

USE or DROP the synonym

Package

BIND, REBIND, FREE, COPY, DROP, EXECUTE, or DROP the package

Plan

BIND, REBIND, FREE, or EXECUTE the plan

Alias

DROP the alias

Distinct type

USE or DROP a distinct type

User-defined functions

EXECUTE, ALTER, DROP, START, STOP, or DISPLAY a user-defined function

Stored procedure

EXECUTE, ALTER, DROP, START, STOP, or DISPLAY a stored Procedure

If the OWNER option is used:

  • Any user can name the primary or any secondary ID.

  • An ID with the BINDAGENT privilege can name the grantor of that privilege.

  • An ID with SYSCTRL or SYSADM authority can name any authorization ID on a BIND command, but not on a REBIND command.

If the OWNER option is omitted:

  • BIND— The primary ID becomes the owner.

  • REBIND— The previous owner retains ownership.

Unqualified Names

A plan or package can contain SQL statements that use unqualified table and view names. For static SQL, the default qualifier for those names is the owner of the plan or package. However, you can use the QUALIFIER option of the BIND command to specify a different qualifier.

For plans or packages that contain static SQL, using the BINDAGENT privilege and the OWNER and QUALIFIER options gives you considerable flexibility in performing bind operations. For plans or packages that contain dynamic SQL, the DYNAMICRULES behavior determines how DB2 qualifies unqualified object names.

For unqualified distinct types, user-defined functions, stored procedures, and trigger names in dynamic SQL statements, DB2 finds the schema name to use as the qualifier by searching schema names in the CURRENT PATH special register. For static statements, the PATH bind option determines the path that DB2 searches to resolve unqualified distinct types, user-defined functions, stored procedures, and trigger names.

However, an exception exists for ALTER, CREATE, DROP, COMMENT ON, GRANT, and REVOKE statements. For static SQL, specify the qualifier for these statements in the QUALIFIER bind option. For dynamic SQL, the qualifier for these statements is the authorization ID of the CURRENT SQLID special register.

Plan Execution Authorization

The plan or package owner must have authorization to execute all static SQL statements that are embedded in the plan or package. These authorizations do not need to be in place when the plan or package is bound, nor do the objects that are referred need to exist at that time.

A bind operation always checks whether a local object exists and whether the owner has the required privileges on it. Any failure results in a message. To choose whether the failure prevents the bind operation from completing, use the VALIDATE option of BIND PLAN and BIND PACKAGE, and also the SQLERROR option of BIND PACKAGE. If you let the operation complete, the checks are made again at run-time. The corresponding checks for remote objects are always made at run-time.

Authorization to execute dynamic SQL statements is also checked at run-time.

For more information on plans and packages refer to Chapter 11, Application Development Overview.

Catalog Table Information for Object Access

Table 3.4 contains information about privileges and authorities that are currently being held on various objects in the DB2 Sub system.

Table 3.4. Catalog Information for Object Access

DB2 Catalog Table

Information

SYSIBM.SYSCOLAUTH

Updating column authorizations

SYSIBM.SYSDBAUTH

Database privileges

SYSIBM.SYSPLANAUTH

Plan privileges

SYSIBM.SYSPACKAUTH

Package privileges

SYSIBM.SYSRESAUTH

Buffer pool, storage group, collection, table space, and distinct type usage privileges

SYSIBM.SYSROUTINEAUTH

User-defined functions and stored procedure privileges

SYSIBM.SYSSCHEMAAUTH

Schema privileges

SYSIBM.SYSTABAUTH

Tables and view privileges

SYSIBM.SYSUSERAUTH

System authorities

Controlling Access with Views

Through the use of views, we can control what data a user can see. This can be certain columns, certain rows, or even a combination. This is a way, in addition to granting privileges and authorities, that we can further restrict access to data. We do this by creating a view that allows users to see only certain columns or rows, and then allowing them access only to the view, not the base table.

In the following example we are allowing the user of the view to see only test scores for candidates who took the test at test center 001:

CREATE VIEW TEST01
AS
SELECT SCORES
FROM DB2CERT.TEST_TAKEN
WHERE TCID=001


033 - Auditing

Chapter 3. Controlling Data Access

Auditing

This chapter provides answers to some fundamental auditing questions. Foremost among them are these:

  1. Who is privileged to access what objects?

  2. Who has actually accessed the data?

Answers to the first question are found in the DB2 catalog, which is a primary audit trail for the DB2 subsystem. Most of the catalog tables describe the DB2 objects, such as tables, views, table spaces, packages, and plans. Several other tables (every one with the characters "AUTH" in its name) hold records of every grant of a privilege or authority on different types of object. Every record of a grant contains the name of the object, the ID that received the privilege, the ID that granted it, the time of the grant, and other information. You can retrieve data from catalog tables by writing SQL queries.

Audit Trace

The audit trace is another primary audit trail for DB2. The trace can record changes in authorization IDs for a security audit and changes that are made to the structure of data (such as dropping a table) or data values (such as updating or inserting records) for an audit of data access.

The trace also allows you to audit access attempts by unauthorized IDs, the results of GRANT and REVOKE statements, the mapping of Kerberos security tickets to RACF IDs, and other activities of interest to auditors.

The DB2 audit trace can tell who has accessed data. When started, the audit trace creates records of actions of certain types and sends them to a named destination.

Some of the types of information that can be obtained by the audit trace records include:

  • The ID that initiated the activity

  • The LOCATION of the ID that initiated the activity (if the access was initiated from a remote location)

  • The type of activity and the time the activity occurred

  • The DB2 objects that were affected

  • Whether access was denied

  • Who owns a particular plan and package

Note

With the trace, it can also be determined which primary ID is responsible for the action of a secondary ID, when that information might not appear in the catalog.

Whether the request is from a remote location or the local DB2, it can be audited. The authorization ID on a trace record for a remote request is the ID that is the final result of any outbound translation, inbound translation, or activity of an authorization exit routine; that is, it is the same ID to which you have granted access privileges for your data. Requests from your location to a remote DB2 are audited only if an audit trace is active at the remote location. The output from the trace appears only in the records at that location.

Trace Details

The audit trace does not record everything. The actual changed data is recorded in the DB2 log. If an agent or transaction accesses a table more than once in a single unit of recovery, only the first access is recorded, and then only if the audit trace is started for the appropriate class of events.

Some utilities are not audited. The first access of a table by LOAD is audited, but access by COPY, RECOVER, and REPAIR is not. Access by standalone utilities, such as DSN1CHKR and DSN1PRNT, is not audited. For more information on these utilities refer to Chapter 7, Maintaining Data.

Note

Everything comes at a cost. Auditing does have some overhead and can produce more data than necessary.

When the trace is started, the events to audit are chosen by giving one or more numbers, to identify classes of events. The trace records are limited to 5000 bytes, so descriptions that contain long SQL statements might be truncated. Table 3.5 shows the available classes and the events they include (for additional information on trace classes refer to Chapter 10, Performance Monitoring and Tuning).

Auditing Specific IDs

The audit trace can be started for a particular plan name, a particular primary authorization ID, or a combination of both. Having audit traces on at all times can be useful for IDs with SYSADM authority, for example, because they have complete access to every table. If you have a network of DB2 subsystems, you might need to trace multiple authorization IDs for those users whose primary authorization ID is translated several times.

Starting/Stopping the Trace

In order to start the audit trace, you would execute the following command:

-START TRACE (AUDIT)CLASS (4, 6) DEST (GTF)LOCATION (*)
COMMENT ("Trace data changes; include text of dynamic DML statements.")

This example would start a trace that audits data changes and will include the text of any dynamic SQL. In order to stop this trace you would simply issue the following command:

-STOP TRACE (AUDIT)CLASS (4, 6) DEST (GTF)

For more information on DB2 Commands, refer to Chapter 2, Environment. This command would simply stop the last trace started. If there is more than one trace executing, the –DISPLAY TRACE command can be using to identify a particular trace by a number.

Table 3.5. Audit Trace Classes and Events

Class

Events Traced

1

Access attempts that DB2 denies because of inadequate authorization. This class is the default.

2

Explicit GRANT and REVOKE statements and their results. The class does not include implicit grants and revokes.

3

CREATE, ALTER, and DROP operations affecting audited tables, and their results. The class includes the dropping of a table caused by DROP TABLESPACE or DROP DATABASE and the creation of a table with AUDIT CHANGES or AUDIT ALL. ALTER TABLE statements are audited only when they change the AUDIT option for the table.

4

Changes to audited tables. Only the first attempt to change a table, within a unit of recovery, is recorded. (If the agent or the transaction issues more than one COMMIT statement, the number of audit records increases accordingly.) The changed data is not recorded, only the attempt to make a change. If the change is not successful and is rolled back, the audit record remains; it is not deleted. This class includes access by the LOAD utility. Accesses to a dependent table that are caused by attempted deletions from a parent table are also audited. The audit record is written even if the delete rule is RESTRICT, which prevents the deletion from the parent table. The audit record is also written when the rule is CASCADE or SET NULL, which can result in deletions cascading to the dependent table.

5

All read accesses to tables that are identified as AUDIT ALL. As in class 4, only the first access within a DB2 unit of recovery is recorded, and references to a parent table are audited.

6

The bind of static and dynamic SQL statements of the following types:

  • INSERT, UPDATE, DELETE, CREATE VIEW, and LOCK TABLE statements for audited tables. Except for the values of host variables, the entire SQL statement is contained in the audit record.

  • SELECT statements to tables that are identified as AUDIT ALL. Except for the values of host variables, the entire SQL statement is contained in the audit record.

7

Assignment or change of an authorization ID, through an exit routine (default or user-written) or a SET CURRENT SQLID statement, through either an outbound or inbound authorization ID translation, or because the ID is being mapped to a RACF ID from a Kerberos security ticket.

8

The start of a utility job, and the end of each phase of the utility.

9

Various types of records that are written to IFCID 0146 by the IFI WRITE function.

Note

The audit trace can be set up to start automatically when DB2 is started. This can be done by options on the panel DSNTIPN when DB2 is installed. This option allows you to set AUDIT TRACE to NO, YES, or a list of audit trace classes.

Auditing a Table

In order for the audit trace to be effective, you must first choose whether or not to audit a table by specifying an option of the CREATE and ALTER statements. The example below shows how to indicate that you want to audit changes:

CREATE TABLE DB2CERT.TEST_TAKEN
(CID SMALLINT NOT NULL
      ....
IN DB2CTDB.CERTTS
AUDIT CHANGES

The options are AUDIT CHANGES|ALL|NONE. The default is NONE—no auditing. In order to turn auditing off at the table level, you would simply do an ALTER with AUDIT NONE.

Note

You cannot audit auxiliary tables or catalog tables.


034 - Summary

Chapter 3. Controlling Data Access

Summary

This chapter discussed a number of topics relating to accessing data. We discussed security with the subsystem, with data sets, and then within DB2. Subsystem security is handled in a number of ways such as via IMS, CICS, and Kerberos and RACF.

Securing access at the data set level also needs to be considered in some situations since DB2 stores its data into individual data sets that can be accessed outside of DB2.

We talked about authorization IDs, primary and secondary, and how they are assigned.

We talked about several of the authorization levels within DB2: SYSADM, SYSCNTL, DBADM, DBCNTL, PACKADM, etc. We took a look at the types of privileges each one of these different authorities possesses. Ownership of objects also comes with inherited authorities and privileges that can also be granted to other authorization IDs.

We also examined the granting and revoking of database object privileges using the GRANT and REVOKE SQL statements.

Finally, we talked about the DB2 audit trace. This trace allows us to carefully monitor critical tables to see who is manipulating the data, or, in some very sensitive cases, who is simply trying to access the data.

All of these different levels of security can work together in order to keep our data and our subsystem safe.


035 - Additional Resources

Chapter 3. Controlling Data Access

Additional Resources

  • IBM DB2 UDB for OS/390 DB2 Administration Guide, SC26-9931


036 - Part II: Using SQL

Part II: Using SQL

4 Database Objects

5 Manipulating Database Objects

6 Advanced SQL


037 - Chapter 4. Database Objects

Part II. Using SQL

Chapter 4. Database Objects

  • Creating Database Objects

  • Data Types

  • Tables and Table Spaces

  • Aliases and Synonyms

  • Constraints

  • Views

  • Indexes

DB2 is a relational database that consists of one or more tables that are made up of rows and columns. The tables are created and accessed using the industry standard SQL (Structured Query Language). Data in these tables is accessed via content, not location. This chapter looks at the structures in a DB2 subsystem and those that make up a DB2 database.

Many of these objects can be directly referenced from a SQL statement; therefore, it is important to understand their purpose.

SQL is divided into three major categories:

  • DDL (Data Definition Language)

    • Used to create, modify, or drop database objects

  • DML (Data Manipulation Language)

    • Used to select, insert, update, or delete database data (records)

  • DCL (Data Control Language)

    • Used to provide data object access control

As SQL has evolved, many new statements have been added to provide a more complete set of data access methods. Later in this chapter, the use of DDL (Data Definition Language) to create database objects is discussed.


038 - Understanding Data Structures

Chapter 4. Database Objects

Understanding Data Structures

A database is an organized collection of related objects. Structured Query Language (SQL) is used throughout the database industry as a common method of issuing database queries. SQL is considered a language, composed of statements, functions, and data types. A SQL statement is used to access database objects using relational operations. Before we examine the SQL language, we need to understand some DB2 terminology. We will be referring to the basic data structures or objects that are defined for each DB2 database. These objects include:

  • Tables

    • Data types

    • Aliases

    • Synonyms

  • Views

  • Indexes

    • Keys

  • Table spaces

  • Index spaces

  • Databases

  • Storage groups

Figure 4.1 shows the relationship between the DB2 objects.

Figure 4.1. A hierarchy of DB2 structures

Tables

A table is an unordered set of data records. It consists of columns and rows. Each column is based on a data type. Tables, once created and populated with data, are referenced in the FROM clause of the SQL statements. There are three types of tables:

  • Permanent (base) tables

  • Auxiliary tables

  • Temporary (declared or global) tables

We talk about only tables created with a DDL statement in this chapter. These tables are created using the CREATE TABLE, CREATE AUXILIARY TABLE, or CREATE GLOBAL TEMPORARY TABLE statement and each is a logical representation of the way the data is physically stored on disk. We discuss the declared temporary tables in "Advanced SQL."

Data Types

Data types are used to define the type of data that will occupy each column in a DB2 table. They will also provide the length of column. Data that does not match the data type defined to the column will not be allowed in that column. There are different types of data such as string, numeric, date, time, ROWIDs, LOB, and user-defined distinct data types. Depending on the data type, the length can be fixed or varying. You can categorize these data types into two categories: built-in and user-defined.

Data types that are defined by DB2 are referred to as built-in. The other type, user-defined, allows you define your own type of data to be stored in the column. These user-defined data types are based upon existing built-in data types.

Aliases

An alias is basically a pointer to another table, and is a substitute for the three-part name of a table or view. The alias can be qualified by an owner ID and the table referenced can be on the local site or on a remote site. Aliases are often used in place of any three-part named table in order to make the references portable, since DB2 on OS/390 is the only server that supports three-part name syntax. Aliases are not dropped if the table they are pointing to is dropped.

Synonyms

Synonyms are used to refer to a table using a different name, or to refer to another owner's table as if you were the owner. It is basically a private pointer to a table, can be referenced only by its owner, and is not allowed to be qualified.

Unlike aliases, a synonym can be used only to refer to a table in the same subsystem in which it was defined. If the table is dropped, so is the synonym.

Views

Views are alternate ways of viewing data in one or more base table(s) or other views. A view is not a physical object and does not store actual data. It is an entry in the DB2 catalog that, when accessed, executes a SQL statement that retrieves data from other tables(s) or views. It can be used to limit access to certain kinds of data, create a customized view of data for users, or allow for alterations of tables without effecting application programs. Views can be read only if necessary or they can be updateable, insertable, or deletable. There are some restrictions to using views.

Indexes

Indexes are ordered sets of pointers that are associated with a table. Indexes can be created on permanent tables or declared temporary tables. The index is based on the values of one or more columns in the table. The index values can be either ascending or descending. Each table can have one or more index, which can also be used to enforce uniqueness in the data values and clustering sequence. Indexes are used to improve performance when accessing data. If an index is used, large scans of data can be avoided.

Indexes are physical objects that are created in index spaces which are data sets like those that support the table spaces. The index space is automatically created in the same database as the table on which it is defined.

There are two types of indexes: partitioned and non-partitioned. A partitioned index is created on a partitioned table space and is used to define the allowable values for each partition. A non-partitioned index is used on tables defined in segmented, simple table spaces, or as an alternate index on a partitioned table space.

Note

The maintenance overhead of indexes can negatively impact the performance of INSERT, UPDATE, and DELETE statements.

Keys

Many keys can be defined on a table. A key is a named ordered set of columns on one table. A composite key is a key made of several columns. Each key will play a particular role for a given table.

Table Spaces

Table spaces are objects that consist of one or more data sets that are used to store DB2 data. Tables are created in table spaces and you can have one or many tables in a table space, depending on the type of table space defined.

There are four types of table spaces: simple, segmented, partitioned, and LOB. Both simple and segmented table spaces allow for multiple tables, but only segmented should be used due to the many performance issues.

  • Simple table spaces

    • These table spaces can contain one or more tables; however, the rows of the different tables are not kept on separate pages, which can cause concurrency and space usage issues.

  • Segmented table spaces

    • Segmented table spaces can contain one or more tables and the space is divided into same-size segments. Each segment will contain rows from only one table.

  • Partitioned table spaces

    • A partitioned table space divides the data into several separate data sets (partitions). There can only be one table in a partitioned table space. You can have up to 254 partitions, each up to 64 GB. Each partition can have different characteristics such as volume placement and free space.

  • LOB table spaces

    • LOB table spaces are required to hold large object data. LOB table spaces are associated with the table space which holds the logical LOB column.

Index Spaces

When an index is created, an index space is also created. Index spaces are a set of VSAM linear data sets that hold index data, and they are implicitly associated with the database that contains the table the index is defined on.

Databases

Databases in DB2 are a collection of table spaces and index spaces. The database on the OS/390 platform does not have any physical storage characteristics, but acts more like an umbrella over all of its dependent objects.

Storage Groups

Storage groups are used to define a list of volumes to DB2 for its use when creating physical objects such as table spaces and index spaces. The storage is allocated when the tables are loaded with data. Storage groups are created using DDL.


039 - Managing Database Objects

Chapter 4. Database Objects

Managing Database Objects

To create, modify, or delete objects in a database, SQL Data Definition Language (DDL) is used.

Using SQL Data Definition Language (DDL)

The DDL contains four main SQL statements:

  • CREATE

  • ALTER

  • DROP

  • DECLARE

The CREATE Statement
CREATE <database object>....

The CREATE statement is used to define database objects. Database objects are used for different purposes. Some are used to define a condition or relationship (index, trigger), and others are a logical representation of the data as it is physically stored on disk (table, table space).

The following database objects can be created with the CREATE statement:

  • ALIAS

  • AUXILIARY TABLE (LOBs)

  • DATABASE

  • DISTINCT TYPE (User-defined data type)

  • FUNCTION (User-defined functions)

  • GLOBAL TEMPORARY TABLE

  • INDEX

  • PROCEDURE (Stored procedures)

  • STOGROUP (Storage groups)

  • SYNONYM

  • TABLE

  • TABLESPACE

  • TRIGGER

  • VIEW

The creation of any database object using DDL will result in an update to the system catalog tables. Special database authorities or privileges are required to create database objects.

Database objects are used for different purposes. Some are reused to define a condition or relationship (index, trigger), and others are a logical representation of the data as it is physically stored on disk (table, tablespace).

The DECLARE Statement
DECLARE <database object>....

The DECLARE statement is very similar to the CREATE statement, except that one of the objects it can create is a temporary table. Temporary tables are used only during the duration of an application or stored procedure or connection. The table does not cause any logging or contention against the system catalog tables and is very useful for working with intermediate results. This table must be placed into an existing TEMP database. This TEMP database should have several segmented table spaces created within it. A single DECLARED table cannot span table spaces.

The creation of a temporary table will not result in any update to the system catalog tables, so locking, logging, and other forms of contention are avoided with this object.

DECLARED tables can be DROPPED and ALTERED, but no other database objects (such as views or triggers) can be created to act against them. Temporary tables do allow for the specification of a partitioning key.

Once a table is declared, it can be referenced like any other SQL table. The SQL in the following example shows a table being declared and then used in a subsequent SQL statement.

For more information on declared temporary tables, refer to Chapter 11.

The DROP Statement
DROP <database object>....

The DROP statement is used to remove definitions from the system catalog tables (and hence the database itself). Since rows in the system catalog tables cannot be directly deleted, the DROP statement is used to remove data rows from these tables. Since database objects can be dependent on other database objects, the act of dropping an object will result in dropping any object that is directly or indirectly dependent on that object. Any plan or package that is dependent on the object deleted from the catalog on the current server, will be invalidated. You can drop any object created with the CREATE <database object> and the DECLARE <table> statements.

The ALTER Statement
ALTER <database object>....

The ALTER statement allows you to change some characteristics of the database object. Any object being altered must already exist in the database. The database objects that can be altered are:

  • Database

  • Function

  • Index

  • Stored Procedure

  • Stogroup

  • Table

  • Table space

Every time you issue a DDL statement, the system catalog tables will be updated. The update will include a creation or modification timestamp and the authorization ID of the creator (modifier).

Let's look in detail at some of the objects that can be created. We will cover data types, tables, table spaces, views, indexes, databases, and storage groups.

Data Types

Data types are used to specify the attribute of the columns when creating a table. Before discussing a table or other objects, we have to understand the various data types supplied by DB2 or created by the users (User-Defined Data Types).

First let us look at the built-in data types supplied by DB2.

DB2-Supplied Data Types

When the database design is being implemented, any of these data types can be used. Data is stored in DB2 tables that are composed of columns and rows. Every DB2 table is defined by using columns. These columns must be one of the built-in DB2 data types or user-defined data types. Every DB2-supplied data type belongs to one of three major categories:

  • Numeric

  • String (binary, single byte, double byte)

  • Datetime

The valid built-in DB2 data types are as follows:

Built-in data types
Datetime types
Time TIME
Timestamp TIMESTAMP
Date DATE
String types
Character types
Fixed length CHAR
Varying length VARCHAR
CLOB
Graphic
Fixed length GRAPHIC
Varying length VARGRAPHIC
DBCLOB
Varying length binary BLOB
Signed numeric types
Exact
Binary integer
16 bit SMALLINT
32 bit INTEGER
Decimal
Packed DECIMAL
Approximate
Floating point
Single precision REAL
Double precision DOUBLE
Row identifier ROWID

Numeric Data Types

The six DB2 data types that can be used to store numeric data are:

  • SMALLINT

  • INTEGER

  • DECIMAL/NUMERIC

  • FLOAT (REAL or DOUBLE)

  • REAL

  • DOUBLE

These data types are used to store different numeric types and precisions. The precision of a number is the number of digits used to represent its value. The data is stored in the DB2 database using a fixed amount of storage for all numeric data types. The amount of storage required increases as the precision of the number goes up.

You must also be aware of the range limits of the data types and the corresponding application programming language when you are manipulating these numeric fields.

Some data values are of the integer type by nature, such as the number of test candidates. It would be impossible to have a number representing a number of people that contains fractional data (numbers to the right of the decimal). On the other hand, some values require decimal places to accurately reflect their value, such as test scores. These two examples should use different DB2 data types to store their values (SMALLINT and DECIMAL, respectively).

Numeric values should not be enclosed in quotation marks. If they are, the value is treated as a character string. Even if a field contains numbers in its representation, a DB2 numeric data type should be used to represent the data only if arithmetic operations should be allowed.

Small Integer (SMALLINT)

A small integer uses the least amount of storage in the database for each value. An integer does not allow any digits to the right of the decimal.

The data value range for a SMALLINT is –32768 to 32767. The precision for a SMALLINT is 5 digits (to the left of the decimal). Two bytes of database storage are used for each SMALLINT column value.

Integer (INTEGER)

An INTEGER takes twice as much storage as a SMALLINT but has a greater range of possible values.

The range value for an INTEGER data type is –2,147,483,648 to 2,147,483,647. The precision for an INTEGER is 10 digits to the left of the decimal.

Four bytes of database storage are used for each INTEGER column value.

Decimal (DECIMAL/NUMERIC)

A DECIMAL or NUMERIC data type is used for numbers with fractional and whole parts. The DECIMAL data is stored in a packed format.

The precision and scale must be provided when a decimal data type is used. The precision is the total number of digits (range from 1 to 31), and the scale is the number of digits in the fractional part of the number. For example, a decimal data type to store currency values of up to $1 million would require a definition of DECIMAL (9,2). The terms NUMERIC, DECIMAL, or DEC can all be used to declare a decimal/numeric column. If a decimal data type is to be used in a C program, the host variable must be declared as a double. A DECIMAL number takes up p/2 + 1 bytes of storage, where p is the precision used. For example, DEC (8,2) would take up 5 bytes of storage (8/2 + 1), whereas DEC (7,2) would take up only 4 bytes (truncate the division of p/2).

Single-Precision Floating-Point (REAL/FLOAT)

A REAL data type is an approximation of a number. The approximation requires 32 bits or 4 bytes of storage. To specify a single-precision number using the REAL data type, its length must be defined between 1 and 21 (especially if the FLOAT data type is used, as it can represent both single- and double-precision and is determined by the integer value specified).

Double-Precision Floating-Point (DOUBLE/FLOAT)

A DOUBLE or FLOAT data type is an approximation of a number. The approximation requires 64 bits or 8 bytes of storage. To specify a double-precision number using the FLOAT data type, its length must be defined between 22 and 53.

String Data Types

This section discusses the string data types that include:

  • CHAR

  • VARCHAR

  • CLOB

  • GRAPHIC

  • VARGRAPHIC

  • DBCLOB

  • BLOB

Note

The syntax of LONG VARCHAR and LONG VARGRAPHIC is still supported but the alternative syntax of VARCHAR (integer) and VARGRAPHIC (integer) is preferred. DB2 will translate the LONG definitions into the other format prior to actually defining the table. Therefore, only the VARCHAR() and VARGRAPHIC() definitions should be used.

Fixed-Length Character String (CHAR)

Fixed-length character strings are stored in the database using the entire defined amount of storage. If the data being stored always has the same length, a CHAR data type should be used.

Using fixed-length character fields can potentially waste disk space within the database if the data is not using the defined amount of storage. However, there is overhead involved in storing varying-length character strings.

The length of a fixed-length string must be between 1 and 255 characters. If you do not supply a value for the length, a value of 1 is assumed.

Varying-Length Character String (VARCHAR)

Varying-length character strings are stored in the database using only the amount of space required to store the data and a 2-byte prefix to hold the length. The individual names, in our example, are stored as varying-length strings (VARCHAR) because each person's name has a different length (up to a maximum length of 30 characters).

If a varying-length character string is updated and the resulting value is larger than the original, it may not fit on the same page, and the row will be moved to another page in the table, leaving a marker in the original place. These marker data records are known as indirect reference rows. Too many of these records can cause significant performance degradation, since multiple pages (IOs) are required to return a single data record.

A VARCHAR column has the restriction that it must fit on one database page. This means that a 4-KB page would allow a VARCHAR up to 4,046 characters long (defined as VARCHAR(4046)), an 8-KB page would be up to 8,128 characters long, and so on up to a 32-KB page with a maximum column length of 32,704 bytes. This means that you must create a table space for this table that can accommodate the larger page size, and you must have sufficient space in the row to accommodate this string.

Note

Character strings on the OS/390 are stored in the database without a termination character. Depending on the non-OS/390 development environment, a null-terminator may or may not be appended to the end of a character string when the data is stored or retrieved.

Character Large Object (CLOB)

Character large objects are varying-length SBCS (single-byte character set) or MBCS (multibyte character set) character strings that are stored in the database. CLOB columns are used to store text greater than 32-KB. The maximum size for each CLOB column is 2-GB (2 gigabytes—1 byte). Since this data type is of varying length, the amount of disk space allocated is determined by the amount of data in each record. Therefore, you should create the column specifying the length of the longest string.

Note

The FOR BIT DATA clause can be used following a character string column definition. During data exchange, code page conversions are not performed. Rather, data is treated and compared as binary (bit) data.

Double-Byte Character Strings (GRAPHIC)

The GRAPHIC data types represent a single character using 2 bytes of storage. The GRAPHIC data types include: GRAPHIC (fixed length—maximum 127 DBCS characters), VARGRAPHIC (varying length—maximum 32704 DBCS characters for 32-KB pages), and DBCLOB.

Note

VARCHAR data types over 255 bytes are similar to CLOB data types. (Both types have usage restrictions.)

Double-Byte Character Large Objects (DBCLOB)

Double-byte character large objects are varying-length character strings that are stored in the database using 2 bytes to represent each character. There is a code page associated with each column. DBCLOB columns are used for large amounts (>32KB) of double-byte text data such as Japanese text. The maximum length should be specified during the column definition because each data record will be variable in length.

Binary Large Objects (BLOB)

Binary large objects are variable-length binary strings. The data is stored in a binary format in the database. There are restrictions when using this data type, including the inability to sort using this type of column. The BLOB data type is useful for storing nontraditional relational database information. The maximum size of each BLOB column is 2-GB (2 gigabytes—1 byte). Since this data type is of varying length, the amount of disk space allocated is determined by the amount of data in each record, not by the defined maximum size of the column in the table definition.

Large Object Considerations

Traditionally, large unstructured data was stored somewhere outside the database. Therefore, the data could not be accessed using SQL. Besides the traditional database data types, DB2 implements data types that will store large amounts of unstructured data. These data types are known as Large Objects (LOBs). Multiple LOB columns can be defined for a single table. DB2 provides special considerations for handling these large objects. You can choose not to log the LOB values to avoid large amounts of data being logged.

There is a LOG option that can be specified during the CREATE TABLESPACE statement for each AUXILIARY TABLE holding LOB column data to avoid logging any modifications.

If you want to define a LOB column greater than 1-GB, you must specify the LOG NO option.

In a database, you may choose to use BLOBs for the storage of pictures, images, or audio or video objects, along with large documents. BLOB columns will accept any binary string without regard to the contents.

If you want to manipulate textual data that is greater than 32KB in length, use CLOB or a character large object data type. For example, if each test candidate were required to submit his or her resume, the resume could be stored in a CLOB column along with the rest of the candidate's information. There are many SQL functions that can be used to manipulate large character data columns.

Date and Time Data Types

There are three DB2 data types specifically used to represent dates and times:

  • DATE

    • This data type is stored internally as a (packed) string of 4 bytes. Externally, the string has a length of 10 bytes (MM-DD-YYYY; this representation can vary and is dependent on the DSNHDECP settings at installation time. DB2 also allows for creation of customized date formats by means of an exit routine).

  • TIME

    • This data type is stored internally as a (packed) string of 3 bytes. Externally, the string has a length of 8 bytes (HH-MM-SS; this representation may vary depending on the DSNHDECP setting).

  • TIMESTAMP

    • This data type is stored internally as a (packed) string of 10 bytes. Externally, the string has a length of 26 bytes (YYYY-MM-DD-HH-MM-SS-NNNNNN).

From the user perspective, these data types can be treated as character or string data types. Every time you need to use a datetime attribute, you will need to enclose it in quotation marks. However, datetime data types are not stored in the database as fixed-length character strings.

Note

The internal format of date, time, and timestamps is not at all similar to the external format returned via SQL. For example, the internal format of timestamp is 10 bytes, but the external format is 26 characters.

DB2 provides special functions that allow you to manipulate these data types. These functions allow you to extract the month, hour, or year of a date-time column. The date and time formats correspond to the site default.

Therefore, the string that represents a date value will change depending on the default format. In some countries, the date format is DD/MM/YYYY, whereas in other countries, it is YYYY-MM-DD. You should be aware of the default format used by your site to use the correct date string format. If an incorrect date format is used in a SQL statement, an SQL error will be reported. There are scalar functions that will return date, time, and timestamp columns in formats other than the default.

As a general recommendation, if you are interested in a single element of a date string, say month or year, always use the SQL functions provided by DB2 to interpret the column value. By using the SQL functions, your application will be more portable.

Note

TIMESTAMP fields use the most storage, but they contain the most accurate time since they include microseconds.

Note

Timestamps cannot be guaranteed to be unique since the granularity is only to microseconds, and it is possible for two events to ask the system timer for the timestamp within one microsecond. The 8-byte system clock is unique, however, because it appends the time value with additional bits that make it unique.

We stated that all datetime data types have an internal and external format. The external format is always a character string. Let us examine the various datetime data type formats available in DB2.

Date String (DATE)

There are a number of valid methods of representing a DATE as a string. Any of the string formats shown in the following table (Table 4.1) can be used to store dates in a DB2 database. When the data is retrieved (using a SELECT statement), the output string will be in one of these formats, or can be returned in any format specifically specified.

Additionally, there are many scalar functions in DB2 to return date information, such as:

  • DATE

  • DAY

  • DAY OF MONTH

  • DAY OF WEEK

  • DAY OF WEEK_ISO

  • DAY OF YEAR

Table 4.1. Date Formats

International Standards Organization ISO

YYYY-MM-DD

IBM USA Standard USA

MM/DD/YYYY

IBM European Standard EUR

DD.MM.YYYY

Japanese Industrial Standard JIS

YYYY-MM-DD

  • JULIAN_DAY

  • MONTH

  • QUARTER

  • WEEK

  • WEEK_ISO

  • YEAR

Time String (TIME)

There are a number of valid methods for representing a TIME as a string. Any of the string formats in the following table can be used to store times in a DB2 database. When data is retrieved, the external format of the time will be one of the formats shown in Table 4.2 below.

Additionally, there are many scalar functions in DB2 to return time information, such as:

  • HOUR

  • MINUTE

  • SECOND

  • TIME

Table 4.2. Time Formats

International Standards Organization ISO

HH.MM.SS

IBM USA Standard USA

HH:MM AM or PM

IBM European Standard EUR

HH.MM.SS

Japanese Industrial Standard JIS

HH:MM:SS

Timestamp String (TIMESTAMP)

The timestamp data type has a single default external format. Timestamps have an external representation as YYYY-MM-DD-HH.MM.SS.NNNNNN (Year-Month-Day-Hour-Minute-Seconds-Microseconds). However, there are several scalar functions that can manipulate the output format, especially the TIMESTAMP_FORMAT function which returns any type of string, up to 255 characters, based on a user-defined template.

Additionally, there are many scalar functions in DB2 to return timestamp information besides all those listed for DATE and TIME, such as:

  • MICROSECOND

  • TIMESTAMP

  • TIMESTAMP_FORMAT

User-Defined Data Types

User-defined types (UDTs) allow a user to extend the data-types that DB2 understands in a database. DB2 for OS/390 and z/OS supports only the user-defined distinct type. The user-defined data types (UDTs) can be created on an existing data type or on other user-defined data types. UDTs are used to define further types of data being represented in the database. If columns are defined using different UDTs based on the same base data type, these UDTs cannot be directly compared. This is known as strong typing. DB2 provides this strong data typing to avoid end-user mistakes during the assignment or comparison of different types of real-world data. For more information on how to create and use UDTs, refer to Chapter 13, Advanced Functions.

Null Value Considerations

A null value represents an unknown state. Therefore, when columns containing null values are used in calculations, the result is unknown. All of the data types discussed in the previous section support the presence of null values. During the table definition, you can specify that a valid value must be provided. This is accomplished by adding a phrase to the column definition. The CREATE TABLE statement can contain the phrase NOT NULL following the definition of each column. This will ensure that the column contains a known data value.

Special considerations are required to properly handle null values when coding a DB2 application. DB2 treats a null value differently from other data values.

To define a column not to accept null values, add the phrase NOT NULL to the end of the column definition:

CREATE TABLE t1 (c1 CHAR (3) NOT NULL)

From the example above, DB2 will not allow any null values to be stored in the c1 column. In general, avoid using nullable columns unless they are required to implement the database design. There is also overhead storage you must consider. An extra byte per nullable column is necessary if null values are allowed.

Note

Relational databases allow null values. It is important to remember that they can be appropriate for your database design.

NOT NULL with Default

When you insert a row into a table and omit the value of one or more columns, those columns may either be populated using a null value (if the column is defined as nullable) or a defined default value (if you have specified this to be used). If the column is defined as not nullable, the insert will fail unless the data has been provided for the column. DB2 has a defined default value for each of the DB2 data types, but you can provide a default value for each of the columns. The default value is specified in the CREATE TABLE statement. By defining your own default value, you can ensure that the data value has been populated with a known value.

Now, all the INSERT statements that omit the DEPT column will populate the column with the default value of 10. The COMM column is defined as WITH DEFAULT.

In this case, you can choose at insert time between null or the default value of 15. To ensure that the default value is being used during an INSERT operation, the keyword DEFAULT should be specified in the VALUES portion of the INSERT statement.

The following example shows two examples of inserting a record with user-defined default values:

CREATE TABLE Staff
(ID SMALLINT NOT NULL,
NAME VARCHAR (9),
DEPT SMALLINT not null with default 10,
JOB CHAR (5),
YEARS SMALLINT,
SALARY DECIMAL(7,2),
COMM DECIMAL (7,2) not null with default 15);

In this case, both cause the same result.

INSERT INTO STAFF
values (360,'Purcell', DEFAULT,'SE', 8, 20000, DEFAULT);
INSERT INTO STAFF (ID, NAME, JOB, YEARS, SALARY)
values (360, 'Purcell', 'SE', 8, 20000, DEFAULT);

    The result is
    ID     NAME      DEPT   JOB  YEARS  SALARY    COMM
    ------ --------- ------ ---- ------ --------- ---------
    360    Purcell   10     SE   8      20000.00  15.00
    1 record(s) selected.
Identity Column

The previous section discussed how columns can be populated with values if no value was supplied by the user. It is also possible to have DB2 generate sequence numbers or other values as part of a column during record insertion.

In the majority of applications, a single column within a table represents a unique identifier for that row. Often this identifier is a number that is sequentially updated as new records are added.

In DB2, a feature exists that will automatically generate this value on behalf of the user. The following example shows a table definition with the EMPNO field being automatically generated as a sequence number:

CREATE TABLE EMPLOYEE
(EMPNO INT GENERATED ALWAYS AS IDENTITY,
NAME CHAR (10));

INSERT INTO EMPLOYEE(NAME) VALUES 'YEVICH';

SELECT * FROM EMPLOYEE;
EMPNO       NAME
----------- ----------
1           YEVICH

If the column is defined with GENERATED ALWAYS, then the INSERT statement cannot specify a value for the EMPNO field. By default, the numbering will start at 1 and increment by 1. The starting and increment values can be modified as part of the column definition:

CREATE TABLE EMPLOYEE
(EMPNO INT GENERATED ALWAYS AS
IDENTITY (START WITH 100, INCREMENT BY 10),
NAME CHAR (10));

    INSERT INTO EMPLOYEE (NAME) VALUES 'YEVICH' , ' LAWSON';

    SELECT * FROM EMPLOYEE;
    EMPNO  NAME
    ---------- ----------
       100 YEVICH
       110 LAWSON

In addition, the default value can be GENERATED BY DEFAULT, which means that the user has the option of supplying a value for the field. If no value is supplied (using the DEFAULT keyword), DB2 will generate the next number in sequence.

Note

To be sure that the number is truly unique, a unique index on the identity column is required.

One additional keyword is available as part of IDENTITY columns. As a DBA, you can decide how many numbers should be "pregenerated" by DB2 if you specify this number in the CACHE option. This can help reduce catalog contention since DB2 will store the next n numbers in memory rather than go back to the catalog tables to determine which number to generate next.

Identity columns are restricted to numeric values (integer or decimal) and can be used in only one column in the table definition.

CREATE TABLE EMPLOYEE
(EMPNO INT GENERATED ALWAYS AS IDENTITY,
NAME CHAR (10),
SALARY INT,
BONUS INT,
PAY INT);

    INSERT INTO EMPLOYEE (NAME, SALARY, BONUS, PAY) VALUES
('YEVICH', 20000, 2000, 22000);

SELECT * FROM EMPLOYEE;
EMPNO   NAME       SALARY     BONUS      PAY
------- ---------- ---------- ---------- ----------
1       YEVICH     20000      2000       22000

The EMPNO is generated as an IDENTITY column.

ROWID

A row ID is a value that uniquely identifies a row in a table. A ROWID column enables queries to be written that navigate directly to a row in the table. ROWID column values are generated by DB2 unless supplied, but they must be unique. If a value is supplied, it must be a valid DB2 generated value. The internal format is 17 bytes and contains bit data. The external length is 40 bytes.

Unicode Support in DB2

Unicode support is another very important feature. Support for Unicode will help with support across multinational boundaries. It is an encoding scheme that allows for the representation of codepoints and characters of several different geographies and language. Unicode character encoding standard is a fixed-length, character encoding scheme that includes characters from most of the world's languages. Unicode characters are usually shown as "U+xxxx" where xxxx is the hexadecimal code of the character. Each character is 16 bits wide, allowing for the support of 65,000 characters. The normal support that is provided is the UCS-2/UTF-8 standard. With UCS-2 or Unicode encoding, ASCII and control characters are also two bytes long, and the lead byte is zero. Since extraneous NULLs may appear anywhere in the string, this could be a problem for ASCII. UTF-8 is a transformation algorithm that is used to avoid the problem for programs that rely on ASCII code. UTF-8 transforms fixed-length UCS characters into variable-length byte strings. ASCII characters are represented by single-byte codes, but non-ASCII characters are two or three bytes long. UTF-8 transforms UCS-2 characters to a multi-byte codeset.

The UCS-2 code page is being registered as code page 1200. When new characters are added to a code page, the code page number does not change. Code page 1200 always refers to the current version of Unicode/UCS-2, and has been used for UCS-2 support in DB2 UDB on many of the other platforms. UTF-8 has been registered as CCSID 1208 (Code Page 1208) and is used as the multi-byte code page number for DB2's UCS-2/UTF-8 support.

DB2 UDB supports UCS-2 as a new multi-byte code page. CHAR, VARCHAR, LONG VARCHAR, and CLOB data, are stored in UTF-8, and GRAPHIC, VARGRAPHIC, LONG VARGRAPHIC, and DBCLOB data, are stored in UCS-2.

Databases are created in the default encoding scheme unless specified. CCSID can be used to specifically detail Unicode.

CREATE DATABASE dbname CCSID UNICODE

A UCS-2 database allows connection from every single-byte and multi-byte code page supported. Code page character conversions between the client's code page and UTF-8 are automatically performed by DB2. Data in graphic string types, is always in UCS-2 and does not go through code page conversions. While some client workstations have a limited subset of UCS-2 characters, the database allows the entire repertoire of UCS-2 characters.

All supported data types are also supported in a UCS-2 database. In a UCS-2 database, all identifiers are in multi-byte UTF-8. Therefore, it is possible to use any UCS-2 character in identifiers where the use of a character in the extended character set is allowed by DB2. This feature will also allow UCS-2 literals to be specified either in GRAPHIC string constant format using the G'...', or N' ...' format or as a UCS-2 hexadecimal string, using the UX' ...' or GX' ...' format.

Selecting the Correct Data Type

Knowledge of the possible data values and their usage is required in order to be able to select the correct data type. Specifying an inappropriate data type when defining the tables can result in:

  • Wasted disk space

  • Improper expression evaluation

  • Performance considerations

A small checklist for data type selection is shown below in Table 4.3.

Remember that you need to create page sizes that are large enough to contain the length of a row in a table. This is particularly important for tables with large character columns.

When using character data types, the choice between CHAR and VARCHAR is determined by the range of lengths of the columns. For example, if the range of column length is relatively small, use a fixed character with the maximum length. This will reduce the storage requirements and could improve performance.

Tables

Tables consist of columns and rows that store an unordered set of data records. Tables can have constraints to guarantee the uniqueness of data records, maintaining the relationship between and within tables, and so on. A constraint is a rule that the database manager enforces. There are three types of constraints:

Table 4.3. Selecting a Data Type

Is the data variable in length?

VARCHAR

If the data is variable in length, what is the maximum length?

VARCHAR

Do you need to sort (order) the data?

CHAR, VARCHAR, NUMERIC

Is the data going to be used in arithmetic operations?

DECIMAL, NUMERIC, REAL, DOUBLE, FLOAT, INTEGER, SMALLINT

Does the data element contain decimals?

DECIMAL, NUMERIC, REAL, DOUBLE

Is the data fixed in length?

CHAR

Does the data have a specific meaning (beyond DB2 base data types)?

USER DEFINED TYPE

Is the data larger than what a character string can store, or do you need to store non-traditional data?

CLOB, BLOB, DBCLOB

  • Unique constraint

    • Ensures the unique values of a key in a table. Any changes to the columns that comprise the unique key are checked for uniqueness.

  • Referential constraint

    • Enforces referential constraints on insert, update, and delete operations. It is the state of a database in which all values of all foreign keys are valid.

  • Check constraint

    • Verifies that changed data does not violate conditions specified when a table was created or altered.

Unique Constraints

A unique constraint is the rule that the values of a key are valid only if they are unique within the table. Each column making up the key in a unique constraint must be defined as NOT NULL. Unique constraints are defined in the CREATE TABLE statement or the ALTER TABLE statement using the PRIMARY KEY clause or the UNIQUE clause. When used, the table is marked as unavailable until you explicitly create an index for any key specified as UNIQUE or PRIMARY KEY, unless processed by the schema processor, in which case DB2 will implicitly create all necessary indexes.

A table can have any number of unique constraints; however, a table cannot have more than one unique constraint on the same set of columns.

The enforcement of the constraint is through the unique index. Once a unique constraint has been established on a column, the check for uniqueness during multiple row updates is deferred until the end of the update (deferred unique constraint).

A unique constraint can also be used as the parent key in a referential constraint.

Referential Integrity

Referential integrity allows you to define required relationships between and within tables. The database manager maintains these relationships, which are expressed as referential constraints, and requires that all values of a given attribute or table column also exist in some other table column. Figure 4.2 shows an example of the referential integrity between two tables.

Figure 4.2. Referential Integrity Between Two Table

This constraint requires that every employee in the EMPLOYEE table must be in a department that exists in the DEPARTMENT table. No employee can be in a department that does not exist.

A unique key is a set of columns in which no two values are duplicated in any other row. Only one unique key can be defined as a primary key for each table. The unique key may also be known as the parent key when referenced by a foreign key.

A primary key is a special case of a unique key. Each table can have only one primary key. In this example, DEPTNO and EMPNO are the primary keys of the DEPARTMENT and EMPLOYEE tables.

A foreign key is a column or set of columns in a table that refers to a unique key or primary key of the same or another table. A foreign key is used to establish a relationship with a unique key or primary key and enforces referential integrity among tables. The column WORKDEPT in the EMPLOYEE table is a foreign key because it refers to the primary key, column DEPTNO, in the DEPARTMENT table.

A parent key is a primary key or unique key of a referential constraint. A parent table is a table containing a parent key that is related to at least one foreign key in the same or another table. A table can be a parent in an arbitrary number of relationships. In this example, the DEPARTMENT table, which has a primary key of DEPTNO, is a parent of the EMPLOYEE table, which contains the foreign key WORKDEPT.

A dependent table is a table containing one or more foreign keys. A dependent table can also be a parent table. A table can be a dependent in an arbitrary number of relationships. For example, the EMPLOYEE table contains the foreign key WORKDEPT, which is dependent on the DEPARTMENT table that has a primary key.

A referential constraint is an assertion that non-null values of a designated foreign key are valid only if they also appear as values of a unique key of a designated parent table. The purpose of referential constraints is to guarantee that database relationships are maintained and data entry rules are followed.

Enforcement of referential constraints has special implications for some SQL operations that depend on whether the table is a parent or a dependent. The database manager enforces referential constraints across systems based on the referential integrity rules. The rules are INSERT rule, DELETE rule, and UPDATE rule. However, only the delete rules are explicitly defined.

INSERT Rules

The INSERT rule is implicit when a foreign key is specified. You can insert a row at any time into a parent table without any action being taken in the dependent table.

You cannot insert a row into a dependent table unless there is a row in the parent table with a parent key value equal to the foreign key value of the row that is being inserted unless the foreign key value is null. If an INSERT operation fails for one row during an attempt to insert more than one row, all rows inserted by the statement are removed from the database.

DELETE Rules

When you delete a row from a parent table, the database manager checks to see if there are any dependent rows in the dependent table with matching foreign key values. If any dependent rows are found, several actions can be taken. You determine which action will be taken by specifying a delete rule when you create the dependent table.

  • RESTRICT or NO ACTION

    • This rule prevents any row in the parent table from being deleted if any dependent rows are found. If you need to remove both parent and dependent rows, delete the dependent rows first.

  • CASCADE

    • This rule will automatically delete the row from the dependent table when the parent row is deleted.

  • SET NULL

    • This option will set the value of the foreign key to NULL (providing that it allows nulls). Other parts of the row will remain unchanged.

UPDATE Rules

The database manager prevents the update of a unique key of a parent row. When you update a foreign key in a dependent table and the foreign key is defined with the NOT NULL option, it must match some value of the parent key of the parent table.

Check Constraints

Table-check constraints will enforce data integrity at the table level. Once a table-check constraint has been defined for a table, every UPDATE and INSERT statement will involve checking the restriction or constraint. If the constraint is violated, the data record will not be inserted or updated, and an SQL error will be returned.

A table-check constraint can be defined at table creation time or later using the ALTER TABLE statement.

The table-check constraints can help implement specific rules for the data values contained in the table by specifying the values allowed in one or more columns in every row of a table. This can save time for the application developer since the validation of each data value can be performed by the database and not by each of the applications accessing the database.

When a check constraint is created or added, DB2 will perform a syntax check on it. A check constraint cannot contain host variables or special registers.

The check constraint's definition is stored in the system catalog tables, specifically the SYSIBM.SYSCHECKS and SYSIBM.SYSCHECKDEP table.

Adding Check Constraints

When you add a check constraint to a table that contains data, one of two things can happen:

  • All the rows meet the check constraint.

  • Some or all the rows do not meet the check constraint.

In the first case, when all the rows meet the check constraint, the check constraint will be created successfully. Future attempts to insert or update data that does not meet the constraint business rule will be rejected.

When there are some rows that do not meet the check constraint, the check constraint will not be created (i.e., the ALTER TABLE statement will fail), depending on the CURRENT RULES setting. The ALTER TABLE statement, which adds a new constraint to the EMPLOYEE table, is shown below. The check constraint is named check_job. DB2 will use this name to inform us which constraint was violated if an INSERT or UPDATE statement fails.

ALTER may be used to change a constraint definition, but it will place the table in a check pending state. For example, once a table is in a check pending state, the use of SELECT, INSERT, UPDATE, and DELETE is disallowed on a table. In order to reset the Check Pending (CHKP) flag, you will need to run the CHECK DATA utility. For more information on this utility, refer to Chapter 8.

ALTER TABLE EMPLOYEE
ADD CONSTRAINT check_job
CHECK (JOB IN ('Engineer' , 'Sales' , 'Manager'));

Note

It is a good idea to label every constraint (triggers, table-check, or referential integrity). This is particularly important for diagnosing errors that might occur.

Modifying Check Constraints

As check constraints are used to implement business rules, you may need to change them from time to time. This could happen when the business rules change in your organization. There is no special command used to change a check constraint. Whenever a check constraint needs to be changed, you must drop it and create a new one. Check constraints can be dropped at any time, and this action will not affect your table or the data within it.

When you drop a check constraint, you must be aware that data validation performed by the constraint will no longer be in effect. The statement used to drop a constraint is the ALTER TABLE statement.

The following example shows how to modify the existing constraint. After dropping the constraint, you have to create it with the new definition.

ALTER TABLE EMPLOYEE DROP CONSTRAINT check_job;
ALTER TABLE EMPLOYEE
ADD CONSTRAINT check_job
CHECK (JOB IN ('OPERATOR' , 'CLERK'));
Creating Tables

The CREATE TABLE statement allows you to define a new table. The definition must include its name and the attributes of its columns. The definition may include other attributes of the table such as its primary key or check constraints.

Once the table is defined, column names and data types cannot be modified. However, new columns can be added to the table (be careful when adding new columns since default data values will be used for existing records).

The RENAME TABLE statement can change the name of an existing table. The maximum number of columns that a table can consist of is 750. This maximum will NOT vary depending on the data page size on the OS/390. DB2 supports 4-KB, 8-KB, 16-KB, and 32-KB data page sizes. Table 4.4 shows the maximum number of columns in a table and maximum length of a row by page size.

Note

If an EDITPROC was defined on a table, the above maximum row sizes would decrease by 10 bytes in each case.

Note

If the table is a dependent table in an RI structure, then the maximum number of columns is 749.

Tables are always created within a table space. Users can specify the table space name in which the table will be created or DB2 will create one implicitly. In the below example, DB2 will implicitly create a table space in the HUMANDB database because no table space name was provided. The name of the table space will be derived from the table.

Table 4.4. Page Size Limitations

4-KB Page

8-KB Page

16-KB Page

32-KB Page

Max. # of Columns

750

750

750

750

Max. Row Length (bytes)

4056

8138

16330

32714

CREATE TABLE DEPARTMENT
(DEPTNUMB SMALLINT NOT NULL,
DEPTNAME VARCHAR (20),
MGRNO SMALLINT,
PRIMARY KEY(Deptnumb)
IN DATABASE DB2CERT

Note

The keyword DATABASE is optional; if left off, DB2 will still implicitly create the table space in the DB2CERT database.

Note

Indexes will have their own index spaces created when they are created.

After the table is created, user data can be placed into the table using one of these methods:

  • INSERT statement

  • LOAD Utility

  • DSN1COPY

Note

The NO LOG option is available only for LOB table spaces.

Note

If you really need a temporary table that is used only during the duration of a program, use the DECLARE GLOBAL TEMP TABLE statement instead. This will result in no catalog contention, no logging, and no lock contention.

The following example shows sample CREATE TABLE statements. This sample creates two tables. This definition includes unique constraints, check constraints, and referential integrity. In this example:

  • The DEPARTMENT table has a primary key that consists of column DEPTNUMB.

  • The EMPLOYEE table has a check constraint that says JOB should be Sales, Mgr, or Clerk.

  • The default value is defined for the column HIREDATE in the EMPLOYEE table.

  • The EMPLOYEE table has a primary key that consists of column ID.

  • A referential constraint is defined between the DEPARTMENT table and the EMPLOYEE table.

  • The EMPLOYEE table is created in the HUMANTS table space in the HUMANDB database.

CREATE TABLE DEPARTMENT
(DEPTNUMB SMALLINT NOT NULL,
DEPTNAME VARCHAR (20),
MGRNO SMALLINT,
PRIMARY KEY (DEPTNUMB))
IN HUMANDB.HUMANTS;

CREATE TABLE EMPLOYEE
(ID SMALLINT NOT NULL,
NAME VARCHAR (9) NOT NULL,
DEPT SMALLINT,
JOB CHAR (5) CHECK (JOB IN
('SALES' , 'MGR' , 'CLERK')),
HIREDATE DATE WITH DEFAULT CURRENT DATE,
SALARY DECIMAL (7,2),
COMM DECIMAL (7,2),
PRIMARY KEY (ID),
FOREIGN KEY (DEPT) REFERENCES DEPARTMENT (DEPTNUMB)
ON DELETE RESTRICT)
IN HUMANDB.HUMANTS;

Note

The table definition will be marked incomplete because no indexes are created at this time to enforce the primary key.

Auxiliary Tables

LOB data is not actually stored in the table in which it is defined. The defined LOB column holds information about the LOB while the LOB itself is stored in another location. The normal place for this data storage is a LOB table space defining the physical storage that will hold an auxiliary table related to the base column and table.

Because the actual LOB is stored in a separate table, one performance consideration might be that you have a large variable character column in use (that is infrequently accessed); you may be able to convert it to a LOB so it is kept separately, and this could speed up table space scans on the remaining data because fewer pages would be accessed.

Null is the only supported default value for a LOB column, and if the value is NULL, then it will not take up space in the LOB table space. The examples below show how to create a base table with a LOB and an auxiliary table to support it.

CREATE TABLE DB2USER1.CANDIDATE
 (CID           CANDIDATE_ID    NOT NULL,
...
  CERT_APP      CHAR (1) NOT NULL WITH DEFAULT,
  PHOTO         BITMAP,
  PRIMARY KEY (CID)
  IN DB2CERT.CERTTS);

CREATE AUX TABLE CAND_PHOTO
  IN DB2CERT.CERTPIC
  STORES DB2USER1.CANDIDATE
  COLUMN PHOTO;
Copying a Table Definition

It is possible to create a table using the same characteristics of another table (or a view). This is done through the CREATE TABLE LIKE statement.

The name specified after LIKE must identify a table or view that exists at the current server, and the privilege set must implicitly or explicitly include the SELECT privilege on the identified table or view. An identified table must not be an auxiliary table. An identified view must not include a column that is considered to be a ROWID column or an identity column.

The use of LIKE is an implicit definition of n columns, where n is the number of columns in the identified table or view. The implicit definition includes all attributes of the n columns as they are described in SYSCOLUMNS with a few exceptions such as identity attributes (unless the INCLUDING IDENTITY clause is used).

The implicit definition does not include any other attributes of the identified table or view. For example, the new table will not have a primary key, foreign key, or check constraint. The table is created in the table space implicitly or explicitly specified by the IN clause, and the table has any other optional clause only if the optional clause is specified.

Below is an example of the CREATE LIKE statement:

CREATE TABLE NEW_DEPT LIKE DEPARTMENT
IN DATABASE HUMANDB
Modifying a Table

After creating a table, the ALTER TABLE statement enables you to modify existing tables. The ALTER TABLE statement modifies existing tables by:

  • Adding one or more columns to a table

  • Adding or dropping a primary key

  • Adding or dropping one or more unique or referential constraints

  • Adding or dropping one or more check constraint definitions

  • Altering the length of a VARCHAR column

  • Enabling auditing of the table

  • Renaming a table

Some of the attributes of a table can be changed only after the table is created. The below example shows how to add a check constraint to the DEPARTMENT table.

ALTER TABLE DEPARTMENT ADD CHECK (DEPTNUM > 10)
Removing a Table

When you want to remove a table, issue this statement:

DROP TABLE EMPLOYEE

Note

Any objects that are directly or indirectly dependent on this table are deleted or made inoperative, for example, indexes, triggers, and views. Whenever a table is deleted, its description is deleted from the catalog and any packages that reference the object are invalidated.

Table Spaces

Data is stored in table spaces that are comprised of one or many VSAM data sets. There are four types of table spaces:

  • Simple

  • Segmented

  • Partitioned

  • LOB

Simple Table Space

Simple table spaces are the default but normally not the optimal. In a simple table space, you can have more than one table in the table space. If you have several tables in the table space, you can have rows from different tables interleaved on the same page; therefore, when a page is locked, you are potentially locking rows of other tables.

Segmented Table Space

Normally, in the cases where a table is not partitioned, a segmented table space is used, not simple. A segmented table space organizes pages of the table space into segments, and each segment will contain only the rows of one table. Segments can be composed of 4 to 64 pages each (in multiples of 4), and each segment will have the same number of pages.

There are several advantages to using segmented table spaces. Since the pages in a segment will contain only rows from one table, there will be no locking interference with other tables. In simple table spaces, rows are intermixed on pages, and if one table page is locked, it can inadvertently lock a row of another table just because it is on the same page. When you have only one table per table space, this is not an issue; however, there are still several benefits to having a segmented table space for one table such as:

  • If a table scan is performed, the segments belonging to the table being scanned are the only ones accessed; empty pages will not be scanned.

  • If a mass delete or a DROP table occurs, segment pages are available for immediate reuse (after the commit), and it is not necessary to run a REORG utility.

  • Mass deletes are much faster for segmented table space, and they produce less logging.

  • The COPY utility will not have to copy empty pages left by a mass delete.

  • When inserting records, some read operations can be avoided by using the more comprehensive space map of the segmented table space.

  • By being able to safely combine several tables in a table space, you can reduce the number of open data sets necessary, which reduces the amount of memory required in the subsystem.

When using a segmented table space to hold more than one table, make sure the tables have very similar characteristics in all categories, including size, volatility, locking needs, compression, and backup/recovery strategies.

There are some guidelines for how many tables to have in a segmented table space based upon the number of pages in the table space, but number or pages is not the only consideration. Table 4.5 shows some general guidelines.

The SEGSIZE is what tells DB2 on OS/390 how large to make each segment for a segmented table space, and it will determine how many pages are contained in a segment. The SEGSIZE will vary, depending on the size of the table space. Recommendations are as follows:

Number of Pages | SEGSIZE
28 or less | 4 to 28
between 28 and 128 | 32
128 or more | 64

Note

A segmented table space is a single data set and is limited to 2 GB.

Table 4.5. Table Space Design Considerations

Number of Pages

Table Space Design

>100,000

Consider partitioning

>10,000

One table segmented table space

>128 < 10,000

Multiple table segmented table spaces

<128

Multiple table segmented table spaces

The following example shows how to create a segmented table space with a segment size of 32:

CREATE TABLESPACE CERTTS
IN DB2CERT
USING STOGROUP CERTSTG
 PRIQTY 52
 SECQTY 20
ERASE NO
LOCKSIZE PAGE
BUFFERPOOL BP6
CLOSE YES
SEGSIZE 32;

Note

The SEGSIZE parameter cannot be ALTERed.

Partitioned Table Space

There are several advantages to partitioning a table space. For large tables, partitioning is the only way to store large amounts of data, but partitioning also has advantages for tables that are not necessarily large. DB2 allows us to define 254 partitions up to 64 GB each. Non-partitioned table spaces are limited to 64 GB of data. You can take advantage of the ability to execute utilities on separate partitions in parallel. This also gives you the ability to access data in certain partitions while utilities are executing on others. In a data-sharing environment, you can spread the workload across members in order to access data in separate partitions. You can also spread your data over multiple volumes and need not use the same storage group for each dataset belonging to the table space. This also allows you to place frequently accessed partitions on faster devices.

The following shows an example of how to create a partitioned table space with 2 partitions:

CREATE TABLESPACE CERTTSPT
IN DB2CERT
USING STOGROUP CERTSTG
   PRIQTY 100
   SECQTY 120
   ERASE NO
NUMPARTS 2
(PART 1
COMPRESS YES,
PART 2
FREEPAGE 20)
ERASE NO
LOCKSIZE PAGE
CLOSE NO;

Note

It is possible to use the ALTER statement to change the partitioning key ranges for rebalancing.

LOB Table Spaces

A LOB table space needs to be created for each column (or each column of each partition) of a base LOB table (table with a LOB column). This table space will contain the auxiliary table. This LOB table space has a structure that can be up to 4,000 TB in size. This is a storage model used only for LOB table spaces. This linear table space can be up to 254 data sets, which can be up to 64 GB. The LOB table space is basically implemented in the same fashion as pieces are implemented for NPIs (discussed later in this chapter). We can have 254 partitions of 64 GB, each allowing for a total of 16 terabytes, and we will have one for each LOB column (up to 254 partitions), which makes up to 4,000 terabytes possible. Figure 4.3 shows the physical implementation of a LOB.

Figure 4.3. Physical LOB Storage

Partitioned base tables can each have different LOB table spaces. A LOB value can be longer than a page in a LOB table space; it can also span pages.

Below is an example of creating a LOB table space:

CREATE LOB TABLESPACE CERTPIC
  IN DB2CERT
  USING STOGROUP CERTSTG
      PRIQTY 3200
      SECQTY 1600
  LOCKSIZE LOB
  BUFFERPOOL BP16K1
  GBPCACHE SYSTEM
  LOG NO
  CLOSE NO;

Note

If a table contains a LOB column and the plan or package is bound with SQLRULES (STD), then DB2 will implicitly create the LOB table space, the auxiliary table, and the auxiliary index. DB2 will choose the name and characteristics for these, implicitly creating these objects. This is not a good idea; naming standards and placement of these objects are critical for management and performance.

Creating Table Spaces

The CREATE TABLESPACE statement is used to define a simple, segmented, or partitioned table space on the current server. There is a large number of parameters used in defining table spaces, with significant options depending on the type of table space. The major table space parameters are:

  • DSSIZE

    • This specifies that the maximum size of any partition in a partitioned table space can exceed the 4-GB limit up to a size of 64-GB.

  • LOB

    • This specifies that the table space is a LOB table space, used to hold only LOB values.

  • USING clause

    • This specifies whether or not storage groups or a user-defined table space is being defined.

      • USING STOGROUP specifies that DB2 will manage the data sets and there will also be a PRIQTY and SECQTY to define the allocations for the data sets.

      • USING VCAT specifies that the data sets will be user-defined.

  • FREEPAGE and PCTFREE

    • These parameters specify the amount of free space to be left when a table space is loaded or reorganized.

  • GBPCACHE

    • This is used in a data sharing environment to specify if changed/all or no pages of a table space or partition are written to the group buffer pools.

  • NUMPARTS

    • This specifies the number of partitions in a partitioned table space, with a maximum of 254 allowed.

  • SEGSIZE

    • This specifies the number of pages in a segment for segmented table spaces.

  • BUFFERPOOL

    • This specifies which buffer pool the table space is assigned to.

  • LOCKSIZE

    • This specifies the size of locks used within the table space and in certain cases can specify the threshold at which lock escalation occurs. The allowable lock sizes are:

      • ANY

      • TABLESPACE

      • TABLE

      • PAGE

      • ROW

      • LOB

  • MAXROWS

    • Specifies the maximum number of rows that are allowed on a page up to a maximum of 255.

  • LOCKPART

    • Specifies whether or not selective partition locking is to be used when locking a partitioned table space.

  • COMPRESS

    • Specifies whether or not data compression is used for rows in the table space or partition.

  • TRACKMOD

    • Specifies whether DB2 tracks modified pages in the space map pages in the table space or partition.

Page Sizes

There are four page sizes available for use:

BP0-BP49 | 4K pages
BP8K0-BP8K9 | 8K pages
BP16K0-BP16K9 | 16K pages
BP32K0–BP32K9 | 32K pages

The 8-KB, 16-KB, and 32-KB page sizes are logical constructs of actual physical 4-KB pages. Index page sizes are only 4-KB pages and work file (DSNDB07) table space pages can only be 4-KB or 32-KB.

With the larger page sizes, we can achieve better hit ratios and have less I/O because we can fit more rows on a page. For instance, if we have a 2200-byte row (maybe for a data warehouse), a 4K page would be able to hold only one row, but if an 8K page were used, three rows could fit on a page, one more than if 4K pages were used and one less lock also, if required.

The page size is defined by the buffer pool chosen for the table space (i.e., BP8K0 supports 8K pages).

Free Space

The FREEPAGE and PCTFREE clauses are used to help improve the performance of updates and inserts by allowing free space to exist on table spaces or index spaces. Performance improvements include improved access to the data through better clustering of data, less index page splitting, faster inserts, fewer row overflows, and a reduction in the number of REORGs required. Some trade-offs will include an increase in the number of pages (and therefore more auxiliary storage needed), fewer rows per I/O, less efficient use of buffer pools, and more pages to scan. As a result, it is important to achieve a good balance for each individual table space and index space when deciding on free space, and that balance will depend on the processing requirements of each table space or index space. When inserts and updates are performed, DB2 will use the free space defined, and by doing this it can keep records in clustering sequence as much as possible. When the free space is used up, the records must be located elsewhere, and this is when performance can begin to suffer. Read-only tables do not require any free space and tables with a pure insert-at-end strategy generally don't require free space. Exceptions to this would be tables with VARCHAR columns or tables using compression that are subject o updates.

The FREEPAGE amount represents the number of full pages inserted between each empty page during a LOAD or REORG of a table space or index space. The trade-off is between how often reorganization can be performed versus how much disk can be allocated for an object. FREEPAGE should be used for table spaces so that inserts can be kept to the optimal page as close as possible. For indexes, FREEPAGE should be used for the same reason, except improvements would be in terms of keeping index page splits nearby the original page instead of being placed at the end of the index. FREEPAGE is useful when inserts are sequentially clustered.

PCTFREE is the percentage of a page left free during a LOAD or REORG. PCTFREE is useful when you can assume an even distribution of inserts across the key ranges. It is also needed in indexes to avoid all random inserts causing page splits.

Compression

Using the COMPRESS clause of the CREATE TABLESPACE and ALTER TABLESPACE SQL statements allows for the compression of data in a table space or in a partition of a partitioned table space.

Note

Indexes and LOB table spaces are not compressed.

In many cases, using the COMPRESS clause can significantly reduce the amount of DASD space needed to store data, but the compression ratio achieved depends on the characteristics of the data. The DSN1COMP utility determines how well your data will compress (for more information on DSN1COMP refer to Chapter 7).

Compression allows us to get more rows on a page and therefore see many of the following performance benefits, depending on the SQL work load and the amount of compression:

  • Higher buffer pool hit ratios

  • Fewer I/Os

  • Fewer getpage operations

  • Reduced CPU time for image copies

There are also some considerations for processing cost when using compression.

  • Compressing data can result in a higher processor cost, depending on the SQL work load. However, if you use IBM 's synchronous data compression hardware, processor time is significantly less than if you use just the DB2-provided software simulation or an edit or field procedure to compress the data.

  • The processor cost to decode a row using the COMPRESS clause is significantly less than the cost to encode that same row. This rule applies regardless of whether the compression uses the synchronous data compression hardware or the software simulation built into DB2.

  • The data access path which DB2 uses affects the processor cost for data compression. In general, the relative overhead of compression is higher for table space scans and less costly for index access.

The example below shows a table space created with compression:

CREATE TABLESPACE CERTTSPT
IN DB2CERT
USING STOGROUP CERTSTG
   PRIQTY 100
   SECQTY 120
   ERASE NO
COMPRESS YES
ERASE NO
LOACKSIZE PAGE
Modifying Table Spaces

After creating a table space, the ALTER TABLESPACE statement enables you to modify existing table spaces. The ALTER TABLESPACE statement modifies many of the table space parameters such as:

  • Change the bufferpool assignment

  • Change the lock size or the lock escalation threshold

  • Change the specifications for a single partition

  • Change any of the space definitions

  • Turn compression off and on

  • Change how pages are cached in group buffer pools

  • Change logging for LOB table spaces

  • Change the USING VCAT

Note

If the USING VCAT is ALTERed, the table space must be stopped first.

In the following example, the ALTER TABLESPACE is used to change the buffer pool assignment and the size of the locks used.

ALTER TABLESPACE DB2CERT.CERTTS
   BUFFERPOOL BP4
   LOCKSIZE ROW
Removing Table Spaces

When you want to remove a table space, the DROP TABLESPACE statement is used to delete the object. This will remove any objects that are directly or indirectly dependent on the table space. It will also invalidate any packages or plans that refer to the object, and will remove its descriptions and all related data from the catalog.

DROP TABLESPACE DB2CERT.CERTTS

Views

Views are logical tables that are created using the CREATE VIEW statement. Once a view is defined, it may be accessed using DML statements such as SELECT, INSERT, UPDATE, and DELETE as if it were a base table. A view is a temporary table and the data in the view is available only during query processing.

With a view, you can make a subset of table data available to an application program and validate data that is to be inserted or updated. A view can have column names that are different from the names of corresponding columns in the original tables. The use of views provides flexibility in the way the application programs and end-user queries look at the table data.

A sample CREATE VIEW statement is shown below. The original table, EMPLOYEE, has columns named SALARY and COMM. For security reasons, this view is created from the ID, NAME, DEPT, JOB, and HIREDATE columns. In addition, we are restricting access on the column DEPT. This definition will show only the information of employees who belong to the department whose DEPTNO is 10.

CREATE VIEW EMP_VIEW1
(EMPID,EMPNAME,DEPTNO,JOBTITLE,HIREDATE)
       AS SELECT ID,NAME,DEPT,JOB,HIREDATE FROM EMPLOYEE
 WHERE DEPT=10;

After the view has been created, the access privileges can be specified. This provides data security since a restricted view of the base table is accessible. As we have seen above, a view can contain a WHERE clause to restrict access to certain rows or can contain a subset of the columns to restrict access to certain columns of data.

The column names in the view do not have to match the column names of the base table. The table name has an associated schema as does the view name. Once the view has been defined, it can be used in DML statements such as SELECT, INSERT, UPDATE, and DELETE (with restrictions). The database administrator can decide to provide a group of users with a higher level privilege on the view than the base table.

A view is an alternative way of looking at data in one or more tables. It is basically a SQL SELECT statement that is effectively executed whenever the view is referenced in a SQL statement. Since it is not materialized until execution, operations such as ORDER BY, WITH clause, and OPTIMIZE FOR clause have no meaning.

Views with Check Option

If the view definition includes conditions (such as a WHERE clause) and the intent is to ensure that any INSERT or UPDATE statement referencing the view will have the WHERE clause applied, the view must be defined using WITH CHECK OPTION. This option can ensure the integrity of the data being modified in the database. An SQL error will be returned if the condition is violated during an INSERT or UPDATE operation.

The example below is an example of a view definition using the WITH CHECK OPTION.

The WITH CHECK OPTION is required to ensure that the condition is always checked. You want to ensure that the DEPT is always 10. This will restrict the input values for the DEPT column. When a view is used to insert a new value, the WITH CHECK OPTION is always enforced.

CREATE VIEW EMP_VIEW2
 (EMPID,EMPNAME,DEPTNO,JOBTITLE,HIREDATE)
        AS SELECT ID,NAME,DEPT,JOB,HIREDATE FROM EMPLOYEE
  WHERE DEPT=10
 WITH CHECK OPTION;

If the view in the above example is used in an INSERT statement, the row will be rejected if the DEPTNO column is not the value 10. It is important to remember that there is no data validation during modification if the WITH CHECK OPTION is not specified. If the view in the previous example is used in a SELECT statement, the conditional (WHERE clause) would be invoked and the resulting table would contain only the matching rows of data. In other words, the WITH CHECK OPTION does not affect the result of a SELECT statement.

The WITH CHECK OPTION must not be specified for the following views that are read-only.

Nested View Definitions

If a view is based on another view, the number of predicates that must be evaluated is based on the WITH CHECK OPTION specification. If a view is defined without WITH CHECK OPTION, the definition of the view is not used in the data validity checking of any insert or update operations. However, if the view directly or indirectly depends on another view defined with the WITH CHECK OPTION, the definition of that super view is used in the checking of any insert or update operation.

Modifying a View

In order to modify a view you simply drop and re-create it. You cannot alter a view.

Removing a View

To remove a view you simply use the following:

DROP VIEW EMP_VIEW2

Indexes

An index is a list of the locations of rows sorted by the contents of one or more specified columns. Indexes are typically used to improve the query performance. However, they can also serve a logical data design purpose. For example, a unique index does not allow the entry of duplicate values in columns, thereby guaranteeing that no rows of a table are the same. Indexes can be created to specify ascending or descending order by the values in a column. The indexes contain a pointer, known as a record id (RID), to the physical location of the rows in the table.

These are three main purposes for creating indexes:

  • To ensure uniqueness of values

  • To improve query performance

  • To ensure a physical clustering sequence

More than one index can be defined on a particular base table, which can have a beneficial effect on the performance of queries. However, the more indexes there are, the more the database manager must work to keep the indexes up-to-date during update, delete, and insert operations. Creating a large number of indexes for a table that receives many updates can slow down processing.

Note

Indexes are also stored in underlying VSAM data sets, just like table spaces. In order to make a correlation between the indexes that you created (or DB2 created), you can look in the INDEXSPACE column in the SYSINDEXES catalog table.

We will take a look at some of the most important parameters of the CREATE INDEX statement.

  • TYPE 2

    • This specifies a type 2 index. The TYPE 2 clause is not required. A type 2 index is always created.

  • UNIQUE

    • This prevents the table from containing two or more rows with the same value of the index key.

  • USING clause

    • This specifies whether or not storage groups or a user-defined index space is being defined.

    • USING STOGROUP specifies that DB2 will manage the data sets and there will also be a PRIQTY and SECQTY to define the allocations for the data sets.

    • USING VCAT specifies that the data sets will be user-defined and there are no space allocations defined in this statement.

  • FREEPAGE and PCTFREE

    • These parameters specify the amount of free space to be left when an index space is built or reorganized.

  • GBPCACHE

    • This is used in data sharing to specify which pages of an index space are written to the group buffer pools.

  • DEFINE

    • This specifies whether or not to create the underlying data sets.

  • CLUSTER

    • This specifies whether or not it is a clustering index.

  • PART

    • This identifies the partition number.

  • VALUES

    • This identifies the values of the keys to be contained in the partition.

  • BUFFERPOOL

    • This specifies which buffer pool the table space is assigned to.

  • PIECESIZE

    • This identifies the size of the pieces (data sets) used for an NPI.

  • COPY

    • This specifies whether or not the index can be image copied (for more information on copying indexes, refer to Chapter 8).

Type 2

Type 2 indexes are created by default. There is no locking on Type 2 indexes and they do not have subpages. Type 1 indexes should not be used and, depending on the release of DB2, are unsupported.

Deferring Physical Definition

An index can be defined with the DEFINE NO option (DEFINE YES is the default). This is done to specify an index and defer the actual physical creation. The data sets will not be created until data is inserted into the index. This option is helpful in order to reduce the number of physical data sets.

Clustering Index

On the S/390, it is generally important to control the physical sequence of the data in a table. The CLUSTER option is used on one and only one index on a table and specifies the physical sequence. If not defined, then the first index defined on the table in a non-partitioned table space is used for the clustering sequence.

There is an option in defining table spaces called MEMBER CLUSTER, which is useful in a data sharing environment. When this is specified, each member gets its own space map but with 199 pointers instead of 2000. In that case, DB2 will choose to locate the data based on available space when a SQL INSERT statement is used in an attempt to avoid each member trying to acquire a page p-lock on the space map page. This results in better insert performance but clustering of the underlying data may suffer resulting in more frequent REORGs.

Partitioning Index

For a partitioned table space a clustering index is required, and it specifies key ranges to be used for each partition. This requires using one part clause for each partition, specifying the highest value for the partition. The length of the limit key is the same as the length of the partitioning index. The below example shows how to define a cluster index to be used for partitioning. In this example, the data is partitioned by the TCID—entries with a key value up to 300 will go into partition 1 and entries with a key value up to 500 will go into partition 2:

CREATE UNIQUE INDEX DB2USER1.TESTCNTX
  ON DB2USER1.TEST_CENTER
   (TCID ASC)
  USING STOGROUP CERTSTG
  PRIQTY 512
  SECQTY 64
  ERASE NO
  CLUSTER
    (PART 1 VALUES (300),
     PART 2 VALUES (500))
  BUFFERPOOL BP3
  CLOSE YES;

Just like the partitioned table space, the partitioning index can be made up of several data sets. Each partition can have different attributes (i.e., some may have more free space than others).

Unique Index and Non-Unique Index

A unique index guarantees the uniqueness of the data values in a table's columns. The unique index can be used during query processing to perform faster retrieval of data. The uniqueness is enforced at the end of the SQL statement that updates rows or inserts new rows. The uniqueness is also checked during the execution of the CREATE INDEX statement. If the table already contains rows with duplicate key values, the index is not created.

An example of creating a unique index is shown below:

CREATE UNIQUE INDEX DB2USER1.TESTIX
  ON DB2USER1.TEST
   (NUMBER ASC)
  USING STOGROUP CERTSTG
  PRIQTY 512
  SECQTY 64
  ERASE NO
  CLUSTER

A non-unique index can also improve query performance by maintaining a sorted order for the data.

Depending on how many columns are used to define a key, you can have one of the following types:

  • An atomic key is a single column key.

  • A composite key is composed of two or more columns.

  • The following are types of keys used to implement constraints:

    • A unique key is used to implement unique constraints. A unique constraint does not allow two different rows to have the same values on the key columns.

    • A primary key is used to implement entity integrity constraints. A primary key is a special type of unique key. There can be only one primary key per table. The primary key column must be defined with the not null option.

    • A foreign key is used to implement referential integrity constraints. Referential constraints can reference only a primary key or unique constraint. The values of a foreign key can have only values defined in the primary key or unique constraint they are referencing or NULL values. (A foreign key is not an index.)

Note

DB2 uses unique indexes and the NOT NULL option to maintain primary and unique key constraints.

Unique Where Not Null Index

This is a special form of a unique index. Normally in a unique index, any two null values are taken to be equal. Specifying WHERE NOT NULL will allow any two null values to be unequal.

Null Values and Indexes

It is important to understand the difference between a primary key and a unique index. DB2 uses two elements to implement the relational database concept of primary and unique keys: unique indexes and the NOT NULL constraint. Therefore, unique indexes do not enforce the primary key constraint by themselves since they can allow a null value. Null values are unknown, but, when it comes to indexing, a null value is treated as equal to all other null values (with the exception of the UNIQUE WHERE NOT NULL index). You cannot insert a NULL value twice if the column is a key of a unique-index because it violates the uniqueness rule for the index.

Non-Partitioning Indexes

Non-partitioning indexes(NPIs) are indexes that are used on partitioned tables. They are not the same as the clustered partitioning key which is used to order and partition the data, but rather they are for access to the data. NPIs can be unique or non-unique. While you can have only one clustered partitioning index, you can have several NPIs on a table, if necessary. NPIs can be broken apart into multiple pieces (data sets) by using the PIECESIZE clause on the CREATE INDEX statement. Pieces can vary in size from 254K to 64GB—the best size will depend on how much data you have and how many pieces, you want to manage. If you have several pieces, you can achieve more parallelism on processes, such as heavy INSERT batch jobs, by alleviating the bottlenecks caused by contention on a single data set.

The example below shows how to create an NPI with pieces:

CREATE UNIQUE INDEX DB2USER1.TESTCN2X
  ON DB2USER1.TEST_CENTER
  (CODE ASC)
  USING STOGROUP CERTSTG
  PIECESIZE 512K;
LOB Indexes

An index must be created on an auxiliary table for a LOB. The index itself consists of 19 bytes for the ROWID and 5 bytes for the RID. Due to this fact, it is always unique. No LOB columns are allowed in the index. The example below shows the CREATE statement for an auxiliary index.

CREATE INDEX DB2CERT.PHOTOIX
  ON DB2USER1.CAND_PHOTO
  USING VCAT DB2USER1
  COPY YES;

Note

No columns are specified because the auxiliary indexes have implicitly generated keys.

General Indexing Guidelines

Indexes consume disk space. The amount of disk space will vary depending on the length of the key columns, and whether the index is unique or non-unique. The size of the index will increase as more data is inserted into the base table. Therefore, consider the disk space required for indexes when planning the size of the database. Some of the indexing considerations include:

  • Primary and unique key constraints will always create a unique index.

  • It is usually beneficial to create indexes on foreign key constraint columns.

  • It is beneficial to always create a clustering index.

Modifying an Index

The ALTER INDEX statement will allow you to change many of the characteristics on the index such as:

  • The bufferpool assignment

  • The specifications for a single partition

  • Any of the space definitions

  • Whether or not the index is to be copied

The example below shows how to change the buffer pool assignment for an index:

ALTER INDEX TESTCN2X BUFFERPOOL BP1;
Removing an Index

When you want to remove an index, issue the following statement:

DROP INDEX TESTCN2X;

Databases

A database is a collection of table space, index spaces, and the objects with them. There are also a couple of different types of databases that are used for special purposes: A WORKFILE database holds the DB2 work files used for sorting and other activities, or a TEMP database holds temporary tables as defined by a DECLARE TEMP statement (refer to Chapter 11 for more information on declared temporary tables). For these special databases, you can also specify which data sharing member it is for since each member must have its own.

You can also specify the coding scheme for the data in the database (ASCII, CCSID, EBCDIC, UNICODE).

Creating a Database

Below is an example of the creation of the DB2CERT database. The BUFFERPOOL parameter lets us specify that any objects created in this database without a buffer pool assigned will default to buffer pool BP7. INDEXBP is for the same reason, but provides a default buffer pool for indexes.

CREATE DATABASE CERTTS
STOGROUP CERTSTG
BUFFERPOOL BP7
INDEXBP BP8;
Modifying a Database

The default buffer pools, encoding scheme, and the stogroup can be changed for a database. The following is an example ALTER DATABASE:

ALTER DATABASE DB2CERT
BUFFERPOOL BP4;
Removing a Database

It is very easy to remove a database. It is a simple DROP statement (provided all the appropriate authorities are in place). When a database is dropped, all dependent objects are dropped.

DROP DATABASE DB2CERT;

Storage Groups

Storage groups are used to list the DASD volumes that will be used to store the data. They can contain one or many volumes. Storage groups can work with or without SMS (System Managed Storage).

If table spaces or index spaces are defined using a storage group (identified in the USING clause in the CREATE INDEX and CREATE TABLESPACE statements), they are considered to be DB2 managed and DB2 will create them allowing you to specify the PRIQTY and SECQTY for the data set allocations. Otherwise, they are considered to be user-managed and must be defined explicitly through ICF (Integrated Catalog Facility).

Creating a Storage Group
CREATE STOGROUP CERTSTG
VOLUME (*) VCAT DB2USER1;

Note

The (*) here indicates that SMS will manage the volumes to be used.

Modifying a Storage Group

We can add or remove volumes within a storage group. This is done via the ALTER.

ALTER STOGROUP CERTSTG ADD VOL1;
Removing a Storage Group

Storage groups can be removed by using the DROP statement. This can be done only if there is no table space or index space using it.

DROP STOGROUP CERTSTG;


040 - Database Design and Implementation

Chapter 4. Database Objects

Database Design and Implementation

The best way to understand data type selection is to design a database and implement the design using DB2. We will create a database that can be used to schedule and track the results of a certification program. This database will be used to illustrate many aspects of the SQL language and features of DB2. You have already seen many examples from this database throughout this chapter.

This database will be used to schedule test candidates' exams and, following the completion of the test, it will contain the candidates' test scores.

The database and its application will need to perform the following tasks:

  1. Insert/update/delete testing center information.

  2. Insert/update/delete test information.

  3. Insert/update/delete test candidate information.

  4. Guarantee a uniquely identified test name, regardless of the test number.

  5. Schedule a candidate to take a test.

  6. Update candidate test scores once exams have been completed.

  7. Determine which candidates qualify for certification.

  8. Generate various reports on the candidates and tests.

The database will be named DB2CERT. The data to be stored in the DB2CERT database can easily be grouped into three reference tables and a fourth table used to relate the other tables. The primary relationship can be defined as a test candidate takes a specific test at a test center.

Figure 4.4 shows the relationships within the problem domain. The rectangles represent the base tables: CANDIDATE, TEST_CENTER, and TEST. The fourth table is a relationship table called TEST_TAKEN.

Figure 4.4. Tables for the DB2CERT Database

DB2CERT Database Table Descriptions

The following is a list of tables that will be used in the DB2CERT database:

  • The CANDIDATE table stores information about each test candidate for the DB2 Certification Program. Data such as candidate name, address, candidate ID, and phone number will be stored in this table. A data record represents a single test candidate (person).

  • The TEST_CENTER table stores information about the test centers where a candidate can take a DB2 Certification exam. Data such as the test center name, address, number of seats at the test center, test center ID, and its phone number will be stored in this table. A data record represents a single test center location.

  • The TEST table stores information about each of the DB2 Certification exams. Data such as the test name, type, test ID, cut score (passing percentage), and the length of each test will be stored in this table. A data record represents a single test. For our example, there are three tests in the DB2 Certification Program; therefore, there are only three data records in this table. A test name must be uniquely identified.

  • The TEST_TAKEN table associates the records from the other three tables. It serves the dual purpose of scheduling tests and tracking each test result. Data such as the candidates' test scores, date taken, start time, and seat number will be stored in this table. This will be the most active of the four tables since multiple exams must be taken by each candidate to become certified, and each test taken will have a corresponding data record in this table.

Once you have defined the tables and their relationships, the following should be defined:

  1. User-defined data types

  2. Columns or attributes for the tables

  3. (Optional) primary keys (PK) for the tables

  4. (Optional) unique keys for the tables

  5. (Optional) foreign keys (referential constraints) for the tables

  6. (Optional) table check constraints for the tables

  7. (Optional) triggers for the database

In Figure 4.5, the database design is shown. The rectangles represent the entities or tables. The columns or attributes are shown as ellipses. Note that some of the columns are derived columns. A derived column is a column that represents a concept and not a physical attribute of an object. The derived columns are included in the model since their values will be populated by the database using a constraint mechanism.

Figure 4.5. DB2 Certification Database Entity Relationship Diagram

We must map the attributes shown in Figure 4.5 to DB2 as supported data types or user-defined data types. To demonstrate some of the powerful features of DB2, we have decided to create distinct types for many of the attributes. Figure 4.6 also shows a basic entity diagram for the database.

Figure 4.6. DB2 Certification Database Entity Basic Relationship Diagram

It is beneficial to have a primary key defined for each of the tables since this will ensure uniqueness of the data records. The attributes that are underlined will be used as primary keys. We will also create unique keys to illustrate their use. Unique key attributes are double-underlined.

In the previous section, we mentioned that there will be four tables in the DB2CERT database. However, the design shown in Figure 4.5 has only three tables defined. They are shown as rectangles. There is an implied table defined in the relationship candidate takes a test. A table is required to store each occurrence of a candidate taking a certification test.

We will impose a restriction on the candidates: They can take the test only once on any given day. A candidate can take different tests on the same day, but not the same test. With this restriction in place, we will define a primary key for the TEST_TAKEN table as a composite key including NUMBER (test ID), CID (candidate ID), and DATE_TAKEN (the date of the test). By defining the primary key as a combination of these three values, we can enforce this constraint.

The diamond shapes are used to describe the relationships between the tables, including the parent-child relationship. For example, there is a one-to-many relationship between the CANDIDATE and the TEST_TAKEN table because a single candidate can take many tests. This relationship is shown by denoting the values of 1 and N (many) on the appropriate side of the diamond. The primary and foreign keys can also be seen in Figure 4.7.

Figure 4.7. DB2 Certification Database Referential Integrity

The database design shown in Figure 4.5 is just one type of diagramming technique. A logical database design can be represented in a number of ways, but it should not dictate the physical implementation of the database. We have included it here because it will be used in many of the SQL statements throughout the rest of the book.

Defining User-Defined Data Types

If you want to utilize user-defined data types, they must exist in your database before they can be referenced in a CREATE TABLE statement. We have decided to create a number of user-defined data types as shown in the example below. The CANDIDATE_ID, TEST_ID, CENTER_ID, and PHONE data types are all based on the fixed-length character (CHAR) data type. These attributes were chosen to be user-defined because they have meaning in their structure, and they should not be used in expressions with other character data types. For example, a telephone number data type could be defined and then a user-defined function could be created to extract the area code. The function would be used only for phone data types.

We also decided to create some numeric user-defined data types, including the data types score and minutes.

   CREATE DISTINCT TYPE CANDIDATE_ID AS CHAR (8)
WITH COMPARISONS
   CREATE DISTINCT TYPE TEST_ID AS CHAR (6)
WITH COMPARISONS
   CREATE DISTINCT TYPE CENTER_ID AS CHAR (4)
WITH COMPARISONS
   CREATE DISTINCT TYPE PHONE AS CHAR (10)
WITH COMPARISONS
   CREATE DISTINCT TYPE SCORE AS DECIMAL (6,2)
WITH COMPARISONS
   CREATE DISTINCT TYPE MINUTES AS SMALLINT
WITH COMPARISONS

Defining Tables and Columns

Designing a database involves many considerations. We will examine only some of these considerations in this book. If we use the database design in Figure 4.5 as a starting point, we can start creating database objects.

The first step in creating a database is to issue the run a CREATE DATABASE statement, such as the one below.

CREATE DATABASE CERTTS
STOGROUP CERTSTG
BUFFERPOOL BP7
INDEXBP BP8;

Once the database has been created, we can start creating objects. We can create distinct (user-defined) data types as we did previously. Then we can start creating tables (if explicit table spaces are to be used, they must be created first, but for this example we will assume they are already created). Refer to the DB2CERT DDL in the appendix for the table space CREATEs.

The database design had a number of attributes shown. Each of these attributes will be a column in the table definitions.

Every DB2 table contains one or more columns. The tables and their corresponding columns are given names. In the previous sections, we discussed all of the data types that can be used for column definitions.

Data is placed in a DB2 table using the SQL statement INSERT or UPDATE. (The LOAD utility is also an option.) Usually, it is desirable for each column, or data value, to have a value. Sometimes, there is no value provided for a column during the INSERT statement. If the column is defined as NOT NULL, the INSERT statement will fail. If a default value is defined, it will be stored.

The table being created in the following DDL is called DB2CERT. CANDIDATE and it contains 15 columns. Each column is given a name and a data type. There are three user-defined data types used in the DB2CERT.CANDIDATE table. These data types are CANDIDATE_ID and PHONE. There are also constraints defined for the valid values for some of the columns. For example, the null constraint is specified for all of the columns except HPHONE, WPHONE, and INITIAL.

CREATE TABLE DB2CERT.CANDIDATE (
   CID CANDIDATE_ID NOT NULL,
  LNAME VARCHAR (30) NOT NULL,
  FNAME VARCHAR (30) NOT NULL,
  INITIAL CHAR (1),
  HPHONE PHONE,
  WPHONE PHONE,
  STREETNO VARCHAR (8) NOT NULL,
  STREETNAME VARCHAR (20) NOT NULL,
  CITY VARCHAR (30) NOT NULL,
  PROV_STATE VARCHAR (30) NOT NULL,
  CODE CHAR (6) NOT NULL,
  COUNTRY VARCHAR (20) NOT NULL,
  CERT_DBA CHAR (1) NOT NULL WITH DEFAULT 'N',
  CERT_APP CHAR (1) NOT NULL WITH DEFAULT 'N',
  CONSTRAINT UNIQUE_CANDIDATE PRIMARY KEY (CID))
  IN DB2CERT.CERTTS;

Keys

Keys are a special set of columns defined on a table. They can be used to uniquely identify a row or to reference a uniquely identified row from another table. Keys can be classified either by the columns they are composed of or by the database constraint they support.

Depending on how many columns are used to define a key, you can have one of the following types:

  • An atomic key is a single column key.

  • A composite key is composed of two or more columns.

The following are types of keys used to implement constraints:

  • A unique key is used to implement unique constraints. A unique constraint does not allow two different rows to have the same values in the key columns.

  • A primary key is used to implement entity integrity constraints. A primary key is a special type of unique key. There can be only one primary key per table. The primary key column must be defined with the NOT NULL option.

  • A foreign key is used to implement referential integrity constraints. Referential constraints can reference only a primary key or unique key. The values of a foreign key can have only values defined in the primary key or unique key they are referencing or the NULL value.

Defining Primary Keys

It is sometimes beneficial to define a primary key for each of your DB2 tables since this will guarantee the uniqueness of a column value or group of column values (composite key). In the previous CREATE statement, the primary key for the table candidate is defined as the column CID (candidate ID). By specifying this column as a primary key, DB2 will create a system-unique index if one does not already exist.

Let us look at the other tables representing the tests and the test centers. In the following CREATE statements, the TEST and TEST_CENTER tables are shown. These tables each have a primary key defined.

In our example, the primary key constraint was given a name (UNIQUE_ TEST and UNIQUE_CENTER) for referencing purposes. If a name is not provided, DB2 will assign a system-generated name to the constraint.

  CREATE TABLE DB2CERT.TEST (
  NUMBER TEST_ID NOT NULL,
  NAME VARCHAR (50) NOT NULL,
  TYPE CHAR (1) NOT NULL,
  CUT_SCORE SCORE NOT NULL,
  LENGTH MINUTES NOT NULL,
  TOTALTAKEN SMALLINT NOT NULL,
  TOTALPASSED SMALLINT NOT NULL,
  CONSTRAINT UNIQUE_TEST PRIMARY KEY (NUMBER),
  CONSTRAINT UNIQUE_TEST_NAME UNIQUE (NAME)
  CONSTRAINT TEST_TYPE CHECK (TYPE IN ('P','B')))
  IN DB2CERT.CERTTS

CREATE TABLE DB2CERT.TEST_CENTER (
  TCID CENTER_ID NOT NULL,
  NAME VARCHAR (40) NOT NULL,
  STREETNO VARCHAR (8) NOT NULL,
  STREETNAME VARCHAR (20) NOT NULL,
  CITY VARCHAR (30) NOT NULL,
  PROV_STATE VARCHAR (30) NOT NULL,
  COUNTRY VARCHAR (20) NOT NULL,
  CODE CHAR (6) NOT NULL,
  TYPE CHAR (1) NOT NULL,
  PHONE PHONE NOT NULL,
  NO_SEATS SMALLINT NOT NULL,
  CONSTRAINT UNIQUE_CENTER PRIMARY KEY (TCID))
  IN DB2CERT.CERTTS
Defining Unique Keys

Unique keys can be used to enforce uniqueness on a set of columns. A table can have more than one unique key (index) defined. The TEST table definition uses a unique constraint (UNIQUE_TEST_NAME) on column NAME to ensure that a test name is not used twice. There is also a primary key constraint on the column NUMBER to avoid duplicate test numbers.

Having unique constraints on more than one set of columns of a table is different from defining a composite unique key that includes the whole set of columns. For example, if we define a composite primary key on the columns NUMBER and NAME, there is still a chance that a test name will be duplicated using a different test number.

Note

A unique index needs to be always created for primary (if one does not already exist) or unique key constraints (unless using the schema processor). If you define a constraint name, it will be used to name the index; otherwise, a system-generated name will be used for the index.

Defining Foreign Keys

A foreign key is a reference to the data values in another table. There are different types of foreign key constraints. Let us look at the remaining table in the DB2 Certification database and, in particular, its foreign key constraints. We have one composite primary key defined and three foreign key constraints.

The primary key is defined as the columns CID, TCID, and NUMBER. The foreign key constraints will perform the following:

  • If a record in the candidate table is deleted, all matching records in the TEST_TAKEN table will be deleted (DELETE CASCADE).

  • If a test center in the TEST_CENTER table is deleted, all of the matching records in the TEST_TAKEN table will be deleted (DELETE CASCADE).

  • If a test in the test table is deleted and there are matching records in the TEST_TAKEN table, the DELETE statement will result in an error (DELETE RESTRICT).

Note

A foreign key constraint always relates to the primary or unique constraint of the table in the references clause.

CREATE TABLE DB2CERT.TEST_TAKEN (
  CID CANDIDATE_ID NOT NULL,
  TCID CENTER_ID NOT NULL,
  NUMBER TEST_ID NOT NULL,
  DATE_TAKEN DATE NOT NULL WITH DEFAULT,
  START_TIME TIME NOT NULL,
  FINISH_TIME TIME NOT NULL,
  SCORE SCORE,
  SEAT_NO CHAR(2) NOT NULL,
  CONSTRAINT NUMBER_CONST
  PRIMARY KEY (CID,TCID,NUMBER),
  FOREIGN KEY (CID)
         REFERENCES DB2CERT.CANDIDATE ON DELETE CASCADE,
  FOREIGN KEY (TCID)
         REFERENCES DB2CERT.TEST_CENTER ON DELETE CASCADE,
  FOREIGN KEY (NUMBER)
         REFERENCES DB2CERT.TEST ON DELETE RESTRICT)
  IN DB2CERT.CERTTS

Defining parent-child relationships between tables is known as declarative referential integrity because the child table refers to the parent table. These constraints are defined during table creation or by using the ALTER TABLE SQL statement. DB2 will enforce referential constraints for all INSERT, UPDATE, and DELETE activity.

Note

This database implementation created only the tables and UDTs. The complete DDL for the stogroup, databases, tablespaces, tables, and indexes can be found in the appendix.


041 - Summary

Chapter 4. Database Objects

Summary

In this chapter, we concentrated on the SQL Data Definition Language (DDL). DDL is used to create, modify, and remove database objects. There are three main statements in DDL: CREATE, ALTER, and DROP. If you want to use a DB2 database, you may learn DDL first to create some database objects. There are many kinds of objects in a DB2 database. Some of them are created by a DB2 command, and some of them are created by DDL statements.

Here we focused primarily on data types, tables, views, and indexes among the database objects created by DDL. A data type is used to specify the attribute of columns in a table. DB2 has two kinds of data types: built-in data types and user-defined data types. The built-in type is a DB2-supplied data type and falls into three main categories:

  • Numeric

  • String (including large object [LOB])

  • Datetime

The user-defined data type enables you to create your application-oriented data type based on the built-in data type.

A table consists of columns and rows and stores an unordered set of records. Each column has a data type as one of the attributes. A table itself can have some rules called CONSTRAINTS to guarantee the uniqueness of records or maintain the relationship between and within tables. It helps application programmers to evaluate the records or maintain the consistency between the tables. A view may also reduce some application development workload. A logical table is created and is based on the physical table or other views. A view can be created to limit access to sensitive data while allowing more general access to other data. An index is one of the most important objects for performance. It can also be used to guarantee the uniqueness of each record. We also covered some of the options in the CREATE and ALTER statements.


042 - Additional Resources

Chapter 4. Database Objects

Additional Resources

  • IBM DB2 UDB for OS/390 SQL Reference, SC26-9944.

  • IBM DB2 UDB for OS/390 Administration Guide, SC26-9931.


043 - Chapter 5. Manipulating Database Objects

Part II. Using SQL

Chapter 5. Manipulating Database Objects

  • Data Manipulation Language

  • Retrieving Data

  • Inserting Data

  • Updating Data

  • Deleting Data

In the previous chapter, we discussed the definitions of various database objects using the Data Definition Language (DDL). In this chapter, we start manipulating the database objects using the portion of SQL known as Data Manipulation Language (DML). We will be populating (inserting) data into the database and retrieving the data using many powerful methods. Depending on the sophistication of the database users, they can use SQL to query the database. The majority of the SQL statements within a DB2 application involves DML statements. Therefore, application developers must understand the various methods of inserting, updating, and retrieving data from the database. We will start with simple retrieval statements and gradually introduce more complex methods of data manipulation. The DB2 Certification database will be used for most of the examples. There are four main DML SQL statements we consider: SELECT, INSERT, UPDATE, and DELETE.


044 - Data Retrieval

Chapter 5. Manipulating Database Objects

Data Retrieval

SQL is based on mathematical principles, specifically set theory and relational algebra. The data is stored in the database as unordered sets of data records. SQL is a set-oriented language and many of its language elements are directly related to relational algebraic terms, such as PERMUTATION, PROJECTION, RESTRICTION, and JOIN.

A set of data is represented in a DB2 database as a table or a view and is stored in a DB2 table without regard to order. To retrieve data in a particular order, an ORDER BY phrase must be added to a SELECT statement. Similarly, if the data is to be grouped, then a GROUP BY phrase must be added to the statement.

Now, let's review the DB2CERT database design defined in the previous chapter and manipulate data using various SQL statements. There are three main tables: CANDIDATE, TEST, and TEST_CENTER. Each of these tables represents a set of records that corresponds to a test candidate (person), a test, and a test center (location).

There is an associative table, known as the TEST_TAKEN table, that is used to reflect the relationships among the three main tables. The TEST_TAKEN table is used to schedule the test candidates and also to maintain their test scores.

Note

The longest SQL statement is 32,765 bytes.

Note

Remember that to execute any operation, one must have the necessary privileges.

Retrieving the Entire Table

The most basic of all retrieval commands involves the SELECT statement with no other operators other than the name of the table. This SQL statement retrieves all of the candidates who have taken a DB2 Certification exam. The information requested is contained in the table TEST_TAKEN.

SELECT * FROM DB2CERT.TEST_TAKEN

SQL is a data access language that consists of language statements and clauses. There are many optional clauses that can be used to modify the output. The output of a SELECT statement is known as a result set or result table. The results from the SELECT statement are shown next.

In this example using * for column selection, the columns will be returned to the user in the order in which they are defined in the table.

CID TCID NUMBER DATE_TAKEN START_TIME FINISH_TIME SCORE PASS_FAIL SEAT_NO
--- ---- ------ ---------- ---------- ----------- ----- --------- -------
111 TX01 500     01/01/2000 11:30:00 12:30:00     65    Y            1
111 TX01 501     02/02/2000 10:30:00 11:45:00     73    Y            1
111 TX01 502     03/03/2000 12:30:00 13:30:00     67    Y            1
222 TR01 500     01/01/2000 14:00:00 15:30:00     55    N            2
222 TR01 502     01/02/2000 09:00:00 10:15:00     53    N            2
222 TR01 502     02/18/2000 10:00:00 11:30:00     75    Y            2
333 TX01 500     03/01/2000 11:30:00 13:00:00     82    Y            2
333 TX01 501     12/29/2000 14:00:00 -            -     -            1
333 TX01 502     03/02/2000  14:00:00 14:30:00    92    Y            1
9 record(s) selected.

In SQL, the asterisk or star character (*) is used to indicate that all columns of a table are being referenced. In this example, the SQL statement refers to all of the columns defined for the DB2CERT.TEST_TAKEN table.

If the table is altered and a new column is added to the table definition, the result set would contain the new column.

Note

Adding a new column to an existing table will result in default values being populated for the existing rows (only true if the newly added column is defined as not null; otherwise the value will be null for existing rows).

Since the output of the SQL statement using the asterisk(*) character varies according to the table definition, it is recommended that you specify all of the column names you want to see in the SELECT statement.

We could have obtained the same result as the statement using SELECT * with the following SQL statement:

SELECT CID, TCID, NUMBER,DATE_TAKEN,START_TIME,
FINISH_TIME,SCORE,SEAT_NO
FROM DB2CERT.TEST_TAKEN;

Note

The asterisk (*) character is used to refer to all of the columns defined for a table. The order of the columns in the result table is the same order as specified in the CREATE TABLE or CREATE VIEW statement.

What is returned to the user is known as the result set. If the result set is large, it is advisable to be sure to filter the data using a WHERE predicate.

The FROM clause is required for the DML SQL statement since it describes the location (table or view) of the data. Our example references a single table called DB2CERT.TEST_TAKEN. The SELECT and FROM clauses are required in all data retrieval statements. The list of columns following the SELECT keyword is referred to as the select list.

Projecting Columns from a Table

Projection is a relational operation that allows you to retrieve a subset of the defined columns from a table. The next example restricts the output from the select command so that only the candidate ID, test center, and test number attributes from the TEST_TAKEN table are shown.

SELECT CID,TCID,NUMBER FROM DB2CERT.TEST_TAKEN

The output of this SELECT statement is shown below.

CID TCID NUMBER
--- ---- ------
111 TX01 500
111 TX01 501
111 TX01 502
222 TR01 500
222 TR01 502
222 TR01 502
333 TX01 500
333 TX01 501
333 TX01 502
9 record(s) selected.

The order of the columns in the result table will always match the order in the select list. The order of the columns as they were defined in the CREATE TABLE or CREATE VIEW statement is ignored when a select list is provided in the SQL statement. In this example, the order of the columns is similar to the order in the CREATE TABLE statement, since the CID column was defined prior to TCID and NUMBER columns.

Changing the Order of the Columns

Permutation is the relational operation that allows you to change the order of the columns in your result table. Permutation is used every time you select columns in an order different from the order defined in the CREATE TABLE statement. For example, to display the test center ID prior to the candidate IDs and the test number, you could execute the following:

SELECT TCID,CID,NUMBER FROM DB2CERT.TEST_TAKEN

The result of this SELECT statement specifies a select list in a different order from that which was defined in the table shown below.

TCID CID NUMBER
---- --- ------
TX01 111 500
TX01 111 501
TX01 111 502
TR01 222 500
TR01 222 502
TR01 222 502
TX01 333 500
TX01 333 501
TX01 333 502
9 record(s) selected.

Note

We refer to the output of a SELECT statement as the result table because the output of all SELECT statements can be considered a relational table.

Restricting Rows from a Table

Restriction is a relational operation that will filter the resulting rows of a table. Restriction can be accomplished through the use of predicates defined in an SQL WHERE clause. To restrict the result set, we need to add a WHERE clause to the SQL statement.

Note

A predicate is a condition placed on the data. The result of the condition is TRUE, FALSE, or UNKNOWN.

The WHERE clause specifies conditions or predicates that must be evaluated by DB2 before the result table is returned to the end-user. There are many valid types of predicates that can be used. In the following example, the equality (=) predicate is being used to restrict the records to only those candidates who have taken a DB2 Certification test at the test center TR01.

SELECT TCID,CID FROM DB2CERT.TEST_TAKEN
WHERE TCID ='TR01'

The WHERE clause also accepts other comparison operators, such as greater than (>), less than (<), greater than or equal to (>=), less than or equal to (<=), and not equal to (<>).This statement is an example of a basic predicate. A basic predicate compares two values. In addition, more complex predicates, such as LIKE, BETWEEN, and IN are also valid, and will be discussed later.

Note

Trying to execute this SQL statement example will result in a data type compatibility error, because the column tcid is defined with a user-defined distinct data type (UDT).

Predicate Evaluation for UDTs

The column tcid was defined as a user-defined distinct data type (UDT). To make the comparison in the WHERE clause valid, a casting function needs to be used. This technique is shown below:

SELECT TCID, CID FROM DB2CERT.TEST_TAKEN
WHERE TCID=CAST('TR01' AS DB2CERT.CENTER_ID)

Predicate evaluation requires that the data types be compatible (same data type or a compatible data type). We can accomplish the data type conversion (cast) using one of two methods:

  • The CAST expression

  • A casting function

Refer to Chapter 15 for more information on casting for UDTs.

Restricting Rows Using Multiple Conditions

It is possible to combine multiple conditions (predicates) in a single SQL statement. The predicates can be combined using Boolean operators, such as the AND or OR operators. These operators allow you to combine multiple conditions in a single SQL statement. The order of the predicate evaluation will not affect the result set (known as set closure). The DB2 optimizer will decide the order in which the conditions are applied to maximize query performance based on statistical information and other factors.

The next example retrieves the records for the test candidates who took a test at test center TR01 and achieved a score greater than 65. The rows that satisfy the predicates are known as the qualifying rows. The following example is an SQL statement using multiple predicates:

SELECT TCID,CID,SCORE
FROM DB2CERT.TEST_TAKEN
WHERE TCID= CAST('TR01' AS DB2CERT.CENTER_ID)
AND SCORE > 65

Selecting Columns from Multiple Tables

There are basically two operations that combine columns from multiple tables into a single SQL statement. These operations are:

  • Cartesian product

  • Join

Cartesian Product

A cartesian product is a relational operation that will merge all the values from one table with all the values from another table. This operation is not used frequently because the result table can be very large.

The number of rows in the result table is always equal to the product of the number of rows in the qualifying rows for each of the tables being accessed.

The following example is a cartesian product of all test numbers and test names from the TEST table, with all candidates from the TEST_TAKEN table. First select from the TEST table.

SELECT NUMBER,NAME FROM DB2CERT.TEST

NUMBER NAME
------ --------------------------------------------------
500 DB2 Fundamentals
501 DB2 Administration
502 DB2 Application Development
3 record(s) selected.

Next, select all candidates from the TEST_TAKEN table.

SELECT CID,TCID FROM DB2CERT.TEST_TAKEN

CID TCID
--- ----
222 TR01
222 TR01
222 TR01
111 TX01
111 TX01
111 TX01
333 TX01
333 TX01
333 TX01
9 record(s) selected.

Then combine the two tables to form a cartesian product result table.

SELECT DB2CERT.TEST_TAKEN.NUMBER,CID,TCID
FROM DB2CERT.TEST_TAKEN,DB2CERT.TEST

NUMBER CID TCID
------ --- ----
500    111 TX01
501    111 TX01
502    111 TX01
500    222 TR01
502    222 TR01
502    222 TR01
500    333 TX01
501    333 TX01
502    333 TX01
500    111 TX01
501    111 TX01
502    111 TX01
500    222 TR01
502    222 TR01
502    222 TR01
500    333 TX01
501    333 TX01
502    333 TX01
500    111 TX01
501    111 TX01
502    111 TX01
500    222 TR01
502    222 TR01
502    222 TR01
500    333 TX01
501    333 TX01
502    333 TX01
27 record(s) selected.

There are two tables referenced in the FROM clause of this query. The tables are separated by commas. There is no relationship expression in the FROM clause. This type of query results in a cartesian product.

The result table is a representation of all possible combinations of the input tables. The test table has three rows and the TEST_TAKEN table has nine rows. Therefore, the select statement shown above returns 27 rows. Note the first column name in this query. It is necessary to fully qualify the column name by providing the schema name and table name with the column name because this column exists in both the TEST table and TEST_TAKEN table. In this case, we needed to specify that the number column is to be retrieved from the DB2CERT.TEST_TAKEN table and not the DB2CERT.TEST table.

By adding a predicate to a cartesian product SQL query, the result table can be a more useful representation of the data. In the example below, the query represents all of the tests that were taken by the candidate whose ID is 111.

SELECT DB2CERT.TEST_TAKEN.NUMBER,CID,TCID
FROM DB2CERT.TEST_TAKEN,DB2CERT.TEST
WHERE CID=CAST('111' AS DB2CERT.CANDIDATE_ID)

NUMBER CID TCID
------ --- ----
500    111 TX01
500    111 TX01
500    111 TX01
501    111 TX01
501    111 TX01
501    111 TX01
502    111 TX01
502    111 TX01
502    111 TX01
9 record(s) selected.

Adding a WHERE clause to your query does not always provide the desired result. In the example shown above, you want to know all of the tests that were taken by the candidate whose ID is 111, and the query returns nine rows. But we know from previous queries that the candidate took only three tests. The query in the example above has a WHERE clause to filter out the candidates whose ID is 111 from the TEST_TAKEN table, but there was no filter on the TEST_CENTER table. Therefore, the result of the query would always be a multiple of the number of testing centers. Usually, when multiple tables are referenced, you should include a cross-table relationship using a table merge or join method, as shown in the following example. We will examine table join methods further in the next section.

SELECT DB2CERT.TEST_TAKEN.NUMBER,CID,TCID
FROM DB2CERT.TEST_TAKEN,DB2CERT.TEST
WHERE CID= CAST('111' AS DB2CERT.CANDIDATE_ID)
AND DB2CERT.TEST_TAKEN.NUMBER=DB2CERT.TEST.NUMBER

NUMBER CID TCID
------ --- ----
500    111 TX01
501    111 TX01
502    111 TX01
3 record(s) selected.

To avoid data redundancy, it is recommended that the database tables be normalized. Following a normalization process, a number of related tables will exist. To satisfy some of the required queries, the tables must be reconstructed. The tables are reconstructed temporarily using a table join strategy to produce a single-result table.

The result tables in the previous examples usually provided candidate ID numbers and not the complete names of the test candidates. The candidate IDs are stored in the TEST_TAKEN table, and the full names are stored in the CANDIDATE table. To obtain the name of a candidate, the data must be retrieved from the CANDIDATE table using a relationship or join strategy.

Consider an example that will list the name and phone numbers of candidates who were registered to take a DB2 Certification test in 2000. To accomplish this, we need to select data from two different tables:

  • DB2CERT.CANDIDATE

  • DB2CERT.TEST_TAKEN

Let's retrieve a list of candidate names, phone numbers, and IDs from the CANDIDATE table. The candidate names were stored in multiple columns to allow for easy retrieval by last name.

SELECT FNAME,INITIAL,LNAME,HPHONE,CID
FROM DB2CERT.CANDIDATE

The output of this example follows. Pay special attention to the values in the CID column. It will be used as the join column in the next example.

FNAME    INITIAL   LNAME        HPHONE        CID
-------  -------   -----------  ----------    ---
Bill     B         Wong         1115551234    111
George             Baklarz      2226543455    222
Susan    M         Visser       4442314244    333
Glen     R         Sheffield    5552143244    444
Jim      G         Stittle      6662341234    555
Kevin    W         Street       7773142134    666
Bert     F         Nicol        8886534534    777
Paul     C         Zikopoulos   9992112212    888
8 record(s) selected.

Now let's retrieve the ID numbers of those candidates who were registered to take the test in 2000.

SELECT DISTINCT CID FROM DB2CERT.TEST_TAKEN
WHERE YEAR(DATE_TAKEN) = 2000

CID
---
222
111
333
3 record(s) selected.

The candidate IDs in the TEST_TAKEN table must correspond to a candidate ID in the CANDIDATE table because of the declarative referential integrity constraints. The parent table in the relationship is the CANDIDATE table and the child table (dependent table) is the TEST_TAKEN table.

The result table from the above query does not include the test candidate 444 since that candidate did not have a test scheduled for 2000. We need to join the two result tables based on the candidate ID values. This column is known as the join column.

Note

Query performance can significantly improve if the join columns are appropriately indexed.

The following single query will satisfy the end-user requirement.

SELECT DISTINCT FNAME,INITIAL,LNAME,HPHONE
FROM DB2CERT.TEST_TAKEN,DB2CERT.CANDIDATE
WHERE YEAR(DATE_TAKEN) = 2000
AND DB2CERT.TEST_TAKEN.CID=DB2CERT.CANDIDATE.CID

A table join requires a predicate that includes an expression based on columns from the tables referenced in the FROM clause. This is known as a join predicate. The FROM clause has not changed from the cartesian product examples. The only difference is in the join predicate (TEST_TAKEN.CID = CANDIDATE.CID). This kind of join operation is also known as an INNER JOIN. An inner join displays only the rows that are present in both of the joined tables.

The table names needed to be explicitly stated because there is a column named cid in both of the referenced tables. When multiple tables are being accessed in a single query, any selected columns which occur in more than one table must be qualified with the table name.

Note

An error will occur if the columns being referenced are ambiguous (not properly qualified).

There is a maximum number of base tables of 225 in a single SQL statement; however, you are limited to 15 tables in a join (the FROM clause), except for a star join.

Using Correlation Names

If each of the columns needed to be fully qualified with the table name, such as tableschema.tablename.columnname, the queries would become very large and cumbersome to work with. Fortunately, there is an easier way to qualify the ambiguous columns resulting from a multitable SELECT statement.

The columns can be qualified using a correlation name. A correlation name is a temporary alias for the tables referenced in an SQL statement. We rewrite the previous query using correlated names as follows:

SELECT DISTINCT FNAME,INITIAL,LNAME,HPHONE
FROM DB2CERT.TEST_TAKEN TT, DB2CERT.CANDIDATE C
WHERE YEAR(DATE_TAKEN) = 2000
AND TT.CID = C.CID

The correlation name immediately follows the name of the table as stated in the FROM clause. In this example, the correlated name for the TEST_TAKEN table is TT and the correlated name for the CANDIDATE table is C.

The correlated names are accessible within the SQL statement only. Following the execution of the SQL statement, the correlation name is no longer defined. Once a correlation name has been defined, it can be referenced in the rest of the query instead of the table name.

Note

Use simple, easy-to-remember correlation names. Table initials are good candidates for correlation names.

Sorting Your Output

We have been retrieving data from one or more tables. The order of the result table has not been specified in any of the SQL statements. Therefore, the data is retrieved in an undetermined order if there is no ORDER BY clause in the SQL statement.

The following example produces a list of the test candidates in alphabetical order by last name for the candidates who have taken a DB2 Certification test at the TR01 test center.

SELECT LNAME,INITIAL,FNAME
FROM DB2CERT.CANDIDATE C,DB2CERT.TEST_TAKEN TT
WHERE C.CID=TT.CID
AND TCID=DB2CERT.CENTER_ID('TR01')
ORDER BY LNAME

This example contains a new clause, ORDER BY. After the ORDER BY clause, you can list the columns that will specify the sort order and the type of sort.

The SQL can be modified so that the output is changed to descending order by last name and a secondary order column on the first name in ascending order.

SELECT LNAME,FNAME,HPHONE
FROM DB2CERT.CANDIDATE C,DB2CERT.TEST_TAKEN TT
WHERE C.CID=TT.CID
AND TCID=DB2CERT.CENTER_ID('TR01')
ORDER BY LNAME DESC, FNAME

In this example, the DESC keyword that follows the LNAME column indicates that the result table should be in descending order based on the last name. More than one record can have the same last name. This situation is quite common. There is a second column specified in the ORDER BY clause, FNAME. There is no keyword specifying the sort sequence based on the FNAME column. Therefore, the default ordering sequence (ascending) is used.

The next example contains three columns: LNAME, FNAME, and HPHONE. You can reference the column that should be used to sort the data using the column name or by specifying its position in the SELECT list. Using the column position is very useful when the column in the select list is made up of derived columns (calculated columns) that have no explicit name.

SELECT LNAME,FNAME,HPHONE
FROM DB2CERT.CANDIDATE C,DB2CERT.TEST_TAKEN TT
WHERE C.CID=TT.CID
AND TCID=DB2CERT.CENTER_ID('TR01')
ORDER BY 1 DESC, 2

In this example, the sort order is specified using the column position. Therefore, the query result is exactly the same as the previous example.

You can also assign an alternative column name using column renaming. The assigned name can then be referenced in the ORDER BY clause, and it appears as the column heading where appropriate.

Note

The ORDER BY clause must succeed all subselects in the fullselect statement. The only clauses which can succeed the ORDER BY clause are the READ-ONLY, OPTIMIZE, WITH, QUERYNO, and FETCH-FIRST clauses.

Derived Columns

There are some cases when you will need to perform calculations on the data. The SQL language has some basic mathematical and string functions built in. Mathematical operations include standard addition, subtraction, multiplication, and division.

The calculation can be defined in the WHERE clause of the SQL statement or the SELECT list. Suppose that you need to calculate a passing rate for a DB2 test. The passing rate is defined as the percentage of candidates who pass the test (totalpassed*100/totaltaken). The following SQL statement will accomplish this for us for test number 500:

SELECT NUMBER, TOTALPASSED*100/TOTALTAKEN
FROM TEST
WHERE NUMBER='500'

In this example, the second column of the output list is a calculated column. Remember that you must use the column position if you want to use this calculated column for the ORDER BY clause unless you name it (as we now discuss).

Note

Occasionally, the results of a derived column may not display as expected. The example using TOTALPASSED*100/TOTALTAKEN will result in a value of 66 being retrieved. Since both the TOTALPASSED and TOTALTAKEN columns are integers, the final result is also an integer and the fractional part is discarded. If this is not your desired result, you should use other functions (like decimal) to change the way the calculation is done or displayed.

Naming Derived/Calculated Columns

You can specify a column name for any expression. By providing the derived (calculated) column with a name, the ORDER BY clause can reference the derived name to allow for a more readable SQL statement.

The following SQL calculates the percentage of people who have passed the DB2 Certification exams and orders the output in descending order of the passing rate:

SELECT NUMBER,TOTALPASSED*100/TOTALTAKEN AS PASSEDRATE
FROM TEST
ORDER BY PASSEDRATE DESC

The AS clause is used to rename the default name of an element in the select list. In this example, we are giving the name of PASSEDRATE to the result of the division of columns TOTALPASSED by TOTALTAKEN. The named column is used in the query to specify the column that should be used for the sorting of the output.

Functions

In DB2, there are different types of functions provided. For example, two types of functions provided by DB2 are scalar and column functions (a third type of function called a table function is discussed in Chapter 7, Advanced SQL).

  • Scalar functions (also known as row functions) provide a result for each row of the result table. A scalar function can be used in any place where an expression is allowed.

  • Column functions (also known as vector functions). They work with a group of rows to provide a result. The group is specified using a FULLSELECT and optionally grouped using the GROUP BY clause.

In this section, we introduce you to some of the SQL functions provided with DB2. SQL functions are categorized by their implementation type. Either the functions are built-in or they are extensions of DB2 and are known as user-defined function (UDFs).

  • Built-in functions are defined within the SQL standards, and they are provided by DB2. These can be either scalar, column, or table functions.

  • User-defined functions (UDFs) are not defined within the SQL standards because they are extensions of the current SQL language. These functions can be developed by a DB2 administrator or application developer. Once the UDFs have been created, they can be invoked by any end-user with the proper privileges. For more information about UDFs refer to Chapter 15, Advanced Functions.

Scalar Functions

Scalar functions are applied to each row of data, and there is a per-row result provided. If we wanted to retrieve only the first three digits of telephone numbers for each candidate, we could use a scalar function. The function that will be used is called SUBSTR. The arguments for this function include a string data type column, a beginning offset, and length. The output data type and attribute of the function depend on the input data type and attribute.

The following example retrieves the telephone area code for the column WPHONE:

SELECT LNAME, SUBSTR(CHAR(WPHONE),1,3)
FROM DB2CERT.CANDIDATE

The SUBSTR function is a scalar function. In this example, SUBSTR returns a character string of three characters.

The result string corresponds to the first three characters of the WPHONE column. This function is known as a string function because it works with any string data type. If we wanted to provide the output column with a meaningful name, we could provide an alias, as was done for calculated columns.

In the example above, the substring starts from the beginning of the string because we indicate one (1) as the second parameter of the function. The length of the resulting string is indicated in the third argument. In our example, the length is three. Note that the data type of the WPHONE column is phone, so a casting function is used to convert the phone data type to the char data type.

The following query will provide the month when the exam was taken. The input for this function is a DATE string and the output is a character string.

SELECT FNAME, MONTHNAME(DATE_TAKEN)
FROM CANDIDATE C, TEST_TAKEN TT
WHERE C.CID=TT.CID
Column Functions

Column functions provide a single result for a group of qualifying rows for a specified table or view. Many common queries can be satisfied using column functions where they include common tasks, such as finding the smallest value, the largest value, or the average value for a group of data records.

In the following example, you can obtain the maximum length of time of any of the DB2 Certification exams:

SELECT MAX(SMALLINT(LENGTH)) FROM TEST

Because the column LENGTH is specified as the minutes data type, a casting function is required to convert the minutes data type to the smallint data type in this example.

If we added a WHERE clause to this example, the maximum would represent the maximum length for the qualifying rows, since the predicate is used to filter the data prior to the application of the MAX function.

This next example calculates the average of the number of seats for all of the test centers. Notice the column function AVG is used in this example:

SELECT AVG(NOSEATS) FROM TEST_CENTER

DB2 does provide many more built-in functions. If you are interested in calculating statistical information, you can use Business Intelligence-related functions, such as STDDEV and VARIANCE.

Grouping Values

Many queries require some level of aggregated data. This is accomplished in SQL through the use of the GROUP BY clause.

The following SQL obtains the average number of seats for each country:

SELECT COUNTRY, AVG(NOSEATS) FROM TEST_CENTER
GROUP BY COUNTRY

This SQL statement obtains the average number of seats per country and the GROUP BY clause tells DB2 to group together the rows that have the same values in the columns indicated in the group by list. In our example, we are grouping countries into subsets. As the subsets are created, DB2 calculates the average of each of those groups or subsets, in this case, by each country.

When you combine column functions and other elements, such as column names, scalar functions, or calculated columns, you must use the GROUP BY clause. In this case, you must include every element that is not a column function in the group by list. The only elements that can be omitted in the GROUP BY list are constant values.

The next SQL statement obtains a list that includes the average cut score and minimum test length for the DB2 Certification exams. We group this list by the type of exam as follows:

SELECT TYPE, AVG(INTEGER(CUT_SCORE)),
MIN(SMALLINT(LENGTH))
FROM TEST
GROUP BY TYPE

It is possible to sort the output of the previous example using an ORDER BY clause.

Restricting the Use of Sets of Data

Up to now, we have discussed how to restrict output based on row conditions. With SQL, it is also possible to restrict that output using vector functions and the GROUP BY clause.

Suppose you want a list of all the test centers that have administered more than five DB2 Certification exams. To make it easier to understand, let's first get the number of tests that have been taken in each test center.

SELECT TCID, COUNT(*) FROM TEST_TAKEN GROUP BY TCID

We use the COUNT column function to get the total number of tests that have been taken in each test center. When you use an asterisk (*) with the COUNT function, you are indicating that you want the number of rows in a table that meet the criteria established in the SQL statement. In this example, we are grouping by TCID because we have a number of occurrences for all the test centers in the TEST_TAKEN table. The TEST_TAKEN table has an entry for every DB2 Certification exam that has been taken.

Finally, the output is restricted to only those test centers that have administered more than four exams.

SELECT TCID FROM TEST_TAKEN
GROUP BY TCID HAVING COUNT(*) > 4

This example introduces the HAVING clause. The HAVING clause is equivalent to the WHERE clause for groups and vector functions. The HAVING clause will restrict the result set to only the groups that meet the condition specified in it.

In our example, only the test centers that have administered more than four DB2 Certification exams will be displayed.

Eliminating Duplicates

When you execute a query, you might get duplicate rows in the answer set. The SQL language provides a special clause to remove the duplicate rows from your output.

The following SQL generates a list of names and phone numbers for all the candidates who have taken a test. In the following example, we eliminate the duplicate rows from our output list using the DISTINCT clause.

SELECT DISTINCT FNAME,WPHONE,HPHONE
FROM CANDIDATE C,TEST_TAKEN TT
WHERE C.CID=TT.CID

The DISTINCT clause can also be used with the COUNT function. When you use DISTINCT inside a COUNT function, it will not count the duplicate entries for a particular column.

The example below allows you to count how many different test centers have candidates registered.

SELECT COUNT(DISTINCT CHAR(TCID)) FROM TEST_TAKEN

This example provides the number of test centers that are registered in the TEST_TAKEN table. Remember that all the candidates who have registered for DB2 Certification exams are stored in this table.

Make sure that you understand the difference between COUNT(*) and COUNT(DISTINCT colname). They are very similar in syntax but differ in function.

Searching for String Patterns

SQL has a powerful predicate that allows you to search for patterns in character strings columns. This is the LIKE predicate. Suppose you want to generate a list of the candidates whose first name starts with the letter G.

SELECT fname,lname,wphone,hphone FROM candidate
WHERE fname LIKE 'G%' ORDER BY lname,fname

In this query, we are using a wildcard character with the LIKE predicate.

In SQL, the percent character (%) is a substitute for zero or more characters. The search string (G%) can be substituted with names like George, Gary, Ginger, and so on (since the percent character can substitute zero or more characters, the search string can also be a single letter G).

The percent character can be used in any place in the search string. It also can be used as many times as you need it. The percent sign is not case-sensitive, so it can take the place of uppercase or lowercase letters. However, the constant characters included in your search string are case-sensitive.

Another wildcard character used with the LIKE predicate is the underline character (_). This character substitutes one and only one character. The underline character can take the place of any character. However, the underline character cannot be substituted for an empty character.

The previous SQL can be modified to include all candidates' names and the telephone numbers for those candidates whose name has a lowercase letter "a" as its second letter.

SELECT FNAME,LNAME,WPHONE,HPHONE FROM CANDIDATE
WHERE FNAME LIKE '_A%' ORDER BY LNAME,FNAME

This example uses two wildcard characters that work with the LIKE predicate. The search string, in this example, can include names, such as Paul, Gabriel, or Natalie. (The first character may be any character, the lowercase letter "a" is the second character in the string, and the string ends with any number of characters.)

Searching for Data in Ranges

SQL also offers us a range operator. This operator is used to restrict rows that are in a particular range of values.

Consider the requirement to list those candidates whose scores in the DB2 Certification exam are between 60 and 75.

SELECT DISTINCT FNAME,LNAME,WPHONE,HPHONE
FROM CANDIDATE C, TEST_TAKEN TT
WHERE C.CID=TT.CID
AND INTEGER (SCORE) BETWEEN 60 AND 75

The BETWEEN predicate includes the values that you specify for searching your data. An important fact about the BETWEEN predicate is that it can work with character ranges as well.

In addition to the score requirement, this example modifies the SQL to include only those candidates whose last name begins with a letter between B and G.

SELECT DISTINCT FNAME,LNAME,WPHONE,HPHONE
FROM CANDIDATE C, TEST_TAKEN TT
WHERE C.CID=TT.CID
AND INTEGER(SCORE) BETWEEN 60 AND 75
AND LNAME BETWEEN 'B' AND 'G'

In this example, the second BETWEEN predicate contains character values. We need to specify the GZ value to include all the possible names that start with the letter G.

This was done assuming that the letter Z is the last possible value in the alphabet.

Searching for Null Values

NULL values represent an unknown value for a particular occurrence of an entity. We can use a NULL value in the cases where we don't know a particular value of a column.

Let's say that we want a list of all those candidates whose score is not yet input. This condition is represented with a NULL value.

SELECT FNAME,LNAME,WPHONE,HPHONE
FROM CANDIDATE C, TEST_TAKEN TT
WHERE C.CID=TT.CID AND SCORE IS NULL

The IS predicate is used to search for the NULL value in this example. Remember that the NULL value means "unknown." Because it has no particular value, it can't be compared with other values. You can't use conditional operands, such as equal (=), with null values.

Searching for Negative Conditions

The BETWEEN, IS, and LIKE predicates always look for the values that meet a particular condition. These predicates can also be used to look for values that don't meet a particular criterion.

The NOT predicate can be used to look for the opposite condition, combined with the LIKE, BETWEEN, and IS predicates, to accomplish negative searches, as shown in the following example. This example has a LIKE predicate combined with the NOT predicate. We want a list of those candidates whose last names do not start with the letter S.

SELECT DISTINCT FNAME,LNAME,WPHONE,HPHONE FROM CANDIDATE
WHERE LNAME NOT LIKE 'S%'
ORDER BY LNAME,FNAME

The next example has a BETWEEN predicate combined with the NOT predicate. We want the list of those candidates whose score, in any test, is not in the range 60 to 75.

SELECT DISTINCT FNAME,LNAME,WPHONE,HPHONE
FROM CANDIDATE C, TEST_TAKEN TT
WHERE C.CID=TT.CID
AND INTEGER(SCORE) NOT BETWEEN 60 AND 75

In this example, the NOT predicate will exclude all the values that are in the range 60 to 75.

Negation can also be applied to the NULL value. This SQL produces a report that searches for those candidates who have a seat number assigned. This is expressed with a NOT NULL value.

SELECT DISTINCT FNAME,LNAME,WPHONE,HPHONE
FROM CANDIDATE C, TEST_TAKEN TT
WHERE C.CID=TT.CID AND SEAT_NO IS NOT NULL

Note

The NOT operator can also be used to negate the standard comparison operators, '=', '<', '<=', '>' and '>='.

Searching for a Set of Values

In SQL, it is possible to establish a restriction condition based on a set of values. Suppose that you need a list of the test centers that have candidates registered for the DB2 Fundamentals test and for the DB2 Application Development test. This can be queried with the following statement:

SELECT DISTINCT NAME,PHONE
FROM TEST_CENTER TC, TEST_TAKEN TT
WHERE TC.TCID=TT.TCID
AND CHAR(NUMBER) IN ('500','502')

The IN clause is used to denote a set of values. In this example, we are using a constant set of values. In this particular case, the SQL statement could also be written using the OR operator to restrict the test numbers.

You can also use the NOT predicate with the IN clause. In this case, the condition will be true when a value is not present in the set of values provided to the IN clause.

You can use as many values as you want in the IN clause, within the defined limits of the size of a SQL statement.

Advanced Selection Functionality

In Chapter 7, Advanced SQL, we will look at more of the functionality and power of the SELECT statement. We will cover topics such as:

  • Subqueries

  • Inner and Outer Joins

  • Nested Table Expressions

  • CASE Expressions

  • Table Functions

  • Row Expressions

  • Unions


045 - Data Modification

Chapter 5. Manipulating Database Objects

Data Modification

Up to now, we have discussed basic SELECT statements. The SELECT statement allows you to retrieve data from your database tables and assumes that data has been previously loaded into the tables. Now we will concentrate on getting data into the database tables using SQL. There are three main statements that can be used to add and change data stored in a DB2 database table. They are the INSERT, DELETE, and UPDATE SQL statements.

To perform these operations, you must have the required privileges on the tables being accessed. Usually, these privileges are more strictly enforced since they can allow the end-user to modify data records.

Inserting Data Records

To initially populate a DB2 table with data, the INSERT statement can be used to store one data record at a time. The statement can be targeted to insert data directly into a base table, or a view can be used instead. If a view is being used as the target, remember that it is the base table where the actual data is being stored.

Every row that is populated using the INSERT statement must adhere to table-check constraints, data type validation, dynamic (trigger) constraints, and referential integrity constraints. An SQL error will occur if any of these conditions is violated during the processing of the INSERT statement.

Note

Remember that you must have the necessary view or table privileges to perform an INSERT statement.

The first example is a simple INSERT statement. This statement will insert the data for the DB2 Data Propagation (#508) exam into the TEST table.

INSERT INTO DB2CERT.TEST
(NUMBER,NAME,TYPE,CUT_SCORE,LENGTH,
TOTALTAKEN,TOTALPASSED) VALUES
('508','DB2 DATA PROPAGATION','P',NULL,90,0,0);

In this example, we specify all the column names and their corresponding values for this data record. Following the VALUES portion of the statement, we include all of the data values for this record.

In the VALUES clause, the number and order of the inserted elements must match the number and order of the column names defined in the INSERT statement. However, the order of the columns doesn't have to match the order in which they are defined in the table. For those columns that don't require a value, you can indicate null or default values. In this example, we are using the NULL value for the CUT_SCORE column.

Note

The number of elements following the VALUES clause must match the number of names in the insert column list.

Depending on your column definition, the DEFAULT value can cause to be inserted a system-defined default, a user-defined default, or NULL. Be aware that if the column doesn't accept nulls (NOT NULL) and wasn't defined as WITH DEFAULT, you will receive an error message when using the DEFAULT value. This error is because the default value for those columns not using the WITH DEFAULT option is the NULL value.

When you want to insert values into all the columns of a table, you do not have to provide the column names in the INSERT statement. This example is shown next.

INSERT INTO DB2CERT.TEST VALUES
('508','DB2 DATA PROPAGATION','P',DEFAULT,90,79,11)

This method will work only if you specify a value for all the columns in a table. If you miss one of the columns of the table, DB2 will not allow you to insert the row into the table. The DEFAULT keyword used in this example will insert the default value for the CUT_SCORE column.

Note

Remember that, depending on the column definition, the default value could be a user-defined default value, a system-defined default value, or NULL.

Inserting Data into Specific Columns

There are times when you need to add data to specific columns. Every column that is not included in the INSERT statement will receive its default value.

This operation can be accomplished only if the omitted columns accept nulls or have a default value definition. This means that you must specify a value for the columns defined as NOT NULL. This restriction excludes columns defined as NOT NULL WITH DEFAULT.

Let's insert a row into the TEST_TAKEN table. In the following example, we will insert data only for the columns CID, TCID, NUMBER, and SEAT_NO.

INSERT INTO DB2CERT.TEST_TAKEN
(CID,TCID,NUMBER,SEAT_NO)
VALUES('888','TR01','508','1')

Remember that columns defined using WITH DEFAULT that are not listed in the INSERT statement will receive the NULL value or a default value.

Note

The TEST_TAKEN table has some referential integrity with other tables. If you want to insert a record into a TEST_TAKEN table, the appropriate values should be inserted into the other three tables in advance.

Inserting a Set of Values

Using SQL, you can insert the result of a SELECT statement into a different table. The SQL statement that generates the resulting set must follow these rules:

  • The number of columns from the SELECT statement must equal the number of columns in the INSERT column list.

  • The data type of each of the columns in the SELECT list must match the data type of those columns in the INSERT list.

  • Column names can be omitted from the INSERT list only if values are inserted into all the columns in the table.

  • Only columns defined to allow NULL or defined as NOT NULL WITH DEFAULT can be omitted from the INSERT list.

  • A subquery in the SELECT, as source for the input, can contain the same table as the table to be inserted, although the source table cannot be the same as the INSERT table.

In some situations, it might be useful to create tables that are duplicates of others so that you can do multiple calculations against them. The next example uses a table called TEST_PASSED, which is a copy of the TEST_TAKEN table. This new table will be used to extract the information about those candidates who have passed any of the DB2 Certification exams.

INSERT INTO DB2CERT.TEST_PASSED
(CID,TCID,NUMBER,
DATE_TAKEN,START_TIME,FINISH_TIME,
PASS_FAIL,SCORE,SEAT_NO)
(SELECT CID,TCID,NUMBER,
DATE_TAKEN,START_TIME,FINISH_TIME,
PASS_FAIL,SCORE,SEAT_NO
FROM DB2CERT.TEST_TAKEN
WHERE PASS_FAIL='P')

The select list used in the fullselect in the above example can also be substituted by a select asterisk (*). This is possible because the TEST_PASSED table has the same column structure as does the TEST_TAKEN table. However, to keep this query isolated from future table modifications, it is recommended that you use the select list instead of the asterisk.

When you use a SELECT statement to insert data into a table, you must enclose it between parentheses. You can also use a table expression to insert values into a table or view using the INSERT clause.

Inserting Large Amounts of Data

Using the SELECT or VALUES statements to insert data into a table can be very useful. However, it is not recommended to load large amounts of data into a table using the INSERT statement because the transaction logging overhead can become unmanageable.

DB2 provides you with the LOAD utility which is designed to move large amounts of data into a table. The LOAD utility is described in Chapter 7 Maintaining Data.

Updating Data Records

So far, we have looked at the INSERT statement as a method of moving data into your DB2 table. You may want to update only a column with values for a group of data records. There is an SQL UPDATE statement that can be used to specify the column and its new values. A table or a view can be referenced as the target for the UPDATE statement.

Note

Remember that you must have the correct privileges in order to perform the UPDATE operation.

The UPDATE statement can be used in two forms:

  • Searched update. This type of UPDATE statement is used to update one or more rows in a table. It requires a WHERE clause to establish the update condition (which rows are to be updated).

  • Positioned update. This kind of UPDATE statement is always embedded into a program. It uses cursors to update the row where the cursor is positioned. As the cursor is repositioned using the FETCH statement, the target row for the UPDATE statement changes.

We focus on searched updates in this chapter. Similar to the INSERT statement, all of the database constraint mechanisms are enforced during an UPDATE statement. There can be specific update constraint triggers and referential integrity constraints that are different from the insert constraints.

For example, the following is a transaction that updates candidate ID 888 exam day for the DB2 Fundamentals certification exam:

UPDATE DB2CERT.TEST_TAKEN
SET DATE_TAKEN=DATE_TAKEN + 3 DAYS
WHERE CHAR(CID) ='888'
AND NUMBER=TEST_ID('500')

In this example, we are using an operation known as a labeled duration to add three days to the original date.

It is very important that you provide the proper WHERE clause to avoid updating unintended data records. In this example, we needed to specify the predicate NUMBER=TEST_ID(500) to avoid changing the date for any of the other tests that the candidate can be scheduled for.

Note

DB2 labeled durations for data types include: YEARS, MONTHS, DAYS, HOURS, MINUTES, SECONDS, and MICROSECONDS.

The UPDATE statement can also be used with fullselects. In this case, the fullselect must return a row with exactly the same number of columns and column data types of the row that will be updated. Observe that this fullselect must return only one row.

Let's update a row using a SELECT statement to set the new value. Candidate ID 888 decides to take the DB2 Fundamentals test today in the test center located in Toronto, Canada.

UPDATE DB2CERT.TEST_TAKEN
SET (DATE_TAKEN,TCID)=
(SELECT CURRENT DATE,TCID FROM DB2CERT.TEST_CENTER
WHERE SUBSTR(CITY,1,7)='TORONTO'
AND COUNTRY='CANADA')
WHERE CHAR(CID)= '888' AND NUMBER=TEST_ID('500');

In the above example, we are updating two different columns in the same operation. These columns are indicated in the parentheses following the SET clause.

After indicating which columns are going to be updated, we use a SELECT statement to retrieve the current date (today) and the test center ID for the test center located in Toronto, Canada. Notice that the last WHERE clause in the statement will restrict the rows that will be updated.

Note

If you forget the update WHERE clause in a searched update, all of the data in your table will be updated.

The SQL statement that will update the DATE_TAKEN and TCID columns is known as a row fullselect. This name is given because it returns only one row. Observe that the scalar fullselect can be considered a special case of a row fullselect.

Note

CURRENT DATE is a DB2 special register that gives the system date. Others include: CURRENT TIME, CURRENT TIMESTAMP, and USER (the authid).

Updating Large Amounts of Data

There are times when you need to update a large number of rows of a particular table. This can be accomplished by issuing a searched update. However, this also could allocate a large amount of transactional log space. You can accomplish updates using positioned updates, where you can easily control the commit frequency.

Removing Data

There are many methods available to remove data from a DB2 database. To remove all of the data within a database, perform the DROP DATABASE command. This may remove more data than you intended because the entire database, including its configuration, will be physically removed.

It is also possible to remove data using the DROP TABLESPACE or DROP TABLE statements. These statements are usually issued only by the SYSADM or DBADM, since they will remove large amounts of data. If you want to remove all of the data records from a table, it is easier and quicker to perform the DROP TABLE statement. Even easier, and much faster is using the LOAD utility with the REPLACE option with an empty input file.

If the table is dropped, it must be re-created before any data can be populated again in the table.

To remove a single data record or a group of records from a table, the DELETE statement should be used. The syntax of the DELETE statement is different from the SELECT or INSERT statements because columns cannot be selected, only rows can be deleted.

The DELETE statement can also be used with views. However, there are restrictions on the type of views that can be used within a DELETE statement.

Note

Remember that you must have the necessary privileges over a table to perform the DELETE operation.

In general, there are two kinds of DELETE statements:

  • Searched delete— This DELETE statement is used to delete one or multiple rows from a table. It can use a WHERE clause to establish the delete condition.

  • Positioned delete— This kind of DELETE operation is always embedded into a program. It uses cursors to delete the row where the cursor is positioned.

In this section we will focus on the searched delete.

The following SQL statement deletes candidates who don't have a telephone number loaded into the table. We will use a searched delete to accomplish this task.

DELETE FROM DB2CERT.CANDIDATE
WHERE HPHONE IS NULL
AND WPHONE IS NULL

This example uses a WHERE clause to delete the data that meets a specific criterion.

To verify the result of the DELETE statement, you can issue a SELECT statement with the same WHERE clause. If the DELETE was successful, the SELECT will return an empty set.

A delete can also become more sophisticated by using subselects. The next SQL statement deletes all the candidates who took the DB2 Certification exams in February of any given year.

DELETE FROM DB2CERT.CANDIDATE
WHERE CID IN (SELECT CID FROM DB2CERT.TEST_TAKEN
WHERE MONTH(DATE_TAKEN)=2)

In this example, we are using a sub-select to retrieve the CID values of the candidates who took a DB2 Certification exam in the month of February. This list will be used to search for the candidates we want to delete.

Deleting All the Rows in a Table

You can delete all the rows in a table if you don't specify a search condition in your DELETE statement. You must be aware of the implications of this type of statement. However, this is not the only way to delete all the rows in a table. You can also delete all the rows in a table if all the rows meet the search condition.

Deleting all the rows in a table by using a DELETE statement may not be the most efficient method. This kind of statement can consume a lot of log space when your tables are large.

Note

For segmented table spaces, a mass delete or drop table will only log the space map pages.


046 - View Classification

Chapter 5. Manipulating Database Objects

View Classification

Now that we have examined various SQL DML statements, let's take a closer look at views. We have already discussed creating views. Now we'll examine the different types of views. Views are classified by the operations they allow, but are generally referred to as either read-only or not read-only views. The referential and check constraints are treated independently. They do not affect the view classification.

For example, you may not be able to insert a value into a table because of a referential constraint. If you create a view using that table, you also cannot insert that value using the view. However, if the view satisfies the rules for a non read-only view, it will still be considered an insertable, updateable, or deleteable view. This is because the insert restriction is located on the base table, not on the view definition.

Read-Only Views

Depending on how a view is defined, a view can be read-only, or can be the object of an INSERT, UPDATE, or DELETE. A view is read-only if one of the following statements is true of its definition:

  • The first FROM clause identifies more than one table or view, or identifies a table function.

  • The first select specifies the keyword DISTINCT.

  • The outer fullselect contains a GROUP BY clause.

  • The outer fullselect contains a HAVING clause.

  • The first SELECT clause contains a column function.

  • It contains a subquery such that the base object of the outer fullselect, and of the subquery, is the same table.

  • The first FROM clause identifies a read-only view.

A read-only view cannot be the object of an INSERT, UPDATE, or DELETE statement. A view that includes GROUP BY or HAVING cannot be referred to in a subquery of a base predicate. The following is an example of a read-only view:

CREATE VIEW READ_ONLY_VIEW
(NAME,WORK_PHONE,HOME_PHONE)
AS
SELECT DISTINCT FNAME,WPHONE,HPHONE
FROM DB2CERT.CANDIDATE C, DB2CERT.TEST_TAKEN TT
WHERE C.CID=TT.CID

The view above is a read-only view because it uses the DISTINCT clause and the SQL statement involves more than one table.

Non Read-Only Views

A view must meet the rules listed above to be considered a non read-only view. Below is an example of a view that can be used for a DELETE statement.

CREATE VIEW DELETABLE_VIEW
(TCID,CID,NUMBER,DATE_TAKEN,START_TIME,SEAT_NO,SCORE)
AS
SELECT TCID,CID,NUMBER,DATE_TAKEN,
START_TIME,SEAT_NO,SCORE
FROM DB2CERT.TEST_TAKEN
WHERE TCID=CENTER_ID('TR01')

A view that can be the object of an UPDATE statement is a special case of a non read-only view as at least one of its columns must be updateable. A column of a view can be updated when all of the following rules are true:

  • The view is not a read-only view.

  • The column resolves to a column of a base table and the FOR READ ONLY or FOR FETCH ONLY option is not specified.

Even though the following view definition uses constant values that cannot be updated, it is a non read-only view and at least you can update one of its columns. Therefore, it can be the object of an UPDATE statement.

CREATE VIEW UPDATABLE_VIEW
(TCID,CID,NUMBER,CURRENT_DATE,CURRENT_TIME,SEAT_NO,SCORE)
AS
SELECT TCID,CID,NUMBER,CURRENT DATE,
CURRENT TIME,SEAT_NO,SCORE
FROM DB2CERT.TEST_TAKEN
WHERE CHAR(TCID)='TX01'

A view can be the object of an INSERT statement when all of its columns are updateable. Also, all of the columns that do not have a default value must be specified in the INSERT statement. The row being inserted must contain a value for each column in the view definition. The following is a view that can be used to insert rows:

CREATE VIEW INSERTABLE_VIEW
(TEST_NUMBER,TEST_NAME,TOTAL_TAKEN)
AS
SELECT NUMBER,NAME,TOTALTAKEN FROM DB2CERT.TEST

The view shown above would appear to allow inserts. However, an attempt to insert the view will fail. This is because there are columns in the base table that don't accept null values. Some of these columns are not present in the view definition.

When you try to insert a value using the view, DB2 will try to insert a NULL into a NOT NULL column. This action is not permitted.

Note

Remember, the constraints defined on the base table are independent of the operations that can be performed using a view.


047 - Summary

Chapter 5. Manipulating Database Objects

Summary

In this chapter, we have discussed the Data Manipulation Language (DML). DML has four primary statements: SELECT, UPDATE, INSERT, and DELETE. These statements enable all database object data manipulation.

The knowledge of basic SQL is mandatory for a DB2 database administrator and application developer. However, SQL is very powerful language, and the level of SQL will vary depending on the user's primary activity.

In this chapter, we cover many of the basic functions of SQL statements. We looked at how to:

  • retrieve rows

  • sort the result set

  • restrict the result with some conditions

  • retrieve rows from more than one table at a time

  • add a row to a table

  • remove a record

  • change the value of the table

  • and more.

If you are a business analyst, you may expect more analytical, statistical information from DB2. DB2 supports very powerful SQL functions for various business needs. We'll talk about some advanced SQL in the next chapter.


048 - Additional Resources

Chapter 5. Manipulating Database Objects

Additional Resources

DB2 UDB for OS/390 DB2 SQL Reference, SC26-9014.

DB2 UDB for OS/390 Application Programming and SQL Guide, SC26-9004.


049 - Chapter 6. Advanced SQL

Part II. Using SQL

Chapter 6. Advanced SQL

  • Subqueries

  • Unions

  • Joins

  • Nested Table Expressions

  • CASE Expressions

  • Row Expressions

  • OLAP and Star Joins

This chapter will cover some the more advanced features found in DB2's SQL. We will start with discussions on the constructs of subqueries, unions, and joins. Then we will examine nested table expressions, the CASE expression with its if-else logic, row expressions, and OLAP features.


050 - Subqueries

Chapter 6. Advanced SQL

Subqueries

A subquery is a SQL statement specifying a search condition that contains either a subselect or a fullselect, and is used inside another SQL statement. A subquery can include search conditions of its own. Each of these search conditions can also contain subqueries. Any SQL statement can contain a hierarchy of subqueries. The SQL query block that contains subqueries is at a higher level than the subqueries it contains. This is important for how columns are referenced between levels.

A subquery can contain search conditions that reference not only columns of the tables identified by its FROM clause, but also columns of tables identified at any higher level. A reference to a column of a table at a higher level is called a correlated reference.

When specified in a search condition, many operands can be used; one of the more common is the IN clause. Consider the difficulty of producing a report on the number of DB2 Certification exams if the name and number of the exams are unknown. The following SQL produces a report that includes all the test centers that have candidates registered for the DB2 Certification exams.

In the example, we'll use the word DB2 as the search string to find the numbers of the DB2 Certification program exams:

SELECT DISTINCT NAME,PHONE
FROM TEST_CENTER TC, TEST_TAKEN TT
WHERE TC.TCID=TT.TCID
AND NUMBER IN
(SELECT NUMBER FROM TEST
WHERE NAME LIKE 'DB2%')

In this example, the subquery appears as part of the IN clause. In the subquery, we are retrieving all the numbers for those tests that have the word DB2 in their name. As you can see in the last example, the subquery is a standard SQL statement. The only difference here is that the subquery is used as a restriction condition. You will never see its output. We are using the subquery only to create a list of values that will be used later by the outer SELECT statement.

The subquery used in this example is known as an noncorrelated subquery. A noncorrelated subquery is one where the values retrieved by the subquery are not directly related to the rows processed by the outer SELECT statement, or any query block at a higher level. A correlated subquery is a query in which the subquery references values of the outer SELECT containing it, or any subselect at a higher level.

In the next example, you want to count how many candidates are registered in each test center. This time you want to display the name of the center near the number of candidates. We use a correlated subquery to accomplish this.

SELECT TC.NAME, COUNT(*)
FROM TEST_CENTER TC,TEST T
WHERE TC.TCID IN
(SELECT TCID FROM TEST_TAKEN TT
WHERE TT.NUMBER=T.NUMBER)
GROUP BY TC.NAME

Observe the WHERE clause in the subquery in this example. It is making reference to a table that is listed in the outer FROM clause.

If you write this query as a noncorrelated subquery, you will have different results. In this example, you need to use a correlated subquery to be sure that all the rows of the TEST_TAKEN table are counted. When you use a noncorrelated subquery, you will count only one occurrence of the test number for each test center.


051 - Unions

Chapter 6. Advanced SQL

Unions

The UNION operation lets you combine the results of two or more different SQL statements into one answer set. You can combine many different tables or SQL statements using the UNION operator; the only restriction is that every table or SQL statement must have the same type, number, and order of columns.

Suppose you wanted to combine the minimum and maximum scores for each of the DB2 Certification program exams and add a string constant that indicates which values are the maximum and minimum.

SELECT NUMBER,'MINIMUM:', MIN(INTEGER(SCORE))
FROM TEST_TAKEN
GROUP BY NUMBER
UNION
SELECT NUMBER,'MAXIMUM:', MAX(INTEGER(SCORE))
FROM TEST_TAKEN
GROUP BY NUMBER
ORDER BY NUMBER,2

The UNION operator shows you the results of two or more separate queries as a single one. In our example, the first query calculates the minimum score of the TEST_TAKEN table. Then the second query calculates the maximum score value. Both queries have the same type, order, and number of columns.

In the above example, the two SQL statements are very similar. However, you can combine very different queries using the UNION operator. Just remember the restriction about the resulting rows.

The UNION operator removes duplicate rows from the resulting set. However, there will be times when you'll need to list all the rows processed by your SQL statements.

SQL provides you with an operator clause that allows you to keep all the rows involved in a UNION operation. This is the ALL clause. Let's create a list of all the first names and last names in our candidate table. In this next example, we want all the first names that start with a letter G and all the last names for the candidates who have taken the DB2 Administration exam.

This example cannot be processed with an OR operator because we are not interested in the first names of those candidates who have taken the DB2 Administration exam. Since there can be a first name that is the same as a last name, we will use a UNION ALL operator to show all the rows.

SELECT FNAME FROM CANDIDATE
WHERE FNAME LIKE 'G%'
UNION ALL
SELECT LNAME FROM CANDIDATE C,TEST_TAKEN TT
WHERE C.CID = TT.CID
AND CHAR(NUMBER) = '501'

Note

Always try to code a UNION ALL. Code a UNION only when duplicates are not desired. The UNION ALL offers better performance. However, you can't always substitute a UNION for a UNION ALL.

Note

A UNION can be used between any combination of subselects and fullselects, and in subqueries, views, and table expressions.


052 - Nested Table Expressions

Chapter 6. Advanced SQL

Nested Table Expressions

A nested table expression is a special kind of subquery. This subquery is used in the FROM clause of an SQL statement. Nested table expressions will create logical temporary tables that are known only in the SQL statement that defines them. They may or may not result in a materialized local table being created as the SQL may be merged with the outer SQL statement.

These subqueries can be considered temporary views, and are also sometimes referred to as inline views. You can use nested table expressions to select from a grouped table or to obtain the same results that you expect from a view.

Consider the problem of obtaining the maximum average score for the DB2 Certification program exams. To gather this result, you must first obtain the averages and then you must select the maximum value from that list.

In the following example, we use a nested table expression to accomplish this request:

SELECT MAX(AVG_SCORE)
FROM (
   SELECT NUMBER,
   AVG(INTEGER(SCORE)) AS AVG_SCORE
   FROM TEST_TAKEN
   GROUP BY NUMBER) AS AVERAGES

In this example the nested subselect will create a temporary table that will be used by the outer SELECT to obtain the maximum average score. This temporary table is called AVERAGES.

The number column is included in the subquery to be able to gather the average for each one of the exams. After the subquery is completed, the outer SELECT will be able to obtain the maximum value of the averages calculated in the nested table expression.

An advantage of using nested table expressions over views is that nested table expressions exist only during the execution of the query, so you don't have to worry about their maintenance. They reduce contention over the system catalog tables, and since they are created at execution time, they can be defined using host variables.

Note

The TABLE clause can also be used to denote that the subquery following it will create a temporary table. This keyword is also used to denote user-defined TABLE functions. Refer to Chapter 15, Advanced Functions, for more information on UDFs.


053 - Joins

Chapter 6. Advanced SQL

Joins

The heart of SQL in a relational system is the ability to perform relational joins. Joins are specified by using more than one table name in the FROM clause of a subselect. A joined table specifies an intermediate result table that is the result of either an inner join or an outer join.

Note

The maximum number of tables that can be joined by DB2 is 15 (or 255 if star join is selected), but there can be up to 225 table references in a SQL statement.

Inner Joins

The join operation used most often, which is the one we have been using in the book exercises, is known as an INNER JOIN. An inner join combines each row of the left table with every row of the right table keeping only the rows where the join condition is true. The result table may be missing rows from either or both of the joined tables. The following example shows an inner join without using the INNER JOIN syntax, which means that the inner join is implicitly defined. In the following example, we want to find the average score of DB2 tests taken by US citizens in the small test centers year by year:

SELECT T.NAME, YEAR(TT.DATE_TAKEN) AS YEAR,
   AVG(INTEGER(TT.SCORE)) AS AVGSC
FROM TEST T, TEST_TAKEN TT, TEST_CENTER TC,
   CANDIDATE C
WHERE C.CID = TT.CID AND
   TC.TCID = TT.TCID AND
   T.NUMBER = TT.NUMBER AND
   T.NAME LIKE 'DB2%' AND
   C.COUNTRY = 'USA' AND
   TC.NOSEATS < 10
   GROUP BY T.NAME, TT.DATE_TAKEN;

If the inner join used the explicit syntax, it would appear as:

SELECT T.NAME, YEAR(TT.DATE_TAKEN) AS YEAR,
   AVG(INTEGER(TT.SCORE)) AS AVGSC
FROM TEST T INNER JOIN TEST_TAKEN TT INNER JOIN
   TEST_CENTER TC INNER JOIN CANDIDATE C
ON C.CID = TT.CID
   AND TC.TCID = TT.TCID
   AND T.NUMBER = TT.NUMBER
WHERE T.NAME LIKE 'DB2%'
  AND C.COUNTRY = 'USA'
  AND TC.NOSEATS < 10
GROUP BY T.NAME, TT.DATE_TAKEN;

There is more on the reasons for this syntax in the next section on outer joins.

Outer Joins

Now, we will talk about a different kind of join operation: the outer joins. The result set of an inner join consists only of those matched rows that are present in both joined tables. What happens when we need to include those values that are present in one or another joined table, but not in both of them? This is the case when we need to use an OUTER JOIN operation. Outer joins are designed to generate an answer set that includes those values that are present in joined tables and those that are not. There are different kinds of outer joins as we will see.

Before getting into the details of the outer join, we will examine the explicit syntax used to code joins between tables. Let's start with an inner join coded with this syntax.

The following join example produces an answer set containing the first name, the phone number, and the highest score for each candidate in the TEST_TAKEN table:

SELECT FNAME, WPHONE, MAX(INTEGER(SCORE))
FROM DB2CERT.CANDIDATE C
INNER JOIN DB2CERT.TEST_TAKEN TT ON C.CID=TT.CID
GROUP BY FNAME, WPHONE

In this syntax, you indicate the tables that will be joined, along with the join operation, and the join columns that are required.

Observe the INNER JOIN operator in the example above. It belongs to the FROM clause of the statement. The INNER JOIN operator specifies that an inner join operation will be used for the statement. The keyword ON is used to specify the join conditions for the tables being joined. In our example, the join condition is based on the join columns, CID of the CANDIDATE table, and CID of the TEST_TAKEN table.

The explicit join syntax also allows you to specify an outer join, as we will see in the next sections.

Left Outer Join

A LEFT OUTER JOIN operation, also known as left join, produces an answer set that includes the matching values of both joined tables and those values present only in the left joined table. The left joined table is the one used in the left part of the LEFT OUTER JOIN operator when coding the join operation.

We have been requested to generate a report that includes the first name, the phone number, and the highest score for all the candidates present in the CANDIDATE table.

If an inner join is used, as is shown in the last example, the report will include only data of those candidates present in the TEST_TAKEN table.

The request could be solved using some SQL statements already discussed; however, the construction will be complex. We will use the left outer join to satisfy the request, as the following example shows:

SELECT FNAME, WPHONE, MAX(INTEGER(SCORE))
FROM DB2CERT.CANDIDATE C
   LEFT OUTER JOIN DB2CERT.TEST_TAKEN TT
ON C.CID=TT.CID
GROUP BY FNAME, WPHONE

Observe the syntax used to indicate a left outer join. The LEFT OUTER JOIN operator is used to indicate the left outer join operation. In this example, the answer set includes those candidates not present in the TEST_TAKEN table. The MAX(INTEGER(SCORE)) column will show nulls for those candidates.

Right Outer Join

A RIGHT OUTER JOIN operation, also known as right join, produces an answer set that includes the matching values of both joined tables and those values present only in the right joined table. The right joined table is the one used in the right part of the RIGHT OUTER JOIN operator when coding the join operation.

The example using a right outer join is shown below:

SELECT NAME, COUNT(DISTINCT CHAR(TT.CID))
FROM DB2CERT.TEST_TAKEN TT
RIGHT OUTER JOIN DB2CERT.TEST T
ON TT.NUMBER = T.NUMBER

In this example, all test names present in the TEST table and the number of candidates who scheduled or took each test are requested. Notice there may be some tests for which no candidate was scheduled. You cannot report such tests using a inner join statement; however, you can do it using a right outer join.

Note

You can also use LEFT JOIN to indicate a left outer join operation.

Full Outer Join

The FULL OUTER JOIN operation produces an answer set that includes the matching values of both joined tables and those values not present in one or the other of the tables.

To show a full outer join operation, we will create two sample tables: CITY and COUNTRY. They show the relationship between a city and a country (CITY table) and a country and a continent (COUNTRY table). The CITY table is designed to have countries that are not in the COUNTRY table. The COUNTRY table is also designed to have countries that are not in the CITY table. The contents of both tables are shown below.

CITY_NAME               COUNTRY_NAME
---------------         ---------------
Sydney                  Australia
London                  England
Dublin                  Ireland
Florence                Italy
Milan                   Italy
Mexico City             Mexico
Lima                    Peru
Toronto                 Canada
Vienna                  Austria
Hannover                Germany

COUNTRY_NAME            CONTINENT
---------------         ------------------------
Australia               Australian Continent
England                 European Continent
Ireland                 European Continent
Italy                   European Continent
Mexico                  American Continent
Austria                 European Continent
South Africa            African Continent
Spain                   European Continent

We want to show all the countries, cities, and the continents that are in the tables. Therefore, we are using a full outer join, as the following example shows:

SELECT CTRY.CONTINENT, CTRY.COUNTRY_NAME,
CTY.COUNTRY_NAME, CTY.CITY_NAME
FROM COUNTRY CTRY
   FULL OUTER JOIN CITY CTY
ON CTY.COUNTRY_NAME=CTRY.COUNTRY_NAME
ORDER BY CTRY.CONTINENT,
   CTY.COUNTRY_NAME,
   CTY.CITY_NAME

The result of the query is as follows:

CONTINENT             COUNTRY_NAME   COUNTRY_NAME   CITY_NAME
--------------        ------------   ------------   ---------
African Continent     South Africa   -              -
American Continent    Mexico         Mexico         Mexico City
Australian Continent  Australia      Australia      Sydney
European Continent    Austria        Austria        Vienna
European Continent    England        England        London
European Continent    Ireland        Ireland        Dublin
European Continent    Italy          Italy          Forence
European Continent    Italy          Italy          Milan
European Continent    Spain          -              -
-                     -              Canada         Toronto
-                     -              Germany        Hannover
-                     -              Peru           Lima

As shown in the above example, the rows that have a null value were added by the outer join operation. The COUNTRY_NAME column is shown twice to see those countries present in the COUNTRY table that are not present in the CITY table and vice versa.

Combining Outer Joins

Up to now we have discussed each outer join operation separately. Now we will show a more complex example combining two outer joins in one single query.

Let's display all the candidates and all the tests with their respective scores.

To create this query, we need two outer joins. The first outer join will obtain all candidates and their scores, including candidates who did not schedule or take any tests. The second outer join will retrieve all the tests present in the TEST table even if no candidate scheduled or took those tests.

SELECT C.CID, T.NAME, SCORE
FROM (DB2CERT.CANDIDATE C
LEFT OUTER JOIN DB2CERT.TEST_TAKEN TT
ON TT.CID = C.CID)
   FULL OUTER JOIN DB2CERT.TEST T
   ON TT.NUMBER = T.NUMBER
ORDER BY C.CID

The first outer join is enclosed in parentheses. The parentheses are used for readability and to denote that the left outer join will be resolved first. This left outer join gathers all the candidate IDs. We need only a left outer join here. Because of referential integrity constraints, table TEST_TAKEN can have only candidates present in the CANDIDATE table.

The second part is a full outer join. With this outer join, we take all the tests taken by the candidates, the result of the left outer join, and join them with all the tests in the TEST table. We need a full outer join this time even though the TEST_TAKEN table can have only test numbers that are present in the TEST table, because the left table of this outer join, which is the result of the first join, may include NULL values as a test number. The TEST table does not have null values, so we need to use a full outer join. The results are shown below.

CID NAME                               SCORE
--- ------------------------------     -----
111 DB2 Fundamentals                   65
111 DB2 Administration                 73
111 DB2 Application Development        67
222 DB2 Fundamentals                   55
222 DB2 Application Development        53
222 DB2 Application Development        75
333 DB2 Fundamentals                   82
333 DB2 Administration                 -
333 DB2 Application Development        92
444 -                                  -
555 -                                  -
666 -                                  -
777 -                                  -
888 -                                  -
14 record(s) selected.

Joins Vs. Denormalization

One issue that needs to be examined when discussing joins is that of denormalization. Sometimes there is the perceived need to take tables always joined together and put them into a single table, in an effort to minimize the joins and improve performance. Since this is generally applied to tables in a one-to-many relationship, it results with multiple occurrences of the same data. This introduces update anomalies, the situation where a single piece of data occurs multiple times. When there is the attempt to update it, all occurrences must be found and there is no fixed way to determine the number or location of the multiple occurrences.


054 - CASE Expressions

Chapter 6. Advanced SQL

CASE Expressions

You can add some logic to your SQL statements and output using CASE expressions. Consider the generation of a list of those candidates who have passed the DB2 Fundamentals exam.

In the report, you want to print the score of the tests, but instead of printing the numeric score, you want to print a message. If the score is below the CUT_SCORE, you want to print 'Not Passed.' If it is between the CUT_SCORE and 90, you want to print 'Passed,' and if the score is above 90, you want to print 'Excellent.' The following SQL statement using a CASE expression accomplishes this:

SELECT FNAME,LNAME,
   CASE
      WHEN INTEGER(SCORE) < 65 THEN 'NOT PASSED'
      WHEN SCORE <= SCORE(90) THEN 'PASSED'
      ELSE
      'EXCELLENT'
   END
FROM CANDIDATE C, TEST_TAKEN TT
WHERE C.CID = TT.CID
AND CHAR(NUMBER)='500'

The SQL statement presented above provides string messages based on the conditions of the CASE expression. In this example, the score column features a numeric value, but we are using it to produce a character string.

The order of the conditions for the CASE expression is very important. DB2 will process the first condition first, then the second, and so on. If you do not pay attention to the order in which the conditions are processed, you might be retrieving the same result for every row in your table. For example, if you coded the <= SCORE (90) option before the < 65, all the data that is lower than 91, even 64 or 30, will display the message Passed. CASE expressions can also be used in places other than select lists.

Note

You must use the end keyword to finish a CASE statement.

Using CASE Expressions in Functions

CASE expressions can be embedded as functions parameters. This allows you to pass different parameters to the function in a single pass of the data. Suppose that the TEST_TAKEN table is very large and we have the following requirements:

  • The number of tests taken with a score higher than 90

  • The number of tests taken with a score of 90

  • The number of tests taken with a score lower than 70

  • The number of DB2 Fundamentals exams taken

Without the use of CASE expressions, this will require four different queries that will read the entire table. We want to do this in one single pass of the data, because the table is very large.

The query will use four count functions, each one evaluating different criteria using a CASE expression.

SELECT COUNT (CASE WHEN INTEGER (SCORE) > 90 THEN 1
   ELSE NULL
   END) AS MOREGB90,
   COUNT (CASE WHEN INTEGER (SCORE) = 90 THEN 1
   ELSE NULL
   END) AS EQUALGB90,
   COUNT (CASE WHEN INTEGER (SCORE) < 70 THEN 1
   ELSE NULL
   END) AS MINORGB70,
   COUNT (CASE WHEN NUMBER=TEST_ID('500') THEN 1
   ELSE NULL
   END) AS EQUALGB500
FROM DB2CERT.TEST_TAKEN;

This type of query may be useful when performing data inspection analysis. Notice that the four different requirements are solved in a single pass of the data. The query was created using a different column function for each one of the conditions presented as a requirement. The conditions are evaluated in the CASE expression inside each function. When the condition evaluates true, it will return a value of 1 and the row will be counted. When the condition evaluates false, the CASE expression will return a null value and the row will not be counted.


055 - Row Expressions

Chapter 6. Advanced SQL

Row Expressions

Row expressions are an extension to predicates. They allow more than one set of comparisons in a single predicate using a subquery that returns more than one column and even more than one row. This predicate will be true when all three columns on the left equal the three values in any single row returned in the result set from the subquery. This will also allow the use of quantified predicates with row expressions.

SELECT * FROM TABLE
WHERE (COL1, COL2, COL3) IN
   (SELECT COLA, COLB, COLC
    FROM TABLE
    UNION ALL
    SELECT COLX, COLY, COLZ
    FROM ANOTHER_TABLE)


056 - OLAP Features

Chapter 6. Advanced SQL

OLAP Features

Databases normally hold large amounts of data that can be updated, deleted, queried, and inserted on a daily basis. Databases in which data is constantly updated, deleted, and inserted are known as Online Transaction Processing (OLTP) systems.

Databases that hold large amounts of data and do not have a heavy transaction work load, but do have a large number of concurrent queries executing all the time, are known as Decision Support Systems (DSS). Certain decision support systems have fewer queries, but each query can be very complex. These allow users to examine the data from different perspectives by performing Online Analytical Processing (OLAP).

The functionality of the database is required to provide multidimensional views of relational data, without a significant performance effect. DB2 provides a star join feature to assist in the processing of star schemas and their derivatives.

STAR Schemas

The concept of a STAR schema is illustrated in the following figure. A business view of a highly normalized database often requires a number of attributes associated with one primary object. Each of these attributes is contained in separate tables.

The following points are characteristic of a STAR schema design:

  • There is a large fact table that contains data relating to the dimension tables. In Figure 6.1, the fact table is the TEST_TAKEN table. It contains detailed information on each test taken, including exactly which test was taken, in which center the test was taken, and who took the test.

    Figure 6.1. STAR Schema in the DB2CERT Database

  • There are a number of small dimension tables that typically hold descriptive information about an entity that has a small number of rows. In Figure 6.1, the dimension tables are TEST, CANDIDATE, and TEST_CENTER.

  • The primary keys of the dimension tables involved in the STAR schema supply foreign key entries in the fact table. The concatenation of foreign keys from the dimension tables usually forms a small subset of the fact table. In Figure 6.1, the foreign keys are Candidate ID, Center ID, and Test ID.

This approach allows as few attributes as possible to be stored in the fact table. The benefit of this is that the fact table is usually very large and therefore any data duplication in this table would be very costly in terms of storage and access times. If the DB2CERT database were used to store information on a university's entrance examinations, for example, the TEST_TAKEN table could grow enormously.

OLAP schemas, such as the STAR schemas, are frequently used for large databases. These schemas make it very important to access the data in these databases in an optimal manner. Otherwise, the joins involved in the schemas may result in poor performance.

OLAP Indexes

A typical STAR schema includes a large number of indexes. This is due to the ad hoc nature of queries in an OLAP environment. Such an environment is typically not subjected to constant insert or update activity and therefore does not have to suffer from significant performance degradation as a result of index maintenance.

The prime consideration of indexes in an OLAP environment is to facilitate the joining of tables and the ordering of output. This is particularly important for the fact table where multiple indexes are defined, especially on foreign key columns relating to the dimension tables. The benefit of multiple indexes in this environment is improved query performance against the fact table. The indexes defined on the tables could either be single-column or multicolumn indexes.

There are also certain issues to be considered when using multiple indexes in the OLAP environment. The first is that multiple indexes will require a certain amount of space, depending on the number of columns in each index and the size of the tables. The second is that there will be a significant one-time cost when building indexes, perhaps during a bulk load.

Star Joins

A typical query against databases designed with the star schema would consist of multiple local predicates referencing values in the dimension tables and contain join predicates connecting the dimension tables to the fact table, as shown in the following example. These types of queries are called star joins.

SELECT T.NAME, YEAR(TT.DATE_TAKEN) AS YEAR,
   AVG(INTEGER(TT.SCORE)) AS AVGSC
FROM TEST T, TEST_TAKEN TT, TEST_CENTER TC,
   CANDIDATE C
WHERE C.CID = TT.CID AND
   TC.TCID = TT.TCID AND
   T.NUMBER = TT.NUMBER AND
   T.NAME LIKE 'DB2%' AND
   C.COUNTRY = 'CANADA' AND
   TC.NOSEATS < 10
GROUP BY T.NAME, YEAR(TT.DATE_TAKEN);

In this example, we want to find the average score of DB2 tests taken by Canadian citizens in the small test centers year by year. A star join query is difficult to execute efficiently because no single join predicate reduces the cardinality of the fact rows significantly, although the combination of join predicates results in a small answer set. The more dimension tables involved in the join result, the more potential there is to plan an inefficient access path. To execute star join queries efficiently, multiple index access can be performed by the optimizer, or a special star join method can be used.

Star Join

The star join feature can be turned on by setting the value of subsystem parameter STARJOIN to 1, or if the cardinality of the fact table to the largest dimension table meets the requirements specified by the value of the subsystem parameter. The values of STARJOIN and cardinality requirements are:

  • -1 star join is disabled. This is the default.

  • 1 star join is enabled. The one table with the largest cardinality is the fact table. However, if there is more than one table with this cardinality, star join is not enabled.

  • 0 star join is enabled if the cardinality of the fact table is at least 25 times the cardinality of the largest dimension that is a base table that is joined to the fact table.

  • n star join is enabled if the cardinality of the fact table is at least n times the cardinality of the largest dimension that is a base table that is joined to the fact table, where 2 <= n <= 32768.

To access the data in a star schema, you write SELECT statements that include join operations between the fact table and the dimension tables; no join operations exist between dimension tables. When the query meets the following conditions, that query is a star schema:

  • The query references at least two dimensions.

  • All join predicates are between the fact table and the dimension tables, or within tables of the same dimension.

  • All join predicates between the fact table and dimension tables are equi-join predicates.

  • All join predicates between the fact table and dimension tables are Boolean term predicates.

  • No correlated subqueries cross dimensions.

  • No single fact table column is joined to columns of different dimension tables in join predicates.

  • After DB2 simplifies join operations, no outer join operations exist.

  • The data type and length of both sides of a join predicate are the same.

When this join is performed, it would be identified by an "S" in the JOINTYPE column of the PLAN_TABLE.


057 - Summary

Chapter 6. Advanced SQL

Summary

In this chapter, we have looked at some of the advanced features of the SQL language. This includes a look at all the different types of joins: inner, left outer, right outer, full outer. Star joins were also examined that are used in support of star schemas.

SQL can become very useful for solving complex business problems by using such features as CASE expressions, nested table expressions, UNIONs, and subqueries. These features allow us to push a lot of business logic into the SQL statement.


058 - Additional Resources

Chapter 6. Advanced SQL

Additional Resources

  • IBM DB2 UDB for OS/390 SQL Reference, SC26-9014.

  • IBM DB2 UDB for OS/390 Application Programming and SQL Guide, SC26-9004.


059 - Part III: DB2 UDB for OS/390 and z/OS Administration

Part III: DB2 UDB for OS/390 and z/OS Administration

7 Maintaining Data

8 Database Recovery

9 Data Sharing

10 Performance Monitoring and Tuning


060 - Chapter 7. Maintaining Data

Part III. DB2 UDB for OS/390 and z/OS Administration

Chapter 7. Maintaining Data

  • Loading Data

  • Unloading Data

  • Reorganizing Data

  • Gathering Statistics

  • Checking Data Consistencies

  • Resolving Exceptions

In this chapter, the techniques of how to populate and extract DB2 data using the LOAD and UNLOAD utilities will be examined. The different options for using these utilities will be examined along with a discussion of some of the performance options.

We will also take a close look at the REORG utility. There are many options available on the REORG utility and we will take a look at many of those. In addition we will also examine the parallelism available for objects that are involved in the REORG process.

Additional maintenance topics will include gathering statistics via the RUNSTATS utility and examining the data and statistics of the database management system.

There are also utilities that help ensure that the data is always in a consistent state. These CHECK utilities will be discussed.

We also look at how to resolve restrictive and advisory states.


061 - Data Movement

Chapter 7. Maintaining Data

Data Movement

In order to get data into the DB2 tables, we can perform DML INSERTs; however, for large amounts of data this is not a feasible option. For populating tables with large amounts of data, we need to use the LOAD utility.

For removing data from a DB2 table, we also have the option of doing DML DELETEs, but here again this is not optimal for large amounts of data. We have a couple of options for unloading data: DSNTIAUL, UNLOAD utility, and REORG DISCARD or UNLOAD EXTERNAL.

Load Utility

The LOAD utility is used to load one or more tables of a table space. This utility loads records into the tables and builds or extends any indexes defined on them. If the table space already contains data, you can choose whether you want to add the new data to the existing data or replace the existing data. The loaded data is processed by any edit or validation routine associated with the table, and any field procedure associated with any column of the table. The LOAD utility will load data coming from a sequential data set into one or more tables in the same table space. Since data is coming in from a non-DB2 source, all integrity checking must be performed, such as entity integrity, referential integrity, and check integrity.

Note

Triggers are not fired during LOAD operations.

The output from LOAD DATA consists of a loaded table space or partition, a discard file of rejected records, and a summary report of errors encountered during processing (generated only if you specify ENFORCE CONSTRAINTS or if the LOAD involves unique indexes).

LOAD Phases

The LOAD utility consists of several phases:

  • ULTILINIT

    • Performs initialization and startup.

  • RELOAD

    • Loads the record types and writes temporary file records for indexes and foreign keys. Check constraints are checked for each row. One pass through the sequential input data set is made. Internal commits are taken to provide commit points at which to restart in case the operation should halt in this phase. Creates inline copies if you specified the COPYDDN or RECOVERYDDN keywords. If SORTKEYS is used, a subtask is started at the beginning of the RELOAD phase to handle the work of sorting the keys. The sort subtask initializes and waits for the main RELOAD phase to pass its keys to SORT. The RELOAD phase loads the data, extracts the keys, and passes them in memory for sorting. At the end of the RELOAD phase, the last key is passed to SORT, and record sorting completes. PREFORMAT for table spaces occurs at the end of the RELOAD phase. SORT for sorting of temporary file records occurs before creating indexes or validating referential constraints, if indexes or foreign keys exist.

      The SORT phase is skipped if all the following conditions apply for the data processed during the RELOAD phase:

      • There is not more than one key per table.

      • All keys are the same type (index key, index foreign key, non-indexed foreign key).

      • The data being loaded or reloaded is in key order (if a key exists).

      • The data being loaded or reloaded is grouped by table and each input record is loaded into one table only.

If the SORTKEYS keyword is used, SORT passes the sorted keys in memory to the BUILD phase, which builds the indexes.

  • BUILD

    • Creates indexes from temporary file records for all indexes defined on the loaded tables, detects any duplicate keys, and performs preformatting of index pages.

  • SORTBLD

    • If you specify a parallel index build, all activities that normally occur in both the SORT and BUILD phases occur in the SORTBLD phase instead. When parallelism is used during index build, you will have multiple SORTBLD tasks active at the same time.

  • INDXVAL

    • Corrects unique index violations from the information in SYSERR, if any exist.

  • ENFORCE

    • Checks referential constraints, performs corrections of violations, reports them to SYSERR.

  • DISCARD

    • Copies records causing errors from the input data set to the discard data.

  • REPORT

    • Generates a summary report, if you specified ENFORCE CONSTRAINT or if load index validation is performed.

  • UTILTERM

    • Performs any necessary clean-up.

Loading Data

In order to load data into a DB2 table, we must supply an input data set and a target table. The input data set must match the target table. The following example loads data from the data set specified by the DB2LDS DD statement into the TEST_TAKEN table.

LOAD DATA INDDN DB2LDS
   INTO TABLE DB2USER1.TEST_TAKEN

Rules can also be selectively added to the LOAD criteria to only LOAD rows that meet a specified criteria. An example is shown below:

LOAD DATA INDDN DB2LDS
   INTO TABLE DB2USER1.TEST_TAKEN
   WHEN (1:3)='300'

The below example shows how to map the position in the file to the columns in the table.

LOAD DATA RESUME YES
   INTO TABLE DB2USER1.TEST WHEN (1:1)='P'
   (NUMBER POSITION (1:6) CHAR,
   NAME POSITION (7:57) CHAR,
   TYPE POSITION (58:59) CHAR,
   CUTSCORE POSITION (60:68) DECIMAL EXTERNAL,
   LENGTH POSITION (69:74) SMALLINT,
   TOTALTKN POSITION (75:80) SMALLINT,
   TOTALPAS POSITION (81:86) SMALLINT)

This example shows how to identify the table that is to be loaded, the fields within the input record, and the format of the input record. The fields need to be mapped to the columns of the table that is being loaded, else errors will occur.

Add To or Replace

The REPLACE option on the LOAD utility can be used to replace data in a single-table table space or in a multiple-table table space. All of the data can be replaced in the table space by using this option. If you want to preserve the records that are already in the table, there is the RESUME option.

If an object is in REORG pending status, you can perform a LOAD REPLACE of the entire table space (which resets REORG pending status). In this situation, no other LOAD operations are allowed.

Note

LOAD REPLACE or PART REPLACE with LOG YES logs only the reset and not each deleted row. If you need to see what rows are being deleted, use the SQL DELETE statement.

In the following example, we show how to replace data in one table in a single-table table space:

LOAD DATA
   REPLACE
      INTO TABLE DSN861.DEPT
      (DEPTNO POSITION (1)CHAR(3),
       DEPTNAME POSITION (5)VARCHAR
       )
   ENFORCE NO

Care must be taken when using LOAD REPLACE on a multiple-table table space, because LOAD works on an entire table space at a time. In order to replace all rows in a LOAD multiple-table table space, you have to work with one table at a time, using the RESUME YES option on all but the first table. For example, if you have two tables in a table space, the following steps need to be performed. The first step is to use LOAD REPLACE on the first table. This will empty out the table space and replace just the data for the first table. Next, use LOAD with RESUME YES on the second table. This adds the records for the second table without destroying the data in the first table.

If just one table in a multi-table table space needs to be replaced, then all rows need to be deleted from the table; use LOAD with RESUME YES.

If you need to add data to a table rather than replace it, use LOAD with RESUME YES. The RESUME keyword specifies whether data is to be loaded into an empty or a non-empty table space. RESUME NO loads records into an empty table space. RESUME YES loads records into a non-empty table space.

If RESUME NO is specified and the target table space is not empty, no data is loaded. If RESUME YES is specified and the target table space is empty, data is loaded.

Note

LOAD always adds rows to the end of the existing rows, but index entries are placed in key sequence.

Note

You can run a LOAD REPLACE on a table space that has pages on the LPL and it will reset the status; however, you cannot run a LOAD RESUME on the object.

To delete all the data in a table space, specify the LOAD REPLACE without loading any records. This is an efficient way of clearing a table space. In order to do this, the input data set should be specified in the JCL as DD DUMMY.

The LOAD REPLACE method for mass deletes is efficient due to the following reasons:

  • LOAD REPLACE does not log any rows.

  • LOAD REPLACE redefines the table space.

  • LOAD REPLACE retains all views and privileges associated with a table space or table.

  • LOG YES can be used to make the LOAD REPLACE recoverable.

  • LOAD REPLACE will replace ALL TABLES in the table space.

Loading Ordered Rows

The LOAD utility loads records into a table space in the order in which they exist in the input stream. LOAD does not sort the input stream or insert records in sequence with existing records, even if there is a clustering index. To achieve clustering when loading an empty table or replacing data, sort the input stream before loading the data.

Note

When adding data to a clustered table, consider reorganizing the table afterwards.

Loading Partitions

By using the PART clause of the INTO TABLE option, only the specified partitions of a partitioned table are loaded. If the PART keyword is omitted, the entire table will be loaded. The REPLACE and RESUME options can be specified separately by partition. The following example loads data into the first (and replaces existing data) and adds data to the second partition of the employee table. Records with '0' in column 1 replace the contents of partition 1; records with '1' in column 1 are added to partition 2; all other records are ignored. (The example, simplified to illustrate the point, does not list field specifications for all columns of the table.)

LOAD DATA
INTO TABLE DB2USER1.CANDIDATE PART 1 REPLACE WHEN (1)= '0'
(CID POSITION (1:9) CHAR(9),
LNAME POSITION (10:40) VARCHAR(30),
...
)
INTO TABLE DB2USER1.CANDIDATE PART 2 RESUME YES WHEN (1)='1'
(CID POSITION (1:9) CHAR(9),
LNAME POSITION (10:40) VARCHAR(30),
...
)

Note

If you are not loading columns in the same order as in the CREATE TABLE statement, you must code field specifications for each INTO TABLE statement.

The next example assumes that the data is in separate input data sets. The data is already sorted by partition, so the WHEN clause of INTO TABLE is not necessary. The RESUME YES option placed before the PART option will not allow concurrent partition processing during the execution of the utility.

LOAD DATA INDDN CERTLD1 CONTINUEIF(72:72)='X'
RESUME YES
INTO TABLE DB2USER1.CANDIDATE REPLACE PART 1
LOAD DATA INDDN CERTLD2 CONTINUEIF(72:72)='X'
RESUME YES
INTO TABLE DB2USER1.CANDIDATE REPLACE PART 2

The following example allows partitioning independence when loading more than one partition concurrently:

LOAD DATA INDDN SYSREC LOG NO
INTO TABLE DB2USER1.CANDIDATE PART 2 REPLACE

Note

LOAD INTO PART x is not allowed if an identity column is part of the partitioning index.

Concurrent Access

If another table needs to access data during a LOAD utility, you can specify SHRLEVEL CHANGE on a LOAD RESUME with the LOG YES option.

This option effectively combines the speed and performance of the LOAD utility with the availability and access offered by INSERT processing. It operates similar to an SQL INSERT program. It claims instead of draining for best concurrent access.

It is LOG YES only (will not require a COPY afterward). Locking problems are avoided through internal monitoring of the commit scope. You can also run it in parallel for partitioned table spaces.

Conversion

The LOAD utility converts data between compatible data types. The table below (Table 7.1) identifies the allowable data conversions and the defaults used when the input data type is not specified in a field specification of the INTO TABLE statement.

Table 7.1. LOAD Utility Data Conversions

Input Data Type

Output Data Type

Numeric Data Conversion

SMALLINT

INTEGER

DECIMAL

FLOAT

SMALLINT

Default

Allowed

Allowed

Allowed

INTEGER

Allowed

Default

Allowed

Allowed

DECIMAL

Allowed

Allowed

Default

Allowed

FLOAT

Allowed

Allowed

Allowed

Default

Character Data Conversion

BLOB /CLOB

CHAR

VARCHAR

LONG VARCHAR

CHAR

Allowed

Default

Allowed

Allowed

CHAR MIXED

Allowed

VARCHAR

Allowed

Allowed

Default

Default

Time Data Conversion

DATE

TIME

TIMESTAMP

DATE EXTERNAL

Default

TIME EXTERNAL

Default

TIMESTAMP

EXTERNAL

Allowed

Allowed

Default

Graphic Data Conversion

DBCLOB

GRAPHIC

VARGRAPHIC

LONG VARGRAPHIC

GRAPHIC

Allowed

Default

Allowed

Allowed

VARGRAPHIC

Allowed

Allowed

Default

Default

Referential Integrity

The LOAD utility will not load a table with an incomplete definition. If the table has a primary key, then the unique index on that key must exist. If any table named to be loaded has an incomplete definition, the LOAD job will terminate.

By default, LOAD enforces referential constraints (ENFORCE CONSTRAINTS). During this process several errors could occur. Some of those errors could include:

  • Input file may have duplicate values of a primary key.

  • Input file may have invalid foreign-key values (values not in the primary key of the corresponding parent table).

  • Target table might lack primary key values (values of foreign keys in dependent tables).

A primary index must be a unique index, and must exist if the table definition is complete. Therefore, when a parent table is loaded, there must be at least a primary index.

Note

An error data set, and probably also a map data set and a discard data set, will be needed for the LOAD utility for RI errors.

A dependent table has the constraint where the values of its foreign keys must be values of the primary keys of corresponding parent tables. By default, LOAD enforces that constraint in much the same way as it enforces the uniqueness of key values in a unique index. First, it loads all records to the table; subsequently, it checks their validity with respect to the constraints, identifies any invalid record by an error message, and deletes the record. The record can optionally be copied to a discard data set.

If a record fails to load because it violates a referential constraint, any of its dependent records in the same job also fail. For example, suppose that the sample project table and project activity tables belong to the same table space, that you load them both in the same job, and that some input record for the project table has an invalid department number. Then, that record fails to be loaded and does not appear in the loaded table; the summary report identifies it as causing a primary error.

But the project table has a primary key, the project number. In this case, the record rejected by LOAD defines a project number, and any record in the project activity table that refers to the rejected number is also rejected. The summary report identifies those as causing secondary errors. If you use a discard data set, both types of error records are copied to it.

The deletion of invalid records does not cascade to other dependent tables already in place. Suppose now that the project and project activity tables exist in separate table spaces, and that they are both currently populated and possess referential integrity. Further suppose that the data in the project table is now to be replaced (using LOAD REPLACE) and that the replacement data for some department was inadvertently not supplied in the input data. Records referencing that department number might already exist in the project activity table. LOAD, therefore, automatically places the table space containing the project activity table (and all table spaces containing dependent tables of any table being replaced) into CHECK pending status.

The CHECK pending status indicates that the referential integrity of the table space is in doubt; it might contain records that violate a referential constraint. There are severe restrictions on the use of a table space in CHECK pending status; typically, you run the CHECK DATA utility to reset this status.

ENFORCE NO

If the ENFORCE NO option is used, the LOAD utility will not enforce referential constraints. But the result is that the loaded table space might violate the constraints. The following is an example of running without enforcing constraints:

LOAD DATA INDDN DB2LDS ENFORCE NO
INTO TABLE DB2USER1.TEST_TAKEN

Note

LOAD places the loaded table space in CHECK (CHKP) pending status. If you use REPLACE, all table spaces containing any dependent tables of the tables that were loaded are also placed in CHECK pending status.

Correcting Violations

The referential integrity checking in LOAD can delete only incorrect dependent rows which were input to LOAD. Deletion is not always the best strategy for correcting referential integrity violations.

For example, the violations may occur because parent rows do not exist. In this case, it is better to correct the parent table, not to delete the dependent rows. Therefore, and in this case, ENFORCE NO would be more appropriate than ENFORCE CONSTRAINTS. After the parent table is corrected, CHECK DATA can be used to reset the CHECK pending status.

LOAD ENFORCE CONSTRAINTS is not equivalent to CHECK DATA. LOAD ENFORCE CONSTRAINTS deletes any rows causing referential constraint violations. CHECK DATA detects violations and optionally deletes such rows. CHECK DATA checks a complete referential structure, although LOAD checks only the rows being loaded. There is more information on the CHECK DATA utility later in this chapter.

Note

When loading referential structures with ENFORCE CONSTRAINTS, parent tables should be loaded before dependent tables.

Note

Be aware that running the LOAD utility on a table space does not activate triggers defined on tables in the table space.

Loading ROWID Columns

Columns defined as ROWID can be designated as input fields using the LOAD field specification syntax diagram. LOAD PART is not allowed if the ROWID column is part of the partitioning key. Columns defined as ROWID can be designated as GENERATED BY DEFAULT or GENERATED ALWAYS. With GENERATED ALWAYS, DB2 always generates a Row ID.

Columns defined as ROWID GENERATED BY DEFAULT can be set by the LOAD utility from input data. The input field must be specified as a ROWID. No conversions are allowed. The input data for a ROWID column must be a unique, valid value for a row ID. If the value of the row is not unique, a duplicate key violation will occur. If such an error occurs, the load will fail.

In this case, the duplicate values will need to be discarded and the load will need to be retried with a new unique value, or allow DB2 to generate the value of the row ID.

The DEFAULTIF attribute can be used with the ROWID keyword. If the condition is met, the column will be loaded with a value generated by DB2. The NULLIF attribute cannot be used with the ROWID keyword, because row ID columns cannot be null.

Note

A ROWID column that is defined as GENERATED ALWAYS cannot be included in the field specification list, because DB2 generates the ROWID value automatically.

Loading a LOB Column

LOB columns are treated by the LOAD utility as varying-length data. The length descriptor for a LOB column must be 4 bytes. When the input record is greater than 32KB, you might have to load the LOB data separately.

Free Space

When loading into a non-segmented table space, LOAD leaves one free page after reaching the FREEPAGE limit, regardless of whether the records loaded belong to the same or different tables.

When loading into a segmented table space, LOAD leaves free pages, and free space on each page, in accordance with the current values of the FREEPAGE and PCTFREE parameters. Those values can be set by the CREATE TABLESPACE, ALTER TABLESPACE, CREATE INDEX, or ALTER INDEX statements. LOAD leaves one free page after reaching the FREEPAGE limit for each table in the table space.

Inline Copies

Another feature of the LOAD utility is the ability to perform an inline copy. An inline copy is, for the most part, equivalent to a full image copy taken with SHRLEVEL REFERENCE. The only difference between an inline copy and a regular full image copy is that data pages and space map pages may be out of sequence or repeated, and if a compression dictionary was built during the LOAD, the pages will be duplicated. These differences, however, should be negligible in terms of the amount of space required for the copy data set, and the copy is still valid for recovery.

The inline copy increases the availability of your data because after the data has been loaded and the inline copy taken, the table space is not left in a copy-pending (COPY) status (even if you specify LOG NO) and the data is ready to be accessed. You can take multiple image copies with this feature as well (with a maximum of two primary and two secondary image copies allowed).

Inline Statistics

Statistics can be collected during the LOAD utility (with RESUME NO or REPLACE). This would eliminate the need for executing the RUNSTATS utility after the LOAD. This is done by using the STATISTICS keyword to gather catalog statistics for the table space.

LOAD STATISTICS
INTO TABLE DB2USER1.TEST_TAKEN

For more information on RUNSTATS, refer to Data Statistics later in this chapter.

SORTKEYS

Using the SORTKEYS option for the LOAD utility allows for elimination of the SYSUT1 and SORTOUT temporary data sets for sorting the key/RID pairs for foreign keys or for indexes provided that have been supplied a nonzero estimate for the key/rid pairs to be sorted. In this case, DFSORT exits are used to pass the unsorted key/RID pairs for the foreign keys or indexes to DFSORT and to accept the sorted key/RID pairs again rather than making the unsorted key/RID pairs available via the SYSUT1 data set and receiving the sorted key/RID pairs via the SORTOUT data set.

When using the SORTKEYS, the sorting of the key/RID pairs is done in parallel with the loading of the rows, the building of the indexes, and (if inline statistics are specified) the collection of statistics. The parallelism is achieved by groups of OS/390 subtasks. A task group can contain two or three tasks depending on whether or not inline statistics are to be established. These subtasks are referred to as the SORT task, BUILD task, and STATISTICS task, respectively.

DB2 can allocate multiple task groups of two or three subtasks each to sort the key/RID pairs and to establish the indexes in parallel with each other. The degree of parallelism used depends on the specifications and the DD statements provided for sort work data sets, the number of indexes to build, and the amount of storage available.

If the sort work data sets are dynamically allocated by DFSORT (i.e., SORTDEVT has been specified), the degree of parallelism is determined by them. If manual allocation of the sort work data sets (and sort message data sets) is chosen, the degree of parallelism is determined by the data sets that have been allocated.

PREFORMAT

When DB2's preformatting delays impact the performance or execution time consistency of high INSERT applications and the table size can be predicted for a business processing cycle, LOAD PREFORMAT or REORG PREFORMAT might be a technique to consider. This technique will be of value only if DB2's preformatting causes a measurable delay with the INSERT processing or causes inconsistent elapsed times for INSERT applications. It is recommended that a performance assessment be conducted before and after LOAD or REORG PREFORMAT is used to quantify its value in your environment.

PREFORMAT is a technique used to avoid DB2 having to preformat new pages in a table space during execution time. This might eliminate execution time delays but adds the preformatting cost as setup prior to the application's execution. LOAD or REORG PREFORMAT primes a new table space and prepares it for INSERT processing. When the preformatted space is utilized and DB2 has to extend the table space, normal data set extending and preformatting occurs.

Preformatting for INSERT processing may be desirable for high INSERT tables that will receive a predictable amount of data allowing all the required space to be allocated prior to the application's execution. This would be the case for a table that acts as a repository for work items coming into a system that are subsequently used to feed a backend task that processes the work items.

Preformatting of a table space containing a table used for query processing may cause a table space scan to read additional empty pages, extending the elapsed time for these queries. LOAD or REORG PREFORMAT is not recommended for tables that have a high ratio of reads to inserts if the reads result in table space scans.

Note

The PREFORMAT option can also be used on the REORG utility.

Parallel Index Builds

LOAD builds all the indexes defined for any table being loaded. At the same time, it checks for duplicate values of any unique index key. If there are any duplicate values, one of the corresponding rows is loaded. Error messages identify the input records that produce duplicates; optionally, the records are copied to a discard data set. At the end of the job, a summary report lists all errors found. For unique indexes, any two null values are taken to be equal, unless the index was created with the UNIQUE WHERE NOT NULL clause. In that case, if the key is a single column, it can contain any number of null values, though its other values must be unique.

Neither the loaded table nor its indexes contains any of the records that might have produced an error. Using the error messages, you can identify faulty input records, correct them, and load them again. If you use a discard data set, you can correct the records there and add them to the table with LOAD RESUME.

Use parallel index build to reduce the elapsed time for a LOAD job by sorting the index keys and rebuilding multiple indexes in parallel, rather than sequentially. Optimally, a pair of subtasks processes each index; one subtask sorts extracted keys while the other subtask builds the index. LOAD begins building each index as soon as the corresponding sort emits its first sorted record. LOAD uses parallel index build if all of the following conditions are true: When there is more than one index to be built, the LOAD utility specifies the SORTKEYS keyword (along with a non-zero estimate of the number of keys) in the utility statement. There is also a choice to either allow the utility to dynamically allocate the data sets needed by SORT, or provide the necessary data sets in the job.

LOAD Parallelism for Partitioned Table Spaces

LOAD parallelism helps when dealing with short windows in which to load a lot of data. This allows us to LOAD multiple partitions of a partitioned table space in parallel in the same job. Large loads have been a problem in the past when NPIs (non-partitioned indexes) were involved, often causing the DBA to have to drop and re-create the NPIs in order to get large data loads done. Multiple tasks in a single job can be used to load the partitions in parallel, and a single job can be submitted with several input files to be loaded in parallel. The performance is fast and the contention on the NPI is eliminated. The number of parallel load tasks will be determined by the number of CPUs and virtual storage available, as well as the number of threads available. The following is an example of the necessary syntax:

LOAD  INTO TABLE tab1 PART 1 INDDN infile1
      INTO TABLE tab1 PART 2 INDDN infile2

Unloading Data

DSNTIAUL

You can use the DB2 unload program DSNTIAUL to unload data. It can also create control statements for use with the DB2 LOAD utility. This program can be found in the DSNSAMP data set.

UNLOAD Utility

The UNLOAD utility unloads data from one or more source objects to one or more BSAM sequential data sets in external formats. The source of the data can be:

  • DB2 table spaces

  • DB2 image copy data sets (full or incremental)

    • Image copies that were taken with the CONCURRENT keyword are not usable for the UNLOAD utility.

  • Copies taken with DSN1COPY

This utility will allow for an unload of rows from an entire table space, specific partitions, or individual tables to unload. It can also just unload specific columns of a table by using a field specification list. If a table space is partitioned, all of the selected partitions can be unloaded into a single data set, or you can unload each partition in parallel into physically distinct data sets. Multiple tables from the same table space can be unloaded in the same job; however, the unload does not have the ability to perform joins on tables. If the tables for unload are not specified, then all the tables in the table space are unloaded.

The UNLOAD utility also has other capabilities such the ability to change data types of selected columns, change the order of the columns, use sampling to get a cross-section of data, specify conditions for row selection, and limit the number of rows to be unloaded.

The output records written by the UNLOAD utility are compatible as input to the LOAD utility. This provides the ability to reload the original table or different tables with the data from the UNLOAD.

Output from UNLOAD will consist of one or more of the following:

  • An unloaded table space or partition

  • A discard file of rejected records

There are three phases to the UNLOAD utility:

  • UTILINIT— Initialization and setup

  • UNLOAD— Unloading records to sequential data sets

  • UTILTERM— Cleanup

During the UNLOAD phase, one pass through the input data set is made. If UNLOAD is processing a table space or partition, DB2 takes internal commits to provide commit points at which to restart in case the operation should halt in this phase.

Below is an example of using the UNLOAD utility:

UNLOAD TABLESPACE DB2CERT.CERTTS
FROM TABLE DB2USER1.TEST_TAKEN
WHEN (CID =300 AND SCORE > 90)
Using REORG to Remove Data

There is also an option in the REORG utility to be able to unload or discard data during the REORG utility execution. The UNLOAD EXTERNAL option will unload selected data and place it into a data set that can then be loaded into a table. The DISCARD option will allow for the selected removal of rows during the REORG utility execution. The next section has more information on the REORG utility and these options.


062 - Data Maintenance

Chapter 7. Maintaining Data

Data Maintenance

The physical distribution of the data stored in tables has a significant effect on the performance of applications using those tables. The way the data is stored in a table is affected by the update, insert, and delete operations on the table. For example, a delete operation may leave empty pages of data that may not be reused later. Also, updates to variable-length columns may result in the new column value's not fitting in the same data page. This can cause the row to be moved to a different page and so produce internal gaps or unused space in the table. As a consequence, DB2 may have to read more physical pages to retrieve the information required by the application.

These scenarios are almost unavoidable. However, as the database administrator, you can use the data maintenance utilities provided in DB2 to optimize the physical distribution of the data stored in your tables.

There are two related utilities or commands that can help you organize the data in your tables. These are:

  • REORG

  • RUNSTATS

Analyzing Data's Physical Organization

It has been discussed that certain SQL operations may produce internal gaps in tables. So the question you may ask is, how can you determine the physical organization of your tables or indexes? How can you know how much space is currently being used and how much is free?

With the information collected from the system catalog tables, you can generally determine the state of any table space or index space.

It is recommended that you establish a data maintenance policy to ensure that the data in your table is stored as efficiently as possible. If you don't, you may discover that your applications start to experience degradation in performance. This may be caused by the poor physical organization of your data, so before this happens, do preventive maintenance on your tables.

REORG Utility

A REORG utility is used to reorganize data in table spaces and in indexes.

Reorganizing Table Spaces

The following occurs during a reorganization of a table space:

  • The data in the table space and the corresponding indexes defined on the table in the table space are also reorganized.

  • The space containing data from dropped tables is reclaimed (if it was not reclaimed before).

  • Free space is reserved based on current settings.

  • New allocation units become effective.

  • Segments are realigned and the rows of the tables are rearranged in clustering-index sequence (except for simple table spaces with multiple tables).

  • Overflow pointers are eliminated.

  • The corresponding indexes are rebuilt from scratch.

The following are some of the important phases in the REORG utility:

  • UTILINIT

    • This phase is the initialization phase; it performs the setup operations for the utility.

  • UNLOAD

    • This phase unloads the rows of the table space and writes to the unload data set which is specified in the UNLDDN parameter in the utility. SYSREC is the default DD name for the unload data set. The unload data set is also referred to as the SYSREC data set. The type of table space, the number of tables in the table space, and whether or not the tables have explicit clustering indexes, will have an effect the sequence in which the rows are unloaded.

  • RELOAD

    • The rows are reloaded from the unload data set into the table space during this phase. The sequence in which the rows were unloaded will be the sequence in which they are contained on the unload data set and subsequently the sequence in which they will be reloaded. All rows for a table were unloaded together for segmented table spaces and are then restored together, resulting in the segments for the table being contiguous. Free space is reserved in the pages and free pages are provided for future insertions during the reloading. This is provided as defined by the active PCTFREE and FREEPAGE values for the table space. As the rows are reloaded into the table space, the keys for the indexes of the table of the table space are extracted and, together with the new RID for the rows, are stored onto the first work data set specified via the WORKDDN parameter. The default DD name for the data set is SYSUT1 (also referred to as the SYSUT1 data set). Since the key/RID pairs for the indexes are extracted as the rows are reloaded, they are in an arbitrary sequence with regard to the key values. Also the key/RID pairs for the various indexes are intermixed.

  • SORT

    • In this phase, the key/RID pairs for the various indexes are sorted, using the DFSORT by index, key, and RID. At the end of this phase, the sorted key/RID pairs are contained on the second work data set (also referred to as the SORTOUT data set) specified in the WORKDDN parameter. The default name for this data set is SORTOUT.

    Note

    DFSORT is mentioned many times in regards to performing sorts in the REORG utility. It can be used for the sorting, or other equivalent sort utilities can also be used.

  • BUILD

    • In this phase, the output of the SORT phase (the sorted key/RID pairs for the different indexes) is used to build the indexes for the tables in the table space.

  • SORTBLD

    • Same as with LOAD if parallel index build is used.

  • UTILTERM

    • In the final phase, the UTILTERM phase performs the cleanup operations, such as releasing virtual storage.

The phases in the REORG utility are performed sequentially. Exploiting many of the other features of the REORG utility will cause many of the phases to be performed in parallel. In the next sections, we look at many of the parameters.

Segmented

If the target table space is segmented, REORG unloads and reloads by table. If an explicit clustering index exists on a table in a segmented table space, that table is unloaded in clustering sequence. If NO explicit clustering index exists, the table is unloaded in physical row and segment order.

For segmented table spaces, REORG does NOT normally have to reclaim space from dropped tables. Space freed by dropping tables in a segmented table space is immediately available if the table space can be accessed when DROP TABLE is executed. If the table space cannot be accessed when DROP TABLE is executed, then REORG reclaims the space for dropped tables. After the execution of the REORG, the segments for each table are contiguous.

Partitioned

The entire table space does not need to be reorganized if it is partitioned. You may choose to simply reorganize one partition, if necessary. If you reorganize a single partition or a range of partitions, all indexes of the table space are affected. Depending on how disorganized the non-partitioning indexes are, they may need to be reorganized as well. The following example shows how to reorganize a partition of a table space:

REORG TABLESPACE DB2USER1.CANDIDATE
PART 3
SORTDATA
SORTDEVT SYSDA
LOB

Reorganizing a LOB table space is a separate task from reorganizing the base table space. A LOB table space that was defined with LOG YES or LOG NO will affect logging while reorganizing a LOB column. Specify LOG YES and SHRLEVEL NONE when you reorganize a LOB table space to avoid leaving the LOB table space in COPY pending status after the REORG.

REORG Options

We will take a look at some of the most important parameters of the REORG utility and talk about how some of them help to achieve parallelism in the REORG phases.

SHRLEVEL

The SHRLEVEL determines the level of access allowed during the REORG. The option of NONE states that, during the unload phase, applications can read but not write to the affected area, and during the reload phase, applications have no access to the data. The REFERENCE option allows for read access during the unload and reload phases. SHRLEVEL CHANGE allows for reading and writing during both the unload and reload phases.

If you needed a more restrictive access (i.e., no access at all by applications or by the end user), the table space should be started in UT (utility mode). These are all offline REORGS. The example below shows a REORG utility that does not allow concurrent writes during the reload:

REORG TABLESPACE DB2USER1.CANDIDATE
SHRLEVEL REFERENCE
SORTDATA

First we will look at the SORTDATA option, but before we look at the benefits of this option, we must look at how the REORG utility works without it.

Several factors come into play when determining the sequence in which the rows of the table space to be reorganized are unloaded. Some of these factors include: the type of table space, the number of tables in the table space, and whether or not the tables have explicit clustering indexes. We will take a look at the differences.

  • Segmented table spaces

    • For a segmented table space, its tables are unloaded one after the other. If a table has an explicit clustering index, meaning that a clustering index was defined with the CLUSTER keyword in the CREATE INDEX DDL, the clustering index will be used to unload the rows of the table in the sequence of the clustering index that was defined. It will then store them in the unload data set. If there is no explicit clustering index defined, then the data is unloaded and sorted in the physical sequence—segment by segment, page by page, then row by row.

  • Partitioned table spaces

    • For these table spaces the rows of the table space, or the individual partitions to be reorganized, are unloaded and stored into the unload data set. This is done using the partitioning index in the sequence defined by the index.

  • Simple table space

    • For simple table spaces, the unloading will depend on the number of tables in the table space. If there are multiple tables in the table space they are unloaded in physical sequence. The rows in the unload data set will have the same physical order as the table space and the rows of the different tables are intermixed. If there is more than one table with an explicit clustering index, the clustering index will be used to unload the rows in the sequence defined by the index. If there is only one table in the table space and it does not have an explicit clustering index, then the table will be unloaded in physical sequence.

When the SORTDATA option is used on the REORG utility control statement, DB2 always unloads the rows in physical sequence. DFSORT (or an equivalent sort utility) is used to sort the rows in index sequence if there is at least one explicit clustering index of the table space and the table space is not a simple table space with multiple tables. The index itself is not used for the sorting. Using the SORTDATA option will help with the performance of the REORG utility. It operates differently for each different table space type.

  • Segmented table space

    • If there is an explicit clustering index on at least one of the tables of the segmented table space, the tables will be unloaded table by table in a physical sequence and the output will be passed to DFSORT for sorting. The rows are then sorted in the sequence of the explicit or implicit clustering index of the tables.

    Note

    The data will not be sorted if there are no indexes on the tables.

    Note

    In the case of REORGs not using SORTDATA, the rows of tables are also sorted if the tables have an implicit clustering index, but not an explicit one. But there must be least one of the tables in the segmented table space with an explicit clustering index; if not, then the tables are unloaded one by one in physical sequence and the rows are not sorted.

  • Partitioned

    • The rows of the entire table space, or the partitions to be reorganized, are unloaded in physical sequence and passed to DFSORT to be sorted in the sequence defined by the partitioning index. However, the partitioned index is not used for the actual sorting.

  • Simple table spaces

    • If the table space contains multiple tables, the rows are unloaded in physical sequence and not sorted. If the table space contains a single table, the rows are unloaded in physical sequence and passed to DFSORT, provided there is an explicit clustering index defined. If not, the rows will not be sorted.

In most cases, physically unloading the rows and sorting them by means of DFSORT is faster than unloading the rows through a clustering index if the CLUSTERRATIO of the index is less that 95% (or the percentage of non-clustered rows (100*FAROFFPOS/CARDF) is greater than 5%). The lower the CLUSTERRATIO, the higher the performance improvements when using SORTDATA. There is no advantage to SORTDATA if the table space does not have an explicit clustering index or a partitioning index or if it is a simple table space with several tables.

The following is an example of using the SORTDATA keyword on the REORG of a partitioned table space:

REORG TABLESPACE DB2USER1.CANDIDATE
PART 3
SORTDATA
SORTDEVT SYSDA
NOSYSREC

The intermediate storage of the rows in the unload data set SYSREC, after the unloading and sorting of the rows by the UNLOAD phase, can be avoided by specifying the NOSYSREC explicitly or implicitly for the REORG utility. NOSYSREC is assumed for read/write online reorganizations (SHRLEVEL CHANGE) covered later in this chapter.

Note

The use of NOSYSREC requires that SORTDATA be used.

By using the NOSYSREC parameter, the UNLOAD phase will not use the unload data set. Therefore, there is no need to specify the UNLDDN parameter or provide a SYSREC DD statement for the utility job.

Rows that must be sorted are passed to the RELOAD phase by the DFSORT exit after they have been sorted. If the rows do not need to be sorted, they are immediately passed to the RELOAD phase. DFSORT will need sort work data sets if the number of rows to be sorted is large. If NONSYSREC is specified, either explicitly or implicitly, and dynamic allocation of the sort work data set is not necessary, you need to provide DD statements with the DD name DATAWKnn (nn = 01 for the first sort work data set, 02 for the second, etc.).

When using NONSYSREC, the DATAWKnn data sets are used for the sorting of the rows, not the SORTWKnn work data sets. DATAWKnn data sets need not be provided if you are not using NOSYSREC and are also not used if SORTDATA has been specified. In this case, the SORTWKnn data sets are used if the sort work data sets are not allocated dynamically.

The use of NOSYSREC will also do away with the I/O for the unload data set and will not cause additional I/O caused by DFSORT. The fact that different sort work data sets are used does not change the I/O behavior of DFSORT. Therefore, the use of NOSYSREC provides performance improvements.

Note

The REORG utility cannot be restarted if NOSYSREC has been specified or is assumed.

SORTKEYS

The use of the SYSUT1 and SORTOUT data sets can be eliminated by the use of the SORTKEYS parameter. You can specify the parameter explicitly or choose SHRLEVEL CHANGE (read/write online REORG), which implies SORTKEYS and you do not need to specify it.

When using SORTKEYS, either implicitly or explicitly, DFSORT exits will be used to pass the unsorted key/RID pairs for the indexes of the table space to DFSORT and to accept the sorted key/RID pairs again, rather than to make the unsorted key/RID pairs available in the SYSUT1 data set and to receive the sorted key/RID pairs in the SORTOUT data set.

If the SORTKEYS parameter is used, the sorting of the key/RID pairs will be done in parallel with the reloading of the rows. Other activities can also be done in parallel, such as the building of the indexes and inline statistics collection. The parallelism in the utility is accomplished by groups of OS/390 subtasks. These task groups contain two or three subtasks (SORT, BUILD, and STATISTICS), depending on whether or not inline statistics are to be established.

DB2 can allocate multiple task groups, 2 or 3 subtasks each, in order to establish the indexes for the tables of the table space in parallel with each other. The degree of parallelism used will depend on the specifications and the DD statements provided for sort work data sets. If the sort work data sets are dynamically allocated (SORTDEVT specified), the degree of parallelism will be determined by DB2. If you manually allocate the sort work data set (and sort message data sets), the degree of parallelism will be determined by the data sets that you have allocated.

Below is an example of executing the REORG utility using SORTKEYS:

REORG TABLESPACE DB2CERT.CERTTS LOG NO SORTDATA SORTKEYS
Parallel Index Build

If SORTKEYS is specified, many of the operations are performed in parallel. These can be multiple sort and index build operations, or multiple SORT and BUILD phases performed in parallel by the subtasks of the allocated task groups. When these operations are done in parallel, this is called the SORTBLD phase. This phase encompasses the sort and build activities for all indexes involved in the reorganization.

The SORTBLD phase, which is jointly performed by the allocated task groups, is partially overlapped with the RELOAD phase. Every index is constructed by a predefined task group. The task group can process multiple indexes. The proper SORT tasks for the indexes pass the key/RID pairs to DFSORT for sorting as the keys for the indexes are extracted from the rows and the new RIDS for the rows are determined. This allows the sorting of the key/RID pairs to be performed in parallel with the RELOAD phase. If multiple task groups are used, the key/RID pairs for the various indexes are then sorted in parallel.

When DFSORT emits the sorted key/RID pairs, the BUILD tasks of the index will begin constructing the index. The building of the indexes will be partially overlapped with the reloading of the table space and the sorting of the key/RID pairs (even of those for the same index). The various indexes are constructed in parallel if multiple task groups are used.

The construction of an index finishes only after the table space has completely been reloaded, despite the fact that the building of the indexes is done in parallel with the reloading of the table space. The task groups have three subtasks if inline statistics have been requested. This third STATISTICS subtask collects the requested statistics as the rows are reloaded and the indexes are built.

Note

Each SORT task will require a sort work data set and a message data set. The message data set can be common to all task groups assigned to the SYSOUT.

Logging

The LOG option specifies whether records are logged during the reload phase of REORG. If the records are not logged, the table space is recoverable only after an image copy has been taken. If you specify COPYDDN, RECOVERYDDN, SHRLEVEL REFERENCE, or SHRLEVEL CHANGE, an image copy is taken during REORG execution.

With LOG YES, the log records during the reload phase. This option is not allowed for any table space in DSNDB01 or DSNDB06, or if the SHRLEVEL REFERENCE or CHANGE options are used. If you specify SHRLEVEL NONE (explicitly or by default), the default is LOG YES. However if you specify LOG NO with SHRLEVEL NONE, then previous image copies may not be candidates for use during certain recoveries.

With LOG NO, DB2 does not log records. This puts the table space in COPY pending status if either of these conditions is true:

  • REORG is executed at the local site, and neither COPYDDN, SHRLEVEL REFERENCE, nor SHRLEVEL CHANGE is specified.

  • REORG is executed at the remote site, and RECOVERYDDN is not specified.

Online Reorganizations (OLR)

Most reorganizations limit or restrict access to the table space that is being reorganized. This is especially true if using SHRLEVEL NONE. During the UNLOAD phase, write access is prohibited, and during the RELOAD phase, any type of access is not allowed. With today's increasing demand for 24-hour service, it becomes less and less acceptable that access is blocked for longer periods of time or at all. To deal with this issue, DB2 supports online REORG, a REORG where online processes can access the table space for most of the time. Online REORG comes in two flavors.

If you specify SHRLEVEL REFERENCE, you will have read access to the table space and access via the associated indexes for most of the time. There is only a small window within the SWITCH phase during which no access is allowed. SHRLEVEL REFERENCE reorganizations are also referred to as read-only online REORGS.

If you specify SHRLEVEL CHANGE, you will have read/write access to the table space and access via the indexes associated with the table space for most of the time. There is a small period at the end of the LOG phase when you will have only read access and a small period within the SWITCH phase during which no access is allowed at all. SHRLEVEL CHANGE REORGS are referred to as read/write online REORGS. The LOG and SWITCH phases exist only for online reorganizations.

Read-Only OLR

Online REORG uses shadow data sets for the table space and the indexes being reorganized. Collectively, the shadows data sets for a table space are referred to as corresponding shadows table space. Similarly, the shadow data sets for an index are referred to as corresponding shadow index or shadow index space. They never exist in the catalog.

During an online reorganization, the rows of the original table space, i.e., of the space being reorganized, are unloaded (UNLOAD phase) and then reloaded into the shadow table space (RELOAD phase).

The reorganized indexes for the table space are not constructed in the original index spaces, but in the shadow index spaces (SORTBLD phase or SORT/BUILD phases).

During the SWITCH phase, the shadow data sets for the table/index spaces replace the original data sets and all access is directed to them. As part of the switching process, the original data sets for the table/index spaces are renamed and the shadow data sets receive their former names.

Since the original table space does not change during a read-only online reorganization, these four phases are sufficient.

For DB2 managed table/index spaces, DB2 automatically creates the shadow data sets in the respective storage groups with the respective active space parameters. It also deletes the original data sets. For user-managed table/index spaces, you must define—using Access Method Services—the data sets for the shadow tables/index space yourself. During the UTILITERM phase, the REORG utility completes the switch by renaming the original data sets. They now receive the data set names which the respective shadow data sets had before the reorganization. It is your responsibility to delete these data sets.

During all phases excluding the SWITCH phase, the rows of the table space can be read and the indexes of the table space can be used. However, during the SWITCH phase, no access to the table space or through the indexes is allowed.

Read/Write OLR

Read/write online REORG(SHRLEVEL CHANGE) does not allow you to specify the parameters SORTDATA, NONSYSREC, or SYSKEYS. It always operates as if these parameters were specified. Thus, it uses DFSORT to sort the rows during the UNLOAD phase and does not use the unload data set. This sorts the key/RID pairs for the indexes in parallel with the RELOAD phase and with the building of the indexes in the shadow index spaces, and if the sort work data sets are dynamically allocated or explicitly allocated for multiple task groups, it builds the indexes in parallel with each other.

Basically the algorithm for read/write online REORG is the same as for read-only online REORG. However, the fact that changes are allowed during the UNLOAD, RELOAD, and SORTBLD phases is reflected by an additional phase, the LOG phase. During the switch, the changes performed throughout these phases are applied to the shadow table/index spaces. Even during most of the LOG phase, the user has full read/write access. Only at the end of the LOG phase, the REORG utility must restrict access to read-only to guarantee that the reorganization comes to an end.

During a read-only online REORG, no access is allowed during the SWITCH phase. Also, a read/write online REORG will always create an inline copy for the table space being reorganized. Therefore, there is no need to create a full image copy after the reorganization of the table space.

Mapping Table

During the LOG phase, changes for the original table space on the DB2 log are applied to the shadow table space and consequently to the shadow index. The RIDs for the DB2 records on the DB2 log point to the original table space. In order to map the changes to the shadow table space, the REORG utility uses a mapping table (actually, a unique index over the mapping table). This table and index must be created prior to the REORG utility execution.

Note

The mapping table must reside in a segmented table space that is not the table space being reorganized.

The four columns in the mapping table are shown below in an example DDL:

CREATE TABLE MAP_TABLE
  (TYPE  CHAR(1) NOT NULL,
   SOURCE_RID CHAR(5) NOT NULL,
   TARGET_XRID CHAR(9) NOT NULL,
   LRSN CHAR(6) NOT NULL)
IN DB2CERT.MAPTS;

CREATE UNIQUE INDEX MAPINX1 ON MAP_TABLE
  (SOURCE_RID ASC,
   TYPE,
   TARGET_XRID,
   LRSN);

The SOURCE_RID contains the RIDs for the rows in the original table space, whereas the column and the TARGET_XRID contain the (extended) RIDs for the rows in the shadow table space. The key of the mapping table index has the same four columns as the mapping table, but in a different sequence.

The name of the mapping table must be specified via the MAPPING TABLE parameter for the REORG utility; it can be any name you choose. An example of using the mapping table is shown below.

REORG TABLESPACE DB2CERT.CERTTS
SHRLEVEL CHANGE
...
MAPPING TABLE DB2USER1.MAP_TABLE

DELETE, INSERT, SELECT, and UPDATE authorization will be needed on the mapping table.

Note

There needs to be a mapping table for each table space table being REORGed in parallel because exclusive locks are taken to prevent concurrent access to the mapping tables.

Only the index over the mapping table is filled; therefore, it is sufficient to allocate as little space as possible to the mapping table. For this index you should assign at least 1.1 * number of rows in TS * 27 bytes, assuming that the entire table space is being reorganized. For REORGS of partitions of the table space, the number of rows in the partitions is to be used instead of the number of rows in the table space.

Controlling Log Iterations

During read/write online reorganizations, application programs or end users may change the data in the original table space up to the end of the LOG phase.

In the first step, the LOG phase applies the changes made during the UNLOAD, RELOAD and SORTBLD phases to the shadow table space and the shadow indexes. During most of the LOG phase, read/write access to the original table space is allowed. Because the data in the original table space may have changed during the first iteration, the second iteration must apply the changes made during the first iteration, a third iteration must apply the changes made during the second iteration, and so on. This could potentially cause the LOG phase to never end.

Hopefully, fewer and fewer changes must be applied with each iteration. However, conceivably an endless number of iterations may be necessary. Therefore, the REORG utility must ultimately limit the access during the LOG phase to read, only to come to a final iteration. After this is done, it must then apply only the changes accumulated during the previous iteration before it can enter the SWITCH phase.

Since the switching to read-only access impacts the operating environment, the REORG utility provides the ability to decide and specify for how long a period this can be tolerated. This is done via the MAXRO parameter.

The REORG utility estimates how long the next iteration will take based on the changes for the previous iteration. If its estimate is lower than or equal to the value specified via the MAXRO parameter, the REORG utility switches to RO access or even to no access allowed, depending on what you have requested, and the last iteration takes place. The actual time of the last iteration may be larger than the estimate or the value you have specified. However, it should not be substantially larger.

If DRAIN WRITERs has been specified (either explicitly or by default), the REORG utility will drain all writers (waits until all units of recovery accessing the table space have been committed) and does not allow new units of recovery to begin. In the case of DRAIN ALL, the REORG utility will drain all the users (waits until all readers and writers are off the table space and does not allow any further read or write access). Readers may have locks (CLAIMS) not being released before they commit. Therefore, it is imperative that (even) long-running read-only programs commit from time to time to allow online reorganizations to succeed.

The default for MAXRO is 300 seconds. MAXRO DEFER causes log processing to continue indefinitely until you change MAXRO by means of the ALTER UTILTY command, until the last iteration is forced by a different condition (LONGLOG), or until the reorganization is terminated.

Note

Specifying a small value for MAXRO will generally cause more iterations to take place and may increase the total elapsed time for the reorganization. A huge value for MAXRO will most likely lead to the second iteration's being the last one.

Controlling Long Log Situations

If there is a small value specified for MAXRO, the log processing of the REORG utility may not catch up with the change activities performed for the table space. This is referred to as the long log condition.

The REORG utility raises a long log condition if the number of log records processed by the next iteration will not be sufficiently lower than the number of log records for the previous iteration and the next log iteration will take longer than the specified MAXRO values (the next iteration will not be the last one).

There is an option to specify what should happen in the case of long log situations. After message DSNU3771 has been issued and the time specified through the DELAY parameter has passed, the action of the LONGLOG parameter is performed. The action can be CONTINUE—continue log processing iterations, TERM—terminate the REORG, or DRAIN—drain writers to force last log iteration (wait until all units of recovery involving the table space have been committed and prevent new units of recovery for the table space).

Note

The time on the DELAY parameter and the actions specified in LONGLOG parameter can be changed by using the ALTER command, if necessary.

REORG DEADLINE

The REORG utility also allows you to control when a REORG must complete. Using the DEADLINE parameter, you can specify a deadline at which time the REORG must finish. The deadline can be an absolute time (a timestamp) or a relative time (a labeled duration expression using CURRENT DATA or CURRENT TIMESTAMP).

If the REORG utility estimates that the SWITCH phase will not complete before the specified deadline, it issues the message that the DISPLAY UTILITY command would issue and terminates the reorganization

The DEADLINE can be changed by means of the ALTER UTILITY command if the reorganization appears that it will not finish before its deadline and you do not want the REORG to be terminated.

Note

The deadline must be extended before it is reached.

Inline Statistics During REORG

After reorganization, the old statistics for the reorganized table space, index or partitions are no longer valid. Neither are the old statistics for the indexes associated with a table space being reorganized. New statistics must be established for these objects to ensure that the DB2 optimizer has the proper statistics for access path determination. For more information on access paths refer to Chapter 12, Performance Monitoring and Tuning.

This is done by executing the RUNSTATS utility after the reorganization, but it can also be performed by the REORG utility for all the associated objects. These inline statistics can be requested by means of the STATISTICS clause. The DB2 catalog tables are updated accordingly and/or the statistics are reported in the output listing for the REORG utility. The following is an example of requesting inline statistics during a REORG:

REORG TABLESPCE DB2CERT.CERTTS
SORTDATA STATISTICS PART 5

By allowing REORG to perform inline statistics, there is no need to run the RUNSTATS utility after the REORG. It is faster to run the statistics inline and they are done in parallel by subtasks during the REORG.

Note

Any necessary rebinds of the plans and packages for application programs will still have to occur to make the statistics effective.

The keywords that you can specify as a part of the STATISTICS clause are the same that you can specify for RUNSTATS, and the functions provided are the same. This includes statistics sampling, which will be discussed later in this chapter.

REORG DISCARD and REORG UNLOAD

The REORG utility can also be used to remove other rows no longer needed. The REORG utility allows you to select the rows you want to remove (in addition to those that must be discarded because they no longer belong to any partition) by means of the DISCARD clause. The DISCARD clause may contain one or more FROM TABLE specifications identifying the tables from which rows should be removed.

In the FROM TABLE specification, you name a table of the table space from which rows should be removed and the selection condition for the removal of the rows (WHEN condition). In the WHEN condition, you can combine basic predicates by means of AND and OR to select the rows to be discarded. The predicates must refer to columns of the specified table.

Note

As the consequence of discarding rows, the check pending (CHKP) conditions will be set for the table space containing dependent tables.

REORG also allows you to unload data using the UNLOAD options. There are various forms of UNLOAD, including CONTINUE, PAUSE, ONLY, and EXTERNAL. CONTINUE tells REORG to continue processing after the data is unloaded and ONLY says to stop and terminate after the unload.

PAUSE tells REORG to stop after the UNLOAD phase. One common use for using the REORG UNLOAD PAUSE is to unload the data from the table space and then remove the extents from user-defined data sets using AMS (Access Method Services). The status of the REORG is recorded in SYSUTIL. The utility could then be restarted at the next phase (RESTART(PHASE)).

The UNLOAD EXTERNAL option to allow you to unload data into a data set that will be in a format that is usable by the LOAD utility.

The examples below show DISCARD and UNLOAD EXTERNAL:

REORG TABLESPACE DB2CERT.CERTTS
   UNLOAD EXTERNAL
   FROM TABLE DB2USER1.TEST_TAKEN
   WHEN (CID = 300)

REORG TABLESPACE DB2CERT.CERTTS
   DISCARD
   FROM TABLE DB2USER1.TEST_TAKEN
   WHEN (CID = 300)

Reorganizing Indexes

The reorganization of an index reclaims fragmented space and improves access performance. Similar options can be specified on the REORG INDEX as were specified on the REORG TABLESPACE, such as degree of access to the data during reorganization, collecting inline statistics using the STATISTICS keyword, access during REORG, preformatting of pages, online reorg features, and the REPORTONLY feature.

Sometimes performance may be improved simply be reorganizing the index, not necessarily the table space.

The REORG INDEX statement will reorganize the entire index (all parts, if partitioned). The REORG INDEX PART n reorganizes PART n of the partitioning index.

The execution phases of REORG INDEX are fairly similar to the REORG TABLESPACE. The phases are:

  • UTILINIT

    • This performs initialization and setup.

  • UNLOAD

    • This unloads index space and writes keys to a sequential data set.

  • BUILD

    • This builds indexes and updates index statistics.

  • LOG

    • This phase processes log iteratively (used only if you specify SHRLEVEL CHANGE).

  • SWITCH

    • This switches access to shadow copy of index space or partition (used only if you specify SHRLEVEL REFERENCE or CHANGE).

  • UTILTERM

    • This phase performs any necessary cleanup.

The following is an example of the REORG INDEX utility with inline statistics:

REORG INDEX DB2USER1.CANDIDATE
SHRLEVEL REFERENCE STATISTICS

Triggering Reorganizations

Due to the fact that running the REORG utility can be quite expensive and disruptive, it is desirable to run the REORG utility only when a table space and indexes absolutely require it.

If the statistics are current in the DB2 catalog tables, REORG can determine whether or not it needs to actually perform the reorganization. The REORG utility allows for specification of criteria indicating when reorganization should be performed. This saves the DBA from having to do the analysis on the statistics and determining if a REORG should be scheduled. The following are the conditions that can be evaluated by the REORG utility:

Table Space REORG Determination

  • Non-partitioned table space

    • If the out-of-sequence references for the explicit clustering index of at least one table of the table space exceed the specified percentage (OFFPOSLIMIT) or if the out-of-sequence references for a clustering index lead to unnecessary data page transitions when retrieving rows in the sequence of the clustering index

  • Entire partitioned table space

    • If the out-of-sequence references for at least one partition of the partitioning index exceed the specified percentage (OFFPOSLIMIT)

  • Range of partitions or a single partition of a partitioned table space

    • If the out-of-sequence reference for at least one of the specified partitions of the partitioned index surpasses the specified percents (OFFPOSLIMIT)

  • Non-partitioned table space

    • If the indirect row reference for the table space exceeds the specified percentage (INDREFLIMIT). Indirect references cause an unnecessary access to a second page for a row.

  • Table space partition

    • If the indirect row references for at least one partition exceed the specified percentage (INDREFLIMIT)

  • Range of partitions or a single part of a partitioned table space

    • If the indirect rows references for at least one of the specified partitions surpass the specified percentage (INDREFLIMIT)

Index REORG Determination

  • Index of a non-partitioned table space

    • If the average gap between two logically successive leaf pages times 100 (i.e., the average number of leaf pages physically located between two logically successive leaf pages times 100) exceeds the specified value (LEAFDISTLIMIT). The specified value is compared with the LEAFDIST value for the index in SYSINDEXPART

  • Partitioning Index

    • If the specified value (LEAFDISTLIMIT) exceeds the LEAFDIST value for at least one partition of the index SYSINDEXPART

  • Partition of a Partitioning Index

    • If the specified values (LEAFDISTLIMIT) exceed the LEAFDIST value for the specified index partition in SYSINDEXPART

Note

If you specify REPORT ONLY on a REORG of a table space, partition, index, or index part, it does not take place regardless of whether or not the limit is surpassed. You just get a report of whether or not you needed a REORG.

The following is an example of how to specify the OFFPOSLIMIT and INDREFLIMIT conditional REORG triggers:

REORG TABLESPACE DB2USER1.TEST_TAKEN
   SORTDATA NOSYSREC SORTKEYS
   COPYDDN SYSCOPY1
   OFFPOSLIMIT
   INDREFLIMIT
   STATISTICS TABLE(ALL)INDEX(ALL)

These options of OFFPOSLIMIT, INDREFLIMIT, and LEAFDIST-LIMIT can also be described in the SQL below:

SELECT CARDF
   , (NEAROFFPOSF + FAROFFPOSF) * 100 / CARDF
FROM SYSIBM.SYSINDEXPART
WHERE CARDF > 0
AND (NEAROFFPOSF + FAROFFPOSF) * 100
    / CARDF > :offposlimit

SELECT CARD
   , (NEARINDREF + FARINDREF) * 100 / CARD
FROM SYSIBM.SYSTABLEPART
WHERE CARD > 0
AND (NEARINDREF + FARINDREF * 100
   / CARD > :indreflimit

SELECT LEAFDIST
FROM SYSIBM.SYSINDEXPART
WHERE LEAFDIST > :leafdistlimit
DBA Analysis for REORG Determination

The DBA can choose not to use the REORG triggers and may choose to analyze the statistics manually to determine if a REORG is necessary.

For any table space, a REORG is needed if any of the following apply (using statistics from the clustering index):

  • Any data set behind the table space has multiple extents

  • CLUSTERRATIO < 90 percent always. However:

    • Very small tables should be kept at 100% clustered (cheap and easy)

    • Medium tables should be REORGed when the CLUSTERRATIO falls below 98%

    • Large tables should be REORGed when the CLUSTERRATIO falls below 95%

    • From 90% to 95% can cause very poor performance

  • (NEARINDREF + FARINDREF) / CARD > 10 percent

  • FAROFFPOS / CARD > 5 percent

  • NEAROFFPOS / CARD > 10 percent

  • DBD growth after successive drops/re-creates in a table space

To check whether your index space needs to be reorganized, review the LEAFDIST column in the SYSINDEXPART catalog table. Large numbers in the LEAFDIST column indicate that there are several pages between successive leaf pages, and using the index will result in additional overhead. In this situation, DB2 may turn off prefetch usage, as well. Reorganizing the index space will solve these problems.

Reorganizing the Catalog and Directory

You can also determine when to reorganize the DB2 catalog table space and index spaces by using the same techniques used for determining when to reorganize application table spaces and index spaces. First, you will want to ensure that statistics are kept current by using RUNSTATS based upon the frequency of changes in the catalog, so that decisions for reorganizations are based on current numbers. A reorganization will also be necessary if the objects are in extents, or if unused space needs to be reclaimed.

Note

Every table space in the DSNDB06 database is eligible for reorganization, but DSNDB06.SYSPLAN cannot be reorganized with SHRLEVEL CHANGE.

DB2 directory reorganizations are also important, because the directory contains critical information regarding internal DB2 control and structures. These are important to DB2 processing because they affect application plans and package execution, utility execution, and database access. If we allow these to become disorganized, it can affect transaction and utility performance.

There are relationships between the DB2 catalog and some of the DB2 directory tables. For instance, you should reorganize the directory table DBD01 when you reorganize catalog table SYSDATABASE. Directory tables SCT01 and SCT02 should be reorganized with SYSPLAN and SYSPACKAGE, respectively.

Rebalancing Partitions

The limit keys for the partitions of a partitioned table space can be changed if the partitions become unbalanced. This is done via the ALTER INDEX SQL statement with the PART clause and the VALUES parameter.

This changes only the definition in the DB2 catalog, but does not actually move any rows in the partitioned table space or any index entries in the partitioned index. DB2 will also set the REORP status for those partitions due to the fact that they may no longer contain the correct rows. The affected partitions (all partitions in the range affected by the ALTER) are no longer accessible.

Note

The index partitions will not be placed in REORP status, but they cannot be used for access to the partitions in REORP status. NPIs also cannot be used for access to the affected data partitions, but if index-only access is needed, they can be used.

In order to redistribute the rows for the affected partitions, you must reorganize all of these partitions at the same time by specifying a range in the PART n:m parameter of the REORG utility control statement. An example is shown below to REORG parts 5 through 10:

REORG TABLESPACE DB2CERT.CERTTS PART 5:10

Note

You cannot specify SHRLEVEL REFERENCE or SHRLEVEL CHANGE if partitions are in REORP. The REORG utility requires that you specify SHRLEVEL NONE (explicitly or default) if at least one of the target partitions is in REORP.

If the limit key for the last partition in key sequence for a large table space (created with LARGE or DSSIZE) is changed, there must be a discard data set if the key range had been reduced regardless of whether or not some of its former rows no longer belong to the key range. The rows no longer belonging to any key range are then placed in the discard data set. This can be specified on the DD name for the discard data set via the DISCARDDN parameter or the default DD name of SYSDISC can be used.

Note

The REORG utility always establishes an inline copy for at least one of the partitions being reorganized that is in REORP status. A single inline copy is established which includes all partitions of the specified range. The DD name for the local primary copy data set must be provided via the COPYDDN parameter or a DD statement with the DD name SYSCOPY must be provided.


063 - Data Statistics

Chapter 7. Maintaining Data

Data Statistics

The system catalog tables contain information about columns, tables, and indexes. They contain such information as the number of rows in a table, the use of space by a table or index, and the number of different values in a column.

However, this information is not kept current. It has to be generated by a utility called RUNSTATS. The statistics collected by the RUNSTATS utility can be used in two ways: to display the physical organization of the data and to provide information that the DB2 optimizer needs to select the best access path for executing SQL statements.

To have efficient access paths to data, current statistics must exist that reflect the actual state of your tables, columns, and indexes. Whenever a dynamic SQL statement is issued, the DB2 optimizer reads the system catalog tables to review the available indexes, the size of each table, the characteristics of a column, and other information to select the best access path for executing the query. If the statistics do not reflect the current state of the tables, the DB2 optimizer will not have the correct information to make the best choice in selecting an access path to execute your query. This becomes more crucial as the complexity of the SQL statements increases. When only one table is accessed without indexes, there are fewer choices available to the optimizer. However, when the SQL statement involves several tables, each one with one or more indexes, the number of choices available to the optimizer increases dramatically.

Choosing the correct access path can reduce the response time considerably. Depending on the size of the tables, the indexes available, and other considerations, the selected access path can affect the response time, which varies from minutes to hours. You may also want to consider the physical and logical design of your database.

The next step in improving performance involves the use of the RUNSTATS utility. It is recommended that you execute RUNSTATS on a frequent basis on tables that have a large number of updates, inserts, or deletes. For tables with a great deal of insert or delete activity, you may decide to run statistics after a fixed period of time or after the insert or delete activity.

An important feature of DB2 is that it allows you to reorganize and use the RUNSTATS utility on the system catalog tables. This feature of DB2 can improve the access plans generated when querying the system catalog tables. DB2 may access these tables when you issue an SQL statement, even though you are referencing only user tables. Therefore, it is very important to have current statistics on the system catalog tables.

Note

It is recommended to use the RUNSTATS utility after a REORG of a table, or specify the usage of inline statistics during the REORG.

RUNSTATS Utility

RUNSTATS is a utility that is critical to a properly tuned DB2 environment. It gathers statistics about DB2 objects—table spaces and indexes. RUNSTATS can also be imbedded in other utilities such as REORG and LOAD, giving us the ability to collect statistics during the execution of a given utility (discussed earlier in this chapter).

The RUNSTATS utility can be executed on a table space and its indexes, or for each object independently, or even for a specific column.

RUNSTATS should initially be executed on all columns after the data is loaded and reorganized. In addition, there are some general guidelines for RUNSTATS execution. In general, RUNSTATS should be executed whenever a table space or index has been reorganized and the statistics have changed significantly, after any heavy update, insert, or delete processing has occurred, before any binding/rebinding of packages of plans, and before any performance tuning that would require monitoring of the statistics in the catalog.

Reporting and Performing Catalog Updates

There is an option to allow RUNSTATS to report on the statistics gathered by using the REPORT NO|YES parameter. We can also control what statistics are collected and whether or not RUNSTATS updates the DB2 Catalog through the following UPDATE options:

  • ALL

    • Will include both space and access path statistics

  • NONE

    • Will not do any updating of the DB2 catalog

  • SPACE

    • Will update only statistics related to space management (i.e., freespace)

  • ACCESSPATH

    • Will update only statistics used by the optimizer

Below is an example of executing the RUNSTATS utility and updating the catalog with only the statistics that are collected for access-path selection. The REPORT option is also used which will route the collected statistics to SYSPRINT.

RUNSTATS TABLESPACE DB2CERT.CERTTS
REPORT YES
UPDATE ACCESSPATH
Access During RUNSTATS Execution

A RUNSTATS utility using SHRLEVEL CHANGE will run without any locking or interference to other processes. This allows you to execute RUNSTATS as often as necessary.

Sampling

The RUNSTATS sampling feature allows you to choose the percent of non-indexed column statistics gathered and will help RUNSTATS to execute faster. The sampling technique can affect the optimizer's choice in access path selection because the sampling must be representative of the data; in the absence of true representation, it would assume a linear distribution of data, which would affect the filter factor and costing done by the optimizer. Below is an example of using sampling of 25 percent:

RUNSTATS TABLESPACE DB2CERT.CERTTS
   TABLE(ALL)SAMPLE 25
   INDEX(ALL)
   SHRLEVEL CHANGE
Key Correlation Statistics

Key correlation statistics provide DB2 the ability to gather statistics where one column's value is related to the value of another column. Without these statistics, only the FIRSTKEYCARD and FULLKEYCARD columns are populated with limited information, and the correlation is on columns for FULLKEYCARD only. Without key correlation statistics, no second or third key cardinality, and multi-key cardinality is considered. This often leads to inaccurate estimation of filter factors and join size, join sequencing, and join methods, which can result in inefficient access-path selection.

Key correlation statistics are collected by RUNSTATS with minimal additional overhead, and can provide CPU and elapsed-time reductions through improved cost and resources estimations. These key correlation statistics play a major role in access-path selection by providing the optimizer columns with information on multicolumn cardinalities and multicolumn frequent values.

This feature gives us the ability to specify the number of columns to collect statistics on (NUMCOLS) and to specify the number of values (COUNT). These keywords are used in the RUNSTATS utility. The KEYCARD parameter indicates that cardinalities for each column, concatenated with all previous key columns, are to be collected.

Using this RUNSTATS feature gives us the option to build the frequency values for critical concatenated key columns, such as the first and second columns, or maybe the first, second, and third columns.

These values are stored in the catalog table and columns as follows:

  • SYSCOLDIST and SYSCOLDISTSTATS

    • TYPE— CHAR(1) NOT NULL DEFAULT 'F'

      • Type of statistic (cardinality or frequent value)

    • CARDF— FLOAT NOT NULL DEFAULT –1

      • Number of distinct values for column group

    • COLGROUPCOLNO— VARCHAR(254) NOT NULL W/DEFAULT

      • Identifies the set of columns

    • NUMCOLUMNS— SMALLINT NOT NULL DEFAULT 1

      • The number of columns in the group

Note

The COLCARDF column in SYSCOLUMNS contains the number of distinct values for a single column.

RUNSTATS on the DB2 Catalog

The RUNSTATS utility needs to be executed on the DB2 catalog, depending on the amount of DDL (Data Definition Language), DML (Data Manipulation Language), and other activities which are occurring that will insert and delete rows in DB2 catalog tables. DB2 will not be able to appropriately optimize queries against the catalog without having current statistics on the table spaces and index spaces. The same principle applies here as with DB2 user-defined objects; a current view of all DB2 objects is also needed to determine the need for and frequency of reorganization of DB2 catalog table spaces and index spaces.

Inline Statistics

In order to avoid having to run a separate RUNSTATS job, inline statistics can be collected when running the utility REORG, LOAD, or REBUILD INDEX. Establishing inline statistics is faster than running RUNSTATS after REORG, LOAD, or REBUILD INDEX because the inline statistics are established as a by-product by separate subtasks of these utilities. There is the option to request a statistics report and/or update the DB2 catalog for the objects involved. For REORG, you can request the statistics for the table space being reorganized and/or the indexes associated with the tables of the table space or for the indexes being reorged.

Note

For the LOAD utility, you can request only inline stats for REPLACE or RESUME and for REBUILD INDEX only for indexes being built.

The inline statistics can be requested using the STATISTICS clause in the REORG, LOAD, or REBUILD utilities. The same parameters as the RUNSTATS utility can be utilized for inline statistics.

Inline statistics cannot be collected for the following:

  • DB2 Catalog or Directory tables with links

  • Non-partitioning indexes if there is only a load or reorganization on individual partitions. DB2 would not know how to apply the partial values to the existing totals for the non-partitioned indexes

  • If you restart the REORG utility using RESTART(CURRENT)

  • If you restart the LOAD or REBUILD INDEX utility

Note

Inline statistics established during LOAD may include the information about rows that have been discarded. If only a few rows have been discarded, this is not a concern, but if a large number of rows have been discarded, the inline stats may be very inaccurate and it may be better to execute a separate RUNSTATS utility.

Historical Statistics

DB2 provides the ability to keep a history of statistics for better proactive performance analysis capabilities. This helps to better monitor objects in terms of growth over time, along with other information to determine if objects need to change. The keyword of HISTORY in the RUNSTATS, REORG, LOAD, and REBUILD utilities is used to identify that historical statistics are to be collected.

REORG INDEX.....HISTORY
RUNSTATS TABLESPACE.....HISTORY

The historical statistics are kept in nine different DB2 catalog tables:

  • SYSIBM.SYSCOLDIST_HIST

  • SYSIBM.SYSCOLUMNS_HIST

  • SYSIBM.SYSINDEXPART_HIST

  • SYSIBM.SYSINDEXES_HIST

  • SYSIBM.SYSINDEXSTATS_HIST

  • SYSIBM.SYSLOBSTATS_HIST

  • SYSIBM.SYSTABLEPART_HIST

  • SYSIBM.SYSTABLES_HIST

  • SYSIBM.SYSTABSTATS_HIST

As the statistics age and are no longer needed, they can be deleted with a MODIFY STATISTICS utility.

STOSPACE Utility

The STOSPACE utility collects space information from the VSAM catalogs for storage groups and related table spaces and indexes. This utility then updates the DB2 catalog with information about the amount of space being utilized. This utility is executed against a storage group (for more information about storage groups, refer to Chapter 4).

Example of STOSPACE utility:

STOSPACE STOGROUP CERTSTG


064 - Data Maintenance Process

Chapter 7. Maintaining Data

Data Maintenance Process

The data maintenance process starts with the RUNSTATS utility. After performing RUNSTATS, a DBA can analyze the statistics to determine if a REORG is necessary or the REORG utility can review the statistics collected, and use the REORG triggers to determine if a REORG is needed.

If reorganization is needed, use the REORG utility on the selected objects and then do RUNSTATS and REBIND. You must perform a REBIND on any packages affected by the above operations, so they can take advantage of the benefits of the new physical organization and updated statistics. After performing subsequent update, insert, and delete operations, as part of the data maintenance process, repeat by first executing the RUNSTATS utility.

Establish a routine for RUNSTATS and the REBIND processes. Updated statistics will give you precise information about your database state.

Rebinds

Packages should be rebound when any of the following are true, based on current statistics from the catalog:

  • Changes > 20% (NLEAF, NPAGES, NACTIVE)

  • Clusterratio < 80%, NLEVELS increases > 2

  • HIGH2KEY and LOW2KEY ranges change > 10%

  • Cardinality and Row Count change > 20%

For more information on packages and rebinds, refer to Chapter 11, Application Development.

Modeling a Production Environment

There are a number of columns in the DB2 catalog that can be updated in order for a DBA to model a production environment so that the access paths against production data can be determined even if the data does not physically exist. Refer to Chapter 2, Environment, for more information on the catalog tables and refer to Chapter 10 for updating catalog columns to predict access paths.

Other Data Maintenance Utilities

CHECK Utilities

There are three utilities that are used to check the integrity of data and indexes. These utilities are often required to remove restrictive statuses. They are:

  • CHECK DATA

  • CHECK INDEX

  • CHECK LOB

CHECK DATA

CHECK DATA is an online utility that checks table spaces for violations of referential and table check constraints. It then reports information about any violations that are detected during its execution. The CHECK DATA utility will need to be executed after a conditional restart or a point-in-time recovery on all table spaces where parent and dependent tables might not be synchronized. See Chapter 9, Database Recovery, for more information on recovery. The CHECK DATA utility can be executed against a base table space only, not a LOB table space.

Rows that violate referential or table check constraints can be optionally deleted by the CHECK DATA utility. Any row that violates one or more constraints can also be copied, once, to an exception table.

If any violation of constraints is found, CHECK DATA puts the table space being checked in the CHECK pending status.

The phases of the CHECK DATA utility are:

  • UTILINIT

    • This is the initialization of the utility.

  • SCANTAB

    • Here the foreign keys are extracted. The foreign key index will be used for this if it exists, else a table scan will be performed.

  • SORT

    • During this phase, the foreign keys are sorted, if not extracted from the foreign key index.

  • CHECKDAT

    • This phase looks in primary indexes for foreign key parents, and issues messages to report any errors that are detected. It will report the RID of the row, the table that contained the row, and the constraint name that was violated.

  • REPORTCK

    • This phase copies the error rows into exception tables. It will then delete these rows from the source table if the DELETE YES option was specified.

  • UTILTERM

    • During this phase, any necessary cleanup operations are performed.

An example of using the CHECK DATA utility to check for and delete all constraint violations in table space DB2CERT.CERTTS is shown below:

CHECK DATA TABLESPACE DB2CERT.CERTTS
   FOR EXCEPTION IN DB2USER1.TEST_TAKEN
   USE DB2USER1.EXCP_TT
   DELETE YES

After a successful execution, CHECK DATA will reset the CHECK pending (CHKP) status.

CHECK INDEX

The CHECK INDEX online utility tests whether or not indexes are consistent with the data on which the index is created. The utility then issues warning messages when an inconsistency is found. The CHECK INDEX utility should be executed after a conditional restart or a point-in-time recovery on all table spaces whose indexes may not be consistent with the data.

The CHECK INDEX should also be used before CHECK DATA to ensure that the indexes used by CHECK DATA are valid. This is especially important before using CHECK DATA with DELETE YES. When checking an auxiliary table index, CHECK INDEX verifies that each LOB is represented by an index entry, and that an index entry exists for every LOB.

The CHECK INDEX utility generates several messages that show whether the indexes are consistent with the data. For unique indexes, any two null values are taken to be equal, unless the index was created with the UNIQUE WHERE NOT NULL clause. In that case, if the key is a single column, it can contain any number of null values, and CHECK INDEX does not issue an error message.

CHECK INDEX issues an error message if there are two or more null values and the unique index was not created with the UNIQUE WHERE NOT NULL clause.

The phases of execution of the CHECK INDEX are very simple. It starts with the initialization and setup, then it unloads the index entries, sorts them, and performs a scan of data to validate index entries, and performs any necessary cleanup.

An example of using the CHECK INDEX utility for all indexes in table space CERTTS is shown below:

CHECK INDEX(ALL) TABLESPACE CERTTS
   SORTDEVT 3380
CHECK LOB

The CHECK LOB online utility can be run against a LOB table space to identify any structural defects in the LOB table space and any invalid LOB values. Run the CHECK LOB online utility against a LOB table space that is marked CHECK pending (CHKP) to identify structural defects. If none is found, the CHECK LOB utility turns the CHKP status off.

Run the CHECK LOB online utility against a LOB table space that is in auxiliary warning (AUXW) status to identify invalid LOBs. If none exists, the CHECK LOB utility turns AUXW status off.

Run CHECK LOB after a conditional restart or a point-in-time recovery on all table spaces where LOB table spaces might not be synchronized.

The execution of the CHECK LOB utility is as follows:

  • UTILINIT

    • This is the initialization of the utility.

  • CHECKLOB

    • This phase scans all active pages of the LOB table space.

  • SORT

    • The SORT phase sorts four types of records from the CHECKLOB phase; it reports four times the number of rows sorted.

  • REPRTLOB

    • This phase examines records that are produced by the CHECKLOB phase and sorted by the SORT phase, and issues error messages.

  • UTILTERM

    • During this phase, any necessary cleanup is performed.

An example of executing the CHECK LOB utility against the CERTLBTS follows. It will check for structural defects or invalid LOBs.

CHECK LOB TABLESPACE DB2CERT.CERTLBTS
   EXCEPTIONS 3 WORKDDN SYSUT1,
   SORTOUT SORTDEVT SYSDA
   SORTNUM 4

After a successful execution of this utility, the CHECK pending (CHKP) and auxiliary warning (AUXW) statuses will be reset.

MODIFY

There are two different MODIFY utilities: MODIFY RECOVERY and MODIFY STATISTICS. They are both considered maintenance utilities because they are used to remove unwanted data from the catalog tables. We will take a look at the purpose of each one and when to run them.

Modify Recovery

The MODIFY online utility with the RECOVERY option deletes records from the SYSIBM.SYSCOPY catalog table, related log records from the SYSIBM.SYSLGRNX directory table, and entries from the DBD. Records can be removed that were written before a specific date or they can be removed if they are of a specific age. Records can be deleted for an entire table space, partition, or data set. This MODIFY utility should be run regularly to clear outdated information from SYSIBM.SYSCOPY and SYSIBM. SYSLGRNX. These tables, and particularly SYSIBM.SYSLGRNX, can become very large and take up a considerable amount of space. Performance can be improved for processes that access data from these tables by deleting outdated information from these tables.

The MODIFY RECOVERY utility automatically removes the SYSIBM. SYSCOPY and SYSIBM.SYSLGRNX recovery records that meet the AGE and DATE criteria for all indexes over the table space that were defined with the COPY YES attribute.

The MODIFY RECOVERY deletes image copy rows from SYSIBM. SYSCOPY and SYSIBM.SYSLGRNX. For each full and incremental SYSCOPY record deleted from SYSCOPY, the utility returns a message giving the name of the copy data set.

Note

If MODIFY RECOVERY deletes all of the SYSCOPY records causing the target table space or partition to be unrecoverable, the target object is placed in COPY pending status.

The phases for MODIFY RECOVERY are: the UTILINIT phase for initialization and setup, the MODIFY phase which deletes records, and the UTILTERM phase that performs cleanup.

Below is an example of the MODIFY RECOVERY utility. This example shows how to delete SYSCOPY records by date for a specific table space.

MODIFY RECOVERY TABLESPACE DB2CERT.CERTTS
DELETE DATE(20000414)

The MODIFY RECOVERY utility can also be used to reclaim space in the DBD after a drop of a user table has been performed. The following steps need to be performed to reclaim DBD space:

  1. Commit the drop.

  2. Run the REORG utility.

  3. Run the COPY utility to make a full image copy of the table space.

  4. Run MODIFY RECOVERY with the DELETE option to delete all previous image copies.

MODIFY RECOVERY utility can also be used to improve REORG performance after adding a column to a table due to the fact that after you add a column to a table space, the next REORG of the table space materializes default values for the added column by decompressing all rows of the table space during the UNLOAD phase and then compressing them again during the RELOAD phase. Subsequently, each REORG job for the table space repeats this processing in the UNLOAD and RELOAD phases. The following procedure helps to avoid repeating the compression cycle with each REORG:

  1. Run the REORG utility on the table space.

  2. Run the COPY utility to make a full image copy of the table space.

  3. Run MODIFY RECOVERY with the DELETE option to delete all previous image copies. MODIFY RECOVERY changes the alter added column status only if there are SYSCOPY rows to delete.
Modify Statistics

The MODIFY STATISTICS online utility deletes unwanted statistics history records from the corresponding catalog tables. These statistics history records that were written before a specific date can be removed or records of a specific age can be removed. Delete records can be specified for an entire table space, index space, or index.

The MODIFY STATISTICS utility should be run regularly to clear outdated information from the statistics history catalog tables. By deleting outdated information from these tables, performance can be improved for processes that access data from these tables.

The MODIFY STATISTICS deletes rows from the following catalog tables:

  • SYSIBM.SYSCOLDIST_HIST

  • SYSIBM.SYSCOLUMNS_HIST

  • SYSIBM.SYSINDEXES_HIST

  • SYSIBM.SYSINDEXPART_HIST

  • SYSIBM.SYSINDEXSTATS_HIST

  • SYSIBM.SYSLOBSTATS_HIST

  • SYSIBM.SYSTABLEPART_HIST

  • SYSIBM.SYSTABSTATS_HIST

These are the tables used for collecting historical statistics from a RUNSTATS execution. Refer to RUNSTATS earlier in this chapter for more information.

Note

The DELETE ALL option must be specified to delete rows from the SYSIBM.SYSTABLES_HIST catalog table.

The phases for MODIFY STATISTICS are the same as for the MODIFY RECOVERY: UTILINT, MODIFY, and UTILTERM.

An example of the MODIFY STATISTICS is shown below. This example removes rows for the CERTTS table space that are older that 90 days.

MODIFY STATISTICS TABLESPACE DB2CERT.CERTTS
 DELETE ALL
 AGE 90

Repair

The REPAIR online utility repairs data. The data can be your own data, or data you would not normally access, such as space map pages and index entries.

REPAIR is a means of replacing invalid data with valid data. Be extremely careful using REPAIR. Improper use can damage the data even further. You can use the REPAIR utility to:

  • Test DBDs.

  • Repair DBDs.

  • Reset a pending status (such as COPY pending) on a table space or index.

  • Verify the contents of data areas in table spaces and indexes.

  • Replace the contents of data areas in table spaces and indexes.

  • Delete a single row from a table space.

  • Produce a hexadecimal dump of an area in a table space or index.

  • Reset the level ID.

  • Change the PSID in the header page.

  • Delete an entire LOB from a LOB table space.

  • Dump LOB pages.

  • Rebuild OBDs for a LOB table space.

The potential output from the REPAIR utility consists of a modified page or pages in the specified DB2 table space or index, and a dump of the contents.

The execution phases of REPAIR are simply initialization, perform the repair, and then terminate the utility. The REPAIR utility cannot be restarted.

Note

REPAIR should be used only by a knowledgeable person. Be careful to grant REPAIR authorization only to appropriate people.


065 - Standalone Utilities

Chapter 7. Maintaining Data

Standalone Utilities

There are a number of utilities that can be executed outside of DB2. These are often referred to as standalone utilities or offline utilities.

DSNJLOGF (Preformat Active Log)

When writing to an active log data set for the first time, DB2 must preformat a VSAM control area before writing the log records. The DSNJLOGF utility avoids this delay by preformatting the active log data sets before bringing them online to DB2.

DSNJU003 (Change Log Inventory)

The DSNJU003 standalone utility changes the bootstrap data sets (BSDSs). You can use the utility to:

  • Add or delete active or archive log data sets.

  • Add or delete checkpoint records.

  • Create a conditional restart control record to control the next start of the DB2 subsystem.

  • Change the VSAM catalog name entry in the BSDS.

  • Modify the communication record in the BSDS.

  • Modify the value for the highest-written log RBA value (relative byte address within the log) or the highest-offloaded RBA value.

DSNJU004 (Print Log Map)

The Print Log Map (DSNJU004) utility lists the following information:

  • Log data set name, log RBA association, and log LRSN for both copy1 and copy2 of all active and archive log data sets

  • Active log data sets that are available for new log data

  • Status of all conditional restart control records in the bootstrap data set

  • Contents of the queue of checkpoint records in the bootstrap data set

  • The communication record of the BSDS, if one exists

  • Contents of the quiesce history record

  • System and utility timestamps

  • Contents of the checkpoint queue

In a data sharing environment, the DSNJU004 utility can list information from any or all BSDSs of a data sharing group.

DSN1CHKR

The DSN1CHKR utility verifies the integrity of DB2 directory and catalog table spaces. DSN1CHKR scans the specified table space for broken links, broken hash chains, and records that are not part of any link or chain.

Use DSN1CHKR on a regular basis to promptly detect any damage to the catalog and directory.

DSN1COMP

DSN1COMP estimates space savings to be achieved by DB2 data compression in table spaces. For more information on compression, refer to Chapter 8, Data Storage and Memory Management.

This utility can be run on the following types of data sets containing uncompressed data:

  • DB2 full image copy data sets

  • VSAM data sets that contain DB2 table spaces

  • Sequential data sets that contain DB2 table spaces (for example, DSN1COPY output)

DSN1COMP does not estimate savings for data sets that contain LOB table spaces or index spaces.

Below is an example of the type of output given from a DSN1COMP utility.

DSN194 I DSN1COMP COMPRESSION REPORT
301 KB WITHOUT COMPRESSION
224 KB WITH COMPRESSION
25 PERCENT OF THE BYTES WOULD BE SAVED
1,975 ROWS SCANNED TO BUILD DICTIONARY
4,665 ROWS SCANNED TO PROVIDE COMPRESSION ESTIMATE
4,096 DICTIONARY ENTRIES
81 BYTES FOR AVERAGE UNCOMPRESSED ROW LENGTH
52 BYTES FOR AVERAGE COMPRESSED ROW LENGTH
16 DICTIONARY PAGES REQUIRED
110 PAGES REQUIRED WITHOUT COMPRESSION
99 PAGES REQUIRED WITH COMPRESSION
10 PERCENT OF THE DB2 DATA PAGES WOULD BE SAVED

DSN1COPY

With the DSN1COPY standalone utility, you can copy:

  • DB2 VSAM data sets to sequential data sets

  • DSN1COPY sequential data sets to DB2 VSAM data sets

  • DB2 image copy data sets to DB2 VSAM data sets

  • DB2 VSAM data sets to other DB2 VSAM data sets

  • DSN1COPY sequential data sets to other sequential data sets

These copies can then be used to restore data. The restore can occur on the same DB2 or another DB2 subsystem. This is one way to be able to move data between subsystems.

Using DSN1COPY, you can also print hexadecimal dumps of DB2 data sets and databases, check the validity of data or index pages (including dictionary pages for compressed data), translate database object identifiers (OBIDs) to enable moving data sets between different systems, and reset to 0 the log RBA that is recorded in each index page or data page.

DSN1COPY is compatible with LOB table spaces when you specify the LOB keyword and omit the SEGMENT and INLCOPY keywords.

DSN1LOGP

The DSN1LOGP utility formats the contents of the recovery log for display. The two recovery log report formats are:

  • A detail report of individual log records

    • This information helps IBM Support Center personnel analyze the log in detail.(This book does not include a full description of the detail report.)

  • A summary report helps you to:

    • Perform a conditional restart

    • Resolve indoubt threads with a remote site

    • Detect problems with data propagation

You can specify the range of the log to process and select criteria within the range to limit the records in the detail report. For example, you can specify:

  • One or more units of recovery identified by URID

  • A single database

By specifying a URID and a database, you can display recovery log records that correspond to the use of one database by a single unit of recovery.

DSN1PRNT

With the DSN1PRNT standalone utility, you can print:

  • DB2 VSAM data sets that contain table spaces or index spaces (including dictionary pages for compressed data)

  • Image copy data sets

  • Sequential data sets that contain DB2 table spaces or index spaces

Using DSN1PRNT, you can print hexadecimal dumps of DB2 data sets and databases. If you specify the FORMAT option, DSN1PRNT formats the data and indexes for any page that does not contain an error that would prevent formatting. If DSN1PRNT detects such an error, it prints an error message just before the page and dumps the page without formatting. Formatting resumes with the next page.

Compressed records are printed in compressed format.

DSN1PRNT is especially useful when you want to identify the contents of a table space or index. You can run DSN1PRNT on image copy data sets as well as table spaces and indexes. DSN1PRNT accepts an index image copy as input when you specify the FULLCOPY option.

DSN1PRNT is compatible with LOB table spaces, when you specify the LOB keyword, and omit the INLCOPY keyword.

DSN1SDMP

Under the direction of the IBM Support Center, use the IFC Selective Dump (DSN1SDMP) utility to:

  • Force dumps when selected DB2 trace events occur.

  • Write DB2 trace records to a user-defined MVS data set.


066 - Resolving Restrictive and Advisory States

Chapter 7. Maintaining Data

Resolving Restrictive and Advisory States

DB2 sets a restrictive or advisory status on an object to control access and help ensure data integrity. The list below outlines the restrictive and non-restrictive object states that affect utilities, and the steps required to correct each status for a particular object.

Use the -DISPLAY DATABASE command to display the current status for an object. Below is Table 7.1 that shows all the restrictive states, the objects affected, and what corrective actions need to be taken.

Table 7.2. Restrictive and Advisory States

Status Code

Status Name

Objects Affected

Corrective Action(s)

ACHKP

Auxiliary CHECK Pending

Base table space, LOB table spaces

1. Update or delete invalid LOB using SQL

2. Run CHECK DATA utility to verify the validity of LOBs

AUXW

Auxiliary Warning

Base Table Space

1. Update or delete invalid LOBs using SQL

2. Run CHECK DATA utility to verify the validity of LOBs

LOB Table Space

1. Update or delete invalid LOBs using SQL

2. Run CHECK LOB utility to verify the validity of the LOB

CHECKP

CHECK Pending

Table space, base table space

Check and correct RI constraints using the CHECK DATA utility

Partitioning index, non-partitioning index, index on auxiliary table LOB table space

Run CHECK INDEX on index—if errors: run REBUILD INDEX utility Use CHECK LOB utility—if errors:

1. Correct defects found in LOB table space with REPAIR utility

2. Run CHECK LOB again

COPY

COPY Pending

Table space, table space partition

Take an image copy (best action), or use—START DATABASE (db) SPACENAM(ts) ACCESS FORCE or run REPAIR and reset COPY flag.

GRECP

GBP Recover Pending

Table space, index space

RECOVER the object, or use START DATABASE command

ICOPY

Informational COPY Pending

Partitioned index, non-partitioned index, index on auxiliary table

Copy the affected index

LPL

Logical Page List

Table spaces, index space

* START DATABASE ACCESS R/W or R/O

* Run RECOVER or REBUILD INDEX utility

* Run LOAD REPLACE

* DROP the object

RBDP

REBUILD Pending

Physical or logical index partition

Run the REBUILD or RECOVER utility on the affected index partition

RBDP*

Logical partitions of non-partitioning indexes

Run REBUILD INDEX PART or RECOVER utility on the affection

PSRBD

Non-partitioning index, index on the auxiliary table

Run REBUILD INDEX ALL, the RECOVER utility, or run REBUILD INDEX

The following actions also reset the REBUILD status:

* LOAD REPLACE with table space or partition

* REPAIR SET INDEX with NORBDPEND on index part—however, does not correct inconsistencies.

* Start database ACCESS FORCE—however, does not correct inconsistencies

* Run REORG INDEX SORTDATA on index

RECP

RECOVER Pending

Table space

Run the RECOVER utility on the affected object

Table space partition

Recover the logical partition

Index on the auxiliary table

Run REBUILD INDEX, RECOVER INDEX, or REORG SORTDATA

The following actions also reset the RECOVER status:

* LOAD REPLACE with table space or partition

* REPAIR SET TABLESPACE or INDEX with NORCVRPEND on index part—however, does not correct inconsistencies

* Start database ACCESS FORCE – however, does not correct inconsistencies

REFP

Refresh Pending

Table space, or index space

* Run a LOAD REPLACE

* Object will also be in RECP or RBDP status—will need appropriate action taken

REORP

REORG Pending

Table space

Perform one of the following:

* LOAD REPLACE entire table space

* REORG TABLESPACE SHRLEVEL NONE

* REORG PART n:m SHRLEVEL NONE

Partitioned Table space

Rows <= 32KB

1. Run REORG TABLESPACE SHRLEVEL NONE SORTDATA

Rows > 32KB

1. Run REORG TABLESPACE UNLOAD ONLY

2. Run LOAD TABLESPACE FORMAT UNLOAD

RESTP

Restart Pending

Table space, partitions, index spaces, physical index partitions

Objects are unavailable until back-out work is complete or until restart is canceled and a conditional restart or cold start is performed

WEPR

Write Error Page Range

Page range in error

Run a RECOVER utility on affected data


067 - Summary

Chapter 7. Maintaining Data

Summary

This chapter dealt with the issues of data movement, organization, and placement in your database. Various utilities in DB2 that load data into database tables and unload data from tables have been reviewed. These included the LOAD, UNLOAD, and DSNTIAUL utilities. We looked at various options for these utilities.

Having data in your tables is only the first step. The utilities and commands in DB2 that allow you to perform data maintenance on your environment were also reviewed. This included analysis of data in tables and indexes by executing REORG with available conditions. This would physically change the organization of your table and index data by then using REORG if the conditions were met. We also looked at how to update the statistics of table and index data using the RUNSTATS utility.

An important component of the data maintenance process is ensuring that your applications are aware of the updated statistics in your environment.

Many of the utilities, or particular circumstances, can put your data into a restrictive or advisory state. We looked at each of these and how to resolve them.


068 - Additional Resources

Chapter 7. Maintaining Data

Additional Resources

  • IBM DB2 UDB for OS/390 Administration Guide, SC26-9931.

  • IBM DB2 UDB for OS/390 Command Reference, SC26-9934.

  • IBM DB2 UDB for OS/390 Utility Guide and Reference, SC26-9945.


069 - Chapter 8. Database Recovery

Part III. DB2 UDB for OS/390 and z/OS Administration

Chapter 8. Database Recovery

  • Database Recovery Concepts

  • Logging

  • Image Copies

  • Data Recoveries

  • Disaster Recoveries


070 - Database Recovery Concepts

Chapter 8. Database Recovery

Database Recovery Concepts

DB2 has several methods for recovering data in the event of failures, or when errors occur, or when disaster strikes. DB2 allows you to recover data to the current state or to the state in an earlier point in time.

Many of the objects in DB2 can be recovered using various methods. You can recover:

  • table spaces

  • index spaces

  • indexes

  • partitions of a table space

  • individual data sets

  • pages within an error range

  • individual pages

In order to insure consistent operation and integrity to data, a DBA must develop a strategy for several different recovery scenarios. These scenarios can include, but are not limited to:

  • recovery from application failures

  • hardware failures

  • system failures

  • space failures

It is possible to recover the DB2 catalog and directory, data, and objects to a point of consistency. This point of consistency can be the situation that is most current (called "to current" or to the end of the log) or to a prior point in time where the data was still consistet.

With the current technology, hardware failures rarely occur. Most likely you have to perform a recover because the data was logically corrupted (e.g., errors in program logic that delete too much or update runs that have been run twice). You can recover from failures only if you have taken the proper backups (image copies and log records). DB2 has its own copy utility that will allow you to save the data by producing an image copy. If you save all data, this is called a full copy; if you save only the changes since the last copy, this is called an incremental copy. DB2 for OS/390 and Z/OS does its backup and recovery on a table space level or on an index space level. This means that if you perform a recovery to a table space and go back in time, then ALL tables in this table space go back in time. It is not possible to perform a backup and recover at a more granular object level (e.g., table), but you can recover at a more granular level, down to an individual page.

Note

DB2 will save only data and not the layout of your structures. Also, if you drop your structures, then all recovery information is lost and recovery cannot be performed. Very often when you change your application, you have to drop the structures and re-create them. In this case, you have to develop special scenarios where you have to unload and reload the tables.

Unit of Work

A unit of work is a series of recoverable changes within an application sequence. It is also referred to as a unit of recovery or a commit scope.

An application can have several units of work within it. If an error occurs, you can perform a rollback (back out of changes) of all the changes within the unit of work.

The only rollback that DB2 can perform is on a unit of work. Once a unit of work is committed, it cannot be rolled back in time. An example of a unit of work that can't be rolled back is a mass delete that will update only the segment table, which is the only thing logged for a mass delete. After the mass delete is committed, the data no longer exists and changes cannot be rolled back. The only information the log contains is the single update to the segment table.


071 - Logging

Chapter 8. Database Recovery

Logging

When changes are made to the data in tables, which subsequently changes the table space and index spaces, DB2 writes one or more records to its log so that a back out of the changes can be performed if the unit of work fails. DB2 can also use this information to "re-apply" changes that may have been lost when recovering to a previous point in time. The primary purpose of the active log is to record all changes (inserts, updates, and deletes) made to DB2 objects. The DML (Data Manipulation) statements are recorded in the log as follows:

  • Insert: entire after image of the record is logged called a redo record.

  • Delete: the before image is recorded called an undo record.

  • Update: both the before and after images (undo and redo record) are recorded

Each log record has its own unique identifier. In a parallel sysplex DB2 data sharing environment, it is known as the LRSN (Log Record Sequence Number). When you are operating in a non-data sharing environment, this is known as the log RBA (Relative Byte Address) or basically the offset of the record in the log from the beginning of the log. In a data sharing environment, the LRSN helps to track the sequence of events that occur over multiple members in the data sharing group. Each member has its own log, and if multiple members are making updates to the same data, then the logs must be merged by DB2 during a recovery.

Log Data Sets

DB2 physically records changes in log data sets. Every DB2 subsystem will have a predefined set of active logs on disk. The log records are then written to these active log data sets. When the active log data sets become full, DB2 automatically switches to the next available log data set. After all of the active log data sets have been used, DB2 will wrap around to the first active log. However, since we do not want to lose log records, because we may want to use them for recovery or back out, the active log data sets will be offloaded when they become full. Offloaded active logs are called archive log data sets that can be stored on disk or on cartridge tapes. As with the active log, you can have multiple copies of your archive datasets to protect yourself against failure. In the DB2 DSNZPARMs, you can specify the medium for archive storage (disk or cartridge) and how many archive copies you want to create.

Note

The maximum number of active log datasets is 31.

Note

Recovery from disk archive data sets is much faster and you can take advantage of DB2's ability to process the logs in parallel.

DB2 allows many archive log data sets and you will want to keep them in the event that a recovery is necessary. The retention period of the archive log data sets depends on your image copy frequency and how far you want to go back in time during a point-in-time recovery.

DB2 also provides a dual logging capability to ensure that there are two copies of the active log data sets; in the event that one is lost, the other can be used for recovery. During a recovery, DB2 applies the changes from the active log that are required to recover to the specified point in time. If the records needed to recover are not on the active log any longer, DB2 will call for the appropriate archive log(s).

Note

The maximum number of archive log data sets in the BSDS is 1000.

Note

All production systems should be using dual logs to ensure that data can be recovered successfully.

BSDS

The BSDS (bootstrap data set) is a VSAM data set that contains information regarding the DB2 logs and the records contained in those logs, as well as other information. The DB2 system will record all of the current active and archive log data sets in the BSDS. During the offload processing, DB2 will dynamically allocate a new data set with a unique name. After the offload completes successfully, this data set is recorded in the BSDS and a copy of the BSDS is created; in the case where the archive log is placed on a cartridge, this copy of the BSDS is also placed on the same cartridge as the archive log. During a recovery, DB2 will use the BSDS to find all of the available archive logs. The number of records that the BSDS can contain is determined by the MAXARCH parameter in the DSNZPARM member.

The BSDS should be large enough to record all archive logs. You should have the MAXARCH parameter set high enough for your environment and retain your archive log data sets long enough so that in the event of a recovery, where you may have to go back through older archive logs, the archive log data sets will still exist. If the data set is not recorded in the BSDS anymore, but is still physically available, it is possible to place its entry in the BSDS by using the change log inventory utility (DSNJU003). However, this can be done only when the DB2 subsystem is stopped and therefore will cause an outage. The proper setting for the MAXARCH parameter will depend on how large your archive logs are and the oldest point in time an application is allowed to recover to.

SYSIBM.SYSLGRNX

DB2 uses a catalog table in the DB2 directory known as SYSIBM. SYSLGRNX that gives it the ability to determine the log records for a table space and recoverable indexes by recording the periods of updates for these objects. There will be a row for the time period that a table space or index spaces had update activity occurring. Recorded in this row is the following:

  • DBID and OBID of the object

  • LRSN and local log RBA of the first update after open

  • LRSN and local log RBA when a pseudo-close occurred (no more update activity)

  • If data sharing, the member which performed the updates will be recorded

This information will help DB2 to determine what log data sets (or parts of the log) will be needed in the event of a recovery and will speed up the recovery process because logs that contain no updates for the object being recovered are skipped. The MODIFY utility can be used to delete outdated information from the tables that store image copy information. It should be executed often and on a scheduled basis, depending on system activity. This will help improve performance for several different processes that access the SYSIBM.SYSCOPY and SYSIBM.SYSLGRNX catalog tables. Due to the nature of the data in these tables, they can grow considerably in size and require a good deal of space. It is recommended, if the SYSCOPY and SYSLGRNX tables are very large, that outdated information be deleted by using the MODIFY utility to delete entries by age. For example, it might be best to have one run of the utility remove rows older than 20 months. If you want to keep information longer, then it is recommended that you unload the tables and also keep the DDL to re-create these tables. This will allow you to re-create very old data, possibly under a different name. The DB2 recovery information is not intended for these scenarios.


072 - Image Copies

Chapter 8. Database Recovery

Image Copies

In order to be able to recover data when there is a problem, you must have a solid recovery plan and a strategy in place for each of the many situations. This plan will include performing image copies on a regular basis. You create image copies of your table spaces and selected index spaces.

Frequency of Image Copies

The frequency with which image copies should be taken depends on the allowable downtime for the primary user of the application. You should determine the clock-time needed if you have to recover your data. If an application cannot be unavailable for more than two hours, you will need to take image copies more frequently than if your application can be unavailable for a complete day. Depending on the currency of data required in the recovery, you will have to determine when image copies can be taken and what types of image copies need to be taken. You may need to have incremental image copies taken throughout the day, or you may have to do only one full copy at the end of the day's processing. There are also situations where an image copy is taken at the beginning of a batch window, especially where there are long running batch processes. This allows for a restore in case something goes wrong during the batch cycle.

There are also a number of technical considerations in image copy frequency. The bare minimum you should allow is half the retention period of the archive log, less one day ([retention period / 2]–1 day). This will guarantee that at any moment there are two valid image copies of the table spaces. You always want two valid copies because of possible media failure, or to reduce the risk of inconsistent data on the image copy. After a drop and re-create of an object, all system retained recovery information is gone. After you run utilities that will re-create the data (REORG, LOAD REPLACE without the LOG YES option), you have to run an image copy to establish a new base for recovery.

Dual Image Copies

DB2 provides for the ability to create dual image copies. If there is an error on the primary image copy and there is no secondary copy of your data to rely on, then a recovery will fail. Even when it is possible to fall back to a previous image copy, substantial time is needed to perform the log apply from the previous image to where the failed image copy was taken. A previous image copy should be regarded as a safety net, in case there are errors in the pages. By un-cataloging the most current image copy, it is possible to force a fallback. This works only if the image copies are cataloged.

In addition to using dual copies, it is possible to make two sets of copies so that there are two copies onsite and two copies offsite. Using just the second copy of a dual copy for offsite backup is not the best choice, because if an onsite copy is bad, then obtaining the offsite copy after a failure may be inconvenient, depending on where the offsite disaster recovery is located. Keeping two copies on the primary site will ensure that there is an immediate backup to use if a copy is bad; the same concept applies when recovering from image copies at the disaster recovery site, where two copies should be maintained also. In order to do this, in the COPY utility you could follow the example below:

COPY TABLESPACE DB2CERT.CERTS COPYDDN(DD1, DD2) RECOVERYDDN(DD3,DD4)...

Note

Version 7 allows the ability to make a copy from an image copy using the COPYTOCOPY utility.

This would ensure that you took two copies for the primary site and two for the recovery site.

Tape vs. DISK Image Copies

The fastest recoveries occur when the input data sets reside on DISK. This applies both to recoveries using image copies and archive logs. However, since DISK space is more expensive then cartridges, there is a trade-off between the cost of an unavailable application vs. the cost of DISK space. Mounting, remounting, and winding of cartridges can take a very long time. If you use the cartridge method and stack the files, you should use the proper JCL options during image copy and recovery so that the cartridge stays mounted and positioned. The handling of stacked files on cartridge can be automated by using TEMPLATE statements in the utility parameters.

Full and Incremental Image Copies

There are two different types of backups available: full copies or incremental copies. A full image copy copies all of the pages in the table space whether it has changed or not (FULL YES is the utility default). Incremental image copies copy only the pages that have changed since the last image copy. You can view the type of copy in the ICTYPE column in the SYSCOPY catalog table along with the database name, the table space name, time of the copy, the start RBA/LRSN, data set name, and the volume serial number (if the image copy is not cataloged). A full image copy will have an ICTYPE or "F" and an incremental will have an ICTYPE of "I".

Incremental copies of data can be useful, but because the procedures involved in taking incremental copies are more complex than those for full image copies (for example, how often to take incremental copies, and when to run the Merge Copy utility), many database administrators choose to take only full image copies.

There is a danger of losing data or extending the recovery time when using incremental image copies. Incremental copies are a substitute for the log. DB2 will try to mount the incremental image copies to skip large parts of the log during the log apply phase. But if this fails, DB2 notes the RBA or LOGPOINT of the last successful incremental and uses archive log from that point on. If the archive log does not exist any more, then from this point on the recovery will fail.

Some administrators use the incremental copy utility for its speed and then run the MERGECOPY utility to merge the last full copy and latest incremental into a new full copy. This procedure combines the best of both worlds: ease of recovery and the speed of incremental image copies.

Note

You might want to reconsider the use of incremental copies if the data is very volatile and has more than five percent of the pages updated between copies because the runtime of the incremental image copy will be longer than a full copy. If runtime is not an issue and the size of the image copy data set is, then an incremental copy might be the best solution.

Note

If the feature for tracking space map changes is turned off (TRACKMOD NO) in the table space definition, then incremental copies are required to perform a table space scan in order to find the individual pages that have changed and need to be copied, and should be avoided. Only full image copies should be used in this case.

The image-copy utility has parameters to determine whether an incremental or a full image copy should be made through the use of the user-defined thresholds. You can use the thresholds in the image-copy utility to control subsequent steps, depending on whether an incremental copy or a full copy was done.

Both full and incremental copies are important for creating backups in a 24 x 7 environment that must always be operational, and that must be able to meet an acceptable recovery window. Full copies should be taken when:

  • A table is newly loaded using LOAD LOG NO **

  • A table is loaded with LOAD RESUME LOG NO **

  • A table is reorganized with REORG LOG NO **

  • Table data has been modified extensively

Note

In many cases (marked by **) DB2 will enforce an image copy by setting the image-copy pending (COPY) condition for the object. It is unwise to ignore this status (e.g., using the REPAIR utility) unless you intend never to update the data again and keep the original input of the utility (e.g., a load in a data warehouse environment).

Incremental copies should be taken when changes have been made to the table since the last full image copy and processing the log would take too much time during the recovery. During the recovery, DB2 requires access to all incremental copies at the same time. If the incremental copies reside on cartridge, this could be a problem because of the number of available units. If DB2 cannot mount all incremental image copies, it will ignore them and start processing log again. You can prevent this situation by running the MERGECOPY utility to merge incremental image copies into a single set.

Copy of Partitions or Data Sets

For both table spaces and index spaces, individual partitions can be backed up by the image-copy utility by specifying the partition number in the DSNUM keyword on the control statement of the COPY utility. Logical partitions of non-partitioning indexes cannot be image copied and if PIECESIZE has been used, the pieces of an index cannot be individually image copied. This DSNUM will be recorded in the SYSCOPY table for each partition or data set that was copied.

If you image copy by partition, then you also have to recover by partition. If you image copy the entire partitioned table space (no use of the DSNUM), then you can recover the entire table space or one or more partitions.

Copy Data Sets

It is advisable to have multiple copies for an image copy in the event that one image copy may become damaged. This can be easily accomplished via the COPYDDN parameter on the COPY utility control card. Here you can specify a primary copy and a secondary, or backup, copy by specifying the DD names for the two copy data sets. This backup copy is intended to be a local backup copy so that during a recovery this can be used in the event that the primary copy is not usable. In many installations, this local backup copy is in addition to having a copy for remote site recovery.

Recommendations

There are a few recommendations for making image copy datasets.

  • Consider using GDG (generation data groups) for backups to generate unique data set names for the image copy.

  • Use DISP=(MOD, CATLG) for the ability to restart the COPY utility.

Note

Remember that GDGs are cataloged at the end of a job (not job step). If a job fails, the data set name is already recorded in SYSIBM.SYSCOPY, but not the OS/390 catalog. The next run will result in duplicate names being rejected by the COPY utility.

Multiple Copy Data Sets

You can specify multiple objects in a single COPY utility, which allows you to take image copies of multiple objects (table space, partitions, etc.) in one job and enables the use of parallelism. This is performed by repeating the object clause as many times as needed (see the example below).

COPY TABLESPACE DB2CERT.CERTTS COPY DDN dsname1
     TABLESPACE DB2CERT.CERTTSPT COPY DDN dsname2
     INDEX DB2USER1.TESTIX COPY DDN dsname3
   PARALLEL (3)

This provides the ability to copy multiple objects using three parallel streams (PARALLEL (3)). DB2 will attempt three steams, but storage constraints may cause this to be lessened or ignored. The parallel option will not work if you are using a cartridge and the copies are stacked. In this case, the copies will still be taken in a serial fashion and the parallel keyword is ignored.

CHANGELIMIT

By using the CHANGELIMIT feature of the COPY utility, it is possible to tell DB2 to look for a specified percentage of changed pages in a table space or partition before actually taking an image copy (full or incremental). You can specify value(s) in the CHANGELIMIT parameter to allow for an incremental image copy or full image copies to be taken only when those values are reached.

Another feature of the COPY utility that can be used to observe changes is the REPORTONLY feature, which would allow you to see if an image copy is recommended because the CHANGELIMIT values were met. This feature could help to report if any changes occurred against an object, possibly for auditing purposes. The following example shows the use of the CHANGELIMIT and REPORTONLY features:

COPY TABLESPACE DB2CERT.CERTTS CHANGELIMIT (5,30) REPORTONLY

This example gives a recommendation for a full image copy but doesn't actually take the image copy because of the REPORTONLY feature. If the percentage of changed pages was greater than 30, it would recommend an incremental image copy if the number of changed pages was greater than 5 percent or less than 30 percent.

There is a possibility that an image copy will not actually be taken when using the CHANGELIMIT option, thus creating an empty copy data set. This is not very desirable, especially with GDGs, because you could lose previous image copies. Using the REPORTONLY option will set the condition code for this job step. The condition code will depend on the advice given and can be used in conjunction with conditional JCL (e.g., full copies go to cartridge and incremental copies to DISK). This option can also be used to prevent empty copy data set.

Access During Image Copy Process

During the taking of an image copy, you can choose whether or not to allow concurrent access. This is done in a couple of ways.

Utility Mode

The first method is to simply start the object in utility mode. This can be done by issuing the START DATABASE command using the ACCESS(UT) option. This can be done for both table spaces or index spaces. Choosing this option will allow only utilities to be executed against the objects. If any other concurrent processes try to access those objects, they will be rejected.

SHRLEVEL

The other method to control concurrent access is through the SHRLEVEL parameter on the COPY utility. The options are CHANGE and REFERENCE.

  • SHRLEVEL REFERENCE: This will force the COPY utility process to wait until all writers are not using the table space or index space. They must commit their changes before the utility can continue. The benefit to this option is that readers are allowed against the objects being copied during the utility execution. However, no updating is allowed.

This option also guarantees that the copies are created in a single COPY utility and do not have any incomplete units of work.

  • SHRLEVEL CHANGE: This option will allow other applications to update the objects during the image copy. This means that some of the changes that were made during the copy may be on the log, but not necessarily in the copy itself. An image copy created by this method is often called a fuzzy copy.

Regardless of the SHRLEVEL that is chosen, DB2 will associate and log RBA or LRSN to the image copy and record this in SYSIBM.SYSCOPY. This is the point in the DB2 log from which DB2 will then need to apply log records during a recovery process.

Inline Copies

After a load of a table space using a LOAD REPLACE, or after reorganizing a table space, you must establish a full image copy so that you can avoid long log apply phases in the event a recovery is necessary, or if you are resolving a COPY pending status. Taking a copy inline during the utility execution is much faster than executing the COPY utility afterwards and avoids the setting of the COPY pending status, allowing for increased availability of the table space.

This copy can easily be taken inline during the execution of the LOAD REPLACE or REORG utility execution. These inline copies are equivalent to a full image copy taken with SHRLEVEL REFERENCE, with a few minor differences such as out-of-order pages or multiple versions. This would possibly cause the inline copy to be slightly larger that a normal full copy. However, the inline copy is just fine for recoveries, can be processed by all DB2 utilities, and can be the basis for future incremental image copies.

Inline copies are requested on the COPYDDN keyword in the LOAD or REORG utility. See below.

Load Utility:

LOAD....REPLACE
   COPYDDN (local-primary,(local-backup))
   (RECOVERYDDN (remote-primary,(remote-backup))

Reorg Utility:

REORG
   COPYDDN (local-primary,(local-backup))
   (RECOVERYDDN (remote-primary,(remote-backup))

MERGECOPY Utility

The MERGECOPY utility merges existing incremental image copies into a single copy to help lessen recovery time. This utility should be executed based upon the amount of exposure perceived and the amount of downtime allowed in a recovery. For recover "to-current" to be effective, running MERGECOPY frequently would be beneficial. If point-in-time recoveries are more likely, MERGECOPY will not be beneficial.

MERGECOPY improves performance for "to-current" recoveries because the full copy and possibly only one incremental copy would be needed to recover, instead of having to read through multiple incremental copies. However, for a point-in-time recovery (TORBA or TOCOPY), MERGECOPY may not have any benefit.

Note

The MERGECOPY utility cannot merge image copies that are from different share levels(SHRLEVEL), and it will also allocate all copies at the same time (like the RECOVER utility). So, if you create more incremental image copies than you have tape drives, you will have a problem.

Using the timeline in the following illustration, if you wanted to recover to May 20, and the MERGECOPY on June 5 has not yet been done, the incremental from May 20 will still be in SYSCOPY and can be used to recover to. After the MERGECOPY, the copy for May 20 will no longer be in SYSCOPY and DB2 must then go back to the full image copy on MAY 1 and do a forward log apply to MAY 20, making the time for recovery longer due to the fact that more of the log will have to be applied.

You can specify whether you want a new full copy or just to merge all of the incremental into one new incremental copy. This is done with the NEWCOPY key word.

MERGECOPY...NEWCOPY YES

or

MERGECOPY...NEWCOPY NO

When you perform a MERGECOPY and create a new incremental copy, all of the rows in SYSIBM.SYSCOPY for the merged copies are deleted.

Note

Another very important reason to merge copy frequently is that if DB2 cannot mount all incremental copies (e.g., the incremental copies are on cartridge and there are not enough units) during a recovery, DB2 will choose to apply changes from the log. This increases the amount of time needed for a recovery, and could possibly cause the recovery to fail because this log is unavailable.

REPORT Utility

You obtain a report that identifies all referential relationships to a given table by using the REPORT utility with the option TABLESPACESET. From this you can determine what table spaces need to be quiesced and image copied together. Keep in mind that this covers only the relationships defined in DB2; there may also be other application-defined relationships between tables that would have to be included in the same recovery scope. The TABLESPACESET option is available for all utilities in combination with TEMPLATES. This will automate utilities if you use DB2 enforced RI.

Index Copy

It is not necessary to have an image copy of the index in order to recover it. An index can be rebuilt from the data in the tables, using the utility REBUILD. You can also recover the index from an image copy. In some cases, it may be faster to recover from an image copy of an index vs. rebuilding it, especially if the index has little or no change activity. The index recovery can also be done in parallel with the table space recovery when you recover an index from an image copy. Once you enable an index for image copies using COPY YES on the CREATE/ALTER INDEX DDL, DB2 now records recovery information in both SYSIBM.SYSCOPY and SYSIBM.SYSLGRNX. There are no changes to the log because indexes are always logged to be able to rollback a unit of work. Copies of indexes can be only full image copies because the indexes do not have a space map to determine the updated pages. There is no concept of incremental image copies for indexes.

Copying the Catalog and Directory

Due to the fact that you start a DB2 system-wide recovery by first recovering the catalog and directory, and no other recoveries are possible before this process is completed, the catalog and directory need to be image copied often, so that the recovery process for these objects is as fast as possible.

Note

If you lose your catalog and/or directory, you will not have an operational DB2 subsystem.

The time required for the recovery of the catalog and directory is crucial and needs to be kept to a minimum. Because DB2 performs the log apply for the catalog and directory, it does not matter when you take the image copy of the catalog. However, it helps if you take a SHRLEVEL REFERENCE image copy just before you cut your last archive, and take both this image and the archive logs to the vault or offsite. This way you make sure that the DB2 catalog contains all updates and that it is also valuable without the archive logs.

It is also important that the ICF(Integrated Catalog Facility) catalog be kept in sync with your DB2 catalog. For example, if the ICF catalog has yesterday's image in a disaster recovery scenario, your DB2 catalog will reference data sets (such as image copies and/or table spaces) that do not exist, according to the ICF catalog. To avoid this problem, make sure that the recovery procedures of the ICF catalog match the DB2 catalog.


073 - Establishing a Point of Consistency

Chapter 8. Database Recovery

Establishing a Point of Consistency

In order to ensure that it is possible to recover to a point in time where everything is consistent and in sync, a point of consistency or a common quiesce point must be established. This point of consistency is to establish a recovery set so that related objects will be recovered to a consistent point in time.

A point of consistency is established by executing the QUIESCE utility. The QUIESCE utility "quiets" the system and writes to the log the time in which there was no activity and everything was in sync. It does this by waiting for current units of work to complete and preventing any new ones from beginning. When the last unit of work has completed, a quiesce point is established. This is recorded for each table space and index space in a recovery set by recording the START_RBA of the quiesce point in SYSIBM.SYSCOPY. This row will also have an ICTYPE of "Q." These objects have a common point to which they can recover.

You determine which objects will be together in establishing this common point by using the TABLESPACESET keyword on the QUIESCE utility. You can also specify particular partitions of a partitioned table space.

QUIESCE TABLESPACESET
         TABLESPACE dbname1.tsname1
         TABLESPACE dbname2.tsname2

Note

By specifying WRITE YES(which is the default), you can ensure that all changed pages of the table spaces are written to DISK.

Note

A QUIESCE utility will drain the table spaces one by one. In a system that is heavily updated and long running units of recovery can be in the way, the QUIESCE utility can cause a serious outage because it will keep the table spaces that it was able to drain locked, causing SQL to fail.

Another way to establish a consistent point for your data is to establish a copy set. If you perform a single copy of multiple objects using the COPY utility with SHRLEVEL REFERENCE, the objects will have the same START_RBA value, which will create a recovery copy set.


074 - Recovery Concepts

Chapter 8. Database Recovery

Recovery Concepts

Now that we have discussed how to make proper backups for our DB2 objects, and how the DB2 log is used, we will take a look at how to use those objects in the event of a recovery. We will also look at the various types of recoveries.

Table Space Recovery

The RECOVER utility is used to recover a table space. It is performed in two major phases: RESTORE and LOGAPPLY.

RESTORE

The RECOVER utility determines which full image copy is needed by looking at the rows in the SYSIBM.SYSCOPY catalog table. This full image copy is taken and merged with any incremental copies that are found in the SYSCOPY table, replacing any updated pages in the full copy. This provides the basis for the restoration of the table space. It is possible to use backups not created by DB2. However, restoring any object is then your own responsibility and you must specify that the RECOVER utility has to perform a LOGAPPLY only.

LOGAPPLY

After the full image copy is brought up to current from the merging of the incremental image copies, the next phase is the LOGAPPLY. This phase will apply any changes that are on the DB2 log that were made after the image copies were taken. The utility reads the logs and applies these changes to the restored table space. It uses the SYSLGRNX table to identify the range on the DB2 log that pertains to the table space that is being recovered. It also uses the START_RBA in SYSCOPY to identify the latest image copy that was used during the RESTORE phase and to know from what point in the log that records must be applied. The LOGAPPLY is very efficient because it can sort the log on page number and then on RBA/LRSN.

Note

The START_RBA will depend on whether the image copy was a SHRLEVEL REFERENCE or SHRLEVEL CHANGE.

Index Space Recovery

You can also recover indexes if they were enabled for image copies in the DDL definition. Using image copies, an index is recovered rather that being rebuilt from the table using the REBUILD utility. There are only full image copies on indexes, no incremental copies. The LOGAPPLY phase works the same way and applies necessary changes from the log since the time of the last image copy. Even if an index was marked to be recoverable, it can still be rebuilt using the REBUILD utility. This utility will extract the index keys from the data, sort the keys, and build the index again, ignoring all recovery information available.

Partial Object Recovery

It is possible to recover part of an object. You can perform recoveries against:

  • a partition

  • a data set of an NPI

  • a single page

  • an error page range

Note

A single table cannot be independently recovered.

Multiple Object Recovery

The same way that it is possible to back up multiple objects in the same job execution, it is also possible to recover multiple objects at the same time. In a single RECOVER statement, you can recover multiple table spaces, index spaces, partitions, or data sets. You simply repeat the TABLESPACE, INDEXSPACE, or INDEX clause:

RECOVER   TABLESPACE dbname1.tsname1
          TABLESPACE dbname2.tsname2
          INDEXSPACE creator1.ixname1
          (PARALLEL (2))

The RESTORE phase is performed in parallel by using the PARALLEL option. In the above example, two streams of parallelism are attempted. The RESTORE phases for the objects are started and executed in parallel. One of the tasks will read the pages from the image copy and the other tasks will write the pages that have been read, and this is done in parallel.

The log is read only once during the LOGAPPLY phase. This phase is common for all of the objects being recovered and does not use parallelism, but it can sort the log to improve performance.

Note

DB2 will lessen the parallel degree or even serialize the recoveries if there is not enough storage to support multiple streams.

Note

Parallel recovery is possible only if the image copies are on DISK; if the copies are stacked on cartridge, then the recovery processes are serialized.

Fallback Recovery

There are times when things can go wrong in recoveries, such as an image copy not being available, being defective, or simply deleted. There are a few choices for DB2 in these cases.

  • Use of local copies

    • If the local primary copy cannot be used, DB2 will try to use the local backup copy. DB2 will not try to use the remote copies because it assumes that they are being keep offsite.

  • Use of incremental copies

    • If an incremental copy cannot be used, DB2 will merge all prior incremental copies with the full copy, ignoring the defective incremental and all subsequent incremental image copies. DB2 will then start the LOGAPPLY phase at the START_RBA of the last incremental that was merged.

  • Invalid full copies

    • If the full copy cannot be used, the DB2 will fall back to a previous full image copy and then merge it with the incremental image copies (if available). If no good full copy exists at all, the table space is recovered from the log, or a DSNU510I error message is issued and the utility terminates in error.

  • Index space invalid copies

    • If no valid index space copies exist, then the same message as the previous point is issued and the utility is terminated. It would then be necessary to perform a rebuild of the index from the table using the REBUILD utility.

REPORT Recovery

Information provided by the REPORT RECOVERY utility can help plan for recovery by providing information on:

  • Recovery history from SYSIBM.SYSCOPY

  • Log ranges from SYSIBM.SYSLGRNX

  • VOL-SER numbers where archive logs and BSDS (Boot Strap Data Set) reside

  • VOL-SER numbers where image copy data sets reside

The information from the REPORT RECOVERY utility helps you identify what image copies, active logs, and archive logs are needed to recover a table space.

Using Fast Log Apply

The RECOVER utility can use a standard log apply process during recoveries or use a Fast Log Apply (FLA). Normal use is the standard log apply process, but the FLA process can be utilized if it has been enabled. This process is enabled if there is a non-zero value for the size of the Log Apply Storage residing in the DBM1 address space. This is a DSNZPARM set on the DSNTIPL installation panel.

The FLA process will use two buffers that are known as FLA buffers. One buffer is for reading the log records and the other is used for applying the log records to the table space or index space. When the read buffer fills, the pages are sorted and then applied (making this now the write buffer), then the older write buffer becomes the new read buffer and begins to fill. These two buffers continue this flipping process until all the records have been read and applied.

FLA also takes advantage of parallelism by using one task per object being recovered for applying the logs and an additional task for reading the next set of log records at the same time.

Point-in-Time Recovery

You can use the TOLOGPOINT parameter in your RECOVERY job in order to recovery to any previous point in time. This parameter is specified to provide the log RBA or LRSN of the point in time for the recovery. This should be a quiesce point that was established for the copy recovery set. This information is obtained from the SYSIBM.SYSCOPY table or by using the REPORT RECOVERY utility.

After a point-in-time recovery has been performed, a row will be inserted into the SYSIBM.SYSCOPY table with an ICTYPE of "P" for each object that was recovered in order to allow future recoveries to skip those associated log ranges.

Note

In a non-data sharing environment, the TORBA keyword can be used, but it is recommended that TOLOGPOINT be used.

Note

In a non-data sharing environment, the TORBA keyword can be used, but it is recommended that TOLOGPOINT be used. In a distributed environment, it can also be used to recover two table spaces in two separate subsystems to the same point in time.

In order to avoid any pending conditions when you recover a table space set to a prior point in time, you will need to ensure that all the related table spaces are in the same RECOVER control statement. You can use the TABLESPACESET keyword in the table space statement.

When a point-in-time recovery is performed for multiple objects in a single RECOVER utility control statement with TOCOPY, there is no LOGAPPLY phase, only a RESTORE phase to the last image copy. Therefore, multiple control statements must be used to prevent a CHKP status being set on the objects where there are RI relations. This will also cause a RBDP or CHKP on the indexes. It is wiser to specify a common START_RBA for a recovery copy set via the TOLOGPOINT parameter.

Pending Conditions

A point-in-time recovery can result in a variety of pending conditions. These are as follows:

  • RBDP– Rebuild pending

    • This status results for indexes that existed on tables that were part of a table space that was recovered. This is for indexes that were not copy enabled. A copy enabled index will get this status only if it was not included in the recovery set.

    • Recovering only certain partitions can result in RDBP* condition on a non-partitioning index rendering it inaccessible.

  • CHKP– Check pending

    • This status results when a table space is recovered that has tables with referential integrity relationships with tables that were not recovered.

    • This status also results when indexes are recovered for table spaces that were not also recovered.

In order to resolve these statuses, certain actions need to be taken. For a RBDP condition, the index will need to be rebuilt via the REBUILD INDEX utility. You will need to rebuild the index if you have a RBDP* status for a logical partition of a non-partitioning index.

For a CHKP, check pending status, the CHECK DATA utility has to be performed. This will find any violations of referential integrity constraints and can optionally resolve them by removing the offending rows, and any directly or indirectly dependent rows. CHKP conditions on an index may indicate that the index is inconsistent with the data in the table. A CHECK INDEX utility can help to resolve these inconsistencies by determining if there are any and, if so, you can run the REBUILD INDEX utility to rebuild the index and get it back in sync with the data.

Note

CHECK INDEX will reset the CHKP flag if there are no inconsistencies detected. If there are inconsistencies between the index and the data, you may want to immediately run the REBUILD INDEX to resolve problems in a timely manner.

Note

If any unexpected pending conditions (especially recovery pending RECP) are encountered, carefully review the utility output. Utilities set and resolve pending conditions while they are processing. If by the end of the utility there are still pending conditions, then maybe something was wrong.

CHECK DATA Utility

This utility will check the table space for violations of table check constraints or referential constraints and report this information. Check constraints would be violated only if they were added after the recovery, so only referential constraints should show up as violated. You can execute the CHECK DATA utility on the entire table space or on individual partitions. It will provide options for handling various violations.

Violations will be reported only in the utility output listing unless the FOR EXCEPTION clause on the utility is specified. If the DELETE NO clause is specified, the utility will report the violations and copy the violators to an exception (shadow) table provided. When DELETE YES is specified, then the violators are removed after they are moved to the exception tables. The rows will have to be moved back to the original tables after they are corrected, but by telling DB2 to delete the violators, the check pending flag will be reset.

NOTE:

It is necessary to rebuild indexes before running the CHECK DATA utility because it uses the primary index and, if available, the index on the foreign key.

Exception Tables

Exception tables will have to be created for the CHECK DATA utility prior to its execution. These tables must have the same columns, in the same order, with the same data type, as the original table, although the column names can be different. The easiest way to do this is via the CREATE TABLE LIKE statement—creating an exception table exactly like the original. Two additional columns can be added to this table: a five-character column that gives you the RID or the original row and a timestamp column that gives you the starting data and time for the CHECK DATA utility.

NOTE:

If a TIMESTAMP column is desired, a RID column must already exist, but you can have the RID column even without the TIMESTAMP column.

Recover Pending

A recover pending (RECP) state can be set on table spaces and index spaces, and it will prevent access to the affected objects. It can be set on indexes after executing a recovery utility to TOCOPY and TORBA on a table space, or from utility abends or terminations of RECOVER, LOAD, or REORG. The condition can be reset by executing a RECOVER, LOAD REPLACE, or REPAIR utility.


075 - Backup and Recovery of the DB2 Catalog and Directory

Chapter 8. Database Recovery

Backup and Recovery of the DB2 Catalog and Directory

Since the DB2 catalog and directory are also comprised of table spaces and index spaces, they can fall subject to failures, such as a media failure. There may be instances where these objects need to be recovered.

There is a specific order to recovering catalog data sets, due to relationships and dependencies in the catalog. This order is as follows:

  1. DSNDB01.SYSUTILX

  2. SYSUTILX indexes

  3. DSNDB01.DBD01

  4. DSNDB06.SYSCOPY

  5. SYSCOPY (3) indexes (IBM only)

  6. DSNDB01.SYSLGRNX

  7. SYSLGRNX Indexes

  8. DSNDB06.SYSDBAUT

  9. SYSDBAUT (3) indexes (IBM only)

  10. DSNDB06.SYSUSER

  11. DSNDB06.SYSDBASE

  12. SYSDBASE indexes (IBM only)

  13. SYSUSER (3) indexes (IBM only)

  14. All other catalog and directory table spaces and indexes

  15. Catalog indexes (user-defined)

  16. System utility table spaces, such as QMF

  17. Communications database

  18. Object and application registration tables

  19. Resource limit specification tables

Note

Never recover the catalog and directory to a point in time unless ALL user data is also recovered to the same point in time; otherwise, it will introduce an inconsistency that will corrupt data and can cause DB2 to fail system-wide.


076 - Disaster Recovery

Chapter 8. Database Recovery

Disaster Recovery

Another type of recovery that must be prepared for is a disaster recovery. There is always the possibility of losing an entire data center and having to recover at another site usually known as the disaster or recovery site. This can only be successful through careful planning and practice.

In order to ensure success for a disaster recovery regular backups of the data and log available must be available at the disaster site. The more current the backups are the better in order to minimize data loss and updates processing. The goal is to be restored and running as soon as possible.

Preparation

There are many steps in preparing for disaster recovery at a recovery site. Some of the basic information needed at this site includes the copies of the data, including the catalog and directory, and the archive log data sets.

In order to prepare a recovery site to be able to recover to a fixed point in time, a weekly copy of the data, possibly with a DFSMS logical volume dump, must be available. This then needs to be sent to the recovery site where it can be restored.

In order to do a recovery through the last archive copy, all of the following objects need to be sent to the recovery site in a timely manner:

  • Image copies of all user data

  • Image copies of catalog and directory

  • Archive logs

  • ICF (Integrated Catalog Facility) EXPORT and list

  • BSDS lists

A disaster site can also be made ready by using the log capture exit in order to capture log data real time and then having that data sent periodically to the recovery site. However, this is not often a viable option due to the overhead of use of the log capture in high-volume environments.

Note

It is possible that you are using special facilities that you have become dependent on (e.g., DFSMS/HSM). In this case, these facilities should be in working order again at the recovery site and completely up-to-date again. If these facilities go back in time, you might be forced to go back in time also. If you use these facilities, then include them in your recovery scenarios.

Items Needed for Disaster Recovery

There are several items that are necessary for a successful disaster recovery that should all be made available at the disaster recovery site.

Image Copies

Image copies of the application data will be required. In the event of a disaster recovery, it is assumed that a copy of the local copies is available at the disaster site. An option exists in the copy utility to make copies to be sent regularly to the remote recovery site. A remote primary and a remote backup copy can be made like the local primary and backup. This is accomplished by specifying the data set names in the DD statements for the RECOVERYDDN parameter in the COPY utility control cards. Those copies can be used for recovery on a subsystem that has the RECOVERYSITE DSNZPARM enabled or if you run a RECOVER using the RECOVERYSITE parameter.

You will also need image copies of the catalog and directory and a listing of the contents of SYSCOPY, which can be obtained via a SQL SELECT.

Note

If the image copies need to be tracked, you will want to catalog them. It is wise to have a single ICF catalog for a DB2 system so all information regarding a system is in sync.

Note

It is not necessary to keep index copies at the disaster site; they can be rebuilt, if necessary.

Archive Logs

Copies of the archive logs need to be made and taken to the disaster recovery site. This can be performed by issuing the ARCHIVE LOG command to archive the current active DB2 log. A BSDS report needs to be created by using the print log map utility in order to have a listing of the archive logs, since DB2 will use the BSDS to find all of the available archive logs during a recovery.

ICF (Integrated Catalog Facility)

It is also necessary to back up the ICF catalog via the VSAM EXPORT command. A list of DB2 entries needs be recorded via the VSAM LISTCAT command. This needs to be done daily and sent to the recovery site.

Note

Very often the ICF catalog is the responsibility of a different department. Make sure that the ICF is always in sync with DB2. Create a single ICF catalog per DB2 system and EXPORT the ICF after the DB2 catalog and directory are image copied.

DB2 Libraries

The DB2 Libraries need to be backed up to tape if they are changed. These are as follows:

  • SMP/E, load, distribution, target libraries, DBRMs, and user applications

  • DSNTIJUZ job which builds the DSNZPARM module and the DECP module

  • Data set allocations for the BSDS, logs, catalog, and directory

Note

It is good practice to record when all these items arrive at the recovery site and to have backups of all documentation.

Minimizing Data Loss

Volume dumps and restores are practiced often as a disaster recovery scenario. There can be significant data loss with this. In order to minimize the data loss, you will want to perform a dump of all the table spaces, logs, and the BSDS while DB2 is up–after issuing an ARCHIVE LOG MODE(QUIESCE) command.

The ARCHIVE LOG command is very useful when performing a DB2 backup in preparation for a remote site recovery. It allows the DB2 subsystem to quiesce all users after a commit point, and capture the resulting point of consistency in the current active log before the archive is taken. Therefore, when the archive log is used with the most current image copy (during an offsite recovery), the number of data inconsistencies will be minimized.

Note

If a -STOP DB2 MODE(FORCE) is in progress, the ARCHIVE command will not be allowed.


077 - LOB Recovery

Chapter 8. Database Recovery

LOB Recovery

There are special recovery considerations for LOBs due to the way LOBs (Large Objects) are stored and are maintained.

Logging

There is an option of whether or not to log changes to LOB columns. This option is specified on the creation of the LOB table space by using the LOG option (LOG YES is the default). LOBs that are larger than 1GB cannot be logged. If you have a LOB defined as LOG NO and then decided to ALTER it to be LOG YES, then the LOB will be placed in Copy Pending status. For LOBs that are defined as LOG NO, the force-at-commit protocol will ensure that the LOB values persist once committed. Even with LOG NO, the changes to the system pages and the auxiliary indexes are logged.

Point-in-Time Recovery

Even though there are no referential constraints between the LOB table space and its associated base table, the LOB table space belongs to the same table space set as the associated base table. A point-in-time recovery will need to recover both the base table and the LOB table space to a common point of consistency. In order to perform this, a point-of-consistency will need to be established by quiescing the table space set or by using the COPY utility to generate a set of SHRLEVEL REFERENCE image copies for the table space set. The table space set will then need to be recovered to either the RBA or the quiesce point or image copy set. A QUIESCE WRITE YES will record the point for the index on the auxiliary table if the index was defined with COPY YES.

AUXW Status

When the LOG NO option has been chosen to not log the LOBs, then any log apply processing required during a recovery will invalidate the LOB values that were recorded after the last restored image copy of the table space, because DB2 does still record when the LOB values were changed, but not the update itself. This will place the LOB table space in AUXW status—Auxiliary Warning, and the invalid LOB values will not be able to be read by SQL.

The table space that was defined with LOG NO will need to be recovered to current by running the CHECK LOB utility on the LOB table space to identify which rows are invalid. The rows are identified by their ROWID. Then SQL is used on the base table to update the invalid LOB values or delete the row that contains the invalid LOB value. A second run of the CHECK LOB utility will be needed to verify the LOB value and then it will reset the AUXW status. Even if the LOB table space is in AUXW status, DML statements can be used on the base table but will fail with a SQLCODE of –904 when they try to read an invalid LOB.


078 - Tracker Site Recovery

Chapter 8. Database Recovery

Tracker Site Recovery

The tracker site option allows for the creation of a separate DB2 subsystem (or data sharing group) that exists only for keeping shadow copies of the primary site's data. The way a tracker site is supported is by transferring the BSDS and archive logs from the primary site to the tracker site. The tracker site periodically runs LOGONLY recoveries to keep shadow data current. If a disaster occurs at the primary site, the trackers site will become the takeover site ( Figure 8.1 ).

Figure 8.1. Tracker Site Recovery

Because the tracker site shadows activity on the primary site, image copies do not have to constantly be shipped. This allows for the tracker site to take control faster.

There are two main reasons for choosing a tracker site:

  1. To minimize the data lost during a disaster

  2. To minimize the amount of time to get access to data during a disaster

It is recommended to use a tracker site if it is important to recover data quickly at a disaster recovery site with minimal data loss. The required is a DSNZPARM to be set in order to support the tracker site option. On the Install Panel DSNTIPO, there is a field called TRACKER SITE (DSNZPARM TRKRSITE). 0 is the default setting. After installing the tracker site subsystem, TRACKER SITE is set to YES in order to enable the tracker site support.

In order to start tracker site support, both sites must be brought in sync. This is accomplished by shutting down the primary DB2 subsystem and taking a DISK dump of everything. This dump is restored on the tracker site. After the tracker site is started, the primary site can be started. If both sites keep connected, then they keep in sync, but if the connection is lost, then the primary site cannot queue the log data and you have to bring both sites in sync again using manual intervention. It is important to make sure that both sites never lose their connection if this option is chosen. A situation to watch for when using this option is the fact that table spaces get out of sync due to utilities with the LOG NO option.


079 - Set Log Suspend/Resume

Chapter 8. Database Recovery

Set Log Suspend/Resume

The -SET LOG SUSPEND/RESUME command can be used to temporarily freeze all DB2 activity. It provides for a fast system level copy of an entire DB2 subsystem. A fast copy can be accomplished via ESS "Shark" FlashCopy or RVA Snapshot. These copies can be used for remote-site recovery or point-in-time recovery. When a -SET LOG SUSPEND command is executed, a system checkpoint is taken (non-data sharing only), log buffers are flushed, a log-write latch is obtained, the BSDS is updated with the highest written RBA, and a message will be issued to the console that DB2 update activity has been suspended. A -SET LOG RESUME command will release the log-write latch, delete the suspended message, and issue a log-resumed message.


080 - Checkpoint Interval

Chapter 8. Database Recovery

Checkpoint Interval

The checkpoint interval is important for DB2 recovery processing. The longer amount of time between checkpoint intervals, the more your DB2 applications are exposed to a longer restart time in case of a system failure. The checkpoint interval is set with the LOGLOAD parameter in the DSNZPARMs. In addition, it can be changed dynamically with the -SET LOG LOGLOAD command. This parameter is based on the number of log records written. You may also specify the interval in terms of minutes with the -SET LOG CHKTIME command.


081 - Minimizing Outages

Chapter 8. Database Recovery

Minimizing Outages

There is an option in DB2 that will issue a warning message when a unit of work has written more log records than a defined threshold without a commit. This will let you identify those applications that would require a long backout and/or recovery in case of system or application failure. This is set through the URCHKTH DSNZPARM. In order to minimize the amount of time it takes to recover from a system failure, we need to ensure that our applications are taking frequent commits.


082 - Data Sharing Recovery

Chapter 8. Database Recovery

Data Sharing Recovery

For information on data sharing recovery, refer to Chapter 9.


083 - Summary

Chapter 8. Database Recovery

Summary

We have looked at what it takes to back up and recover objects in DB2. There are many scenarios that need to be accounted for and a successful recovery can come only from careful planning. Setting up and practicing these steps for recovery is key to knowing whether or not you can achieve your availability requirements.


084 - Additional Recovery Resources

Chapter 8. Database Recovery

Additional Recovery Resources

For additional documentation on recovery topics, refer to:

  • DB2 UDB for OS/390 Administration Guide, SC26-9931.

  • DB2 UDB for OS/390 and Continuous Availability, SG24-5486-00.

  • Using RVA and SnapShot for BI with OS/390 and DB2, SG24-5333-00.

  • DB2 on MVS Platform: Data Sharing Recovery, SG24-2218-00.


085 - Chapter 9. Data Sharing

Part III. DB2 UDB for OS/390 and z/OS Administration

Chapter 9. Data Sharing

  • Data Sharing Components

  • Maintaining Data Integrity

  • Migration

  • Workload Management

  • Sysplex Query Parallelism

  • Recovery Considerations

Data sharing, available in DB2 since Version 4, allows an application to run on one or more DB2 subsystems in a parallel sysplex environment. The applications can read and write to the same data concurrently. Prior to data sharing, DDF was used to access data on other subsystems, or used for other more creative means, such as replication between subsystems.

The subsystems that can share data must belong to a data sharing group. The subsystems in the group are known as members. Up to 32 members are allowed in a data sharing group. Only members in the group can share data and a member can belong to only one group. Figure 9.1 shows a Data Sharing environment.

Figure 9.1. Data Sharing

DB2 data sharing operates in a parallel sysplex environment, which is a cluster of OS/390 systems that can communicate with each other. There are some very important components that allow for this communication to occur, as well as insure the consistency and coherency of the data that is being shared between the subsystems in the group.


086 - Data Sharing Benefits

Chapter 9. Data Sharing

Data Sharing Benefits

The many advantages of using DB2 data sharing include:

  • Improved price/performance by using S/390 microprocessor technology

  • Increased capacity with more power and a higher degree of inter-transaction parallelism available

  • Continuous availability through the ability to hide unplanned outages and planned outages (for example, rolling maintenance), and if a member is lost, you can keep on running

  • Incremental growth (horizontal growth) by adding processors without any disruption

  • Configuration flexibility with the ability to start/stop members as required, separate subsystems by function (such as batch, ad hoc, and OLTP)

  • Ability to split large queries across all CPCs with sysplex query parallelism

  • Flexibility for scheduling existing workloads by cloning CICS region on another MVS and removing the restriction of a CICS application that is able to run on one MVS only

  • Applications can run concurrently on several subsystems, allowing for increases in throughput

  • Reduced need for distributed processing because applications will not have to use DRDA to communicate in order to share data, which eliminates the overhead of DRDA for this purpose

  • Due to the fact that data is shared, affinity and non-affinity workloads can be in the same group, run workloads of different processors, or specifically run a workload on a particular processor

  • Shared Data Architecture (SDA) is the architecture behind DB2 data sharing and is based on coupling technology, which allows for the use of high-speed coupling facility channels, reduced system-to-system communication (message passing), supplying multiple paths to the data for higher availability, and dynamic workload routing based on capacity, not on location. There is no need for data partitioning for growth, and it does not rely on node-to-node communication for resources.


087 - Data Sharing Components

Chapter 9. Data Sharing

Data Sharing Components

There are new hardware components to become familiar with in a data sharing environment. Most are simply components in the parallel sysplex, but they are vital to the operation and success of data sharing. DB2 DBA must learn what is in these components, how they are configured, and how they are used. Many performance and availability issues lie with these components.

Coupling Facility

The coupling facility is one of the most important components of a data sharing environment. It is a specialized piece of hardware with dedicated high-speed links that are connected to each DB2 in the data sharing group. It is the center for communication and sharing of data between the subsystems.

It is important to have at least two coupling facilities for each data sharing group, although only one is required. This is mainly done for availability and capacity reasons. It is also ideal to have at least one of these coupling facilities be a dedicated 9674 processor that runs only the coupling facility control code. This is recommended for both availability and performance. You can use ICMF (Integrated Coupling Migration Facility) where the coupling facility control code would run in an LPAR on a CMOS bipolar machine, but this is not recommended for a production environment. It is fine for a test environment or a disaster recovery site.

A S/390 9674 is a dedicated microprocessor that is used as a dedicated coupling facility. It is a CPC that runs only as a coupling facility in a logical partition with dedicated processor resources. This is the more optimal configuration because a failure of a coupling facility and a connected MVS image (which is more likely when on the same CPC) can lead to extended recovery times. You also get better performance by using dedicated hardware as well as better connectivity to the coupling facility.

ICFs (Internal Coupling Facilities) are a relatively new option for coupling facility configurations. An ICF is an IBM 9672 machine that can be configured with a internal coupling facility. It can use one or more engines depending on the generation of the machine. An ICF is a more attractive alternative to the dedicated 9674, mainly for economic reasons (less expensive). You will still want to have at least one external coupling facility for high availability.

The coupling facility is usually one of the top items in this environment that are significantly undersized in terms of storage. It is recommended that you have at least one GB on each coupling facility to start. Attempting to support a data sharing group with anything less can be very difficult, if not impossible.

You will need enough storage to hold all of the necessary structures and enough room (empty) to hold the structures from the other coupling facility in the group in order to be able to rebuild all the necessary structures from a failing coupling facility, or a coupling facility that has been taken offline for maintenance.The coupling facility consists of three structures very important for DB2 data sharing. These structures are the SCA, LOCK and Group Buffer Pool(s).

SCA (Shared Communication Area)

The SCA structure, also known as the list structure, holds all database status information and system information and any critical information for any recovery situation. It contains a list (array of lists for accuracy) that includes some of the following:

  • Database exception table

  • BSDS information

  • Stopped pagesets

  • LPL entries

  • GRECP entries

  • GCLSN

  • EDM pool invalidations

Lock Structure

The lock structure controls inter-system locking and serialization on records, pages, tablespaces, etc. It also holds all of the global locks. The lock structure is comprised of two parts:

  • Lock list

    • Names of modified resources

    • Lock status of the resource

    • Contains modify and retained locks (discussed later in this chapter)

  • Lock hash table

    • Owning members of modified resources

    • Lock status information

    • Used for quick inter-system lock contention detection

This structure is connected by the IRLM during the DB2 member startup. Sizing of the lock structure is not too difficult; normally, you can start with 32 MB, and for above average workloads (high-volume DML), you could start with 64 MB. If the lock structure is too small, you can have increased lock contention because there are only so many entries for locks to be held.

Group Buffer Pools

The group buffer pools are known as the cache structures. They provide for high performance access to shared data, data coherency, and serialization of updates to data on DASD.

The group buffer pools are shared by all the DB2 subsystem that have a corresponding local buffer pool defined (i.e., if objects are defined in local BP1 and there is a GBP1, then these object can be shared). This is done by providing a corresponding group buffer pool in the coupling facility.

Since we can have up to 80 virtual, or local, buffer pools, we can also have up to 80 group buffer pools. We are, of course, limited by the size of the coupling facility because the group buffer pools will take up space in the coupling facility. Sizing group buffer pools is more of an art that a science. It requires knowledge of how the data in each buffer pool is accessed and a good breakout of objects into individual pools.

The GBP gets allocated the first time it is used and all the pages in it can be shared. The GBP registers data pages and also handles the invalidation of data pages cross-system. There is more information on group buffer pools later in this chapter.

Structures and Policies

The aforementioned structures are defined to the coupling facility via policies. There are different policies used for various definitions in the parallel sysplex environment. Policies (or couple data sets) store information about the systems in the sysplex, XCF groups, and members' definitions and general status information. They are formatted using the MVS IXCL1DSU utility in SYS1.MIGLIB and must be accessible by all members (residing on shared DASD). Below are listed a few that are key to the operation of a data sharing environment.

CFRM–Coupling Facility Resource Management

The CFRM policy defines the structures to the couple facility. This is where you defined the SCA, Lock, and GBP structures. When defining the policies to the coupling facilities, it is important to remember a few things.

Note

It is important to leave room for growth of the structures and you will want to plan carefully what the size should be according to the space available in your coupling facility and account for fail over conditions.

SFM–Sysplex Failure Management

This policy is one to be familiar with because it holds information about the importance of each subsystem in the sysplex. During a coupling facility failure and subsequent structure rebuilds, the REBUILDPERCENT that is contained in the CFRM is compared to the WEIGHT in the SFM policy to determine whether or not to rebuild a structure.

ARM–Automatic Restart Manager

This is an optional policy, but is highly recommended. It is ARM's job to keep specific working running in the event of a failure. The quicker you can get DB2 restarted in the event of a hardware failure or abend, the better. It is important for availability that all retained locks (discussed later in this chapter) are resolved quickly; the only way to do this is to restart the failing member and this is where ARM is critical.

Links

High-speed links are used to connect coupling facilities to processors running the operating systems. There are two types:

  • Multi-mode fiber (50/125 micron) supporting distances up to 1km

  • Single-mode fiber (9/125 or 10/125 micron) supporting distances up to 3km whose characteristics are the same as those used by ESCON XDF

One link between the coupling facility and processor will suffice but this is not recommended for performance and, of course, availability. Use two links between the coupling facility and each processor. These links are not the same as XCF links, which are CTC (channel-to-channel) and are used for inter-MVS communications. Coupling facility links are used for high speed communication between MVS systems to the CF via XES.

Sysplex Timer

The sysplex timer is a required component of the data sharing parallel sysplex environment. You must have at least one (two highly recommended). This timer keeps timestamps of the S/390 processors synchronized for the DB2s in the data sharing group. It is used to determine the order of events regardless of which processor in the Sysplex performed them. For example: the LRSNs can be guaranteed to order events for recovery because they are based on the TOD clock generated by the timer.

Note

The sysplex time should be on a separate power source or UPS. If the timer fails, all MVS systems in the sysplex are placed in non-re-startable wait states (figure x).

XCF

The XCF (Cross System Coupling Facility) is a component of MVS. All DB2 members join an XCF group when they join the data sharing group. Each IRLM will join another XCF group. XCF is used for communication between IRLM and XES, and to notify other members to retrieve database or system status, or control information changes made to SCA. DB2 also uses XCF for some intersystem communication and for processing DB2 commands and utilities within the group.

There are three categories of XCF services:

  • Group services

    • Provide means for requesting information about other members in the same XCF group via SETXCF commands

  • Signaling services

    • Provide means for communicating among members of the group

  • Status monitoring services

    • Members can check their own status and relay this information to others in the group. This also allows for monitoring of other members in the sysplex.


088 - Shared Data

Chapter 9. Data Sharing

Shared Data

In a data sharing environment, there are many items that are shared by all the members in the group. In order to have these available for each member in the group, they must reside on shared DASD. The following is a list of items that need to be on shared DASD:

  • MVS catalog

  • DB2 catalog

  • DB2 directory

  • Couple data sets (SFM, CFRM, ARM, WLM)

  • Shared databases

  • ICF user catalog for shared databases

  • LOG data sets must be shared for read access

    • But log exists for each member

  • BSDS data sets must be shared for read access

    • But BSDS exists for each member

  • Work files

    • Required for queries using Sysplex Query Parallelism

    • Keeps DB2 connected to its work files regardless of where the DB2 has to be restarted

    • No longer DSNDB07


089 - To Share or Not to Share

Chapter 9. Data Sharing

To Share or Not to Share

Just because you are in a data sharing environment does not mean all the data must be shared. Some data can be dedicated to one subsystem. These non-shared objects are defined unique to one member. They can also reside/remain on non-shared DASD if you want (Figure 9.2). It is easy to leave objects dedicated to one subsystem; you simply put the object in a virtual buffer pool that is not backed by a group buffer pool in the coupling facility.

Figure 9.2. Sharing Data

Note

Not sharing data between members may be a consideration if the application is not suited for data sharing or there is no requirement for data to be shared among several DB2s.


090 - Maintaining Data Integrity

Chapter 9. Data Sharing

Maintaining Data Integrity

Data that is defined as "sharable" can be accessed by any DB2 in the group. Several subsystems can read and write simultaneously against the same data. This is controlled via "inter-DB2 Read/Write interest." Changed data, or data with "inter-DB2 R/W interest" is always cached in the group buffer pool; this data becomes "group bufferpool dependent." The coupling facility will control the invalidations of changed pages in all members' local buffer pools.

Consistency of data among the DB2 members in the data sharing group is protected in two ways:

  • Concurrency controls

  • Coherency controls

In order to provide concurrency controls among the DB2 members in the data sharing group, a new locking structure is used. Since tuning locking is extremely critical to performance in a data sharing environment, it is important to understand how it works.

Locking

Data sharing locking is not like locking before. With data sharing we introduce Explicit Hierarchical Locking (EHL). If you were not aware, prior to data sharing you were using Implicit Hierarchical Locking. The only difference is that with EHL, a token is kept that identifies parent/child relationships. A parent lock is a table space/partition lock and a child lock is a page/row lock (Figure 9.3). The benefit of using EHL is the fact that only the most restrictive lock moves to the coupling facility reducing the number calls to coupling facility to control concurrency, which can create a great deal of overhead. Lock avoidance still works with EHL and type 2 indexes work best. This does not lessen the fact that the use of uncommitted read (UR) should still be considered wherever possible.

Figure 9.3. Parent/Child Locks

With EHL, only the most restrictive parent lock is propagated until the time when it is necessary for the child to be propagated (recorded in the coupling facility), thus lessening the amount of lock activity.

Here is an example of how EHL works: If parent lock is a table space in IX (Intent Exclusive) mode, then a child lock in S (Share) mode on a page would not have to be propagated to the coupling facility lock structure because the lock on the parent is more restrictive.

Basically, we lock only what is necessary, and negotiate locks in CF if there is conflict. Child locks will get propagated only if the parent locks are in conflict.

As you can see, there is a lot of new lock terminology with data sharing locking. We will take a brief look at each of these.

Local Locks

Local locks are those locks that we have always known and loved. They are the same as in non-data sharing environment. These are the locks requested on local subsystem providing only intra-DB2 concurrency control.

Global Locks

Global locks are the locks that a DB2 subsystem needs to make known to the group through the coupling facility. These are locks that are propagated to the coupling facility and they provide intra-DB2 and inter-DB2 concurrency control. In a data sharing environment, almost all locks are global.

Whether a lock becomes global will depend on whether the lock request is for a logical (L-Lock) or physical (P-Lock). P-Locks, or physical locks, are owned by a DB2 member and are negotiable. Unlike our normal transaction locks, these locks are not used for concurrency but rather coherency. There are two types of P-Locks (Page Set and Page).

P-Locks

Page Set P-Locks are used to track intersystem interest between DB2 members, and they determine when a page set becomes GBP dependent. These locks will have different modes depending on the level of R/W interest on the page set among the DB2 members. A P-Lock cannot be negotiated if it is retained. They are released when the page set or partitioned data set is closed. There are few P-Locks taken, and they are usually held for long periods of time.

Page P-Locks are used to ensure the physical consistency of a page when it is being modified—these work at subpage level and are used in the same manner as latches in a non-data sharing environment. P-Locks are also used when changes are being made to a GBP dependent space map page.

Page Set P-Lock negotiation takes place when P-Locks are noted as being incompatible. The two members with the incompatible lock will negotiate the lock so that both can still use the object. Since the P-Lock is used for coherency, not concurrency, this negotiation does not sacrifice any data integrity.

The reason for P-lock negotiation is to lessen the number of locks that are propagated to the coupling facility. The most restrictive P-lock is taken first and then, if necessary, negotiated so that another process can have access to the page set. Page set P-locks are used to track interest in a page set and to know when it is necessary to begin propagation of child locks because of the level of interest in the DB2 members for the page set. Basically, P-locks and negotiation can be thought of as "I need to know what you are doing, here is what I am doing, let's find a way to work together, or do we have to take turns?" It can also be thought of as an indicator.

L-Locks

L-Locks, or logical locks, occur in both data sharing (can be local or global) and non-data sharing subsystems. These locks are transaction or program owned. They are non-negotiable locks and work like the normal locks in a single subsystem environment to serialize access to objects. L-Locks are controlled by IRLM of member and are held from update to commit. There are two types of L-locks.

Parent L-Locks

Parent L-locks are at table space or partition level (page set) and are almost always propagated to find if conflict exists with another member.

Child L-Locks

Child L-locks can be on a table, data page, or row. They are based on parent L-Lock conflict. If no conflict, then they are not propagated.

Modify Locks

Modify and retained locks are two more types of locks introduced in data sharing. Modify locks are used to identify a lock on a resource that is being shared (updated). Any active X type (X, IX, SIX) P-lock or L-lock can be considered a modify lock. This lock is kept in Modify Resource List in the Lock Structure in the coupling facility and is kept regardless of group buffer pool dependency of the object. These locks are used to create retained locks if the DB2 member holding the modify lock were to fail.

Retained Locks

Retained locks are modify locks that get converted to retained locks if a member of the group fails. They are necessary to preserve the data integrity in the event of a failure. These locks will be held when a DB2 subsystem fails and the locks belong to the failing member and must be resolved before access to the locked object will be allowed by other members.

This can be a performance/availability bottleneck if proper procedures are not in place for recovering a failed DB2 member. These locks are held at the Global Lock Manager (GLM) level and owned by the Local Lock Manager (LLM), not a transaction. This means that only that DB2 member that had the lock can resolve it, so the subsystem MUST come up to resolve the lock. So, regardless of where a transaction may resume (i.e., another subsystem), the locks are still retained and the data is still not accessible by any process (readers using uncommitted read can still view the data). The DB2 can be restarted on the same MVS or another one in the same group–it does not matter, as long as it comes up.

Note

Each local IRLM also keeps a local copy of retained locks for fast reference so retained locks can survive a coupling facility failure.

Lock Contention

There are three types of locking contention that can occur in a data sharing environment:

  • Global Lock Contention (IRLM/Real)

    • This contention occurs when there is real contention against two resources.

  • False Lock Contention

    • This contention occurs when two locks hash to the same entry in the lock table, but the actual locks are not in contention.

  • XES Contention

    • This occurs because XES interprets locks as only X or S, therefore some locks that are in contention are actually compatible because they are really intent locks.

Note

All forms of lock contention need to be monitored and minimized because the process of contention resolution in a data sharing environment can get expensive and can be detrimental to performance.

Group Buffer Pools

Group buffer pools are structures in the coupling facility in a data sharing environment. They are used to allow for the sharing of data between multiple subsystems. When a page is read by a member, it is read into that member's virtual buffer pool and registered in the group buffer pool. If the page is updated, then a force-at-commit will cache that changed page in the group buffer pool and invalidate the page in any other member's virtual buffer pool. This process, known as cross invalidation, ensures that everyone is working with the most current page. Figure 9.4 shows how pages are registered.

Figure 9.4. Group Buffer Pool Page Registration

Page directory entries are used to check the existence of a page copy in the group buffer pool. They are also used to determine which members will need to be sent XI (Cross Invalidation) messages. Only one directory entry is needed for a page regardless of how many virtual buffer pools the page is cached in.

Interest for a page is registered in the page directory when a page is read by a member into the local buffer pool from disk, or the group buffer pool for a group buffer pool dependent page set. If using CFLEVEL=2, DB2 prefetch can register up to 32 pages with a single CF interaction; otherwise, it is done on a page-by-page basis. When a page set or partition becomes GBP dependent, all changed pages in the local buffer pool are moved synchronously into the GBP. All of these pages, clean and changed, are registered in the directory in the GBP.

The ratio is a setting on the group buffer pool that establishes the number of directory entries to the number of data entries in the GBP. If we do not have enough directory entries (entry for each page read on any DB2 member—only one page registered regardless of number of members with interest), then when a new page needs to be registered, a directory will be reclaimed in order to register a new page. This will then cause a process requiring that page to have to go to disk to reread and register the page. Depending on the number of times this occurs, it can add up to significant overhead. Use the –DISPLAY GROUPBUFFERPOOL command to determine how many times this occurs.

In order to change the ratio, you can issue the -ALTER GROUPBUFFERPOOL command. In the below example, we are changing the ratio to be 20:1 (20 directory entries for every data entry):

-ALTER GROUPBUFFERPOOL (GBP3) RATIO (20)

There are a few situations that can occur which will cause the deregistration of a page. If buffers are stolen from the local buffer pool of a GBP dependent page set, the page is deregistered. If this occurs, it indicates there may be a problem with either the size and/or threshold in the virtual buffer pool because the pages are falling off the LRU queue. This would not be a problem if the page was not referenced, but if it is needed and has to be read back into the virtual buffer pool, it must also be re-registered.

If an existing directory entry must be reclaimed for new work, the page is marked invalid and deregistered, and as a result must then be re-read in from disk. This can happen if the group buffer pool is too small or the ratio is in correct.

Sizing GBPs

Sizing group buffer pools is not like sizing normal virtual buffer pools. The group buffer pools are defined as structures in the coupling facility. They are given an initial size when they are defined in the CFRM policy and, for performance and availability reasons, will need to be created in a coupling facility separate from the coupling facility that holds the Lock and SCA structures

There are some standard rules of thumb for sizing but most are very generic at best. For the best sizing of your group buffer pools, you must have a good understanding of the amount of sharing that will be occurring against the objects that are in the group buffer pool. In other words, you are going to have to worry about object separation in virtual buffer pools even more so when implementing group buffer pools; otherwise, your initial sizing estimates will be rather difficult.

Castout

Since there is no connection between the coupling facility and disk, DB2 must have a way to move changed pages out to disk. It does this through a process called castout. The castout process (performed by castout engines) moves the changed pages from the group buffer pool through a private area in the DBM1 address space (NOT a virtual buffer pool) and from there they are written to disk. Figure 9.5 shows how the castout process works.

Figure 9.5. Castout Process

The castout process is triggered when the number of changed pages exceeds the CLASST threshold, or the number of changed pages exceeds the GBPOOLT threshold, or a psuedo/physical close is performed on a data set by the last updating member. The CLASST threshold is similar to the VDWQT threshold on local buffer pools, and the GBPOOLT is similar to the DWQT threshold. Castout can also be triggered if the GBPCHKPT threshold is reached and a group buffer pool checkpoint is taken.


091 - Performance

Chapter 9. Data Sharing

Performance

So what is new about DB2 performance when you add data sharing? Not much except for the fact that data sharing has thrown a whole new spin on DB2 performance and tuning. From application selection to post-implementation troubleshooting, there are several new places to look for problems. Old performance problems that were acceptable or tolerable in the past are magnified in the data sharing environment. New skills are needed in diverse areas to monitor and tune for overall performance.

What is new about operating in a data sharing environment is the introduction of new hardware and new rules. Contrary to popular belief, data sharing is not just an install option! The introduction of the coupling facility in the Parallel Sysplex data sharing architecture accounts for a whole new set of factors to be concerned with in terms of performance. The coupling facility is unique to DB2 data sharing, and it provides us with many performance benefits over other sharing architectures, but it must be also be cared for. We need to be concerned with activity involving the coupling facility due to the number of accesses DB2 will have to issue to the coupling facility in terms of LOCK/UNLOCK requests, physical directory reads, cache updates, and reads of buffer invalidated data in order to maintain the consistency and coherency of the shared data.

In a ideal world with more processors in the complex, the transaction rate achieved by a single DB2 would be that multiplied by the number of available processors but this is not necessarily true. Due to the requirement of additional buffer management and global locking capability, DB2 and IRLM processing costs are increased and as a result this can decrease the overall transaction rate attainable. The typical overhead seen for data sharing has been around 5-15% after the second member was defined as data sharing. As each member is added, overhead is generally low, but is very dependent on the amount of sharing among the members.

It is key for estimating data sharing performance to understand the overhead involved and also to gain an appreciation of the tuning efforts required to minimize the impact of this overhead. First of all, you must set realistic goals, define performance objects, and, most importantly, tune your current environment. Keep in mind that bad performers will become worse and new problems will surface. The key to a successful implementation is education of those involved in the migration and support of the data sharing environment. This will make the monitoring, tuning, and troubleshooting much less painful.

Note

The majority of all DB2 performance problems in a data sharing environment will be concentrated two areas—locking and buffer pools. However, many times these performance problems are still related to poor application design.


092 - Processing Costs

Chapter 9. Data Sharing

Processing Costs

Processing costs for data sharing will vary due to the degree of data sharing, locking factors, workload characteristics, hardware/software configurations, application design, physical design, and various application options. The processing costs can be controlled to some degree by application and system tuning. Data sharing costs are a function of the processing required, in addition to the normal processing, in order to have concurrency control for inter-DB2 interest and data coherency. Hardware/software costs can include speed of the processors and level of the CFCC (Coupling Facility Control Code), coupling facility structure sizes, coupling facility link configurations, level of hardware, software maintenance, and number of members in the data sharing group. Workload characteristics can include real, false, and XES (Cross System Extended Service), a set of MVS services allowing multiple instances of an application or subsystem to utilize data sharing features through the use of the coupling facility) contention, disk contention, workload dynamics, thread reuse, and application use of lock avoidance.


093 - Movement to Data Sharing

Chapter 9. Data Sharing

Movement to Data Sharing

The movement to a DB2 data sharing environment can be done generally in two different ways. A new install of DB2 can be done giving you the opportunity to start with a clean subsystem to move applications into. This also makes the monitoring of initial data sharing performance easier and at this time new naming standards could be implemented. So while a new install is less painful, it is not often practical. The other option, of course, is to migrate existing subsystems together by enabling a DB2 subsystem as a data sharing subsystem and then adding members to the group. This is much easier for the movement of very large applications and it has less of an impact on distributed processing. It is also the more common scenario for moving to a data sharing environment. The complications, of course, come with the catalog merge process and the measure of application performance as the migration occurs. Whether you decide to do a new install or an enablement/migration of existing subsystems, there will be issues as to how you can effectively measure the performance of data sharing and the impact it is having on your system. Keep in mind that not all applications belong in a data sharing environment. Some applications will still benefit from isolation.

Application Analysis

Application analysis, or selection, is the process of evaluating which applications will benefit from data sharing and belong in a data sharing environment. You will need to determine the application objectives for data sharing in order to set performance objectives. Ask such questions as:

  1. "What is the overall goal to accomplish with the implementation of data sharing?"

  2. "Are we looking to offload CPU cycles?"

  3. "Will we benefit from transaction routing capabilities?"

  4. "Is 24x7 availability the driving requirement?"

These are just a few of several questions that should be addressed in order to implement data sharing with maximum performance benefit achievement in mind.

Current Environment Evaluation

Evaluate your current DB2 environment in terms of both system and applications before moving to data sharing. Even a movement to one-way can expose missed performance problems (although only a few, because interaction with the coupling facility is minimal) and two-way can further magnify them. The time to fix known application/system performance issues is prior to any movement into the data sharing environment. Of course, these same items will still need to be investigated as workload and other factors change in the new data sharing environment. Such items to evaluate are locking activity, application commit frequency, bind parameters, use of standard rule-of-thumb recommendations, ZPARMS, maintenance schedule/hiper application, buffer pools, and recovery/restart procedures.

Migration Issues

There are many issues when migrating to a data sharing environment, but with careful planning and testing the process can be made to go very smoothly. Below are a few issues that seem to require the most planning.

Catalog Merging

Most shops merge existing subsystems. There are some pros and cons with this. The pros include easier movement of large applications and less distributed processing implications. However, the cons include complications with catalog merge process due to the fact that there is no automated way (tool) to help with this process. Depending on the number of objects and the method, this can be a laborious and error-prone process. There are naming convention issues with several objects because you will have existing names to deal with.

You do not want to merge subsystems that do not need shared data and you definitely do not want to merge test and production subsystems. You would want to merge subsystems only if they are split out because of capacity constraints, if they need common data, or if they are currently relying on distributed connections or replication to satisfy needs that could be resolved by moving to a data sharing environment. When merging subsystems, be sure to evaluate the security schemas for both subsystems and ensure that the same level of security will be in place when they are merged.

The actual migration of the catalog is not too bad (compared to the migration of the data, that is!). You first decide which catalog to migrate all other objects to, taking into consideration the items mentioned above. Query the catalog to determine which databases, table spaces, and indexes must be defined in the target system. Then, using DDL, define the objects in the target catalog. Depending on the number of objects to create DDL for, this could become a cumbersome process, especially if your DDL is not current.

Naming Conventions

The establishment of a flexible naming convention is the most important planning event in the process of migrating to a data sharing environment. When done properly, it will reduce operational and administration errors by reducing confusion and allow easy extension to the sysplex. You will want to plan names carefully because some cannot be changed at all and any changes are difficult and often error-prone.

The names need to be unique within the sysplex and there are several categories of names to be decided on, such as group level names, DB2 member level names, IRLM group and member names, subsystem data sets (DB2) and ICF catalog names. Many have to change from current naming convention (this may give your standards people a heart attack). Of course, this will depend on migration options—in particular, if you are starting a data sharing group from scratch or migrating existing systems into the group. For a complete list of items to be named refer to the DB2 UDB for OS/390 Data Sharing Planning and Administration Guide.


094 - Workload Management and Affinity Processing

Chapter 9. Data Sharing

Workload Management and Affinity Processing

One of the biggest advantages of the data sharing environment is the ability to balance your workload across the members in the group. The DB2 subsystems work very closely with the workload manager (WLM) component of OS/390 that allows us the ability to optimally balance incoming work across the subsystems in the group. WLM will manage workloads based upon predefined parameters, according to the characteristics and priorities of the work.

This allows us the ability to give more priority to OLTP transactions over long running batch queries. WLM will balance these workloads by ensuring that the long running batch queries do not consume the resources need by the online transactions. Allow the OLTP transaction to achieve the desired response times.

Deciding how to distribute the workload across the data sharing group will affect the sizing of coupling facility structures as well as have an effect upon certain aspects of hardware configurations.

Data sharing allows us to move parts of a DB2 application workload across processors in the sysplex. All of our processors in the sysplex will have efficient and direct access to the same DB2 data. It is up to you to decide how that workload is going to utilize its resources.

Data sharing allows you to move processing away from processors with capacity constraints. This is probably one of the best and quickest benefits that can be realized by an organization that is constrained and having problems completing its workload because it has simply outgrown its processor capacity.

You can have all members concurrently processing the same data by allowing a transaction to run on any member in the group or just one member accessing data. This decision will directly effect the amount of data sharing overhead you will experience due to the control of inter-DB2 read/write interest among the members to maintain coherency and concurrency. By using affinity processing, you can force an application to run on only one DB2 subsystem; maybe you could run OLTP on one subsystem, batch on another, and ad hoc on a third.


095 - Distributed Processing

Chapter 9. Data Sharing

Distributed Processing

The DB2 data sharing group configuration is transparent to SQL users and programs. When connecting, there is no awareness that a DB2 sysplex exists and the system selects which DB2 member processes the SQL request. There are a few methods for supporting this.

  • Group generic

    • By using VTAM generic resources (group generic method) the client requestor connects to the generic LU name representing all members. Then VTAM controls which DB2 member LU is selected and the workload is balanced at the client level.

  • Member specific

    • This method is the DDF sysplex support for a distributed workload distribution where a client requestor can connect to one or more real DB2 LU names and then the MVS Workload Manager tells the client which DB2 LU names should be used. The workload is balanced at the thread level and can be stopped/started on an individual member basis.

  • Hard coded

    • This support is where the client connects to a single DB2 member LU name. This way the client controls which DB2 member is selected and the workload is balanced at the client level.


096 - Sysplex Query Parallelism

Chapter 9. Data Sharing

Sysplex Query Parallelism

We have the ability to split a single query over several members using sysplex query parallelism. This is a very scalable solution for complex queries for decision support.

Sysplex query parallelism works in much the same multi-tasking way as CPU parallelism. In addition, it gives us the ability to take a complex query and run across multiple members in a data sharing group (Figure 9.6 ). Good candidate queries for sysplex query parallelism are long running, read-only queries, static and dynamic, local and remote, private and DRDA protocols, table space scan and index scan, joins (nested loop, merge scan, hybrid without sort on new table), and sorts. It is best used with isolation level UR (Uncommitted Read) to avoid excess lock propagation to the coupling facility.

Figure 9.6. Sysplex Query Parallelism

A query is issued by a coordinator, who then sends the query to the assistant members in the group. The data is then processed and returned to the coordinator, either by a work file where the coordinator will read each of the assistant's work files, or by XCF links when a work file was not necessary.

Note

There are some additional considerations with sysplex query parallelism because it will use resources in the subsystems where it runs. It will use buffer pools and work files on these subsystems. If these resources are unavailable (or not sufficient), the degree (number of members used to process the query) will be decreased for the query.


097 - Recovery Considerations

Chapter 9. Data Sharing

Recovery Considerations

Recovery is a little different in the data sharing environment and there are several considerations necessary.

Each DB2 member writes to its own recovery logs and BSDSs, sort of a striping effect. Each DB2 member must be able to read the logs and BSDSs of every other member in the group so they must be on shared DASD with appropriate access granted. The reason it must be accessible is because the logs may be required from multiple DB2s to do media recovery. A group restart will require access to all logs (discussed a bit later).

The SCA contains information about all members' logs and BSDSs, and during the backward log recovery phase of DB2 startup, each member updates the SCA with new log information. It is then read by all DB2s during a recovery. The BSDSs back up in the SCA and every member's BSDSs contain the same information, because SCA can fail due to the fact that it is a cache in the coupling facility.

Logs and Recovery

The logs from several DB2s may need to be merged for recovery. The Log Record Sequence Number (LRSN) provides common log record sequencing across members. It is used to control REDO/UNDO records for data sharing and is based off a 6-byte value derived from a sysplex timer timestamp and is store clock instruction based. The RBA is still used for non-data sharing data.

Archive logs on tape should be avoided because recovery will increase by the number of members whose archive logs must be processed. Depending on where archive logs are kept, this can become a lengthy process. Keep archive logs needed at a minimum by having large active logs, incremental copies, and frequent commits. Archiving to DASD is best, but if using tape, never archive logs for more that one DB2 in the data sharing group to the same tape.

During a recovery, DB2 accesses the logs of all DB2 subsystems in the group and merges the log records in sequence by the LRSNs. It then compares the LRSN in the log record to the LRSN on the data page; if larger–the change is applied ( Figure 9.7).

Figure 9.7. LRSNs and Recovery

Recovery Scenarios

We will take a brief look at what it takes to recover from various failure scenarios.

DASD Failure

When recovering from a DASD failure, you will execute the RECOVER utility to restore to the most recent image copy and then apply LRSN (at the time of the image copy) to the end of the logs. This will require merging the logs from all updating members and it is necessary for the logs to be on DASD. If the tape archives must be read, there must be enough available drives for all RECOVERs and there must be no deallocation delay to avoid the tapes not being accessible to other members.

DB2 Failure

When a DB2 fails, the locks are retained and the other members remain active (one of the biggest benefits of a data sharing environment). When DB2 is restarted, forward processing begins from the unit of recovery of the oldest Indoubt UR and the oldest pending write from the virtual pool to GBP. This is not as bad as non-data sharing because only the checkpoint forces data to DASD and data sharing has updated pages forced-at-commit to GBP. The retained locks are freed at the end of the subsequent restart. A DB2 subsystem can be restarted on the same or on a different MVS subsystem.

Coupling Facility Failure

If a coupling facility fails or there is a loss of connectivity to it, a dynamic rebuild of Lock or SCA structure is triggered. This rebuild will be triggered only if the thresholds of the system which lost exceed the rebuild threshold in CFRM, or this can be done via an operator command. When rebuild is caused by storage failure and rebuild fails, then a group restart is necessary. This is where all DB2 members come down and then a coordinated restart of all members is performed to rebuild SCA/Lock from the logs.

Some pages may be marked GRECP (Group Buffer Pool Recover Pending) and require page set recovery. These were pages that were in the GBP when the CF failed. This recovery requires a LOGONLY recovery when no image copy is needed. A -START DATABASE command will start recovery. The GBP checkpoint determines how far back in log to process.

Structure Failures

There are cases where you can lose a structure in the coupling facility. There are different recoveries needed for each.

  • Lock Structure

    • This structure failure is detected by all active members at the time of failure and all members initiate rebuild, but only one rebuild occurs. It will be rebuilt into a backup coupling facility. The rebuild will use the active members to rebuild locks from local information. If there is loss of connectivity to CF, the weights from the SFM will determine if the structure is rebuilt.

  • SCA Structure

    • This failure is also detected by all members and attempts are made to rebuild into a backup coupling facility. During the rebuild, the remaining active members rebuild SCA from local information. If there is loss of connectivity to CF, the weights from the SFM will determine if the structure is rebuilt.

  • Group Buffer Pool Structures

    • The GBP will be rebuilt in the backup CF and if there were pages in the GBP, they will be marked GRECP. To recover page sets marked GRECP, you can issue a -START DATABASE ( ) SPACENAM ( ) ACCESS (RO) OR ACCESS (RW). DB2 merges logs from the oldest modified page LRSN or the oldest pending write to DASD from the GBP to the end of the logs. Or you could run a RECOVER utility, LOAD REPLACE utility, or a DROP TABLE statement which also removes GRECP status. If there is loss of connectivity, DB2 will quiesce GBP dependent page sets and the result will be a SQLCODE –904 to a requesting application. Writes will then go on the LPL list and GBP dependent page sets are marked GRECP. If the GBPs are duplexed, then there is simply a switch made to the secondary GBPs.

Note

Group buffer pool duplexing became available in Version 6 (V5 APAR also). This allows you to have a primary and a secondary group buffer pool and DB2 will write to both. This would do away with having pages become GRECP and having to wait for a rebuild during a failure.


098 - Summary

Chapter 9. Data Sharing

Summary

Data sharing provides us with many benefits and is used in many shops throughout the world supporting some of the most high-volume transaction system to date. Migration to a data sharing environment is relatively easy and can be done with few or no application changes. We have some new components to become familiar with and we also have some new tuning opportunities as well.


099 - Additional Data Sharing Resources

Chapter 9. Data Sharing

Additional Data Sharing Resources

  • SG24-5637-00 OS/390 Parallel Sysplex Configuration, Volume 1: Overview.

  • SG24-5638-00 OS/390 Parallel Sysplex Configuration, Volume 2: Cookbook.

  • SG24-5639-00 OS/390 Parallel Sysplex Configuration, Volume 3: Connectivity.

  • SG24-5373-00 Parallel Sysplex Performance Health Case Study.

  • SG24-5346-00 Parallel Sysplex Continuous Availability—Case Studies.

  • SG24-5329-00 Batch Processing in a Parallel Sysplex.

  • SG24-2218-00 DB2 on MVS Platform: Data Sharing Recovery.

  • SG24-4791-00 DB2 for MVS/ESA Version 4 Data Sharing Implementation.

  • SG24-4611-00 DB2 for MVS/ESA V4 Data Sharing Performance Topics.

  • SG26-9007-00 DB2 UDB for OS/390 Data Sharing: Planning and Administration.


100 - Chapter 10. Performance Monitoring and Tuning

Part III. DB2 UDB for OS/390 and z/OS Administration

Chapter 10. Performance Monitoring and Tuning

  • Planning

  • DB2 Architecture Overview

  • Database Monitoring

  • SQL Monitoring

  • Tuning Buffer Pool and System Parameters

  • Problem Determination

Understanding the performance of the DB2 database management system, its databases, and active applications in a dynamic environment requires monitoring. This means that a database administrator should gather information regarding the usage of the database. An application programmer may also require SQL statement execution information. Gathering database information using DB2's monitoring facilities and information regarding SQL statement processing will be discussed.

In this chapter, the various DB2 facilities for the monitoring and gathering of information that is input into the tuning process will be reviewed. The EXPLAIN process, trace, commands, and other tools used to monitor DB2 database objects and SQL statements will be discussed.

These tools may be used to perform the following tasks:

  • Understand user and application activity within DB2

  • Better understand how an SQL statement is processed

  • Determine the sources and causes of problems

  • Tune buffer pool and system parameters

  • Improve database and application performance


101 - Elements of Performance

Chapter 10. Performance Monitoring and Tuning

Elements of Performance

Performance is the way a computer system behaves given a particular work load. Performance is measured through one or more of the system's response time, throughput, and availability. It is affected by:

  • The resources available

  • How well the resources are utilized

Performance tuning should be undertaken when you want to improve the cost-benefit ratio of your system. Specific situations include:

  • Processing a larger, more demanding work load without increasing processing costs that may include having to acquire additional hardware

  • Obtaining faster system response time, or higher throughput, without increasing processing costs

  • Reducing processing costs without negatively affecting service to the client(s)

Translating performance from technical terms to economic terms is difficult. Performance-tuning costs money through labor and machine resources, so the cost of tuning must be weighed against the benefits tuning may or may not deliver.

Some of these benefits, including less resource usage and the ability to add more users to the system, are tangible, whereas other benefits, such as increased customer satisfaction, are less tangible from a monetary perspective.


102 - Tuning Guidelines

Chapter 10. Performance Monitoring and Tuning

Tuning Guidelines

The following guidelines should be considered in developing an overall approach to performance tuning:

  • Remember the Law of Diminishing Returns

    • Your greatest performance benefits usually come from your initial efforts. Further changes generally produce smaller and smaller benefits and require greater effort.

  • Do Not Tune Just for the Sake of Tuning

    • Tune to relieve identified constraints. If you tune resources that are not the primary cause of performance problems, this can have little or no effect on response time until you have relieved the major constraints, and it can actually make subsequent tuning work more difficult. If there is any significant improvement potential, it lies in improving the performance of the resources that are major factors in the response time.

  • Consider the Whole System

    • You can never tune one parameter or system in isolation. Before you make any adjustments, consider how it will affect the system as a whole.

  • Change One Parameter at a Time

    • Do not change more than one performance tuning parameter at a time. Even if you are sure that all the changes will be beneficial, you will have no way of evaluating how much each change has contributed. You also cannot effectively judge the trade-off you have made by changing more than one parameter at a time. Every time you adjust a parameter to improve one area, you almost always affect at least one other area that may not have been considered.

  • Measure and Reconfigure by Levels

    • For the same reasons that you should change only one parameter at a time, tune one level of your system at a time. You can use the following list of levels within a system as a guide:

      • Hardware

      • Operating system

      • Application server and requester

      • Database

      • SQL statements

      • Application programs

  • Check for Hardware and Software Problems

    • Some performance problems may be corrected by applying service to your hardware, your software, or to both. Do not spend excessive time monitoring and tuning your system when simply applying service may be the solution to the problem.

  • Understand the Problem Before Upgrading Hardware

    • Even if it seems that an additional storage or processor resource could immediately improve performance, take the time to understand where the bottlenecks are. You may spend money on additional disk storage only to find that you do not have the processor resource to exploit it.

  • Put Fallback Procedures in Place Before You Start Tuning

    • Because changes are being made to an existing system, you must be prepared to back out those changes out if they do not have the desired effect or have a negative effect on the system.

Performance Improvement Process

Monitoring and tuning a database and its applications should be performed using the following basic process:

  • Establish performance indicators.

  • Define performance objectives.

  • Develop a performance monitoring plan.

  • Implement the plan.

  • Analyze the measurements. Determine if the objectives have been met. If so, consider reducing the number of measurements to keep to a minimum the amount of resource consumed for monitoring.

  • Determine the major constraints in the system.

  • Decide where you can afford to make trade-offs and which resources can bear an additional load. Most tuning activities involve trade-offs among system resources and various elements of performance.

  • Adjust the configuration of the system. If you think that it is feasible to change more than one tuning option, implement one at a time.

  • Based on the results, start another iteration of the monitoring cycle.

You may want to follow the above process for periodic monitoring or when significant changes occur to the system and/or work load taken on by the system.

How Much Can a System Be Tuned?

There are limits to how much you can improve the efficiency of a system. Consider how much time and money you should spend on improving system performance and how much the spending of additional time and money will help the users of the system.

Your system may perform adequately without any tuning at all, but it probably will not perform to its potential. Each database is unique. As soon as you develop your own database and applications for it, investigate the tuning parameters available and learn how you can customize their settings to reflect your situation. In some circumstances, there will be only a small benefit from tuning a system. In most circumstances, however, the benefit may be significant.

There are wizards available from the DB2 Control Center that assist in tuning the database parameters. The performance Configuration Wizard can be found by clicking the right mouse button on the database, object, or component you want to tune.

If your system encounters performance bottlenecks, it is likely that tuning will be effective. If you are close to the performance limits, and you increase the number of users on the system by about 10%, the response time may rise by much more than 10%. In this situation, you will need to determine how to counterbalance this degradation in performance by tuning your system. However, there is a point beyond which tuning cannot help. At that point, you should consider revising your goals and expectations within that environment. Or you should change your system environment by considering more disk storage, faster/additional processors, additional memory, or faster networking solutions.

A Less-Formal Approach

If you do not have enough time to set performance objectives and to monitor and tune in a comprehensive manner, you can address performance by listening to your users. Find out if they are having performance-related problems. You can usually locate the problem or determine where to start looking for the problem by asking a few simple questions. For example, you can ask your users:

  • What do you mean by slow response? Is it 10% slower than you expect it to be or ten times slower?

  • When did you notice the problem? Is it recent or has it always been there?

  • Do you know of other users who are complaining of the same problem? Are those complaining one or two individuals or a whole group?

  • Are the problems you are experiencing related to a specific transaction or application program?

  • Do your problems appear during regular periods, such as at lunch hour, or are they continuous?


103 - Access Paths and Optimization

Chapter 10. Performance Monitoring and Tuning

Access Paths and Optimization

If you want to know how a query will be executed by DB2, you must analyze its access path, which is the method for retrieving data from a specific table. The EXPLAIN facility will provide information about how DB2 accesses the data to resolve the SQL statements.

Before describing the capabilities and features of the explain facilities, you should understand at a high level how SQL statements are processed by the DB2 database engine. Each SQL statement is analyzed by DB2, and then it is determined how to process the statement during a static bind or when executed dynamically. The method used to retrieve data from tables is called the access plan.

The component within DB2 that determines the access plan to be used is known as the optimizer. During the static preparation of an SQL statement, the SQL compiler is called on to generate an access plan. The access plan contains the data access strategy including index usage, sort methods, locking semantics, and join methods.

The executable form of the SQL statement is stored in the system tables when a BIND command is executed. This is called a DBRM (Database Request Module), which can then be bound into a package. For more on packages, refer to Chapter 12.

Sometimes the complete statement is not known at application development time. In this case, the compiler is invoked during program execution to generate an access plan for the query that can be used by the database manager to access the data. Such an SQL statement is called a dynamic SQL statement. The access plans for a dynamic SQL statement are not stored in the system catalogs. They can be cached in memory in the dynamic SQL cache and will not be re-prepared if the access plans for the dynamic SQL statements already exist in the dynamic SQL cache.

EXPLAIN

EXPLAIN is a monitoring tool that produces information about the following:

  • A plan, package, or SQL statement when it is bound. The output appears in a table you create called PLAN_TABLE, which is also called a plan table. For experienced users, you can use PLAN_TABLE to give optimization hints to DB2. We will look at access path hints later in this chapter.

  • An estimated cost of executing an SQL SELECT, INSERT, UPDATE, or DELETE statement. The output appears in a table you create called DSN_STATEMNT_TABLE, which is also called a statement table.

  • User-defined functions referred to in the statement, including the specific name and schema. The output appears in a table you create called DSN_FUNCTION_TABLE, which is also called a function table.

We will look at the data contained in the three tables mentioned above, and how to use this data to improve query performance.

Gathering EXPLAIN Data

There are two different ways to populate the PLAN_TABLE:

  • Executing the SQL statement EXPLAIN

    • You can populate PLAN_TABLE by executing the SQL statement EXPLAIN. In the statement, specify a single explainable SQL statement in the FOR clause. You can execute EXPLAIN either statically from an application program, or dynamically, using QMF or SPUFI.

  • Binding with the option EXPLAIN(YES)

    • You can populate PLAN_TABLE by executing the SQL statement EXPLAIN. In the statement, specify a single explainable SQL statement in the FOR clause. You can execute EXPLAIN either statically from an application program, or dynamically, using QMF or SPUFI.

Before you can use EXPLAIN, you must create the PLAN_TABLE to hold the results of EXPLAIN. A copy of the statements needed to create the table are in the DB2 sample library, under the member name DSNTESC. (Unless you need the information they provide, it is not necessary to create a function table or statement table to use EXPLAIN.)

Note

DB2 does not automatically delete rows from the PLAN_TABLE. To clear the table of obsolete rows, use DELETE, just as you would for deleting rows from any table.

Examining EXPLAIN Data

EXPLAINs will populate this table and from there you must interpret the data and act accordingly. The following Table 10.1 shows the columns in the PLAN_TABLE.

Table 10.1. PLAN_TABLE Description

Column Name

Description

QUERYNO

A number intended to identify the statement being explained. For a row produced by an EXPLAIN statement, specify the number in the QUERYNO clause. For a row produced by non-EXPLAIN statements, specify the number using the QUERYNO clause, which is an optional part of the SELECT, INSERT, UPDATE, and DELETE statement syntax. Otherwise, DB2 assigns a number based on the line number of the SQL statement in the source program. FETCH statements do not each have an individual QUERYNO assigned to them. Instead, DB2 uses the QUERYNO of the DECLARE CURSOR statement for all corresponding FETCH statements for that cursor. When the values of QUERYNO are based on the statement number in the source program, values greater than 32,767 are reported as 0. Hence, in a very long program, the value is not guaranteed to be unique. If QUERYNO is not unique, the value of TIMESTAMP is unique.

QBLOCKNO

The position of the query in the statement being explained (1 for the outermost query, 2 for the next query, and so forth). For better performance, DB2 might merge a query block into another query block. When that happens, the position number of the merged query block will not be in QBLOCKNO.

APPLNAME

The name of the application plan for the row. Applies only to embedded EXPLAIN statements executed from a plan or to statements explained when binding a plan. Blank if not applicable.

PROGNAME

The name of the program or package containing the statement being explained. Applies only to embedded EXPLAIN statements and to statements explained as the result of binding a plan or package. Blank if not applicable.

PLANNO

The number of the step in which the query indicated in QBLOCKNO was processed. This column indicates the order in which the steps were executed.

METHOD

A number (0, 1, 2, 3, or 4) that indicates the join method used for the step:

0

First table accessed, continuation of previous table accessed, or not used.

1

Nested loop join. For each row of the present composite table, matching rows of a new table are found and joined.

2

Merge scan join. The present composite table and the new tables are scanned in the order of the join columns, and matching rows are joined.

3

Sorts needed by ORDER BY, GROUP BY, SELECT DISTINCT, UNION, a quantified predicate, or an IN predicate. This step does not access a new table.

4

Hybrid join. The current composite table is scanned in the order of the join-column rows of the new table. The new table is accessed using list prefetch.

CREATOR

The creator of the new table accessed in this step, blank if METHOD is 3.

TNAME

The name of a table, created temporary tables, declared temporary tables, materialized view, table expression, or an intermediate result table for an outer join that is accessed in this step, blank if METHOD is 3.

For an outer join, this column contains the created temporary table or the declared temporary table name of the work file in the form DSNWFQB(qblockno). Merged views show the base table names and correlation names. A materialized view is another query block with its own materialized views, tables, and so forth.

TABNO

Values are for IBM use only.

ACCESSTYPE

The method of accessing the new table:

I

By an index (identified in ACCESSCREATOR and ACCESSNAME)

I1

By a one-fetch index scan

N

By an index scan when the matching predicate contains the IN keyword

R

By a table space scan

M

By a multiple index scan (followed by MX, MI, or MU)

MX

By an index scan on the index named in ACCESSNAME

MI

By an intersection of multiple indexes

MU

By a union of multiple indexes

blank

Not applicable to the current row

MATCHCOLS

For ACCESSTYPE I, I1, N, or MX, the number of index keys used in an index scan; otherwise, 0.

ACCESSCREATOR

For ACCESSTYPE I, I1, N, or MX, the creator of the index; otherwise, blank.

ACCESSNAME

For ACCESSTYPE I, I1, N, or MX, the name of the index otherwise, blank.

INDEXONLY

Whether access to an index alone is enough to carry out the step, or whether data too must be accessed. Y=Yes; N=No

SORTN_UNIQ

Whether the new table is sorted to remove duplicate rows. Y=Yes; N=No.

SORTN_JOIN

Whether the new table is sorted for join method 2 or 4. Y=Yes; N=No.

SORTN_ORDERBY

Whether the new table is sorted for ORDER BY. Y=Yes; N=No.

SORTN_GROUPBY

Whether the new table is sorted for GROUP BY. Y=Yes; N=No.

SORTC_UNIQ

Whether the composite table is sorted to remove duplicate rows. Y=Yes; N=No.

SORTC_JOIN

Whether the composite table is sorted for join method 1, 2 or 4. Y=Yes; N=No.

SORTC_ORDERBY

Whether the composite table is sorted for an ORDER BY clause or a quantified predicate. Y=Yes; N=No.

SORTC_GROUPBY

Whether the composite table is sorted for a GROUP BY clause. Y=Yes; N=No.

TSLOCKMODE

An indication of the mode of lock to be acquired on either the new table, or its table space or table space partitions. If the isolation can be determined at bind time, the values are:

IS

Intent share lock

IX

Intent exclusive lock

S

Share lock

U

Update lock

X

Exclusive lock

SIX

Share with intent exclusive lock

N

UR isolation; no lock

If the isolation cannot be determined at bind time, then the lock mode determined by the isolation at run-time is shown by the following values.

NS

For UR isolation, no lock; for CS, RS, or RR, an S lock.

NIS

For UR isolation, no lock; for CS, RS, or RR, an IS lock.

NSS

For UR isolation, no lock; for CS or RS, an IS lock; for RR, an S lock.

SS

For UR, CS, or RS isolation, an IS lock; for RR, an S lock.

The data in this column is right justified. For example, IX appears as a blank followed by I followed by X. If the column contains a blank, then no lock is acquired.

TIMESTAMP

Usually, the time at which the row is processed, to the last .01 second. If necessary, DB2 adds .01 second to the value to ensure that rows for two successive queries have different values.

REMARKS

A field into which you can insert any character string of 254 or fewer characters.

PREFETCH

Whether data pages are to be read in advance by prefetch. S = pure sequential prefetch; L = prefetch through a page list; blank = unknown or no prefetch.

COLUMN_FN_EVAL

When an SQL column function is evaluated. R = while the data is being read from the table or index; S = while performing a sort to satisfy a GROUP BY clause; blank = after data retrieval and after any sorts.

MIXOPSEQ

The sequence number of a step in a multiple index operation.

1, 2, ...n

For the steps of the multiple index procedure (ACCESSTYPE is MX, MI, or MU.)

0

For any other rows (ACCESSTYPE is I, I1, M, N, R, or blank.)

VERSION

The version identifier for the package. Applies only to an embedded EXPLAIN statement executed from a package or to a statement that is explained when binding a package. Blank if not applicable.

COLLID

The collection ID for the package. Applies only to an embedded EXPLAIN statement executed from a package or to a statement that is explained when binding a package. Blank if not applicable.

ACCESS_DEGREE

The number of parallel tasks or operations activated by a query. This value is determined at bind time; the actual number of parallel operations used at execution time could be different. This column contains 0 if there is a host variable.

ACCESS_PGROUP_ID

The identifier of the parallel group for accessing the new table. A parallel group is a set of consecutive operations, executed in parallel, that have the same number of parallel tasks. This value is determined at bind time; it could change at execution time.

JOIN_DEGREE

The number of parallel operations or tasks used in joining the composite table with the new table. This value is determined at bind time and can be 0 if there is a host variable. The actual number of parallel operations or tasks used at execution time could be different.

JOIN_PGROUP_ID

The identifier of the parallel group for joining the composite table with the new table. This value is determined at bind time; it could change at execution time.

SORTC_PGROUP_ID

The parallel group identifier for the parallel sort of the composite table.

SORTN_PGROUP_ID

The parallel group identifier for the parallel sort of the new table.

PARALLELISM_MODE

The kind of parallelism, if any, that is used at bind time:

I

Query I/O parallelism

C

Query CP parallelism

X

Sysplex query parallelism

MERGE_JOIN_COLS

The number of columns that are joined during a merge scan join (Method=2).

CORRELATION_NAME

The correlation name of a table or view that is specified in the statement. If there is no correlation name, then the column is blank.

PAGE_RANGE

Whether the table qualifies for page range screening, so that plans scan only the partitions that are needed. Y = Yes; blank = No.

JOIN_TYPE

The type of an outer join.

F

FULL OUTER JOIN

L

LEFT OUTER JOIN

S

STAR JOIN

blank

INNER JOIN or no join

RIGHT OUTER JOIN converts to a LEFT OUTER JOIN when you use it, so that JOIN_TYPE contains L.

GROUP_MEMBER

The member name of the DB2 that executed EXPLAIN. The column is blank if the DB2 subsystem was not in a data sharing environment when EXPLAIN was executed.

IBM_SERVICE_DATA

Values are for IBM use only.

WHEN_OPTIMIZE

When the access path was determined:

blank

At bind time, using a default filter factor for any host variables, parameter markers, or special registers.

B

At bind time, using a default filter factor for any host variables, parameter markers, or special registers; however, the statement is reoptimized at run-time using input variable values for input host variables, parameter markers, or special registers. The bind option REOPT(VARS) must be specified for reoptimization to occur.

R

At run-time, using input variables for any host variables, parameter markers, or special registers. The bind option REOPT(VARS) must be specified for this to occur.

QBLOCK_TYPE

For each query block, the type of SQL operation performed. For the outermost query, it identifies the statement type. Possible values:

SELECT

SELECT

INSERT

INSERT

UPDATE

UPDATE

DELETE

DELETE

SELUPD

SELECT with FOR UPDATE OF

DELCUR

DELETE WHERE CURRENT OF CURSOR

UPDCUR

UPDATE WHERE CURRENT OF CURSOR

CORSUB

Correlated subquery

NCOSUB

Noncorrelated subquery

TABLEX

Table Expression

UNION

UNION

UNIONA

UNION ALL

BIND_TIME

The time at which the plan or package for this statement or query block was bound. For static SQL statements, this is a full-precision timestamp value. For dynamic SQL statements, this is the value contained in the TIMESTAMP column of PLAN_TABLE appended by 4 zeroes.

OPTHINT

A string that you use to identify this row as an optimization hint for DB2. DB2 uses this row as input when choosing an access path.

HINT_USED

If DB2 used one of your optimization hints, it puts the identifier for that hint (the value in OPTHINT) in this column.

PRIMARY_ACCESSTYPE

Indicates whether direct row access will be attempted first:

D

DB2 will try to use direct row access. If DB2 cannot use direct row access at runtime, it uses the access path described in the ACCESSTYPE column of PLAN_TABLE.

blank

DB2 will not try to use direct row access.

PARENT_QBLOCK

Number that indicates the QBLOCKNO of the parent query.

TABLE_TYPE

The type of new table:

F = Table Function

Q = Temporary Intermediate Result table (not materialized)

T = Table

W = Workfile

Access Path Evaluation

Here we will take a look at how to examine some of the most useful data in the PLAN_TABLE and determine what the DB2 optimizer is using to access the data.

Index Usage
Index Access (ACCESSTYPE is I, I1, N or MX)

If the column ACCESSTYPE in the plan table has one of those values, DB2 uses an index to access the table named in column TNAME. The columns ACCESSCREATOR and ACCESSNAME identify the index.

Multiple Index Access (ACCESSTYPE=M)

Those values indicate that DB2 uses a set of indexes to access a single table. A set of rows in the plan table contains information about the mul-tiple index access. The rows are numbered in column MIXOPSEQ in the order of execution of steps in the multiple index access. (If you retrieve the rows in order by MIXOPSEQ, the result is similar to postfix arithmetic notation.)

Number of Matching Index (MATCHCOLS=n)

If MATCHCOLS is 0, the access method is called a nonmatching index scan. All the index keys and their RIDs are read.

If MATCHCOLS is greater than 0, the access method is called a matching index scan: The query uses predicates that match the index columns.

In general, the matching predicates on the leading index columns are equal or IN predicates. The predicate that matches the final index column can be an equal, IN, or range predicate (<, <=, >, >=, LIKE, or BETWEEN).

The following example illustrates matching predicates:

SELECT *
FROM EMP
WHERE JOBCODE = '5'
AND SALARY > 60000
AND LOCATION ='CA';

INDEX XEMP5 on (JOBCODE,LOCATION,SALARY,AGE);

The index XEMP5 is the chosen access path for this query, with MATCHCOLS = 3. Two equal predicates are on the first two columns and a range predicate is on the third column. Though the index has four columns in the index, only three of them can be considered matching columns.

Index Only Access (INDEXONLY=Y)

In this case, the method is called index-only access. For a SELECT operation, all the columns needed for the query can be found in the index and DB2 does not access the table. For an UPDATE or DELETE operation, only the index is required to read the selected row.

Index-only access to data is not possible for any step that uses list prefetch (PREFETCH = L, or S)

Index-only access is not possible when returning varying-length data in the result set or when a VARCHAR column has a LIKE predicate, unless the VARCHAR FROM INDEX field of installation panel DSNTIP4 is set to YES and plans or packages have been rebound to pick up the change.

If access is by more than one index, INDEXONLY is Y for a step with access type MX, because the data pages are not actually accessed until all the steps for intersection (MI) or union (MU) take place.

When an SQL application uses index-only access for a ROWID column, the application claims the table space or table space partition. As a result, contention may occur between the SQL application and a utility that drains the table space or partition. Index-only access to a table for a ROWID column is not possible if the associated table space or partition is in an incompatible restrictive state. For example, an SQL application can make a read claim on the table space only if the restrictive state allows readers.

Table Access
Table Space Scans (ACCESSTYPE=R, PREFETCH=S)

Table space scan (R=Relational Scan) is most often used for one of the following reasons:

  • Access is through a temporary table. (Index access is not possible for temporary tables.)

  • A matching index scan is not possible because an index is not available or no predicates match the index columns.

  • A high percentage of the rows in the table is returned. In this case, an index is not really useful because most rows need to be read anyway.

  • The indexes that have matching predicates have low cluster ratios and are therefore efficient only for small amounts of data.

Assume that table DB2USER1.CANDIDATE has no index on CID. The following is an example that uses a table space scan:

SELECT *FROM DB2USER1.CANDIDATE WHERE CID = :CIDHV;

In this case, at least every row in the CANDIDATE table must be examined to determine whether the value of CID matches the given value.

Prefetching (PREFETCH = L, S, or blank)

Prefetching is a method of determining in advance that a set of data pages is about to be used and then reading the entire set into a buffer with a single asynchronous I/O operation. If the value of PREFETCH is:

  • S, the method is called sequential prefetch. The data pages that are read in advance are sequential. A table space scan always uses sequential prefetch. An index scan might not use it.

  • L, the method is called list prefetch. One or more indexes are used to select the RIDs for a list of data pages to be read in advance; the pages need not be sequential. Usually, the RIDs are sorted. The exception is the case of a hybrid join (METHOD=4) when the value of column SORTN_JOIN is N.

  • Blank, prefetching is not chosen as an access method. However, depending on the pattern of the page access, data can be prefetched at execution time through a process called sequential detection.

SORT (SORTN_ and SORTC_ )

The plan tables show the reasons why a sort was invoked. Those reasons could include a sort of data rows or a sort of row identifiers (RIDs) in a RID list.

SORTN_JOIN indicates that the new table of a join is sorted before the join. (For hybrid join, this is a sort of the RID list.) When SORTN_JOIN and SORTC_JOIN are both "Y", two sorts are performed for the join. The sorts for joins are indicated on the same row as the new table access.

SORTC_UNIQ indicates a sort to remove duplicates, as might be needed by a SELECT statement with DISTINCT or UNION. SORTC_ORDERBY usually indicates a sort for an ORDER BY clause. But SORTC_UNIQ and SORTC_ORDERBY also indicate when the results of a noncorrelated subquery are sorted, both to remove duplicates and to order the results. One sort does both the removal and the ordering.

A SORTC_GROUPBY would indicate a sort for processing a GROUP BY clause.

To perform list prefetch, DB2 sorts RIDs into ascending page number order. This sort is very fast and is done totally in memory. A RID sort is usually not indicated in the PLAN_TABLE, but a RID sort normally is performed whenever list prefetch is used. The only exception to this rule is when a hybrid join is performed and a single, highly clustered index is used on the inner table. In this case, SORTN_JOIN is "N", indicating that the RID list for the inner table was not sorted.

Note

SORTN_GROUPBY, SORTN_ORDERBY and SORTN_UNIQ are currently not used by DB2.

Nested Loop Join (METHOD=1)

For a Nested loop join, DB2 scans the composite (outer) table. For each row in that table that qualifies (by satisfying the predicates on that table), DB2 searches for matching rows of the new (inner) table. It concatenates any it finds with the current row of the composite table.

If no rows match the current row, then:

  • For an inner join, DB2 discards the current row.

  • For an outer join, DB2 concatenates a row of null values.

Stage 1 and stage 2 predicates eliminate unqualified rows during the join.

Nested loop join is often used if:

  • The outer table is small.

  • Predicates with small filter factors reduce the number of qualifying rows in the outer table.

  • An efficient, highly clustered index exists on the join columns of the inner table.

  • The number of data pages accessed in the inner table is small.

Note

The nested loop join repetitively scans the inner table. That is, DB2 scans the outer table once, and scans the inner table as many times as the number of qualifying rows in the outer table. Hence, the nested loop join is usually the most efficient join method when the values of the join column passed to the inner table are in sequence and the index on the join column of the inner table is clustered, or the number of rows retrieved in the inner table through the index is small.

Merge Scan Join (METHOD=2)

Merge scan join is also known as merge join or sort merge join. For this method, there must be one or more predicates of the form TABLE1.COL1 = TABLE2.COL2, where the two columns have the same data type and length attribute.

DB2 scans both tables in the order of the join columns. If no efficient indexes on the join columns provide the order, DB2 might sort the outer table, the inner table, or both. The inner table is put into a work file; the outer table is put into a work file only if it must be sorted. When a row of the outer table matches a row of the inner table, DB2 returns the combined rows.

A merge scan join is often used if:

  • The qualifying rows of the inner and outer table are large, and the join predicate does not provide much filtering, that is, in a many-to-many join.

  • The tables are large and have no indexes with matching columns.

  • Few columns are selected on inner tables. This is the case when a DB2 sort is used. The fewer the columns to be sorted, the more efficient the sort is.

Hybrid Join (METHOD=4)

The method applies only to an inner join and requires an index on the join column of the inner table. The method requires obtaining RIDs in the order needed to use list prefetch.

In the successive steps, DB2:

  1. Scans the outer table (OUTER).

  2. Joins the outer tables with RIDs from the index on the inner table. The result is the phase 1 intermediate table. The index of the inner table is scanned for every row of the outer table.

  3. Sorts the data in the outer table and the RIDs, creating a sorted RID list and the phase 2 intermediate table. The sort is indicated by a value of Y in column SORTN_JOIN of the plan table. If the index on the inner table is a clustering index, DB2 can skip this sort; the value in SORTN_JOIN is then N.

  4. Retrieves the data from the inner table, using list prefetch.

  5. Concatenates the data from the inner table and the phase 2 intermediate table to create the final composite table.

Hybrid join is often used if:

  • A nonclustered index or indexes is used on the join columns of the inner table.

  • The outer table has duplicate qualifying rows.

Star Join (METHOD = 0,1 and JOIN TYPE = S)

A star schema or star join is a logical database design that is included in decision support applications. A star schema is composed of a fact table and a number of dimension tables that are connected to it. A dimension table contains several values that are given an ID, which is used in the fact table instead of all the values. You can think of the fact table, which is much larger than the dimension tables, as being in the center surrounded by dimension tables; the result resembles a star formation.

To access the data in a star schema, you write SELECT statements that include join operations between the fact table and the dimension tables; no join operations exist between dimension tables. When the query meets the following conditions, that query is a star schema:

  • The query references at least two dimensions.

  • All join predicates are between the fact table and the dimension tables, or within tables of the same dimension.

  • All join predicates between the fact table and dimension tables are equi-join predicates.

  • All join predicates between the fact table and dimension tables are Boolean term predicates.

  • No correlated subqueries cross dimensions.

  • No single fact table column is joined to columns of different dimension tables in join predicates.

  • After DB2 simplifies join operations, no outer join operations exist.

  • The data type and length of both sides of a join predicate are the same.

  • The value of subsystem parameter STARJOIN is 1, or the cardinality of the fact table to the largest dimension table meets the requirements specified by the value of the subsystem parameter.

Parallelism Usage (PARALLELISM_MODE = I, C, or X)

Parallel processing applies only to read-only queries. The values would be:

  • I = Parallel I/O operations

  • C = Parallel CP operations

  • X = Sysplex query parallelism

Non-null values in columns ACCESS_DEGREE and JOIN_DEGREE indicate to what degree DB2 plans to use parallel operations. However, this can change at run-time. Parallelism is discussed later in this chapter.

DSN_STATEMNT_TABLE

This table is populated (if it exists) when EXPLAIN is run. Some of the columns are similar to those in the PLAN_TABLE and some are new and related only to the statement cost.

The columns that are unique to the DSN_STATEMNT_TABLE are:

  • STMT_TYPE — type of SQL statement

  • COST_CATEGORY— how much information was available

  • PROCMS— estimated processor cost in milliseconds

  • PROCSU— estimated processor cost in service units

  • REASON— reasons why COST_CATEGORY may be inaccurate

The COST_CATEGORY determination is affected by such thing as: number of tables, number of rows in the tables, column cardinality, cluster ratio, first key cardinality, full key cardinality, number of leaf pages, number of index levels, host variables, special registers, triggers, UDFs, RI, LOBS, and expressions.

The following Table 10.2 is a detailed description of what is contained in the DSN_STATEMNT table.

Table 10.2. DSN_STATEMNT Table

Column name

Description

QUERYNO

A number intended to identify the statement being explained. If QUERYNO is not unique, the value of EXPLAIN_TIME is unique.

APPLNAME

The name of the application plan for the row, or blank.

PROGNAME

The name of the program or package containing the statement being explained, or blank.

COLLID

The collection ID for the package, or blank.

GROUP_MEMBER

The member name of the DB2 that executed EXPLAIN, or blank.

EXPLAIN_TIME

The time at which the statement is processed. This time is the same as the BIND_TIME column in PLAN_TABLE.

STMT_TYPE

The type of statement being explained. Possible values are:

SELECT

SELECT

INSERT

INSERT

UPDATE

UPDATE

DELETE

DELETE

SELUPD

SELECT with FOR UPDATE OF

DELCUR

DELETE WHERE CURRENT OF CURSOR

UPDCUR

UPDATE WHERE CURRENT OF CURSOR

COST_CATEGORY

Indicates if DB2 was forced to use default values when making its estimates. Possible values:

A

Indicates that DB2 had enough information to make a cost estimate without using default values.

B

Indicates that some condition exists for which DB2 was forced to use default values. See the values in REASON to determine why DB2 was unable to put this estimate in cost category A.

PROCMS

The estimated processor cost, in milliseconds, for the SQL statement. The estimate is rounded up to the next integer value. The maximum value for this cost is 2147483647 milliseconds, which is equivalent to approximately 24.8 days. If the estimated value exceeds this maximum, the maximum value is reported.

PROCSU

The estimated processor cost, in service units, for the SQL statement. The estimate is rounded up to the next integer value. The maximum value for this cost is 2147483647 service units. If the estimated value exceeds this maximum, the maximum value is reported.

REASON

A string that indicates the reasons for putting an estimate into cost category B.

HOST VARIABLES

The statement uses host variables, parameter markers, or special registers.

TABLE CARDINALITY

The cardinality statistics are missing for one or more of the tables used in the statement.

UDF

The statement uses user-defined functions.

TRIGGERS

Triggers are defined on the target table of an INSERT, UPDATE, or DELETE statement.

REFERENTIAL CONSTRAINTS

Referential constraints of the type CASCADE or SET NULL exist on the target table of a DELETE statement.

If the query uses a function, there will be additional information in the DSN_FUNCTION_TABLE. A description of the DSN_FUNCTION_TABLE is seen in Table 10.3.

DSN_FUNCTION_TABLE

Table 10.3. DSN_FUNCTION_TABLE

Column name

Description

QUERYNO

A number intended to identify the statement being explained. If QUERYNO is not unique, the value of EXPLAIN_TIME is unique.

APPLNAME

The name of the application plan for the row, or blank.

PROGNAME

The name of the program or package containing the statement being explained, or blank.

COLLID

The collection ID for the package, or blank.

GROUP_MEMBER

The member name of the DB2 that executed EXPLAIN, or blank.

EXPLAIN_TIME

The time at which the statement is processed. This time is the same as the BIND_TIME column in PLAN_TABLE.

SCHEMA_NAME

The schema name of the function invoked in the explained statement.

FUNCTION_NAME

The name of the function invoked in the explained statement.

SPEC_FUNC_ID

The specific name of the function invoked in the explained statement.

FUNCTION_TYPE

The type of function invoked in the explained statement. Possible values are:

S

Scalar function

T

Table function

VIEW_CREATOR

If the function specified in the FUNCTION_NAME column is referenced in a view definition, the creator of the view. Otherwise, blank.

VIEW_NAME

If the function specified in the FUNCTION_NAME column is referenced in a view definition, the name of the view. Otherwise, blank.

PATH

The value of the SQL path that was used to resolve the schema name of the function.

FUNCTION_TEXT

The text of the function reference (the function name and parameters). If the function reference is over 100 bytes, this column contains the first 100 bytes. For functions specified in infix notation, FUNCTION_TEXT contains only the function name. For example, for a function named /, which overloads the SQL divide operator, if the function reference is A/B, FUNCTION_TEXT contains only /.

Access Path Hints (Optimization Hints)

DB2 provides a facility for telling DB2 how to process a query. This is done by giving DB2 hints. The process of giving hints to DB2 is relatively simple, but determining what those hints should be is not. Giving optimization hints to DB2 is useful in the following situations:

  • To ensure consistency of response times across rebinds and across release migrations

  • To temporarily bypass the access path chosen by DB2

The facility for implementing access path hints requires many tasks to be performed as well as making sure the subsystem has been enabled for hints. Some of the items that will be affected by using hints are:

  • Modifying the SQL to use the hint

  • Updating a PLAN_TABLE with the correct access path information or using an existing one

  • Changing the bind options for those affected

Even after a hint has been established, it is not always possible for the optimizer to use it. As part of the normal bind process, the optimizer will have to evaluate the hint and determine if it is valid. Processes will have to be in place to make sure that rebinds do not change the process. For this reason, hints should not be used unless all else fails in establishing an acceptable access path.

Using Statistics to Model Production Environment

There are statistics in the catalog that the optimizer uses in calculating the costs of SQL to determine the best access path to pick. In building a test environment, it is often helpful to copy statistics from a production environment, or to update the statistics to an approximation of what production might be like. The following list shows the catalog statistics that can be updated and are also used by the optimizer in access path selection.

  • SYSIBM.SYSCOLDIST

    • FREQUENCYF

    • TYPE

    • CARDF

    • COLGROUPCOLNO

    • NUMCOLUMNS

  • SYSIBM.SYSCOLUMNS

    • COLCARDF

    • HIGH2KEY

    • LOW2KEY

  • SYSIBM.SYSINDEXES

    • CLUSTERRATIO

    • FIRSTKEYCARDF

    • FULLKEYCARDF

    • NLEAF

    • NLEVELS

  • SYSIBM.SYSTABLES

    • CARDF

    • NPAGES

    • PCTROWCOMP

  • SYSIBM.SYSTABLESPACE

    • NACTIVE

  • SYSIBM.SYSTABSTATS

    • CARD

    • NPAGES

Note

There are a couple of other statistics used for access path selection that cannot be updated by the user.

Guidelines on Using EXPLAIN Output

There are a number of ways in which analyzing the EXPLAIN data can help you to tune your queries and environment. For example:

  • Are indexes being used? Creating appropriate indexes can have a significant benefit on performance. Using the EXPLAIN output, you can determine if the indexes you have created to help a specific set of queries are being used. In the EXPLAIN output, you should look for index usage in the following areas:

    • Join predicates

    • Local predicates

    • GROUP BY clauses

    • ORDER BY clauses

    • The select list

    You can also use EXPLAIN to evaluate whether a different index can be used instead of an existing index or no index at all. After creating a new index, collect statistics for that index using the RUNSTATS utility and rebind the query. Over time, you may notice, through the EXPLAIN data, that instead of an index scan, a table scan is now being used. This can result from a change in the clustering of the table data. If the index that was previously being used now has a low cluster ratio, you want to:

    • Reorganize your table to cluster the data according to that index.

    • Use the RUNSTATS command to update the catalog statistics.

    • Recompile the query (bind or rebind).

    • Reexamine the EXPLAIN output to determine whether reorganizing the table has affected the access plan.

  • Is the type of access appropriate for the application? You can analyze the EXPLAIN output and look for types of access to the data that, as a rule, are not optimal for the type of application being executed. For example:

    • Online Transaction Processing (OLTP) queries— OLTP applications are prime candidates to use index scans with range-delimiting predicates because they tend to return only a few rows that are queries using an equality predicate against a key column. If your OLTP queries are using a table scan, you may want to analyze the EXPLAIN data to determine the reasons why an index scan was not used.

    • Read-only queries— The search criteria for a read-only type query may be vague, causing a large number of rows to qualify. If the user usually looks at only a few screens of the output data, she may want to try to ensure that the entire answer set need not be computed before some results are returned. In this case, the goals of the user are different from the basic operating principle of the optimizer, which attempts to minimize resource consumption for the entire query, not just the first few screens of data.

For example, if the EXPLAIN output shows that both merge scan join and sort operators were used in the access plan, the entire answer set will be materialized in a temporary table before any rows are returned to the application. In this case, you can attempt to change the access plan by using the OPTIMIZE FOR clause on the SELECT statement. In this way, the optimizer can attempt to choose an access plan that does not produce the entire answer set in a temporary table before returning the first rows to the application.

Visual Explain

Visual Explain is a Graphical User Interface (GUI) utility that gives the database administrator or application developer the ability to examine the access plan determined by the optimizer. It provides:

  • An easy-to-understand display of a selected access path

  • Suggestions for changing an SQL statement

  • An ability to invoke EXPLAIN for dynamic SQL statements

  • An ability to provide DB2 catalog statistics for referenced objects of an access path

  • A subsystem parameter browser with keyword "Find" capabilities

Visual Explain can be used to analyze previously generated EXPLAINs or to gather EXPLAIN data and explain dynamic SQL statements. If the PLAN_TABLE has not been created when you start Visual Explain, it will create it for you. You can invoke Visual Explain from either the Command Center or Control Center.


104 - Designing Indexes for Performance

Chapter 10. Performance Monitoring and Tuning

Designing Indexes for Performance

The following sections show the different access paths provided by indexes. We should ensure that indexes are in place to provide the best access path possible for all critical queries.

Indexes for Efficient Access

DB2 uses the following index access paths:

  • Matching index scan (MATCHCOLS>0)

  • Index screening

  • Nonmatching index scan (ACCESSTYPE=I and MATCHCOLS=0)

  • IN-list index scan (ACCESSTYPE=N)

  • Multiple index access (ACCESSTYPE is M, MX, MI, or MU)

  • One-fetch access (ACCESSTYPE=I1)

  • Index-only access (INDEXONLY=Y)

  • Equal unique index (MATCHCOLS=number of index columns)

Matching Index Scan (MATCHCOLS>0)

In a matching index scan, predicates are specified on either the leading or all of the index key columns. These predicates provide filtering; only specific index pages and data pages need to be accessed. If the degree of filtering is high, the matching index scan is efficient.

Index Screening

In index screening, predicates are specified on index key columns but are not part of the matching columns. Those predicates improve the index access by reducing the number of rows that qualify while searching the index. For example, with an index on T(C1,C2,C3,C4) in the following SQL statement, C3>0 and C4=2 are index screening predicates:

SELECT *FROM T
WHERE C1 =1
AND C3 >0 AND C4 =2
AND C5 =8;
Nonmatching Index Scan (ACCESSTYPE=I and MATCHCOLS=0)

In a nonmatching index scan, no matching columns are in the index. Hence, all the index keys must be examined. Because a nonmatching index usually provides no filtering, only a few cases provide an efficient access path.

IN-List Index Scan (ACCESSTYPE=N)

An IN-list index scan is a special case of the matching index scan, in which a single indexable IN predicate is used as a matching equal predicate. You can regard the IN-list index scan as a series of matching index scans with the values in the IN predicate being used for each matching index scan. The following example has an index on (C1,C2,C3,C4) and might use an IN-list index scan:

SELECT *FROM T
WHERE C1=1 AND C2 IN (1,2,3)
AND C3>0 AND C4<100;
Multiple Index Access (ACCESSTYPE is M, MX, MI, or MU)

Multiple index access uses more than one index to access a table. It is a good access path when:

  • No single index provides efficient access.

  • A combination of index accesses provides efficient access.

RID lists are constructed for each of the indexes involved. The unions or intersections of the RID lists produce a final list of qualified RIDs that is used to retrieve the result rows, using list prefetch. You can consider multiple index access as an extension to list prefetch with more complex RID retrieval operations in its first phase. The complex operators are union and intersection. DB2 chooses multiple index access for the following query:

SELECT *FROM EMP
WHERE (AGE =34)OR
(AGE =40 AND JOB ='MANAGER');

For this query:

  • EMP is a table with columns EMPNO, EMPNAME, DEPT, JOB, AGE, and SAL.

  • EMPX1 is an index on EMP with key column AGE.

  • EMPX2 is an index on EMP with key column JOB.

One-Fetch Access (ACCESSTYPE=I1)

One-fetch index access requires retrieving only one row. It is the best possible access path and is chosen whenever it is available. It applies to a statement with a MIN or MAX column function: The order of the index allows a single row to give the result of the function.

Index-Only Access (INDEXONLY=Y)

With index-only access, the access path does not require any data pages because the access information is available in the index. Conversely, when an SQL statement requests a column that is not in the index, updates any column in the table, or deletes a row, DB2 has to access the associated data pages. Because the index is almost always smaller than the table itself, an index-only access path usually processes the data efficiently.

Equal Unique Index (MATCHCOLS=number of index columns)

An index that is fully matched and unique, and in which all matching predicates are equal-predicates, is called an equal unique index case. This case guarantees that only one row is retrieved. If there is no one-fetch index access available, this is considered the most efficient access over all other indexes that are not equal unique. (The uniqueness of an index is determined by whether or not it was defined as unique.) Sometimes DB2 can determine that an index that is not fully matching is actually an equal unique index case.

Indexes to Help Avoid Sorts

As well as providing selective access to data, indexes can also order data, sometimes eliminating the need for sorting. Some sorts can be avoided if index keys are in the order needed by ORDER BY, GROUP BY, a join operation, or DISTINCT in a column function. In other cases, as when list prefetch is used, the index does not provide useful ordering, and the selected data might have to be sorted.

When it is absolutely necessary to prevent a sort, consider creating an index on the column or columns necessary to provide that ordering. Consider also using the clause OPTIMIZE FOR 1 ROW to discourage DB2 from choosing a sort for the access path. Consider the following query:

SELECT C1,C2,C3 FROM T
WHERE C1 >1
ORDER BY C1 OPTIMIZE FOR 1 ROW;

An ascending index on C1 or an index on (C1,C2,C3) could eliminate a sort.

Not all sorts are inefficient. For example, if the index that provides ordering is not an efficient one and many rows qualify, it is possible that using another access path to retrieve and then sort the data could be more efficient than the inefficient, ordering index.

Index Advisor Facility

The Index Advisor is a management tool that provides assistance in the designing of indexes on tables. It is useful in the following situations:

  • Finding the best indexes for a problem query

  • Finding the best indexes for a set of queries (work load) subject to resource limits that are optionally applied

  • Testing out an index on a work load without having to create the index

  • There are two concepts associated with this facility: work load and virtual indexes.

A work load is a set of SQL statements (SELECT, INSERT, UPDATE, and DELETE) that DB2 has to process over a given period of time. The information in the work load is concerned with the type and frequency of the SQL statements over a given period of time. The index advisor uses this work load information in conjunction with the database information to recommend indexes. The goal of the advising engine is to minimize the total work load cost.

Virtual indexes are indexes that do not exist in the current database schema. These indexes could be either recommendations that the facility has made or indexes that are being proposed but want to model the effect using the advisor facility.

For DB2 for OS/390 and z/OS, this tool works on a set of dynamic SQL statements (only dynamic SQL is currently supported; static will be delivered in a later release). This process is performed by creating a modeling database for DB2 for OS/390 on a DB2 for UNIX, Windows, Linux workstation using configuration parameters to mimic DB2 for OS/390 and z/OS. One way to use this tool is to invoke it through the Control Center.


105 - Query Parallelism

Chapter 10. Performance Monitoring and Tuning

Query Parallelism

In order to reduce elapsed time for a query, DB2 can provide a query with parallel resources, such as several I/O paths or processors. By taking advantage of these resources, queries can run in a shorter period of time allowing more work to be pushed through the system. Parallelism can help improve the performance of I/O and CPU bound read-only queries. It can help queries that are reading large amounts of data, regardless of the filtration.

There will be some overhead associated with the use of parallelism in terms of CPU. DB2 will scale processor intensive work across all available processors. Parallelism can average less than one percent additional CPU overhead for long running queries and less than ten percent for short running queries.

I/O and CPU Parallelism

DB2 can utilize two different methods for achieving query parallelism: I/O or CPU. With I/O parallelism, the goal is to move elapsed time towards CPU time by split data access into equal sequential prefetch streams to bring I/O time down to estimated CPU time. If CPU is estimated at one second, and I/O at three seconds, the three I/O parallel streams of approximately equal size will be started and each I/O stream should cost about one second. This is implemented with a round-robin type of get paging. With current releases of DB2, I/O parallelism is infrequently chosen; instead, the preferred method is CPU parallelism.

The goal of CPU parallelism is to move elapsed time towards CPU time, by splitting queries into equal multiple smaller queries and process those queries in multiple execution units, or parallel tasks. At execution time, DB2 will take into consideration the number of CPUs available, and if there are not enough CPUs to support the degree of parallelism initially chosen by the optimizer, the degree will be degraded.

When Parallelism Is Chosen

DB2 parallelism is decided at both the time of the bind and at run-time. If parallelism is not chosen at bind time, there is no possibility of its being chosen at run-time. Even if parallelism is chosen at bind time, it may not be used at run-time due to several factors. If there is not enough space in the virtual buffer pool to support the requested degree of parallelism, the degree can be reduced from that chosen at bind time or parallelism can be turned of all together. If host variables are used in the SQL query, this can prevent DB2 from determining which partitions will qualify in a query; therefore, the degree chosen for parallelism will be decided at run-time. If the machine on which DB2 is running does not have hardware sort at run-time, parallelism will be disabled. If DB2 determines that a cursor is ambiguous, the cursor can be updated and then parallelism will be disabled. If parallelism is disabled, the query does not fail; DB2 simply uses a sequential plan for access to the data.

During BIND or PREPARE, DB2 chooses the access path best suited for the query and then does a post-optimization step where it will identify the sections of the access path that will benefit most from parallelism. It then identifies the part of the actual query that can be executed in parallel and determines the degree of parallelism to be used.

Sysplex Query Parallelism

Sysplex query parallelism works in much the same multi-tasking way as CPU parallelism; in addition, it gives us the ability to take a complex query and run across multiple members in a data sharing group. Good candidate queries for sysplex query parallelism are long running, read-only queries, static and dynamic, local and remote, private and DRDA protocols, table space scan and index scan, joins (nested loop, merge scan, hybrid without sort on new table), and sorts. It is best used with isolation level UR (Uncommitted Read) to avoid excess lock propagation.

A query is issued by a coordinator who sends the query to the assistant members in the group. The data is then processed and returned to the coordinator either by a work file where the coordinator will read each of the assistants workfiles, or by XCF links when a work file was not necessary.

Queries Best Suited for Parallelism

Queries with the following characteristics will be able to take advantage of parallelism:

  • Long running, read-only queries, both static and dynamic SQL, from both local and remote sites, and when using either private or DRDA protocols

  • Table space scans and index scans

  • Joins

  • Nested loop

  • Merge scan

  • Hybrid without sort on new tables

  • Sorts

  • Aggregate functions

There are only a few places where parallelism will not be considered:

  • Queries that use materialization of views

  • Queries that perform materialization because of nested table expressions

  • Queries performing a merge scan join of more than one column

  • Queries using direct row access

  • The below will cause only sysplex parallelism not to be considered

    • Queries with list prefetch and multiple index access

    • Queries accessing LOB data

Parallelism should not be used if a system is already CPU-constrained, because parallelism would only add to the problem in most situations.

CPU parallelism cannot be used when a cursor is defined WITH HOLD, since this cursor's use is potentially interrupted by a commit, which causes a stoppage in processing.

Short running queries are usually not going to see a great benefit from parallelism (generally a short running query is one that is sub-second). But how often are long running queries separated from short running queries? Well, if you are trying to get the benefits from parallelism and not cause unnecessary overhead where it does not belong, consideration for this type of granularity of processing will need to be done. For example, you could separate the long running queries into a separate package and BIND it DEGREE(ANY) in a different collection and then use the SET CURRENTPACKAGE statement to switch between it and a program bound with DEGREE(1) for shorter queries that are better to let run sequentially.


106 - Database Monitoring

Chapter 10. Performance Monitoring and Tuning

Database Monitoring

DB2 PM

IBM's DB2 PM is a performance analysis tool for DB2. Its primary objective is to report DB2 instrumentation data in a form that is easy to understand and analyze. DB2 PM presents this instrumentation data in the following ways:

  • The Batch report sets present the data you select in comprehensive reports or graphs containing system-wide and application-related information for both single DB2 subsystems and DB2 members of a data sharing group. You can combine instrumentation data from several different DB2 locations into one report. Batch reports can be used to examine performance problems and trends over a period of time.

  • The Online Monitor gives a current "snapshot" view of a running DB2 subsystem, including applications that are running. Its history function displays information about subsystem and application activity in the recent past.

The Statistics reports provide an excellent source of information such as buffer pools, EDM pools, RID processing, and logging, all at a subsystem level.

The Accounting reports provide information about applications for specific time periods. Those are the most popular most useful reports for on-going analysis, trend analysis, and predictive monitoring. There also some more specific reports such as the Lock Detail Analysis report for monitoring locking problems. The more detailed reports will require some additional traces to be turned on and should be used only for problem analysis and not ran on a continual basis.

Both Accounting and Statistics reports have long and short versions. Which is used will depend on the amount of detail needed for analysis. An example of a Short Accounting Report is shown in Figure 10.1.

Figure 10.1. Short Accounting Report.

There are also UTRs—User Tailored Reports that will allow you to tailor reports for more precise analysis.

The on-line portion of the DB2PM product has a very user-friendly GUI interface. Figure 10.2 shows a picture of main screen of the DB2PM GUI interface. It has a look and feel similar to the DB2 Control Center. From here you can monitor all of your DB2 subsystems and your data sharing environment. Figure 10.3 gives you a look at a detail screen.

Figure 10.2. DB2PM GUI—Main Screen.

Figure 10.3. DB2PM GUI—Locking Detail Screen.

Resource Limit Facility

DB2 's resource limit facility (governor) lets you perform the following activities:

  • Set warning and error thresholds by which the governor can inform users (via your application programs) that a certain processing limit might be exceeded for a particular dynamic SELECT, INSERT, UPDATE, or DELETE statement. This is called predictive governing.

  • Stop a currently executing dynamic SQL statement (SELECT, INSERT, UPDATE, or DELETE) that exceeds the processor limit that you have specified for that statement. This is sometimes called reactive governing to differentiate its function from that of predictive governing, a function that is also handled by the resource limit facility. The resource limit facility does not control static SQL statements whether or not they are executed locally or remotely.

  • Restrict bind and rebind activities to avoid performance impacts on production data.

  • Restrict particular parallelism modes for dynamic queries.


107 - Tracing Problems in DB2

Chapter 10. Performance Monitoring and Tuning

Tracing Problems in DB2

These traces can be used for problem determination:

  • DB2 trace

  • IMS attachment facility trace

  • CICS trace

  • Three TSO attachment facility traces

  • CAF trace stream

  • OS/390 RRS trace stream

  • MVS component trace used for IRLM

In this section, we will look at DB2 traces. For information about other trace capabilities, refer to the appropriate product documentation.

DB2 Traces

DB2 trace allows you to trace and record subsystem data and events. There are five different types of trace. DB2 trace can record six types of data: statistics, accounting, audit, performance, monitor, and global. Table 10.4 later in this chapter indicates which IFCIDs are activated for the different types of trace and the classes within those trace types and the information each IFCID returns.

The trace records are written using GTF or SMF records. We will take a look at the types of data DB2 collects for each trace class.

Statistics

The data collected in the statistics trace allows you to conduct DB2 capacity planning and to tune the entire set of DB2 programs. The statistics trace reports information about how much the DB2 system services and database services are used. It is a system-wide trace and should not be used for chargeback accounting. Use the information the statistics trace provides to plan DB2 capacity, or to tune the entire set of active DB2 programs.

Statistics trace classes 1,3,4, and 5 are the default classes for the statistics trace if statistics is specified YES in panel DSNTIPN. If the statistics trace is started using the START TRACE command, then class 1 is the default class.

  • Class 1 provides information about system services and database statistics. It also includes the system parameters that were in effect when the trace was started.

  • Class 3 provides information about deadlocks and timeouts.

  • Class 4 provides information about exceptional conditions.

  • Class 5 provides information about data sharing.

If you specified YES in the SMF STATISTICS field on the Tracing Panel (DSNTIPN), the statistics trace starts automatically when you start DB2, sending classes 1,3,4, and 5 statistics data to SMF. SMF records statistics data in both SMF type 100 and 102 records. IFCIDs 0001,0002,0202,and 0230 are of SMF type 100. All other IFCIDs in statistics trace classes are of SMF type 102. From panel DSNTIPN, you can also control the statistics collection interval (STATISTICS TIME field).

The statistics trace is written on an interval basis, and you can control the exact time that statistics traces are taken.

Accounting

Accounting class 2 data allows you to assign DB2 costs to individual authorization IDs and to tune individual programs. The DB2 accounting trace provides information related to application programs, including such things as:

  • Start and stop times

  • Number of commits and aborts

  • The number of times certain SQL statements are issued

  • Number of buffer pool requests

  • Counts of certain locking events

  • Processor resources consumed

  • Thread wait times for various events

  • RID pool processing

  • Distributed processing

  • Resource limit facility statistics

Accounting times are usually the prime indicator of a performance problem and most often should be the starting point for analysis. DB2 times are classified as follows:

  • Class 1

    • This time shows the time the application spent since connecting to DB2, including time spent outside DB2.

  • Class 2

    • This shows the elapsed time spent in DB2. It is divided into CPU time and waiting time.

  • Class 3

    • This elapsed time is divided into various waits, such as the duration of suspensions due to waits for locks and latches or waits for I/O.

DB2 trace begins collecting this data at successful thread allocation to DB2, and writes a completed record when the thread terminates or when the authorization ID changes.

Accounting data for class 1 (the default) is accumulated by several DB2 components during normal execution. This data is then collected at the end of the accounting period; it does not involve as much overhead as individual event tracing.

On the other hand, when you start classes 2,3,7, or 8, many additional trace points are activated. Every occurrence of these events is traced internally by DB2 trace, but these traces are not written to any external destination. Rather, the accounting facility uses these traces to compute the additional total statistics that appear in the accounting record, IFCID 003,when class 2 or class 3 is activated. Accounting class 1 must be active to externalize the information.

To turn on accounting for packages and DBRMs, accounting trace classes 1 and 7 must be active. Though you can turn on class 7 while a plan is being executed, accounting trace information is gathered only for packages or DBRMs executed after class 7 is activated. Activate accounting trace class 8 with class 1 to collect information about the amount of time an agent was suspended in DB2 for each executed package. If accounting trace classes 2 and 3 are activated, there is minimal additional performance cost for activating accounting trace classes 7 and 8.

If you want information from either, or both, accounting classes 2 and 3, be sure to activate classes 2 and/or 3 before your application starts. If these classes are activated during the application, the times gathered by DB2 trace are only from the time the class was activated.

Accounting trace class 5 provides information on the amount of elapsed time and TCB time that an agent spent in DB2 processing instrumentation facility interface (IFI) requests. If an agent did not issue any IFI requests, these fields are not included in the accounting record.

If you specified YES for SMF ACCOUNTING on the Tracing Panel (DSNTIPN), the accounting trace starts automatically when you start DB2, and sends IFCIDs that are of SMF type 100 to SMF. The accounting record IFCID 0003 is of SMF type101.

Performance

The performance trace provides information about a variety of DB2 events, including events related to distributed data processing. You can use this information to further identify a suspected problem, or to tune DB2 programs and resources for individual users or for DB2 as a whole.

To trace performance data, you must use the -START TRACE(PERFM) command. Performance traces cannot be automatically started.

Audit

This trace provides data that can be used to monitor DB2 security and access to data to ensure that data access is allowed only for authorized purposes. On the CREATE TABLE or ALTER TABLE statements, you can specify whether or not a table is to be audited, and in what manner; you can also audit security information such as any access denials, grants, or revokes for the table. The default causes no auditing to take place. If you specified YES for AUDIT TRACE on the Tracing Panel (DSNTIPN), audit trace class 1 starts automatically when you start DB2. By default, DB2 will send audit data to SMF. For more information on auditing, refer to Chapter 3, Controlling Data Access.

Monitor

The monitor trace records data for online monitoring with user-written programs. This trace type has several predefined classes; those that are used explicitly for monitoring are listed here:

  • Class 1 (the default) allows any application program to issue an instrumentation facility interface (IFI)READS request to the IFI facility. If monitor class 1 is inactive, a READS request is denied. Activating class 1 has a minimal impact on performance.

  • Class 2 collects processor and elapsed time information. The information can be obtained by issuing a READS request for IFCID 0147 or 0148. In addition, monitor trace class 2 information is available in the accounting record, IFCID 0003. Monitor class 2 is equivalent to accounting class 2 and results in equivalent overhead. Monitor class 2 times appear in IFCIDs 0147,0148,and 0003 if either monitor trace class 2 or accounting class 2 is active.

  • Class 3 activates DB2 wait timing and saves information about the resource causing the wait. The information can be obtained by issuing a READS request for IFCID 0147 or 0148. In addition, monitor trace class 3 information is available in the accounting record, IFCID 0003. As with monitor class 2, monitor class 3 overhead is equivalent to accounting class 3 overhead. When monitor trace class 3 is active, DB2 can calculate the duration of a class 3 event, such as when an agent is suspended due to an unavailable lock. Monitor class 3 times appear in IFCIDs 0147,0148, and 0003, if either monitor class 3 or accounting class 3 is active.

  • Class 5 traces the amount of time spent processing IFI requests.

  • Class 7 traces the amount of time an agent spent in DB2 to process each package. If monitor trace class 2 is active, activating class 7 has minimal performance impact.

  • Class 8 traces the amount of time an agent was suspended in DB2 for each package executed. If monitor trace class 3 is active, activating class 8 has minimal impact.

Invoking Traces

Traces can be started automatically via the DB2 install panels, or they can be stopped and started dynamically by using the TRACE commands. The trace commands include:

  • START TRACE

    • Invokes one or more different types of trace.

  • DISPLAY TRACE

    • Displays the trace options that are in effect.

  • STOP TRACE

    • Stops any trace that was started by either the START TRACE command or the parameters specified when installing or migrating.

  • MODIFY TRACE

    • Changes the trace events (IFCIDs) being traced for a specified active trace. Several parameters can be specified to further qualify the scope of a trace. Specific events within a trace type can be traced as well as events within specific DB2 plans, authorization IDs, resource manager IDs and location. The destination to which trace data is sent can also be controlled.

Trace Classes

It is important to have the appropriate classes always gathering information about your DB2 subsystem and its activity. It is a general recommendation that you have SMF Accounting Class 1,2 and 3, and SMF Statistics Class 1,3, and 4 (1 and 3 at a minimum) selected during normal execution. Any other trace classes should not be running at all times due to the fact that they due causes excessive overhead if run for long periods of time.

When executing other traces, it is wise to time the trace to only the IFCIDs necessary for the appropriate performance analysis or problem diagnosis. These traces also should only be run for short periods of time. The example below shows an example of limiting a trace.

-START TRACE(PERFTC1) CLASS(8) IFCID(0221) PLANNAME(CERTPLA)

In the above example, this limits a performance trace to class 8 with IFCID 221 which is used to view the actual degree of parallelism at run time.

IFCIDs

Table 10.4 shows the various IFCIDs that are started for each different type of class within a trace.

Accounting

Table 10.4. Traces classes and IFCIDs

Class

IFCID

Description

1

3

ALL ACCOUNTING

106

SYSTEM PARAMETERS IN EFFECT

239

OVERFLOW FOR PACKAGE ACCOUNTING

2

200

UDF ENTRY/EXIT SIGNAL

232

DB2 THREAD ENTRY/EXIT SIGNAL

3

6

BEGINNING OF A READ I/O OPERATION

7

CC AFTER READ I/O OPERATION

8

BEGINNING OF SYNCHRONOUS WRITE I/O

9

CC OF SYNC OR ASYNC WRITE I/O

32

BEGIN OF WAIT FOR LOG MANAGER

33

END OF WAIT FOR LOG MANAGER

44

LOCK SUSPEND OR IDENTIFY CALL IRLM

45

LOCK RESUME

51

SHARED LATCH RESUME. SERVICEABILITY

52

SHARED LATCH WAIT. SERVICEABILITY

56

EXCL. LATCH WAIT. SERVICEABILITY

57

EXCL. LATCH RESUME. SERVICEABILITY

117

BEGIN THREAD WAIT TIME FOR LOG I/O

118

END THREAD WAIT TIME FOR LOG I/O

127

AGENT READY TO SUSPEND PAGE WAIT

128

PAGE REQUESTOR RESUMED BY I/O INIT.

170

SUSPEND FOR SYNC EXEC.N UNIT SWITCH

171

RESUME AGENT WAITING DB2 SERV. TASK

174

BEGIN ARCHIVE LOG MODE(QUIESCE)

175

END ARCHIVE LOG MODE(QUIESCE)

213

BEGIN OF WAIT FOR CLAIM REQUEST

214

END OF WAIT FOR CLAIM REQUEST

215

BEGIN OF WAIT FOR DRAIN REQUEST

216

END OF WAIT FOR DRAIN REQUEST

226

BEGIN OF SUSPEND FOR PAGE LATCH

227

END OF SUSPEND FOR PAGE LATCH

242

BEGIN WAIT FOR SCHED. STORED PROC.

243

END WAIT FOR SCHED. STORED PROC.

313

MESSAGES FOR LONG-RUNNING URS.

4

151

USER-DEFINED ACCOUNTING TRACE

5

187

ENTRY TO AND EXIT FROM IFI

7

232

DB2 THREAD ENTRY/EXIT SIGNAL

232

FOR PACKAGE/DBRM LEVEL ACCOUNTING

240

EVENT SIGNAL FOR PACKAGE ACCOUNTING

8

6

BEGINNING OF A READ I/O OPERATION

7

CC AFTER READ I/O OPERATION

8

BEGINNING OF SYNCHRONOUS WRITE I/O

9

CC OF SYNC OR ASYNC WRITE I/O

32

BEGIN OF WAIT FOR LOG MANAGER

33

END OF WAIT FOR LOG MANAGER

44

LOCK SUSPEND OR IDENTIFY CALL IRLM

45

LOCK RESUME

51

SHARED LATCH RESUME. SERVICEABILITY

52

SHARED LATCH WAIT. SERVICEABILITY

56

EXCL. LATCH WAIT. SERVICEABILITY

57

EXCL. LATCH RESUME. SERVICEABILITY

117

BEGIN THREAD WAIT TIME FOR LOG I/O

118

END THREAD WAIT TIME FOR LOG I/O

127

AGENT READY TO SUSPEND PAGE WAIT

128

PAGE REQUESTOR RESUMED BY I/O INIT.

170

SUSPEND FOR SYNC EXEC.N UNIT SWITCH

171

RESUME AGENT WAITING DB2 SERV. TASK

174

BEGIN ARCHIVE LOG MODE(QUIESCE)

175

END ARCHIVE LOG MODE(QUIESCE)

213

BEGIN OF WAIT FOR CLAIM REQUEST

214

END OF WAIT FOR CLAIM REQUEST

215

BEGIN OF WAIT FOR DRAIN REQUEST

216

END OF WAIT FOR DRAIN REQUEST

226

BEGIN OF SUSPEND FOR PAGE LATCH

227

END OF SUSPEND FOR PAGE LATCH

241

BEGIN/END SUSPENSION OF PACK/DBRM

242

BEGIN WAIT FOR SCHED. STORED PROC.

243

END WAIT FOR SCHED. STORED PROC.

Audit

Class

IFCID

Description

1

140

AUTHORIZATION FAILURES

2

141

EXPLICIT GRANT AND REVOKES

3

142

CREATES, ALTERS, DROPS—AUDIT

4

143

FIRST ATTEMPTED WRITE AUDITED OBJ.

5

144

FIRST ATTEMPTED READ AUDITED OBJ.

6

145

AUDIT LOG RECORD OF SOME SQL STMTS

7

55

ISSUANCE OF SET CURRENT SQLID

83

END IDENTIFY REQUEST

87

ENDING OF SIGNON REQUEST

169

DISTRIBUTED AUTHID TRANSLATION

312

DCE SECURITY

8

23

UTILITY START INFORMATION

24

UTILITY OBJECT OR PHASE CHANGE

25

UTILITY END INFORMATION

9

146

USER-DEFINED AUDIT TRACE

Monitor

Class

IFCID

Description

1

1

SYSTEM SERVICES

2

DATABASE SERVICES

106

SYSTEM PARAMETERS IN EFFECT

124

CURRENT SQL STATEMENT

129

VSAM CI'S — DB2 RECOVER LOG

147

SUMMARY THREAD STATUS RECORD

148

DETAILED THREAD STATUS RECORD

149

LOCK INFORMATION FOR A RESOURCE

150

LOCK INFORMATION FOR AN AGENT IFCID

202

SYSTEM PARAMETERS

230

DATA SHARING GLOBAL STATISTICS

254

GROUP BUFFER POOL USAGE

306

LOG RECORD RETRIEVAL

316

PREPARED STMT. CACHE STATISTICS

317

PREPARED STMT. CACHE STMT. TEXT

2

232

DB2 THREAD ENTRY EXIT SIGNAL

3

6

BEGINNING OF A READ I/O OPERATION

7

CC AFTER READ I/O OPERATION

8

BEGINNING OF SYNCHRONOUS WRITE I/O

9

CC OF SYN OR ASYNC WRITE I/O

32

BEGIN OF WAIT FOR LOG MANAGER

33

END OF WAIT FOR LOG MANAGER

44

LOCK SUSPEND OR IDENTIFY CALL IRLM

45

LOCK RESUME

51

SHARED LATCH RESUME. SERVICEABILITY

52

SHARED LATCH WAIT. SERVICEABILITY

56

EXCL. LATCH WAIT. SERVICEABILITY

57

EXCL. LATCH RESUME. SERVICEABILITY

117

BEGIN THREAD WAIT TIME FOR LOG I/O

118

END THREAD WAIT TIME FOR LOG I/O

127

AGENT READY TO SUSPEND PAGE WAIT

128

PAGE REQUESTOR RESUMED BY I/O INIT.

170

SUSPEND FOR SYNC EXEC. UNIT SWITCH

171

RESUME AGENT WAITING DB2 SERV. TSK

174

BEGIN ARCHIVE LOG MODE(QUIESCE)

175

END ARCHIVE LOG MODE(QUIESCE)

213

BEGIN OF WAIT FOR CLAIM REQUEST

214

END OF WAIT FOR CLAIM REQUEST

215

BEGIN OF WAIT FOR DRAIN REQUEST

216

END OF WAIT FOR DRAIN REQUEST

226

BEGIN OF SUSPEND FOR PAGE LATCH

227

END OF SUSPEND FOR PAGE LATCH

242

BEGIN WAIT FOR SCHED. STORED PROC.

243

END WAIT FOR SCHED. STORED PROC.

4

155

USER-DEFINED MONITOR TRACE

5

187

ENTRY OR EXIT TO IFI

6

185

DATA CAPTURE INFORMATION

7

232

DB2 THREAD ENTRY/EXIT SIGNAL

232

FOR PACKAGE/DBRM LEVEL ACCOUNTING

240

EVENT SIGNAL FOR PACKAGE ACCOUNTING

8

6

BEGINNING OF A READ I/O OPERATION

7

CC AFTER READ I/O OPERATION

8

BEGINNING OF SYNCHRONOUS WRITE I/O

9

CC OF SYN OR ASYNC WRITE I/O

32

BEGIN OF WAIT FOR LOG MANAGER

33

END OF WAIT FOR LOG MANAGER

44

LOCK SUSPEND OR IDENTIFY CALL IRLM

45

LOCK RESUME

51

SHARED LATCH RESUME. SERVICEABILITY

52

SHARED LATCH WAIT. SERVICEABILITY

56

EXCL. LATCH WAIT. SERVICEABILITY

57

EXCL. LATCH RESUME. SERVICEABILITY

117

BEGIN THREAD WAIT TIME FOR LOG I/O

118

END THREAD WAIT TIME FOR LOG I/O

127

AGENT READY TO SUSPEND PAGE WAIT

128

PAGE REQUESTOR RESUMED BY I/O INIT.

170

SUSPEND FOR SYNC EXEC. UNIT SWITCH

171

RESUME AGENT WAITING DB2 SERV. TSK

174

BEGIN ARCHIVE LOG MODE(QUIESCE)

175

END ARCHIVE LOG MODE(QUIESCE)

213

BEGIN OF WAIT FOR CLAIM REQUEST

214

END OF WAIT FOR CLAIM REQUEST

215

BEGIN OF WAIT FOR DRAIN REQUEST

216

END OF WAIT FOR DRAIN REQUEST

226

BEGIN OF SUSPEND FOR PAGE LATCH

227

END OF SUSPEND FOR PAGE LATCH

241

BEGIN/END SUSPENSION OF PACK/DBRM

242

BEGIN WAIT FOR SCHED. STORED PROC.

243

END WAIT FOR SCHED. STORED PROC.

Performance

Class

IFCID

Description

1

1

SYSTEM SERVICES

2

DATABASE SERVICES

31

EDM POOL FULL CONDITION

42

A CHECKPOINT STARTED

43

A CHECKPOINT ENDED

76

BEGINNING OF END OF MEMORY REQUEST

77

ENDING OF AN END OF MEMORY REQUEST

78

BEGINNING OF AN END OF TASK REQUEST

79

ENDING OF AN END OF TASK REQUEST

102

DETECTION OF SHORT ON STORAGE

103

SETTING OFF OF SHORT ON STORAGE

105

INTERNAL DBID OBID TO DB/TS

106

SYSTEM PARAMETERS IN EFFECT

107

DATA SET OPEN/CLOSE INFORMATION

153

USER-DEFINED EXCEPT-CONDITION TRACE

2

3

ALL ACCOUNTING

68

BEGINNING OF A ROLLBACK REQUEST

69

ENDING OF A ROLLBACK REQUEST

70

BEGIN COMMIT PHASE 2 REQUEST

71

END COMMIT PHASE 2 REQUEST

72

BEGINNING OF CREATE THREAD REQUEST

73

ENDING OF A CREATE THREAD REQUEST

74

BEGINNING OF TERM. THREAD REQUEST

75

ENDING OF A TERM. THREAD REQUEST

80

BEGINNING OF AN ESTABLISH EXIT REQ.

81

ENDING OF AN ESTABLISH EXIT REQUEST

82

BEGIN IDENTIFY REQUEST

83

END IDENTIFY REQUEST

84

BEGIN PHASE 1 COMMIT REQUEST

85

END PHASE 1 COMMIT REQUEST

86

BEGINNING OF SIGNON REQUEST

87

ENDING OF SIGNON REQUEST

88

BEGINNING OF A SYNC REQUEST

89

ENDING OF A SYNC REQUEST

106

SYSTEM PARAMETERS IN EFFECT

174

BEGIN ARCHIVE LOG MODE(QUIESCE)

175

END ARCHIVE LOG MODE(QUIESCE)

3

22

MINIPLANS GENERATED

53

END OF DESCR., COMMIT, RLBCK OR ERR

55

ISSUANCE OF SET CURRENT SQLID

58

END OF SQL STATEMENT EXECUTION

59

START OF FETCH SQL STATEMENT EXEC.

60

START OF SELECT SQL STATEMENT EXEC.

61

START OF INSERT, UPDATE, DELETE SQL

62

START OF DDL STATEMENT EXECUTION

63

SQL STATEMENT TO BE PARSED

64

START PREPARE SQL STATEMENT EXEC.

65

START OPEN CURSOR STATIC/DYN SQL

66

START CLOSE CURSOR STATIC/DYN SQL

92

START AN ACCESS METHOD SERVICES

95

SORT STARTED

96

SORT ENDED

97

ACCESS METHOD SERVICES CMD COMPL.

106

SYSTEM PARAMETERS IN EFFECT

112

ATTRIBUTES PLAN AFTER THREAD ALLOC.

177

SUCCESSFUL PACKAGE ALLOCATION

233

START/END CALL TO USER ROUTINE

237

SET CURRENT DEGREE INFORMATION

272

ASSOCIATE LOCATORS INFORMATION

273

ALLOCATE CURSOR INFORMATION

324

FUNCTION RESOLUTION INFORMATION

325

START/END TRIGGER ACTIVATION

4

6

BEGINNING OF A READ I/O OPERATION

7

COMPLETION CODE AFTER READ I/O

8

BEGINNING OF SYNCHRONOUS WRITE I/O

9

CC OF SYN OR ASYNC WRITE I/O

10

BEGINNING OF ASYNC WRITE I/O

29

START EDM I/O REQ. LOAD DBD OR CT

30

END OF EDM I/O REQUEST

105

INTERNAL DBID OBID TO DB/TS

106

SYSTEM PARAMETERS IN EFFECT

107

DATA SET OPEN/CLOSE INFORMATION

127

AGENT READY TO SUSPEND PAGE WAIT

128

PAGE REQUESTOR RESUMED BY I/O INIT.

226

BEGIN OF SUSPEND FOR PAGE LATCH

227

END OF SUSPEND FOR PAGE LATCH

5

32

BEGIN OF WAIT FOR LOG MANAGER

33

END OF WAIT FOR LOG MANAGER

34

LOG MANAGER WAIT FOR READ I/O BEGIN

35

LOG MANAGER WAIT FOR READ I/O END

36

LOG MANAGER WAIT FOR NON-I/O BEGIN

37

LOG MANAGER WAIT FOR NON-I/O END

38

LOG MGR WAIT ACT. LOG WRITE BEGIN

39

LOG MGR WAIT ACT. LOG WRITE I/O END

40

LOG MANAGER ARCHIVE WRITE I/O BEGIN

41

LOG MANAGER ARCHIVE WRITE I/O END

104

LOG DATA SET MAPPING

106

SYSTEM PARAMETERS IN EFFECT

114

START ARCHIVE READ I/O WAIT

115

END READ ARCHIVE I/O WAIT ON DASD

116

END READ ARCHIVE I/O WAIT ON TAPE

117

BEGIN ARCHIVE READ

118

END ARCHIVE READ

119

BSDS WRITE I/O BEGINNING

120

BSDS WRITE I/O END

228

START ARCHIVE ALLOCATION WAIT

229

END ARCHIVE ALLOCATION WAIT

6

20

LOCKING SUMMARY

44

LOCK SUSPEND OR AN ID. CALL TO IRLM

45

LOCK RESUME

105

INTERNAL DBID OBID TO DB/TS

106

SYSTEM PARAMETERS IN EFFECT

107

DATA SET OPEN/CLOSE INFORMATION

172

UNITS OF WORK INVOLVED IN DEADLOCK

196

LOCK TIMEOUT DETAILS

213

BEGINNING OF WAIT FOR DRAIN LOCK

214

END OF WAIT FOR DRAIN LOCK

218

SUMMARY OF LOCK AVOIDANCE TECHNIQUE

7

21

DETAIL LOCK REQ.ON RETURN FROM IRLM

105

INTERNAL DBID OBID TO DB/TS

106

SYSTEM PARAMETERS IN EFFECT

107

DATA SET OPEN/CLOSE INFORMATION

199

BUFFER POOL DATA SET STATISTICS

223

DETAIL OF LOCK AVOIDANCE TECHNIQUE

8

13

INPUT TO HASH SCAN

14

END OF HASH SCAN

15

INPUT MATCH./NON-MATCH.INDEX SCAN

16

INPUT TO THE FIRST INSERT

17

INPUT TO SEQUENTIAL SCAN

18

END INDEX SCAN, INSERT, SEQ. SCAN

105

INTERNAL DBID OBID TO DB/TS

106

SYSTEM PARAMETERS IN EFFECT

107

DATA SET OPEN/CLOSE INFORMATION

125

RID LIST PROCESSING USAGE

221

PARALLEL DEGREE FOR PARALLEL GROUP

222

PARALLEL GROUP ELAPSED TIME

231

PARALLEL GROUP COMPLETION

305

TABLE CHECK CONSTRAINTS

311

TEMPORARY TABLES

9

26

WORK FILE OBTAINED FOR SORT

27

NUMBER OF ORDERED RECORDS SORT RUN

28

DETAILED SORT INFORMATION

95

SORT STARTED

96

SORT ENDED

106

SYSTEM PARAMETERS IN EFFECT

10

23

UTILITY START INFORMATION

24

UTILITY OBJECT OR PHASE CHANGE

25

UTILITY END INFORMATION

90

COMMAND TEXT OF ENTERED DB2 COMMAND

91

COMPLETION STATUS OF A DB2 COMMAND

105

INTERNAL DBID OBID TO DB/TS

106

SYSTEM PARAMETERS IN EFFECT

107

DATA SET OPEN/CLOSE INFORMATION

108

BEGINNING OF BIND/REBIND

109

END OF BIND/REBIND

110

BEGINNING OF FREE PLAN

111

END OF FREE PLAN

201

STATUS BEF/AFT ALTER BUFFERPOOL

256

ATTRIBUTES BEF/AFT ALTER BUFFERPOOL

11

46

AGENT BEGIN EXEC. UNIT SWITCH

47

NEW SRB EXECUTION UNIT STARTED

48

NEW SRB EXECUTION UNIT COMPLETED

49

BEGIN NEW TCB

50

END NEW TCB

51

SHARED LATCH RESUME

52

SHARED LATCH WAIT

56

EXCLUSIVE LATCH WAIT

57

EXCLUSIVE LATCH RESUME

93

SUSPEND WAS CALLED

94

EVENT RESUMED

106

SYSTEM PARAMETERS IN EFFECT

113

ATTRIBUTES PLAN AFTER AGENT ALLOC.

12

98

BEGIN GETMAIN/FREEMAIN (NONPOOL)

99

END GETMAIN/FREEMAIN (NONPOOL)

100

BEGIN GETMAIN/FREEMAIN (POOL)

101

END GETMAIN/FREEMAIN (POOL)

106

SYSTEM PARAMETERS IN EFFECT

13

11

RESULTS OF A VALIDATION EXIT CALL

12

RESULTS EDIT EXIT CALL ENCODE RECRD

19

RESULTS EDIT EXIT CALL DECODE A ROW

105

INTERNAL DBID OBID TO DB/TS

106

SYSTEM PARAMETERS IN EFFECT

107

DATA SET OPEN/CLOSE INFORMATION

14

67

START OF ACCOUNTING COLLECTION

106

SYSTEM PARAMETERS IN EFFECT

121

ENTRY ALLOCATING DB2 CONNECTION

122

EXIT ALLOCATING DB2 CONNECTION

15

154

USER-DEFINED ROUTINE COND. PERF.

16

157

DRDS INTER. WITH RDS RDI CALL TYPES

158

DRDS INTER. WITH CONVERSATION MGR

159

DRDS REQUESTING LOCATION DATA

160

REQUESTING AGENT DATA

161

SERVING AGENT DATA

162

DISTRIB TRANS. MGR REQ. AGENT DATA

163

DISTRIB TRANS. MGR RESP. AGENT DATA

167

CONVERSATION ALLOC. REQUEST QUEUED

183

DRDS RDS/SCC INTERFACE DATA

17

211

INFORMATION ABOUT CLAIMS

212

INFORMATION ABOUT DRAINS

213

BEGINNING OF WAIT FOR DRAIN LOCK

214

END OF WAIT FOR DRAIN LOCK

215

BEGIN OF WAIT OF CLAIM COUNT TO 0

216

END OF CLAIM COUNT TO GO TO 0

20

249

EDM POOL DBD INVALIDATAION

250

GROUP BUFFER POOL CON/DISCON

251

P-LOCK OPERATIONS

256

ALTER BUFFER POOL COMMAND

257

DETAILS OF IRLM NOTIFY REQUEST

261

GROUP BUFFER POOL CHECKPOINT

262

GBPOOLT CASTOUT THRESHOLD PROCESSIN

267

BEGIN CF STRUCT REBLD/EXPAND/CONTR

268

END CF STRUCTURE REBLD/EXPAND/CONTR

21

255

BUFFER REFRESH DUE TO XI

259

P-LOCK REQUEST/NEGOTIATION REQUEST

263

PAGE SET AND PARTITION CASTOUT DETA

314

AUTHORIZATION EXIT PARAMETERS

327

LANGUAGE ENVIRONMENT RUN-TIME INFO

Statistics

Class

IFCID

Description

1

1

SYSTEM SERVICES

2

DATABASE SERVICES

105

INTERNAL DBID OBID TO DB/TS

106

SYSTEM PARAMETERS IN EFFECT

202

BUFFER POOL ATTRIBUTES

2

152

USER-DEFINED STATISTICS TRACE

3

172

UNITS OF WORK INVOLVED IN DEADLOCK

196

LOCK TIMEOUT DETAILS

250

CONNECT/DISCONNECT FROM GBP

258

DATA SET EXTEND INFORMATION.

330

ACTIVE LOG SHORTAGE

4

191

DATA CAPTURE FOR DDIS ERRORS

192

DDM LEVEL 6A HEADER ERRORS

193

UOW DISPOSITION/SQLCODE MISMATCH

194

INVALID SNA FMH-5 RECEIVED

195

FIRST FAILURE DATA CAPTURE FOR DRDS

203

HEURISTIC DECISION OCCURRED

204

PARTNER COLD START DETECTED

205

INCORRECT LOGNAME/SYNC. PARMS

206

SNA COMPARE STATES PROTOCOL ERROR

207

HEURISTIC DAMAGE OCCURRED

208

SNA SYNC POINT PROTOCOL ERROR

209

SYNC POINT COMMUNICATION FAILURE

210

LOG NAME CHANGED ON WARM START

235

CONDITIONAL RESTART DATA LOSS

236

EXCHANGE LOG NAMES PROTOCOL ERROR

238

DB2 RESTART ERROR

267

START OF CF STRUCTURE REBUILD

268

END OF CF STRUCTURE REBUILD

5

230

DATA SHARING GLOBAL STATISTICS

7

326

WLM DELAY MONITOR SUPPORT


108 - Using DISPLAY Commands

Chapter 10. Performance Monitoring and Tuning

Using DISPLAY Commands

A lot of the information you can get via the Accounting and Statistics reports can also be obtained using the DISPLAY command (for more information on how to issue a command, refer to Chapter 2). The DISPLAY COMMAND can show details about current activity in the subsystem. The following are the different types of DISPLAYs:

  • DISPLAY ARCHIVE

  • DISPLAY BUFFERPOOL

  • DISPLAY DATABASE

  • DISPLAY FUNCTION SPECIFIC

  • DISPLAY GROUP

  • DISPLAY GROUPBUFFERPOOL

  • DISPLAY LOCATION

  • DISPLAY LOG

  • DISPLAY PROCEDURE

  • DISPLAY RLIMIT

  • DISPLAY THREAD

  • DISPLAY TRACE

  • DISPLAY UTILITY

Note

This type of information can also be viewed in the DB2 Control Center.

For more information on all of the options on the DISPLAYs, refer to the IBM DB2 UDB for OS/390 Command Reference.


109 - Buffer Pool and Subsystem Tuning

Chapter 10. Performance Monitoring and Tuning

Buffer Pool and Subsystem Tuning

Here we will discuss some of the performance aspects of some of these areas in DB2 memory. For more details on these objects refer to Chapter 2, Environment.

Buffer Pools

Separation

Buffer pools need to be fine-tuned for best performance. They work best when the data in them has like characteristics so that the thresholds for the write operations and queue management may be tuned according to the behavior or the applications against the data.

Thresholds

There are several thresholds used for buffer pool tuning. They can be different for each buffer pool. The following is a list of thresholds:

  • VPSEQT— sequential steal threshold

  • DWQT— deferred write threshold

  • VDWQT— vertical deferred write threshold

  • VPPSEQT— parallel sequential threshold

  • VPXPSEQT— sysplex parallel sequential threshold

  • VPSTEAL— type of queuing method used

  • VPTYPE— using memory in DBM1 or a data space

  • HPSIZE— size of hiper pool (if used)

  • HPSEQT— sequential steal thresholds for hiper pool

These thresholds vary by buffer pool depending on the type of access to the data. For instance, the thresholds are different for data that is accessed randomly vs. sequentially.

Monitoring

There are a couple of ways to monitor buffer pool usage. You can use the –DISPLAY BUFFERPOOL command to view the settings and current activity in the buffer pool, or you can view monitor reports for a given period of time. Some items to be monitored for buffer pools include: hit ratios, paging, amount of prefetch activity, amount of write activity, and warning thresholds. The following screen in Figure 10.4 shows the output from a –DISPLAY BUFFERPOOL command.

Figure 10.4. DISPLAY BUFFERPOOL Command.

Buffer pools can be easily changed by using the –ALTER BUFFERPOOL statement. This can take effect without a bounce of the DB2 subsystem. For example, if you want to change the sequential steal threshold on BP1, you would issue the following statement:

-ALTER BUFFERPOOL (BP1) VPSEQT (30)

Subsystem Tuning

There are other areas in the subsystem that we can tune. The following are more tuning opportunities in the area of memory usage:

EDM Pool

The EDM pool contains many objects critical to an applications performance. Those objects should be found in the EDM pool when requested. There should be about an 80% hit ratio of found objects. This can be monitored via a performance monitor such as DB2PM. Figure 10.5 shows information regarding the EDM pool usage.

Figure 10.5. EDM pool usage.

This is also critical for the dynamic SQL cache that is in the EDM pool. If the EDM pool is too small, the performance of dynamic SQL can suffer because the statements will not be cached in the pool when an application requests them and they will have to be re-prepared. For more information on dynamic SQL, refer to Chapter 12.

RID Pool

The RID pool needs to be monitored because if failures occur, they can hurt application performance. Figure 10.6 show the types of RID pool failures.

SORT Pool

The larger the sort pool (sort work area) is, the fewer the sort runs produced. If the sort pool is large enough, then the buffer pools and DSNDB07 will not be used. If buffer pools and DSNDB07 are not used, then the better performance will be due to less I/O.

Note

The size of the sort pool and DSNDB07 needs to be large so that large sorts don't have pages written to disk.

Make sure the you assign an adequately sized sort pool to your system and/or avoid really large SQL sorts in programs that need to execute fast. If you overflow to DSNDB07, this is reported in IFCID (traces) and your performance monitor can report on these events.

Other Tuning Parameters

There are several parameters (DSNZPARMS) that can be tuned in the DB2 subsystem. For a complete list, refer to Chapter 2, DB2 Environment.

Figure 10.7. RID pool failures.

    Team-Fly
    Top


110 - Summary

Chapter 10. Performance Monitoring and Tuning

Summary

Understanding the DB2 database environment is an important part of any database administrator's job. To gain this understanding, there are various facilities that can be used.

There are several ways to analyze the database activity. There are graphical monitoring tools which can be used with DB2 Universal Database, such as DB2PM and Visual Explain.

The DB2 optimizer is one of the most advanced in the relational database industry. The optimizer will generate an access plan during query compilation. Access paths are stored in the system catalog tables for static SQL applications. Access paths for dynamic SQL statements are generated at query execution time and stored in memory.

To gain an understanding of the access path (strategy) chosen by the DB2 optimizer, EXPLAIN may be used. EXPLAIN will populate a PLAN_TABLE and optional DSN tables with detailed information for the SQL statements. These tables can then be queried to determine the plan information regarding index usage and other database resources. This graphical version of the access path can be examined using Visual Explain.

In this chapter, the performance topics such as buffer pool tuning, sorting, database parameter tuning, and query parallelism were also discussed.

Diagnostics and problem determination information is provided in DB2. This information comes in the form of SQLCODE, SQLSTATE codes and traces.

Additional resources include:

  • DB2 UDB for OS/390 Command Reference Guide, SC26-9934.

  • DB2 UDB for OS/390 Administration Guide, Sc26-9931.

  • DB2 PM for OS/390 General Information, 6C26-9172.


111 - Part IV: Developing Applications

Part IV: Developing Applications

11 Application Development Overview

12 Development Considerations

13 Advanced Functions

14 Locking and Concurrency


112 - Chapter 11. Application Development Overview

Part IV. Developing Applications

Chapter 11. Application Development Overview

  • Application Environment

  • Embedded SQL

  • Host Variables

  • Precompiling and Binding

  • Plans and Packages

  • Stored Procedure Builder (SPB)


113 - DB2 Application Development Environment

Chapter 11. Application Development Overview

DB2 Application Development Environment

Suppose you are writing an application program to access data in a DB2 database. When your program executes an SQL statement, the program needs to communicate with DB2. When DB2 finishes processing an SQL statement, DB2 sends back a return code, and your program should test the return code to examine the results of the operation.

To communicate with DB2, you need to choose a method. You can use one of these methods:

  • Static SQL

  • Dynamic SQL

  • Open Database Connectivity (ODBC)

  • JDBC application support

ODBC lets you access data through ODBC function calls in your application. You execute SQL statements by passing them to DB2 through a ODBC function call. ODBC eliminates the need for precompiling and binding your application and increases the portability of your application by using the ODBC interface, but the ODBC processor does issue dynamic SQL.

If you are writing your applications in Java, you can use JDBC application support to access DB2. JDBC is similar to ODBC but is designed specifically for use with Java and is therefore a better choice than ODBC for making DB2 calls from Java applications.


114 - Delimiting SQL in a Program

Chapter 11. Application Development Overview

Delimiting SQL in a Program

In order to distinguish DB2 calls from the rest of the program, delimiters are used. The delimiters and methods differ considerably. For all languages using the standard precompiler, however, the methods are very similar. For example, use EXEC SQL and END-EXEC to delimit an SQL statement in a COBOL program:

EXEC SQL
an SQL statement
END-EXEC.

Note

For REXX, precede the statement with EXECSQL. If the statement is in a literal string, enclose it in single or double quotation marks.


115 - Declaring Table and View Definitions

Chapter 11. Application Development Overview

Declaring Table and View Definitions

Before your program issues SQL statements that retrieve, update, delete, or insert data, you should declare the tables and views which your program accesses. To do this, include an SQL DECLARE statement in your program.

You do not have to declare tables or views, but there are advantages if you do. One advantage is documentation. For example, the DECLARE statement specifies the structure of the table or view you are working with, and the data type of each column. You can refer to the DECLARE statement for the column names and data types in the table or view. Another advantage is that the DB2 precompiler uses your declarations to make sure you have used correct column names and data types in your SQL statements. The DB2 precompiler issues a warning message when the column names and data types do not correspond to the SQL DECLARE statements in your program.

A way to declare a table or view is to code a DECLARE statement in the WORKING-STORAGE SECTION or LINKAGE SECTION within the DATA DIVISION of your COBOL program. Specify the name of the table and list each column and its data type. When you declare a table or view, you specify DECLARE table-name TABLE regardless of whether the table-name refers to a table or a view.

For example, the DECLARE TABLE statement for the DB2USER1. TEST table looks like this:

EXEC SQL
DECLARE DB2USER1.TEST
  (NUMBER         CHAR(6)         NOT NULL,
   NAME           VARCHAR(50)     NOT NULL,
   TYPE           CHAR(1)         NOT NULL,
   CUT_SCORE      DECIMAL(6,2),
   LENGTH         SMALLINT        NOT NULL,
   TOTALTAKEN     SMALLINT        NOT NULL))
END-EXEC.

Note

An alternative to coding the DECLARE statement manually is to use DCLGEN, the declarations generator supplied with DB2. DCLGEN is covered later in this chapter.

Note

When a table or view is declared that contains a column with a distinct type, it is best to declare that column with the source type of the distinct type, rather than the distinct type itself. When the column is declared with the source type, DB2 can check embedded SQL statements that reference that column at precompile time.


116 - Host Variables and Host Structures

Chapter 11. Application Development Overview

Host Variables and Host Structures

A host variable is a data item declared in the host language for use within an SQL statement. Using host variables, you can:

  • Retrieve data into the host variable for your application program's use.

  • Place data into the host variable to insert into a table or to change the contents of a row.

  • Use the data in the host variable when evaluating a WHERE or HAVING clause.

  • Assign the value in the host variable to a special register, such as CURRENT SQLID and CURRENT DEGREE.

  • Insert null values in columns using a host indicator variable that contains a negative value.

  • Use the data in the host variable in statements that process dynamic SQL, such as EXECUTE, PREPARE, and OPEN.

A host structure is a group of host variables that an SQL statement can refer to using a single name. You can use host structures in all languages except REXX.

Use host language statements to define the host structures.

Using Host Variables

Any valid host variable name can be used in an SQL statement. Host variables must be declared in the program before they can be used.

Host variables are used by DB2 to:

  • Retrieve data and put it into the host variables for use by the application program.

  • Insert data into a table or UPDATE it from the data in the host variable.

  • Evaluate a WHERE or HAVING clause using the data in the host variable; however, host variables cannot be used to represent a table, column, or view.

To optimize performance, make sure the host language declaration maps as closely as possible to the data type of the associated data in the database. Table, view, or column names can be specified at run-time when using dynamic SQL. We discuss dynamic SQL later in this chapter.

Host variables follow the naming conventions of the host language. A colon (:) must precede host variables used in SQL to tell DB2 that the variable is not a column name. A colon must not precede host variables outside of SQL statements.

In the following example, :CANPHONE and :CIDNO are host variables. The values will be supplied at run-time.

EXEC SQL
   UPDATE CANDIDATE
      SET PHONE = :CANPHONE
      WHERE CID = :CIDNO

The host variables can be used to specify a program data area to contain the column values of a retrieved row or rows.

Using Host Structures

A host structure can be used in place of one or more host variables. In the following example, assume that a COBOL program includes the following SQL statement:

EXEC SQL
   SELECT CID, LNAME, FNAME
   INTO :CIDHV,:LNAMEHV,:FNAMEHV
   FROM DB2USER1.CANDIDATE
   WHERE CID =:CIDHV
END-EXEC.

If you want to avoid listing host variables, you can substitute the name of a structure, say, :CANDHS, that contains :CIDHV, :LNAMEHV, and :FNAMEHV. The example then reads:

EXEC SQL
   SELECT CID, LNAME, FNAME
   INTO :CANDHS
   FROM DB2USER1.CANDIDATE
   WHERE CID =:CIDHV
END-EXEC.

Host structures can be declared manually, or DCLGEN can be used to generate a COBOL record description, PL/I structure declaration, or C structure declaration that corresponds to the columns of a table. We will discuss DCLGEN later in this chapter.

DCLGEN

DCLGEN, the declarations generator supplied with DB2, produces a DECLARE statement for use in a C, COBOL, or PL/I program, so that the declares for the tables do not need to be manually coded.

DCLGEN generates a table declaration and puts it into a member of a partitioned data set that you can include in your program. When you use DCLGEN to generate a table's declaration, DB2 gets the relevant information from the DB2 catalog, which contains information about the table's definition and the definition of each column within the table. DCLGEN uses this information to produce a complete SQL DECLARE statement for the table or view and a matching PL/I, or C structure declaration or COBOL record description.

Note

DCLGEN can be used only on existing tables.

The DCLGEN must be used and the DCLGEN name of the table or view must be supplied before the program is precompiled. In order to use the declarations generated by DCLGEN in the program, the SQL INCLUDE statement is used. The example below shows this INCLUDE statement for the CAND DCLGEN:

EXEC SQL INCLUDE CAND;

DB2 must be active before DCLGEN can be used. It can be started in several different ways:

  • From ISPF through DB2I

    • Select the DCLGEN option on the DB2I Primary Option Menu panel. Next, fill in the DCLGEN panel with the information it needs to build the declarations. Then press ENTER.

  • Directly from TSO

    • To do this, sign on to TSO, issue the TSO command DSN, and then issue the subcommand DCLGEN.

  • From a CLIST

    • Run a CLIST in TSO foreground or background, that issues DSN and then DCLGEN.

  • With JCL

    • Supply the required information, using JCL, and run DCLGEN in batch.

Below is an example of the output from a DCLGEN.

**************************************************************
     DCLGEN TABLE(DB2USER1.TEST)
         LIBRARY(DB2U01.DCLGEN.LIB(TEST))
         LANGUAGE(COBOL)
         APOST
**************************************************************
   EXEC SQL DECLARE DB2USER1.TEST TABLE
      (NUMBER        CHAR(6)         NOT NULL,
      NAME           VARCHAR(50)     NOT NULL,
      TYPE           CHAR(1)         NOT NULL,
      CUT_SCORE      DECIMAL(6,2),
      LENGTH         SMALLINT        NOT NULL,
      TOTALTAKEN     SMALLINT        NOT NULL)
      ) END-EXEC.
**************************************************************
* COBOL DECLARATION FOR TABLE DB2USER1.TEST
**************************************************************
   01  DCLTEST.
       10 NUMBER                PIC X(63).
       10 NAME.
           49 NAME-LEN          PIC S9(4) COMP-4.
           49 NAME-TEXT         PIC X(50).
       10 TYPE                  PIC X(1).
       10 CUTSCRE               PIC S(4)V9(2) COMP-3.
       10 LENGTH                PIC 9(5) COMP.
       10 TOTALTKN              PIC 9(5) COMP.
**************************************************************
*THE NUMBER OF COLUMNS DESCRIBED BY THIS DECLARATION IS 6
**************************************************************

Retrieving a Single Row of Data

The INTO clause of the SELECT statement names one or more host variables to contain the column values returned. The named variables correspond one-to-one with the list of column names in the SELECT list. In the following example, we are retrieving the CID, LNAME, and FNAME from the CANDIDATE table and populating the host variables. You can define a data area in your program to hold each column, then name the data areas with an INTO clause, as in the following example (notice that a colon precedes each host variable):

EXEC SQL
   SELECT CID, LNAME, FNAME
   INTO :CIDHV,:LNAMEHV,:FNAMEHV
   FROM DB2USER1.CANDIDATE
   WHERE CID =:CIDHV
END-EXEC.

In the DATA DIVISION of the program, the host variables CIDHV, LNAMEHV, and FNAMEHV must be declared and need to be compatible with the data types in the columns CID, LNAME, and FNAME of the DB2USER1.CANDIDATE table.

Note

If the SELECT statement returns more than one row, it will result in a –811 error code, and any data returned is undefined and unpredictable.

Retrieving Multiple Rows of Data

If the number of rows returned by DB2 is unknown, or if you expect more than one row to return, then you must use an alternative to the SELECT...INTO statement. The DB2 cursor enables an application to process a set of rows and retrieve one row at a time from the result table. We will take a look at cursors later in this chapter.

Inserting and Updating Data

You can set or change a value in a DB2 table to the value of a host variable. To do this, you can use the host variable name in the SET clause of UPDATE or the VALUES clause of INSERT. This example changes a candidate's home phone number:

EXEC SQL
   UPDATE DB2USER1.CANDIDATE
      SET PHONENO =:HPHONEHV
      WHERE CID =:CID
END-EXEC.

Searching Data

You can use a host variable to specify a value in the predicate of a search condition or to replace a constant in an expression. For example, if you have defined a field called HVCID that contains a candidate number, you can retrieve the name of the candidates whose number is "012345678" with:

MOVE '012345678' TO HVCID.
EXEC SQL
   SELECT LNAME
   INTO :HVLNAME
   FROM DB2USER1.CANDIDATE
   WHERE CID =:HVCID
END-EXEC.


117 - SQL Execution Validation

Chapter 11. Application Development Overview

SQL Execution Validation

A program that includes SQL statements needs to have an area set apart for communication with DB2—an SQL communication area (SQLCA). When DB2 processes a SQL statement in a program, it places return codes in the SQLCODE and SQLSTATE host variables or corresponding fields of the SQLCA.

The return codes indicate whether the statement executed succeeded or failed. Because the SQLCA is a valuable problem-diagnosis tool, it is a good idea to include the instructions necessary to display some of the information contained in the SQLCA in your application programs. For example, the contents of SQLERRD(3)—which indicate the number of rows that DB2 updates, inserts, or deletes—could be useful. If SQLWARN0 contains W, DB2 has set at least one of the SQL warning flags (SQLWARN1 through SQLWARNA). A description of the SQLCA is shown in the following Table 11.1.

Table 11.1. SQLCA Description

Name

Data Type

Purpose

SQLCAID

CHAR(8)

An "eye catcher" for storage dumps, containing the text "SQLCA".

SQLCABC

INTEGER

Contains the length of the SQLCA: 136.

SQLCODE

INTEGER

Contains the SQL return code.

Code

Means

0

Successful execution (though there might have been warning messages).

positive

Successful execution, with an exception condition.

negative

Error condition.

SQLERRML

SMALLINT

Length indicator for SQLERRMC, in the range 0 through 70. 0 means that the value of SQLERRMC is not pertinent.

SQLERRMC

VARCHAR(70)

Contains one or more tokens, separated by X"FF", that are substituted for variables in the descriptions of error conditions.

SQLERRP

CHAR(8)

Provides a product signature and, in the case of an error, diagnostic information such as the name of the module that detected the error. In all cases, the first three characters are "DSN" for DB2 for OS/390.

SQLERRD(1)

INTEGER

Contains an internal error code.

SQLERRD(2)

INTEGER

Contains an internal error code.

SQLERRD(3)

INTEGER

Contains the number of rows affected after INSERT, UPDATE, and DELETE (but not rows deleted as a result of CASCADE delete). Set to 0 if the SQL statement fails, indicating that all changes made in executing the statement were canceled. Set to –1 for a mass delete from a table in a segmented table space. SQLERRD(3) can also contain the reason code of a timeout or deadlock for SQLCODES –911 and –913.

SQLERRD(4)

INTEGER

Generally contains timerons, a short floating-point value that indicates a rough relative estimate of resources required. It does not reflect an estimate of the time required. When preparing a dynamically defined SQL statement, you can use this field as an indicator of the relative cost of the prepared SQL statement. For a particular statement, this number can vary with changes to the statistics in the catalog. It is also subject to change between releases of DB2 for OS/390.

SQLERRD(5)

INTEGER

Contains the position or column of a syntax error for a PREPARE or EXECUTE IMMEDIATE statement.

SQLERRD(6)

INTEGER

Contains an internal error code.

SQLWARN0

CHAR(1)

Contains a W if at least one other indicator also contains a W; otherwise, contains a blank.

SQLWARN1

CHAR(1)

Contains a W if the value of a string column was truncated when assigned to a host variable.

SQLWARN2

CHAR(1)

Contains a W if null values were eliminated from the argument of a column function; not necessarily set to W for the MIN function because its results are not dependent on the elimination of null values.

SQLWARN3

CHAR(1)

Contains a W if the number of result columns is larger than the number of host variables. Contains a Z if fewer locators were provided in the ASSOCIATE LOCATORS statement than the stored procedure returned.

SQLWARN4

CHAR(1)

Contains a W if a prepared UPDATE or DELETE statement does not include a WHERE clause.

SQLWARN5

CHAR(1)

Contains a W if the SQL statement was not executed because it is not a valid SQL statement in DB2 for OS/390.

SQLWARN6

CHAR(1)

Contains a W if the addition of a month or year duration to a DATE or TIMESTAMP value results in an invalid day (for example, June 31). Indicates that the value of the day was changed to the last day of the month to make the result valid.

SQLWARN7

CHAR(1)

Contains a W if one or more nonzero digits were eliminated from the fractional part of a number used as the operand of a decimal multiply or divide operation.

SQLWARN8

CHAR(1)

Contains a W if a character that could not be converted was replaced with a substitute character.

SQLWARN9

CHAR(1)

Contains a W if arithmetic exceptions were ignored during COUNT DISTINCT processing. Contains a Z if the stored procedure returned multiple result sets.

SQLWARNA

CHAR(1)

Contains a W if at least one character field of the SQLCA or the SQLDA names or labels are invalid due to a character conversion error.

SQLSTATE

CHAR(5)

Contains a return code for the outcome of the most recent execution of an SQL statement.

SQLCODE and SQLSTATE

Whenever an SQL statement executes, the SQLCODE and SQLSTATE fields of the SQLCA receive a return code. Although both fields serve basically the same purpose (indicating whether the statement executed successfully), there are some differences between the two fields.

SQLCODE

DB2 returns the following codes in SQLCODE:

  • If SQLCODE = 0, execution was successful.

  • If SQLCODE > 0, execution was successful with a warning.

  • If SQLCODE < 0, execution was not successful.

  • SQLCODE 100 indicates no data was found.

The meaning of SQLCODEs other than 0 and 100 varies with the particular product implementing SQL.

SQLSTATE

SQLSTATE allows an application program to check for errors in the same way for different IBM database management systems. An advantage to using the SQLCODE field is that it can provide more specific information than the SQLSTATE. Many of the SQLCODEs have associated tokens in the SQLCA that indicate, for example, which object incurred an SQL error.

To conform to the SQL standard, the SQLCODE and SQLSTATE (SQLCOD and SQLSTA in FORTRAN) can be declared as standalone host variables. If the STDSQL(YES) precompiler option is specified, these host variables receive the return codes, and an SQLCA does not need to be included in the program.


118 - Using Cursors

Chapter 11. Application Development Overview

Using Cursors

Use a cursor in an application program to retrieve rows from a table or from a result set returned by a stored procedure. This sections looks at how an application program can use a cursor to retrieve rows from a table.

When the SELECT statement is executed, a set of rows is retrieved. The set of rows returned is referred to as the result table or result set. In an application program, there needs to be a way to retrieve one row at a time from the result set into host variables. A cursor performs that function. A program can have several cursors, each of which performs the previous steps.

DECLARE Cursor

To define and identify a set of rows to be accessed with a cursor, issue a DECLARE CURSOR statement. The DECLARE CURSOR statement names a cursor and specifies a SELECT statement. The SELECT statement defines the criteria for the rows that will make up the result table. The simplest form of the DECLARE CURSOR statement is as follows:

EXEC SQL
   DECLARE CERTCUR CURSOR FOR
      SELECT TCID, NAME, CODE
      FROM DB2USER1.TEST_CENTER
      WHERE TCID = :TCIDHV
END-EXEC.
Updating a Column

If any of the columns are to be updated in the rows of the identified table, include the FOR UPDATE clause. The FOR UPDATE clause has two forms: one form for when the columns to be updated are known, and another for when the columns are not known. Use the following form when the columns intended for update are known:

EXEC SQL
   DECLARE CERTCUR CURSOR FOR
      SELECT TCID, NAME, CODE
      FROM DB2USER1.TEST_CENTER
      WHERE TCID = :TCIDHV
      FOR UPDATE OF CODE
END-EXEC.

If a cursor might be used to update any of the columns of the table, use the following form of the FOR UPDATE clause, which allows for updates to any columns of the table that can be updated:

EXEC SQL
   DECLARE CERTCUR CURSOR FOR
      SELECT TCID, NAME, CODE
      FROM DB2USER1.TEST_CENTER
      WHERE TCID = :TCIDHV
      FOR UPDATE
END-EXEC.

Note

DB2 must do more processing when you use the FOR UPDATE clause without a column list than when you use the FOR UPDATE CLAUSE OF clause with a column list. Therefore, if only a few columns of a table are going to be updated, the program can run more efficiently if a column list is included.

A column of the identified table can be updated even though it is not part of the result table. In this case, there is no need to name the column in the SELECT statement. When the cursor retrieves a row (using FETCH) that contains a column value which you want to update, the UPDATE...WHERE CURRENT OF can be used to update the row.

Note

Some result tables cannot be updated—for example, the result of joining two or more tables.

OPENing the Cursor

To tell DB2 you are ready to process the first row of the result table, the OPEN statement must be issued. DB2 then uses the SELECT statement within DECLARE CURSOR to identify a set of rows. If host variables are used in that SELECT statement, DB2 uses the current value of the variables to select the rows. The result table that satisfies the search conditions might contain zero, one, or many rows. The OPEN statement looks like this:

EXEC SQL
   OPEN cursor-name
END-EXEC.

When used with cursors, DB2 evaluates CURRENT DATE, CURRENT TIME, and CURRENT TIMESTAMP special registers once when the OPEN statement executes. There is more information on special registers later in this chapter. DB2 uses the values returned in the registers on all subsequent FETCH statements.

Two factors that influence the amount of time that DB2 requires to process the OPEN statement are:

  • Whether DB2 must perform any sorts before it can retrieve rows from the result table

  • Whether DB2 uses parallelism to process the SELECT statement associated with the cursor (For more information on parallelism refer to Chapter 10, Performance Monitoring and Tuning.)

Executing SQL Statements

One of the following SQL statements is executed using the cursor:

  • A FETCH statement

  • A positioned UPDATE statement

  • A positioned DELETE statement

FETCH Statements

Execute a FETCH statement for one of the following purposes:

  • To copy data from a row of the result table into one or more host variables.

  • To position the cursor before a positioned UPDATE or positioned DELETE operation is performed. The simplest form of the FETCH statement looks like this:

EXEC SQL
   FETCH CERTCUR
   INTO :TCIDHV,:NAMEHV ...
END-EXEC.

The SELECT statement within the DECLARE CURSOR statement identifies the result table from which to fetch rows, but DB2 does not retrieve any data until the application program executes a FETCH statement.

When the program executes the FETCH statement, DB2 uses the cursor to point to a row in the result table. That row is called the current row. DB2 then copies the current row contents into the program host variables that you specified on the INTO clause of FETCH. This sequence repeats each time you issue FETCH, until you have processed all rows in the result table.

The row that DB2 points to when a FETCH statement is executed depends on whether the cursor is declared as a scrollable or non-scrollable (covered later in this chapter).

When querying a remote subsystem with FETCH, consider using block fetch for better performance. Block fetch processes rows ahead of the current row.

Note

Block fetch cannot be used when a positioned UPDATE or DELETE is performed.

Positioned UPDATE

After the program has executed a FETCH statement to retrieve the current row, a positioned UPDATE statement can be used to modify the data in that row. An example of positioned UPDATE is as follows:

EXEC SQL
   UPDATE DB2USER1.TEST_CENTER
      SET CODE = 123456
      WHERE CURRENT OF CERTCUR
END-EXEC.

A positioned UPDATE statement updates the row that cursor-name points to. A positioned UPDATE statement must meet these conditions:

  • The row cannot be updated if the UPDATE violates any unique, check, or referential constraints.

  • An UPDATE statement cannot be used to modify the rows of a created temporary table. However, it can be used to modify the rows of a declared temporary table (covered later in this chapter).

  • If the right side of the SET clause in the UPDATE statement contains a subselect, that subselect cannot include a correlated name for a table that is being updated.

Positioned DELETE

After the program has executed a FETCH statement to retrieve the current row, a positioned DELETE statement can be used to delete that row. An example of a positioned DELETE statement follows:

EXEC SQL
   DELETE FROM DB2USER1.TEST_CENTER
   WHERE CURRENT OF CERTCUR
END-EXEC.

A positioned DELETE statement deletes the row that cursor-name points to. A positioned DELETE statement must meet these conditions:

  • A DELETE statement cannot be used with a cursor to delete rows from a created global temporary table. However, it can be used to delete rows from a declared temporary table.

  • After a row has been deleted, you cannot update or delete another row using that cursor until you execute a FETCH statement to position the cursor on another row.

  • A row cannot be deleted if the delete violates any referential constraints.

CLOSEing the Cursor

If the program is finished processing the rows of the result table, issue a CLOSE statement to close the cursor:

EXEC SQL
   CLOSE CERTCUR
END-EXEC.

This will destroy the result set that was created during the OPEN cursor.

CURSOR WITH HOLD

A held cursor, which is declared WITH HOLD, does not close after a commit operation. A cursor that is not declared WITH HOLD closes after a commit operation. When a cursor is declared, the inclusion or exclusion of the WITH HOLD clause tells DB2 whether or not the cursor is to be held. After a commit operation, a held cursor is positioned after the last row retrieved and before the next logical row of the result table to be returned.

A held cursor can close when:

  • A CLOSE cursor, ROLLBACK, or CONNECT statement is issued.

  • A CAF CLOSE function call or an RRSAF TERMINATE THREAD function call is issued.

  • The application program terminates.

If the program abnormally terminates, the cursor position is lost. To prepare for restart, your program must reposition the cursor.

The following restrictions apply to cursors that are declared WITH HOLD:

  • Do not use DECLARE CURSOR WITH HOLD with the new user sign-on from a DB2 attachment facility, because all open cursors are closed.

  • Do not declare a WITH HOLD cursor in a thread that could become inactive. If you do, its locks are held indefinitely.

There are also some restrictions when using CURSOR WITH HOLD with IMS or CICS programs. (Refer to IBM DB2 Application Programming Guide for more details.)

Note

The use of CURSOR WITH HOLD will also prevent some options in DB2 that are used for performance (i.e., parallelism, drains needed for online reorg).

EXEC SQL
   DECLARE CERTCUR CURSOR WITH HOLD FOR
      SELECT TCID, NAME, CODE
      FROM DB2USER1.TEST_CENTER
      WHERE TCID = :TCIDHV
END-EXEC.

Types of Cursors

Cursors can be scrollable or not scrollable. They can also be held or not held. The following sections discuss these characteristics in more detail.

Scrollable and Non-Scrollable

When you declare a cursor, you tell DB2 whether you want the cursor to be scrollable or non-scrollable by including or omitting the SCROLL clause. This clause determines whether the cursor moves sequentially forward through the data set or has random access to the data set.

Using a Non-Scrollable Cursor

The simplest type of cursor is a non-scrollable cursor. A non-scrollable cursor always moves sequentially forward in the result table. When a cursor is opened, it is positioned before the first row in the result table. When the first FETCH is executed, the cursor is positioned on the first row. The next FETCH statement moves the cursor one row ahead for each FETCH.

After each FETCH statement, the cursor is positioned on the row that was fetched. After an execution of a positioned UPDATE or positioned DELETE operation, the cursor stays at the current row of the result table. Rows cannot be retrieved backward or move to a specific position in a result table with a non-scrollable cursor.

Using a Scrollable Cursor

Scrollable cursors allow for movement to any row in the result table. DB2 uses declared temporary tables for processing scrollable cursors. See section on Declared Temporary Tables later in this chapter for details.

The declaration for a scrollable cursor has the following form:

EXEC SQL
   DECLARE cursor-name sensitivity STATIC SCROLL
      CURSOR FOR
      SELECT column-name-list
      FROM table-name
      WHERE search-condition
END-EXEC

STATIC SCROLL indicates that the cursor is scrollable and follows the static model.

The sensitivity in the DECLARE statement indicates whether changes that are made to the underlying table after the cursor is opened are visible to the result table. Sensitivity can be INSENSITIVE or SENSITIVE.

  • INSENSITIVE

    • This means that no changes to the underlying table after the cursor is opened are visible to the result table.

  • SENSITIVE

    • This means that some or all changes that are made to the underlying table after the cursor is opened are visible to the result table.

The sensitivity clause of the FETCH statement determines which changes to the underlying table are visible to the result table.

When the cursor is opened, the cursor is positioned before the first row. A clause is included in each FETCH statement to tell DB2 where to position the cursor in the result table. The FETCH clause for a scrollable cursor has the following form:

EXEC SQL
   FETCH sensitivity cursor-position cursor-name
   INTO :host-variable1 ,:host-variable2 ...
END-EXEC.

Sensitivity in the FETCH statement indicates whether changes that are made to the underlying table by some means other than the cursor are visible to the result table. Sensitivity can be INSENSITIVE or SENSITIVE. INSENSITIVE means that the only time that changes to the underlying table are visible to the result table is when a positioned UPDATE or positioned DELETE using cursor-name makes those changes. SENSITIVE means that the result table is updated when the underlying tables changes.


119 - Using Dynamic SQL

Chapter 11. Application Development Overview

Using Dynamic SQL

Before you decide to use dynamic SQL, some consideration should be taken for whether using static SQL or dynamic SQL is the best technique for the application.

For most DB2 users, static SQL—embedded in a host language program and bound before the program runs—provides a straightforward, efficient path to DB2 data. Static SQL should be used when the SQL statements which the application needs to execute are known before run-time.

Dynamic SQL prepares and executes the SQL statements within a program, while the program is running. There are four types of dynamic SQL:

  • Embedded dynamic SQL

    • The application puts the SQL source in host variables and includes PREPARE and EXECUTE statements that tell DB2 to prepare and run the contents of those host variables at run-time. The programs that include embedded dynamic SQL must go through the precompile and bind steps.

  • Interactive SQL

    • A user enters SQL statements through SPUFI. DB2 prepares and executes those statements as dynamic SQL statements.

  • Deferred embedded SQL

    • Deferred embedded SQL statements are neither fully static nor fully dynamic. Like static statements, deferred embedded SQL statements are embedded within applications, but like dynamic statements, they are prepared at run-time. DB2 processes deferred embedded SQL statements with bind-time rules. For example, DB2 uses the authorization ID and qualifier determined at bind time as the plan or package owner.

  • Dynamic SQL executed through ODBC functions

    • The application contains ODBC function calls that pass dynamic SQL statements as arguments. There is no need to precompile and bind programs that use ODBC function calls.

Coding Dynamic SQL in Applications

Applications using dynamic SQL use three types of SQL statements:

  • Dynamic SQL for non-SELECT statements

    • Those statements include DELETE, INSERT, and UPDATE. For execution, use either of the following SQL statements:

      • EXECUTE IMMEDIATE

      • PREPARE and then EXECUTE

  • Dynamic SQL for fixed-list SELECT statements

    • A SELECT statement is fixed-list if you know in advance the number and type of data items in each row of the result. For execution, use the following SQL statements:

      • PREPARE (a SQL cursor definition)

      • OPEN the cursor

      • FETCH from the cursor

      • CLOSE the cursor

  • Dynamic SQL for varying-list SELECT statements

    • A SELECT statement is varying-list if you cannot know in advance how many data items to allow for or what their data types are.

      • DECLARE CURSOR for the statement

      • PREPARE the statement

      • DESCRIBE the statement

      • OPEN cursor

      • FETCH using DESCRIPTOR

      • CLOSE cursor

There are many other variations to using dynamic SQL. Host variable can also be used, as well as putting the literal data into the dynamic SQL statement. When using parameter markers in the dynamic SQL statement, to be replaced by the host variables later, the bind feature REOPT(VARS) can also be used to assist the optimizer in picking the right access path, based on the host variable data, at execution time.


120 - Precompile and Bind

Chapter 11. Application Development Overview

Precompile and Bind

DB2 application programs include SQL statements. In order to compile those programs, the SQL statements must be converted into language recognized by the compiler or assembler. The DB2 precompiler or a host-language compiler needs to be used to:

  • Replace the SQL statements in the source programs with compilable code

  • Create a database request module (DBRM), which communicates the SQL requests to DB2 during the bind process

After the source program has been precompiled, a load module is created, possibly one or more packages, and an application plan. Creating a load module involves compiling and link-editing the modified source code that is produced by the precompiler. Creating a package or an application plan, a process unique to DB2, involves binding one or more DBRMs. Figure 11.1 shows a picture of how programs are prepared in DB2.

Figure 11.1. Program Preparation

Precompile

The precompiler "prepares" the source program for compilation by replacing EXEC SQL by a CALL and by putting the SQL in comment.

The precompiler performs the following:

  • Includes DCLGEN member

  • Includes SQLCA

  • Looks for SQL statements and for host variable definitions

  • Verifies the SQL syntax

  • Matches each column and table name in the SQL to the DECLARE TABLE statements

  • Prepares the SQL for compilation or assembly in the host language

  • Produces a DBRM and stores it in a PDS

  • Can be invoked in DB2I or in batch

Note

DB2 is not accessed during this process.

The DBRM (Database Request Module) that is created contains extracted, somewhat modified, parsed SQL source code. It can be stored as a member in a PDS (partitioned data set). One DBRM is created for each precompile and it will then become the input to the BIND process.

The SQL statements are replaced during the precompile process with a call to the DSNHLI module. This call will contain the necessary parameters (DBRM name, timestamp, statement number, address of host variables, and address of SQLCA) in order to locate the access path needed to execute the SQL statement associated with the call when the DBRM and modified source are used together at execution time.

Binding

The bind process establishes a relationship between an application program and its relational data. This step is necessary before a program can be executed. DB2 allows two basic ways of binding a program: to a package, or directly to an application plan.

Note

If the application uses DRDA access to distribute data, then you must use packages. (Refer to Chapter 2 for more information on DRDA.)

When a program is preprocessed for DB2, there are normally two outputs: a modified source program to be passed to a compiler where the original SQL statements have been replaced by statements to invoke the appropriate DB2 interface functions, and another output containing information on how those SQL statements will be used by DB2. This second output, called a DBRM (database request module) must be bound so that all the DB2 functions used in the program can be checked for validity and authorization, and so a proper access path can be determined.

The bind process performs the following:

  • Checks SQL syntax

  • Checks security

  • Compares column/table names against DB2 catalog

  • Builds access path strategy for each SQL statement

These processes are discussed in more detail later in this chapter.

Plans and Packages

A DBRM can be bound in a package or directly into a plan. In addition, packages can be bound into logical groups called collections. These packages or collections can then be bound into a plan. A package can be bound for only a single SQL statement, for a subset of the program, or for an entire program, and it will contain information representing optimized SQL, which may just be an access path for the SQL to use to process the data required, or it could be information representing an SQL statement that was first rewritten by the optimizer.

A plan can be bound for multiple packages, collections, and/or DBRMs. Each package can be bound independently of a plan. If a DBRM is bound to a plan and then is rebound, all DBRMs are rebound as well. The output will contain the access path information for each SQL statement.

When determining the maximum size of a plan, several physical limitations must be considered, including the time required to bind the plan, the size of the EDM pool, and fragmentation. There are no restrictions to the number of DBRMs that can be included in a plan. However, packages provide a more flexible method for handling large numbers of DBRMs within a plan.

Packages are database objects that contain executable forms of SQL statements. These packages contain statements that are referenced from a DB2 application. A package corresponds to a program source module.

Packages are stored in the DB2 catalog tables. The packages contain the DB2 access plan that was selected by DB2 during the BIND or PREPARE process. This type of BIND is known as static binding since it is performed prior to the execution of the SQL statement. Packages cannot be directly referenced in an SQL data manipulation (DML) statement.

Most applications that access a DB2 database will have a package or group of packages stored (bound) in the system catalog tables.

The package is the input to the plan bind using the PKLIST options. A package must be bound into a plan before it is usable.

An example of a plan bind is shown below:

BIND PLAN (certpln) PKLIST (col1.*)
Plan to Package Ratio

It is extremely difficult to justify the use of one plan containing all the packages for an installation or even for just all CICS transactions or batch jobs. Plans need to be granular, the same as all other definitions. Large, cumbersome plans cause performance degradation, buffer pool problems, and EDM pool problems. The number of packages put into a single plan needs to be based on functionality, such as all the packages supporting a particular function of an on-line application.

Advantages of Packages

Packages need to be used based on the application design and objectives. The following are a list of advantages provided by packages:

  • Ease of maintenance:

    • With packages, there is no need to bind the entire plan again when a change is made to one SQL statement. Only the package associated with the changed SQL statement needs to be bound.

  • Incremental development of a program:

    • Binding packages into package collections (discussed later) allows for the ability to add packages to an existing application plan without having to bind the entire plan again.

  • Versioning:

    • The ability to maintain several versions of a plan without using packages requires a separate plan for each version, and therefore separate plan names and RUN commands. This is discussed later in this section.

  • Flexibility in using bind options:

    • The options of BIND PLAN apply to all DBRMs bound directly to the plan. The options of BIND PACKAGE apply only to the single DBRM bound to that package. The package options need not all be the same as the plan options, and they need not be the same as the options for other packages used by the same plan.

  • Flexibility in using name qualifiers:

    • The bind option QUALIFIER can be used to name a qualifier for the unqualified object names in SQL statements in a plan or package. By using packages, this will allow different qualifiers for SQL statements in different parts of an application. In order to redirect SQL statements, for example, from a test table to a production table, all that is required is a rebind.

  • CICS flexibility:

    • With packages, there is no need to have dynamic plan selection and the associated exit routine. A package listed within a plan is not accessed until it is executed. However, it is possible to use dynamic plan selection and packages together. This can reduce the number of plans in an application, and hence less effort to maintain the dynamic plan exit routine.

Collections

A collection is a group of associated packages. If a collection name is included in the package list when a plan is bound, any package in the collection becomes available to the plan. The collection can even be empty when the plan is first bound. Later, packages can be added to the collection, and existing packages can be dropped or replaced, without binding the plan again.

A collection is implicitly created at the first package bind that references a named collection. The name of the package is collection.packagename.

Note

Collections are used in order to group packages by application or function.

The CURRENT PACKAGESET special register will allow for package searches in specific collections and will disregard any other DBRMS. Below is an example of using the special register to search only the "certcol" collection:

EXEC SQL SET CURRENT PACKAGESET = 'certcol'

Without the use of CURRENT PACKAGESET, all the DBRMs would be searched and then collections would be searched.

Note

It is also possible to have two different copies of a program by binding them into two different collections.

Versioning

Maintaining several versions of a plan without using packages requires a separate plan for each version, and therefore separate plan names and RUN commands. Isolating separate versions of a program into packages requires only one plan and helps to simplify program migration and fallback. At precompile time the user can assign versions to the program, which allow multiple copies of the same package in the DB2 system.

For example, separate development, test, and production levels of a program can be maintained by binding each level of the program as a separate version of a package, all within a single plan.

Binding and Rebinding

The BIND PACKAGE subcommand allows you to bind DBRMs individually. It provides the ability to test different versions of an application without extensive rebinding.

Note

Package binding is also the only method for binding applications at remote sites.

Even when DBRMs are bound into packages, all programs must be designated in an application plan. BIND PLAN establishes the relationship between DB2 and all DBRMs or packages in that plan. Plans can specify explicitly named DBRMs, packages, collections of packages, or a combination of these elements. The plan contains information about the designated DBRMs or packages and about the data the application program intends to use. It is stored in the DB2 catalog.

In addition to building packages and plans, the bind process:

  • Validates the SQL statements using the DB2 catalog.

    • During the bind process, DB2 checks the SQL statements for valid table, view, and column names. Because the bind process occurs as a separate step before program execution, errors are detected and can be corrected before the program is executed.

  • Verifies that the process binding the program is authorized to perform the data accessing operations requested by your program's SQL statements.

    • When the BIND command is issued, an authorization ID can be specified as the owner of the plan or package. The owner can be any one of the authorization IDs of the process performing the bind. The bind process determines whether the owner of the plan or package is authorized to access the data which the program requests.

  • Selects the access paths needed to access the DB2 data that the program wants to process.

    • In selecting an access path, DB2 considers indexes, table sizes, and other factors. DB2 considers all indexes available to access the data and decides which ones (if any) to use when selecting a path to the data. BIND PLAN and BIND PACKAGE can be accomplished using DB2I panels, the DSNH CLIST, or the DSN subcommands BIND PLAN and BIND PACKAGE.

Depending upon how the DB2 application was designed, all DBRMs could be bound in one operation, creating only a single application plan. Or, you might bind some or all of your DBRMs into separate packages in separate operations. After that, the entire application must still be bound as a single plan, listing the included packages or collections and binding any DBRMs not already bound into packages.

Note

Regardless of what the plan contains, a plan must be bound before the application can run.

Binding or Rebinding a Package or Plan in Use

Packages and plans are locked when binding or during execution. Packages that run under a plan are not locked until the plan uses them. If you execute a plan and some packages in the package list are never executed, those packages are never locked.

There is no ability to bind or rebind a package or a plan while it is running. However, a different version of a package that is running can be bound.

Options for Binding and Rebinding

There are a few methods used for BINDing and REBINDing packages and plans.

  • The BIND (ADD) option can be used to allow for the creation of a new plan or package.

  • The BIND (REPLACE) is used when the program has been changed.

  • REBINDing is used when a program has not changed but maybe an index has been added or RUNSTATS has been executed and any access path changes need to be accounted for.

Several of the options of BIND PACKAGE and BIND PLAN can affect your program design. For example, there is a bind option that can be used to ensure that a package or plan can run only from a particular CICS connection or a particular IMS region—this cannot be enforced in your code. An example is shown below:

BIND PLAN(CICSONLY)-
   MEMBER(CERTDBRM)-
   ACTION(ADD)-
   ISOLATION(CS)-
   OWNER(DB2USER1)-
   QUALIFIER(DB2USER1)-
   CACHESIZE(0)-
   ENABLE(CICS)CICS(CON1)

Several other options are discussed at length in later chapters, particularly the ones that affect the program's use of locks, such as the option ISOLATION. Table 11.2 shows a list of options for BIND/REBIND and what it is valid for Plans, Packages, and Trigger Packages (REBIND only discussed in Chapter 13). The options chosen are recorded in the SYSPLAN and SYSPACKAGE catalog tables.

Table 11.2. BIND Options

Option

Valid Values

Plan

Pkge

Trgr

ACQUIRE

USE, ALLOCATE

X

X

Determines whether to acquire resources specified in the DBRM at first access or allocation

ACTION

REPLACE, ADD

X-BO

X-BO

REPLACE(RPLVER)

X-BO

REPLACE(RETAIN)

X-BO

Determines whether the object (plan or package) replaces an existing object with same name or is new

CACHESIZE

Value of PLAN AUTH CACHE, Decimal value

X

Determines the size (in bytes) of the authorization cache acquired in the EDM pool for the plan

COPY

Collection-id. package-id, COPYVER

X-BO

Determines that you are copying an existing package and names the package

CURRENTDATA

YES, NO

X

X

X

Determines whether to require data currency for RO and ambiguous cursors when isolation level is CS

CURRENTSERVER

Location-name

X

Determines the location to connect to before running the plan.

DBPROTOCOL

DRDA, PRIVATE

X

X

Specifies which protocol to use when connecting to a remote site that is identified by a three-part name

DEFER

DEFER(PREPARE), NODEFER (PREPARE)

X

X

Determines whether to defer preparation of dynamic SQL that refer to remote objects or to prepare them immediately

DEGREE

1,ANY

X

X

Determines whether to attempt to run a query using parallel processing to maximize performance

DISCONNECT

EXPLICIT, AUTOMATIC, CONDITIONAL

X

Determines which remote connections to destroy during commit operations

DYNAMICRULES

RUN, BIND

X

X

DEFINEBIND, DEFINERUN, INVOKEBIND, INVOKERUN

X

Determines what values apply at run-time for dynamic SQL attributes

ENABLE DISABLE

BATCH, CICS, DB2CALL, DLIBATCH, IMS, IMSBMP, IMSMPP, RRSAF

X

X

REMOTE

X

Determines which connections can use the plan or package

ENCODING

ASCII, EBCIDIC, UNICODE, ccsid

X

X

Specifies the application encoding for all static statements in the plan or package

EXPLAIN

NO, YES

X

X

X

Determines whether or not to populate the PLAN_TABLE with information about the SQL statements

FLAG

I,W,E,C

X

X

X

Determines what messages to display

IMMEDIATE

NO,PH1,YES

X

X

Determines whether immediate writes will be done for updates made to GBP dependent pagesets/partitions

ISOLATION

RR,RS,CS,UR,NC

X

X

X

Determines how far to isolate an application from the effects of other running applications

KEEPDYNAMIC

NO, YES

X

X

Determines whether DB2 keeps dynamic SQL statements at commit points

LIBRARY

dbrm-pds-name (can be multiple for PLAN)

X -BO

X-BO

Determines what PDS to search for DBRMs listed in the member option

MEMBER

dbrm-member-name (can be multiple for PLAN)

X -BO

X-BO

Determines what DBRMs to include in the plan or package

OPTHINT

hint-id

X

X

Controls whether query optimization hints are used for static SQL

OPTIONS

COMPOSITE, COMMAND

X-BC

Specifies which bind options to use for the new package

OWNER

Authorization-id

X

X

Determines the authorization ID or the owner of the object (plan or package)

PACKAGE

Location-name.collection-id.package-id. (version-id)

X

([*])

X-RO

Determines what package or packages to bind or rebind

PATH

Schema-name, USER, (schema-name, (USER)…)

X

X

Determines the SQL path that DB2 uses to resolve unqualified UDTs, functions, and stored procedure name

PKLIST or NOPKLIST

(Location-name.collection-id.package-id,..) PKLIST only

X

Determines what package to include for the package list in the plan

PLAN

Plan-name

X

([*])

X- RO

Determines what plan or plans to bind or rebind

QUALIFIER

Qualifier-name

X

X

Determines the implicit qualifier for unqualified names of objects in the plan or package

RELEASE

COMMIT, DEALLOCATE

X

X

X

Determines when to release resources that the program uses, either at commit or at termination

REOPT

NOREOPT(VARS), REOPT(VARS)

X

X

Determines if access path should be determined at run-time with host variables, parameter markers and special regs.

SQLERROR

NOPACKAGE, CONTINUE

X

Determines whether to create a package if there is a SQL error in it

SQLRULES

DB2, STD

X

Determines if a Type 2 connection can be made according to DB2 rules for an existing connection

VALIDATE

RUN, BIND

X

X

Determines whether to recheck at run-time "not found" and "not authorized" errors found at bind time

[*] BO—BIND only, BC—BIND COPY, RO—REBIND only

Note

If the values in the BIND/REBIND PACKAGE are different from those in the BIND/REBIND PLAN, then the values for the BIND/REBIND PACKAGE will prevail.

Preliminary Steps

Before performing a bind, consider the following:

  • Determine how the DBRMs should be bound (i.e., into packages, directly into plans, or a combination of both methods).

  • Develop a naming convention and strategy for the most effective and efficient use of the plans and packages.

  • Determine when the application should acquire locks on the objects it uses: on all objects when the plan is first allocated, or on each object in turn when that object is first used. (For more details on locking, refer to Chapter 14.)

Invalidations

If an object that a package depends on is dropped, the following occurs:

  • If the package is not appended to any running plan, the package becomes invalid.

  • If the package is appended to a running plan, and the drop occurs outside of that plan, the object is not dropped, and the package does not become invalid.

  • If the package is appended to a running plan, and the drop occurs within that plan, the package becomes invalid.

In all cases, the plan does not become invalid unless it has a DBRM referencing the dropped object. If the package or plan becomes invalid, automatic rebind occurs the next time the package or plan is allocated.

Rebinding a Package

How the collection-id (coll-id) package-id (pkg-id), and version-id (ver-id) on the REBIND PACKAGE subcommand are specified, will determine which packages are bound.

REBIND PACKAGE does not apply to packages for which you do not have the BIND privilege. An asterisk (*) used as an identifier for collections, packages, or versions does not apply to packages at remote sites.

The asterisk can be used on the REBIND subcommand for local packages, but not for packages at remote sites. Any of the following commands rebinds all versions of all packages in all collections, at the local DB2 system, for which you have the BIND privilege.

REBIND PACKAGE (*)
REBIND PACKAGE (*.*)
REBIND PACKAGE (*.*.(*))

Either of the following commands rebinds all versions of all packages in the local collection REGS for which you have the BIND privilege.

REBIND PACKAGE (REGS.*)
REBIND PACKAGE (REGS.*.(*))

Either of the following commands rebinds the empty string version of the package TESTPK in all collections, at the local DB2 system, for which you have the BIND privilege.

REBIND PACKAGE (*.TESTPK)
REBIND PACKAGE (*.TESTPK.())
Package Lists

Using the PKLIST keyword replaces any previously specified package list. Omitting the PKLIST keyword allows the use of the previous package list for rebinding. Using the NOPKLIST keyword deletes any package list specified when the plan was previously bound.

The following example rebinds PLANTK and changes the package list:

REBIND PLAN(PLANTK) PKLIST(GROUP1.*) MEMBER(TEST)

The following example rebinds the plan and drops the entire package list:

REBIND PLAN(PLANTK) NOPKLIST

A list of REBIND subcommands can be generated for a set of plans or packages that cannot be described by using asterisks, using information in the DB2 catalog. You can then issue the list of subcommands through DSN.

One situation in which the technique is particularly useful is in completing a rebind operation that has terminated for lack of resources. A rebind for many objects, say REBIND PACKAGE (*) for an ID with SYSADM authority, terminates if a needed resource becomes unavailable. As a result, some objects are successfully rebound and others are not. If you repeat the subcommand, DB2 attempts to rebind all the objects again. But if you generate a rebind subcommand for each object that was not rebound, and issue those, DB2 does not repeat any work already done and is not likely to run out of resources.

Automatic Rebinding

Automatic rebind might occur if an authorized user invokes a plan or package when the attributes of the data, on which the plan or package depends, have changed, or if the environment in which the package executes changes, has changed. Whether the automatic rebind occurs depends on the value of the field AUTO BIND on installation panel DSNTIPO. The options used for an automatic rebind are the options used during the most recent bind process.

In most cases, DB2 marks a plan or package that needs to be automatically rebound as invalid. A few common situations in which DB2 marks a plan or package as invalid are:

  • When a table, index, or view on which the plan or package depends is dropped

  • When the authorization of the owner to access any of those objects is revoked

  • When the authorization to execute a stored procedure is revoked from a plan or package owner, and the plan or package uses the CALL literal form to call the stored procedure

  • When a table on which the plan or package depends is altered to add a TIME, TIMESTAMP, or DATE column

  • When a created temporary table on which the plan or package depends is altered to add a column

  • When a user-defined function on which the plan or package depends is altered

  • Whether a plan or package is valid is recorded in column VALID of catalog tables SYSPLAN and SYSPACKAGE

In the following cases, DB2 might automatically rebind a plan or package that has not been marked as invalid:

  • A plan or package is bound in a different release of DB2 from the release in which it was first used.

  • A plan or package has a location dependency and runs at a location other than the one at which it was bound. This can happen when members of a data sharing group are defined with location names, and a package runs on a different member from the one on which it was bound.

DB2 marks a plan or package as inoperative if an automatic rebind fails. Whether a plan or package is operative is recorded in column OPERATIVE of SYSPLAN and SYSPACKAGE catalog table.

Whether EXPLAIN runs during automatic rebind depends on the value of the field EXPLAIN PROCESSING on installation panel DSNTIPO, and on whether you specified EXPLAIN(YES). Automatic rebind fails for all EXPLAIN errors except "PLAN_TABLE not found."

Note

The SQLCA is not available during automatic rebind; therefore, some error messages may be available.

Removing a Plan or Package

The only way to remove a plan or package is to use the FREE command. This command will remove the object from the catalog tables and will no longer be available for use. The example below frees all of the packages in the CERTCL collection:

FREE PACKAGE (CERTCL.*)


121 - Application Program Features

Chapter 11. Application Development Overview

Application Program Features

All SQL programs execute as part of an application process. An application process involves the execution of one or more programs, and is the unit to which DB2 allocates resources and locks. Different application processes might involve the execution of different programs, or different executions of the same program. The means of initiating and terminating an application process are dependent on the environment.

COMMIT, ROLLBACK, and SAVEPOINT

More than one application process might request access to the same data at the same time. Locking is used to maintain data integrity under such conditions, preventing, for example, two application processes from updating the same row of data simultaneously. For more information on locking and concurrency, refer to Chapter 14.

DB2 implicitly acquires locks to prevent uncommitted changes made by one application process from being perceived by any other. DB2 will implicitly release all locks it has acquired on behalf of an application process when that process ends, but an application process can also explicitly request that locks be released sooner.

A commit operation releases locks acquired by the application process and commits database changes made by the same process.

DB2 provides a way to back out uncommitted changes made by an application process. This might be necessary in the event of a failure on the part of an application process, or in a deadlock situation. An application process, however, can explicitly request that its database changes be backed out. This operation is called rollback.

The interface used by an SQL program to explicitly specify these commit and rollback operations depends on the environment. If the environment can include recoverable resources other than DB2 databases, the SQL COMMIT and ROLLBACK statements cannot be used. Thus, these statements cannot be used in an IMS, CICS, or WebSphere environment.

Unit of Work

A unit of work is a recoverable sequence of operations within an application process. A unit of work is sometimes called a logical unit of work. At any time, an application process has a single unit of work, but the life of an application process can involve many units of work as a result of commit or full rollback operations.

A unit of work is initiated when an application process is initiated. A unit of work is also initiated when the previous unit of work is ended by something other than the end of the application process. A unit of work is ended by a commit operation, a full rollback operation, or the end of an application process. A commit or rollback operation affects only the database changes made within the unit of work which it ends.

While these changes remain uncommitted, other application processes are unable to perceive them unless they are running with an isolation level of uncommitted read. The changes can still be backed out. Once committed, these database changes are accessible by other application processes and can no longer be backed out by a rollback. Locks acquired by DB2 on behalf of an application process that protects uncommitted data are held at least until the end of a unit of work.

The initiation and termination of a unit of work define points of consistency within an application process. A point of consistency is a claim by the application that the data is consistent. For example, a banking transaction might involve the transfer of funds from one account to another. Such a transaction would require that these funds be subtracted from the first account, and added to the second. Following the subtraction step, the data is inconsistent. Only after the funds have been added to the second account is consistency reestablished. When both steps are complete, the commit operation can be used to end the unit of work, thereby making the changes available to other application processes.

Unit of Recovery

A DB2 unit of recovery is a recoverable sequence of operations executed by DB2 for an application process. If a unit of work involves changes to other recoverable resources, the unit of work will be supported by other units of recovery. If relational databases are the only recoverable resources used by the application process, then the scope of the unit of work and the unit of recovery are the same and either term can be used.

Rolling Back Work

DB2 can back out all changes made in a unit of recovery or only selected changes. Only backing out all changes results in a point of consistency.

The SQL ROLLBACK statement without the TO SAVEPOINT clause specified causes a full rollback operation. If such a rollback operation is successfully executed, DB2 backs out uncommitted changes to restore the data consistency that it assumes existed when the unit of work was initiated. That is, DB2 undoes the work.

Savepoints

Savepoints enable milestones within a transaction (unit of recovery) to be demarked. An external savepoint represents the state of data and schema at a particular point in time. Data and schema changes made by the transaction after the savepoint is set can then roll back to the savepoint, as application logic requires, without affecting the overall outcome of the transaction.

SAVEPOINT name (other options)
ROLLBACK TO SAVEPOINT name

This enables milestones within a unit of recovery and represents the state of the data and schema at a particular point in time. ROLLBACK can be used to restore to a savepoint. This would be useful when a point has been reached during a UNIT-OF-WORK and there is a need to back out without ROLLBACK over the entire UNIT-OF-WORK. The individual SAVEPOINTs can be named and then ROLLBACK can be used to roll back work to whichever point is required, based on the application processing requirements skipping over individual savepoints, if necessary.

Declared Temporary Tables

Declared temporary tables (DTTs) give us the ability to declare a temporary table for use in a program. The DECLARE GLOBAL TEMPORARY TABLE statement defines a temporary table for the current session, not just for a Unit of Work (UOW). The table description does not appear in the DB2 catalog. It is not persistent and cannot be shared (unlike a global temporary table).

This statement can be embedded in an application program or issued through the use of dynamic SQL statements. It is an executable statement that can also be dynamically prepared. Each session that defines a declared global temporary table of the same name has its own unique instantiation of the temporary table. When the session terminates, the temporary table is dropped. With declared temporary tables, some of the locking, DB2 catalog updates, and DB2 restart forward and backward log recoveries that are associated with persistent base tables are avoided. No authority is required to issue the DECLARE GLOBAL TEMPORARY TABLE statement, but there will be an authority required to use the new user temporary table space where the table will be materialized.

Declared temporary tables can be useful for applications that need to extract data from various sources and use them in SQL joins, or for data that needs to be used repetitively, or kept separate from other OLTP processes. They can also be used as staging areas for data that comes from various sources so that the data can be manipulated before storing it permanently in regular tables.

A couple of examples of the syntax for declared temporary tables are shown below.

DECLARE GLOBAL TEMPORARY TABLE SESSION.CERTTEST
    LIKE DB2USER1.TEST
INSERT INTO SESSION.CERTEST
    SELECT * FROM DB2USER1.TEST
    NOT LOGGED

DECLARE GLOBAL TEMPORARY TABLE SESSION.CERTEST
    AS
    (SELECT * FROM DB2USER1.TEST)
DEFINITION ONLY
NOT LOGGED
Usage Considerations

These declared temporary tables can be used as a way to temporarily hold or sort data within a program. DTTs are useful for ROLAP and MOLAP queries for warehouse tools and as staging area for IMS of VSAM data so it is SQL and ODBC accessible. The word SESSION must be the qualifier for a declared temporary table and can be named explicitly in the table name or can be in the QUALIFIER BIND option on the plan/package.

Only Undo records are logged and a the full range of DML (INSERT,UPDATE, SELECT, DELETE) can be performed on them. DTTs are supported by rollback to the savepoint or last commit point. The table exists until thread termination, or if thread reuse is being used, it will exist until it is implicitly dropped (which may or may not be desirable, depending on the application).

No locks are taken (PAGE, ROW, TABLE) on DTTs; however, there are locks taken on table space and DBD in share mode. They also do not require a declared cursor to hold rows across commits.

Static SQL referencing a DTT will be incrementally bound at run-time. The cost associated with a DTT is equivalent to the cost of executing a dynamic SQL statement. High-volume transaction applications need careful evaluation when planning to use DTTs.

Note

Any dynamic SQL statements that are referencing DTTs will not be able to use the dynamic SQL cache.

There are some restrictions when using DTTS. They do not support the following:

  • LOBs or ROWIDs

  • Referential integrity

  • Use in a CREATE TABLE LIKE statement

  • Sysplex query parallelism

  • Dynamic SQL caching

  • ODBC/JDBC functions that rely on the catalog definitions

  • Thread reuse for DDF pool threads

  • Use within triggers

TEMP Database

DTTs are not materialized in DSNDB07, but rather in a segmented TEMP table space. There must be a TEMP database created prior to the creation of the DTTs. This database is not sharable across data sharing members; each member must have its own. There several segmented table spaces will need to be created. DB2 will decide where tables are created. PUBLIC will automatically have authority to create DTTs in these table spaces.

The declared temporary tables cannot span table spaces. This fact does give you the ability to have small table spaces and be able to control size of any DTT that is created. There may be a need to have different tablespaces for 4K, 8K, 16K, and 32K pages. DB2 will choose an appropriate one and if one does not exist, the DECLARE fails.

Note

It is recommended to use the same size for each— spread across volumes.

Note

It is important to size this database accordingly to accommodate growth and concurrent transactions—those transactions that are all using DTTs at a given point in time.

Fetch for Limited Rows

The ability to tell DB2 to fetch only the first or top "n" rows is a requirement in many applications. There are ways in SQL and the application program to do this, however, they are not the most efficient. With the FETCH FIRST clause, DB2 has this functionality with improved performance, as the clause sets a maximum number of rows that can be retrieved through a cursor. The application specifies that it does not want to retrieve more than some fixed number of rows, regardless of how many rows qualified for the result set.

The OPTIMIZE FOR n ROWS clause, which provides additional information to the optimizer about the intent, is different from the FETCH FOR clause. The OPTIMIZE clause could allow the entire answer set to be retrieved. In contrast, the FETCH FOR n ROWS ONLY stops processing after the specified number of rows. This provides direct control to the application programmer on the size of the result table. For example, an implementation limit on the number of rows displayed for an on line screen can now be enforced.

Global Transactions

Global transactions allow for several units of work to share locks if they are all participants in a unit of work. The large unit of work is referred to as a global transaction. This provides a way to avoid deadlocks between transactions. It may be considered if you want to combine transactions into a large transaction (single unit of work) for several reasons. For example, this may be an option if you are adding OO (Object Oriented) interfaces on existing transactions.

This simply is an extension of the distributed unit of work and adds ability to share locks. It supports IMS, RRS, and DDF transactions. These transactions will be identified via a token—global transaction ID, or XID. All of the URs in a global transaction must run on the same subsystem and each part can see uncommitted updates of other parts.

The sharing of locks is limited to normal transaction locks and a LOCK TABLE or partition key updates could still cause deadlocks/timeouts. There are many re-design issues to consider when considering the use of global transactions and there are also additional risks and exposures.


122 - Stored Procedure Builder

Chapter 11. Application Development Overview

Stored Procedure Builder

The Stored Procedure Builder tool is a graphical application that supports the rapid development of DB2 stored procedures written in Java or SQL. The builder can be launched as a separate application on the client workstation, or from the integrated development environments such as VisualAge for Java or Microsoft VisualStudio. The Stored Procedure Builder can be used to create stored procedures on local and remote DB2 servers, and test, execute, and debug stored procedures. Figure 11.2 shows the main integrated development environment in the Stored Procedure Builder.

Figure 11.2. Stored Procedure Builder.

SPB comes with design assistants that guide you through basic design patterns, help you create SQL queries, and estimate the performance cost of invoking a stored procedure. Stored procedures created with SPB are implemented with Java, and all database connections are managed using Java Database Connectivity (JDBC). SQL statements can be specified using JDBC, SQLJ, or the SQL Procedure Language. Once you develop the basic SQL structure using SPB, if you have highly sophisticated stored procedure logic you can use your editor to modify the stored procedure.

SPB provides an easy-to-use development environment for creating, installing, and testing stored procedures so that you can focus on creating your stored procedure logic rather than on the details of registering, building, and installing stored procedures on a DB2 server. Since the SPB helps you create stored procedures in Java, they are highly portable among operating systems. The SQL Procedure Language is also portable among all members of the DB2 Family. Some of the more important features of the SPB are:

  • For new stored procedures:

    • Wizards lead through steps

    • Generate Java SP with static or dynamic SQL

    • Generate SQL SP

  • For existing stored procedures:

    • View and modify

    • Execute

    • One-step build / rebuild


123 - Summary

Chapter 11. Application Development Overview

Summary

In this chapter, we looked at the various components of a DB2 application program. We looked at how to construct a program and have it communicate with DB2. DB2 is able to use data in an application program by means of host variables and host structures. We are able to move through result sets of our data by means of cursors. These cursors must be declared and opened before fetching rows of data is possible. There are a couple type of cursors (scrollable and non-scrollable) that can be used.

In order to have an application be executable in a DB2 environment, it must be precompiled and then bound into a package and a plan. A plan is the actual executable in DB2.

There are also many options for flexibility available through the use of dynamic SQL.

In the next chapter, we will look at some of the various options for coding and executing these applications.


124 - Additional Resources

Chapter 11. Application Development Overview

Additional Resources

  • IBM DB2 UDB for OS/390 Application Programming and SQL Guide, SC26-9933.


125 - Chapter 12. Development Considerations

Part IV. Developing Applications

Chapter 12. Development Considerations

  • SQL Procedure Language

  • Extenders

  • DB2 ODBC

  • JDBC and SQLJ

  • Stored Procedure Builder

There are many features of the DB2 development environment that offer the use of open standards and the newer languages. We will take a look at the SQL Procedure Language for developing stored procedures. We will also examine the Stored Procedure Builder Tool that can be used for implementing stored procedures built with the SQL Procedure Language and also Java. In the area of extended data types and objects, we will discuss the large object extenders for distinct data types such as video, text, and XML.

For the dynamic and portable world, we will look at dynamic SQL programming with the DB2 ODBC interface, as well as the JDBC and SQLJ interfaces for the Java language.


126 - SQL Procedure Language

Chapter 12. Development Considerations

SQL Procedure Language

The SQL Procedure Language is a procedural language that was designed only for writing stored procedures and is available across the entire DB2 family including the AS/400. It is based on the ANSI/ISO standard language SQL/PSM. Its major benefit is that users can create stored procedures very quickly using a simple, easily understood language, without the headaches of precompilers, compilers, link editors, binding, and special authorizations. This is because stored procedures written using SQL Procedure Language will be mostly managed by DB2; this automates the process and allows programmers and users to simply write the logic and pass it off to DB2.

SQL Procedures support multiple parameters for input, output, and input/output as well as returning output results sets to clients. SQL Procedures are defined in the DB2 catalog, and the source is stored there also. The following shows a SQL Procedure that was created from a client workstation using the Stored Procedure Builder tool. It accepts one input parameter and returns a result set to the client.

CREATE PROCEDURE DW.SQLProc1 ( IN gender char(6) )
    SPECIFIC DW.Genders       RESULT SETS 1
    LANGUAGE SQL
P1: BEGIN
    DECLARE gender_value CHAR(1);
    DECLARE bad_gender CONDITION FOR SQLSTATE '99001';
    CASE gender
       WHEN 'MALE' THEN
           SET gender_value = 'M';
       WHEN 'FEMALE' THEN
           SET gender_value = 'F';
       ELSE SIGNAL bad_gender;
    END CASE;
    -- Declare cursor
    DECLARE cursor1 CURSOR WITH RETURN FOR
        SELECT E.EMPNO, E.LASTNAME, E.HIREDATE, D.DEPTNO,
               D.DEPTNAME
        FROM DW.DEPARTMENT D, DW.EMPLOYEE E
        WHERE E.WORKDEPT = D.DEPTNO
          AND E.SEX = :gender_value
        ORDER BY E.WORKDEPT;
    -- Cursor left open for client application
    OPEN cursor1;
END P1

The language itself is primarily SQL (DML and DDL) with local variables, cursors, assignment statements, flow control, and signal/resignal conditions. The real difference with the SQL Procedure Language is how it becomes a stored procedure, since all one does is write a "CREATE PROCEDURE…LANGUAGE SQL...name: BEGIN...END name" DDL statement. All the code is in the body of the create statement. The procedure body includes:

  • Compound statements

  • Declaration statements

  • Assignment statements

  • Conditional statements

  • Iterative control structures

    • Loop

    • Repeat

    • While

  • Exception handling

  • Call another stored procedure

The primary language statements that show the strength of this approach are:

IF, CASE and LEAVE
LOOP, REPEAT, and WHILE
FOR, CALL, and RETURN
GET DIAGNOSTICS
SIGNAL and RESIGNAL

Developing SQL Stored Procedures

A SQL procedure is created using one of the following three methods:

  • Stored Procedure Builder

  • Directly invoking DSNTPSMP

  • Using JCL or CLIST to prepare

When a SQL procedure is built using the Stored Procedure Builder (SPB) tool (discussed later in this chapter), all the "code" is developed on the client workstation, and the completed procedure is passed to DSNTPSMP. DSNTPSMP is the OS/390 SQL Procedure Processor, which is a REXX Stored Procedure, and it is also fully customizable. This processor invokes the following steps:

  • SQL precompile

  • C precompile

  • C compile and prelink

  • Link

  • Procedure definition

  • Bind

Once the procedure is built in this fashion, it is immediately executable. When using the other two methods for developing a SQL procedure, they basically just bypass the SPB, and either directly invoke DSNTPSMP or execute the steps listed above under manual control. There is quite a lot of information on the Web about SQL Procedures and the SPB.

  • General:

    • www.ibm.com/software/data/db2

  • OS/390 z/OS SQL Procedures:

    • www.ibm.com/software/data/db2/os390/sqlproc

  • DB2 Stored Procedure Builder:

    • www.ibm.com/software/data/db2/os390/spb

  • Microsoft Add-ins:

    • www.ibm.com/software/data/db2/udb/ide

  • Redbook:

    • Developing Cross-Platform DB2 Stored Procedures: SQL Procedures and the DB2 Stored Procedure Builder, SG24-5485


127 - Extenders

Chapter 12. Development Considerations

Extenders

Extenders for DB2 help in the use of large objects (LOBs), the base storage for the object relational environment. Extenders are a complete package that defines distinct data types and special functions for many types of large objects including image, audio, video, text, XML, and spatial objects. This allows us not to worry over defining these data types and functions in applications. You can use SQL to manipulate these data types and functions. LOBs can be from relatively small to extremely large, and can be cumbersome to deal with.

Image, Audio, and Video

Image, audio, and video objects are stored as BLOBs in a DB2 database. The image, audio, and video extenders work on the large binary objects. The DB2 extenders provide user-defined functions to store, access, and manipulate image, audio, and video data in a database server. You code requests for these UDFs in your application program using SQL statements in the same way that you request SQL built-in functions. Like built-in functions, UDFs are run in the database server.

The DB2 Video Extender can store as many as three representative frames per shot. By displaying the frames, you get a quick, yet effective view of a video's content. The DB2 Video Extender provides sample programs that demonstrate how to build and display a video storyboard.

Video storyboards give you the opportunity to preview videos before you download and view them. This can save you time and reduce video traffic on the network.

When inserting image data into a table using the DB2IMAGE user-defined functions, many processes are performed for the application automatically. The following code demonstrates using this function:

EXEC SQL INSERT INTO CONSULTANTS VALUES(
    :cons_id,
    :cons_name,
    DB2IMAGE(
        CURRENT SERVER,
        '/YLA/images/current.bmp'
        'ASIS',
        MMDB_STORAGE_TYPE_INTERNAL,
        :cons_picture_tag);

In this particular example, the DB2IMAGE will read all the attributes about the image (height, width, colors, layers, pixels, and more) from the source image file header, in this case, the "current.bmp". All the input is of a standard supported format, and all graphic files contain header information about the structure of the content. The function then creates a unique handle for the image, and records all the information in the support table for administrative control of the image. This table would contain:

  • The handle for the image

  • A timestamp

  • The image size in bytes

  • The comment contained in :cons_picture_tag

  • The content of the image

The content of the image source file is inserted into the LOB table as a BLOB. There is no conversion done and the image is stored in its native format. There is a record in the administrative table that contains all the image-specific attributes, such as the number of colors in the image, as well as a thumbnail-sized version of the image.

In this example, there is use of the storage type constants. Here we used MMDB_STORAGE_TYPE_INTERNAL to store the image into a database table as a BLOB. By using the extenders, we could have stored it elsewhere. If you want to store the object and have its content remain in the original file on the server, you can specify the constant MMDB_STORAGE_TYPE_EXTERNAL. Just because you are using LOBs does not mean that they have to be in DB2 managed tables. The administrative support table for image extenders tells where the LOB is actually stored. This does require Open Edition support services on the S390.

Text

DB2 Text Extender enables programmers to include SQL queries for text documents in their applications. DB2 Text Extender adds the power of full-text retrieval to SQL queries by making use of features available in DB2 UDB for OS/390 and z/OS that let you store unstructured text documents in databases. DB2 Text Extender offers DB2 UDB for OS/390 and z/OS users and application programmers a fast, versatile, and intelligent method of searching through such text documents. DB2 Text Extender's strength lies in its ability to search through many thousands of large text documents at high speed, finding not only what you directly ask for, but also word variations and synonyms. At the heart of DB2 Text Extender is IBM's high-performance linguistic search technology

It allows your applications to access and retrieve text documents in a variety of ways. Your applications can:

  • Search for documents that contain specific text, synonyms of a word or phrase, or sought-for words in proximity, such as in the same sentence or paragraph.

  • Do wildcard searches, using front, middle, and end masking, for word and character masking.

  • Search for documents of various languages in various document formats.

  • Make a "fuzzy" search for words having a spelling similar to the search term. This is useful for finding words even when they are misspelled.

  • Make a free-text search in which the search argument is expressed in natural language.

  • Search for words that sound like the search term.

You can integrate your text search with business data queries. For example, you can code an SQL query in an application to search for text documents that are created by a specific author, within a range of dates, and that contain a particular word or phrase. Using the DB2 Text Extender programming interface, you can also allow your application users to browse the documents. By integrating full-text search into DB2 UDB for OS/390 and z/OS's SELECT queries, you have a powerful retrieval function. The following SQL statement shows an example:

SELECT *
FROM MYTEXTTABLE
WHERE VERSION ='2'
AND DB2TX.CONTAINS (
   DB2BOOKS_HANDLE,
   '"authorization"
    IN SAME PARAGRAPH AS "table"
    AND SYNONYM FORM OF "delete"')=1

DB2TX.CONTAINS is one of several DB2 Text Extender search functions. DB2BOOKS_HANDLE is the name of a handle column referring to column DB2BOOKS that contains the text documents to be searched. The remainder of the statement is an example of a search argument that looks for authorization, occurring in the same paragraph as table, and delete, or any of delete's synonyms.

XML

An XML extender has been added to the DB2 extenders. This will allow for a method of marking the meaning of the data so it can be more easily used. XML is an acronym for Extensible Markup Language. It is extensible in that the language itself is a meta-language that allows you to create your own language depending on the needs of your enterprise.

For Web-based businesses, DB2 V7 provides more native support for XML via a DB2 extender. It is now possible to combine structured XML information with traditional relational data and you can choose whether to store entire XML documents in DB2 as a XML Character Large Object (XMLCLOB), or you can map the XML content as traditional data in relational tables. The XML Extender adds the ability for powerful searching of rich data types of XML element or attribute values, even for nontraditional XML data types.

Incoming XML documents can be decomposed into traditional SQL data types and placed in columns. Traditional SQL data can also be used to compose outgoing XML documents. There are mapping methods implemented to provide for the transformation between XML documents and relational data. This will allow for decomposing an XML document into one or more pieces for storing in tables. Data in existing relational tables can be used to compose XML documents.

By using the XML Extender and its set of defined UDTs, an application can store entire XML documents as column data in an application table. The UDTs also provide for the composition or decomposition of the contents of XML documents from or into an XML collection. These XML collections can be made up of one or more regular DB2 tables. There is a long list of capabilities that these user-defined functions provide.

Net.Data also now has built-in exploitation to generate XML tags rather than manually entering them.


128 - DB2 ODBC Programming

Chapter 12. Development Considerations

DB2 ODBC Programming

DB2 Open Database Connectivity (ODBC) is IBM 's callable SQL interface by the DB2 family of products. It is a "C" and "C++" application programming interface for relational database access, and it uses function calls to pass dynamic SQL statements as function arguments. It is an alternative to embedded dynamic SQL, but unlike embedded SQL, it does not require a precompiler. DB2 ODBC is based on the Microsoft Open Database Connectivity (ODBC) specification, and the X/Open Call Level Interface specification. These specifications were chosen as the basis for the DB2 ODBC in an effort to follow industry standards and to provide a shorter learning curve for those application programmers already familiar with either of these data source interfaces. In addition, some DB2 specific extensions were added to help the DB2 application programmer specifically exploit DB2 features. DB2 ODBC is derived from the ISO Call Level Interface Draft International Standard (ISO CLI DIS) and ODBC Version 2.0.

Embedded Dynamic Vs. Call Level Interface

DB2 ODBC access is by default dynamic but the traditional access types like CAF, DSN, IMS, and CICS can also issue dynamic SQL. Every access to DB2, dynamic or not, needs a plan. Dynamic SQL is just a kind of SQL statement in a plan. The program has a buffer containing a SQL statement, e.g., "DELETE FROM DEPT WHERE DEPTNUMBER=100". Now by issuing an EXEC SQL EXECUTE IMMEDIATE STATEMENT, you can give this statement to DB2. The statement is optimized and executed with this action. It becomes a little more complex when we are using host variables or you want to do a SQL SELECT. Host variables should be indicated in the statement as question marks and you now have to use an EXEC SQL PREPARE STATEMENT and EXEC SQL EXECUTE STATEMENT USING ws-field. The prepare will invoke the optimizer and the result will be a so-called mini-plan. The EXECUTE will execute the prepared statement; if there are host variables, they will be given with the EXECUTE statement. A SQL SELECT is a special case because the program does not know the type and number of columns that will be returned by DB2. As with a "normal" program, you have to create a cursor unless you know that only one row will be returned (a so-called singleton select). The EXEC SQL PREPARE now prepares this cursor. This is where you can issue a EXEC SQL DESCRIBE and DB2 will describe the result set in the SQLDA. The SQLDA is a kind of array where every entry describes the column that will be returned. The program can now allocate a working storage field to receive the data. After this is done, you can do a EXEC SQL OPEN that executes the SELECT statement.

So PREPARE and EXECUTE are normal SQL statements, and the only way to execute a statement is to create a DBRM using the preprocessor and bind them as a plan or package. The bind for PREPARE and EXECUTE will be extremely fast because the optimizer cannot optimize anything for these statements; all the work is done at run-time.

Now why is it that programs that use DB2 ODBC do not have to bind anything? This is because at run-time your program calls the functions of DB2 ODBC. These functions use statements that are part of a standard plan used by this interface. For OS390 this plan is called DSNACLI.

DB2 ODBC is based on the ODBC and CLI standards that are based on a set of APIs (application programming interfaces) that access data sources. ODBC was originally based on an early version of CLI. DB2 provides both ODBC and CLI drivers.

The DB2 ODBC Environment

DB2 ODBC does not support an ODBC driver manager. All API calls are routed through the single ODBC driver that is loaded at run-time into the application address space. DB2 ODBC support is implemented as an IBM C/C++ Dynamic Load Library (DLL). By providing DB2 ODBC support using a DLL, DB2 ODBC applications do not need to link-edit any DB2 ODBC driver code with the application load module. Instead, the linkage to the DB2 ODBC APIs is resolved dynamically at run-time by the IBM Language Environment (LE) run-time support. The DB2 ODBC driver can use either the call attachment facility (CAF) or the Recoverable Resource Manager Services attachment facility (RRSAF) to connect to the DB2 for OS/390 and z/OS address space.

  • If the DB2 ODBC application is not running as a DB2 for OS/390 and z/OS stored procedure, the MVSATTACHTYPE keyword in the DB2 ODBC initialization file determines the attachment facility that DB2 ODBC uses.

  • If the DB2 ODBC application is running as a DB2 for OS/390 and z/OS stored procedure, then DB2 ODBC uses the attachment facility that was specified for stored procedures.

When the DB2 ODBC application invokes the first ODBC function, SQLAllocEnv(), the DB2 ODBC driver DLL is loaded.

DB2 ODBC supports access to the local DB2 for OS/390 and z/OS subsystems and any remote data source that is accessible using DB2 for OS/390 and z/OS Version 7. This includes:

  • Remote DB2 subsystems using specification of an alias or three-part name

  • Remote DRDA-1 and DRDA-2 servers using LU 6.2 or TCP/IP


129 - DB2 and Java

Chapter 12. Development Considerations

DB2 and Java

DB2 for OS/390 and z/OS supports SQLJ and JDBC for Java database applications. In general, Java applications use SQLJ for static SQL, and JDBC for dynamic SQL. However, there are features of JDBC that cannot be used yet in SQLJ, because the standard for SQLJ is still maturing.

JDBC Programming

JDBC (Java database connectivity) is an API that provides a uniform SQL interface to a large selection of relational databases and a common base on which higher-level interfaces and tools can be developed.

On the server side, JDBC supports both fenced and unfenced UDFs and stored procedures in Java. Of course, on OS/390, all stored procedures are fenced to protect the environment.

On the client side, JDBC is considered to be an "equivalent" to ODBC. JDBC is a dynamic SQL interface that has become a standard interface for Java applications needing to access several different relational database systems. Although not as simple as static or embedded SQL, it does provide a perceived level of portability.

JDBC is very similar to DB2 ODBC and is actually built on top of it. JDBC has the same features as DB2 ODBC, with a few more advantages. First, by using JDBC, you can code Java applets to be executed by a Java-enabled browser, allowing your data to be accessed from the Web without additional client software. Second, you can access relational databases by using an object-oriented interface, because JDBC is integrated into the Java language.

A JDBC applet is a program written in Java that can be invoked by any Java-enabled Web browser when a page is displayed. By using the facilities of JDBC, the applet can connect to the database, and retrieve and display data. Use of JDBC applets can allow computers that can access the Internet to access your data on DB2.

SQLJ Programming

SQLJ is an implementation of static SQL to be used with the Java programming language. It was designed by IBM, Oracle, and Sun to eliminate the weaknesses of JDBC. It allows most of the normal SQL language in a Java module, as shown below.

#sql iterator Honors ( String, float);
Honors honor;
#sql (recs) honor =
  { SELECT STUDENT, SCORE FROM GRADE_REPORTS
  WHERE SCORE >= :limit
  AND ATTENDED >= :days
  AND DEMERITS <= :offences
  ORDER BY SCORE DESCENDING };
while (true) {
  #sql {FETCH :honor INTO :name, :grade };
  if (honor.endFetch()) break;
  System.out.println( name + " has grade " + grade );
  }

The static SQL syntax for Java is easier than JDBC and the performance is much better than JDBC. It provides popular language support for binary portability of static SQL across DBMS vendors. With SQLJ, Java Stored Procedures can be developed that can be ported across vendors.

SQLJ supports:

  • INSERT

  • UPDATE

  • DELETE

  • Singleton SELECT

  • Cursor SELECT

  • Calling stored procedures, including result sets

  • COMMIT and ROLLBACK

  • Methods for CONNECT and DISCONNECT

Dynamic SQL is supported only through JDBC. When using both SQLJ and JDBC in a single process, you must share the connection handles and connect state.

With SQLJ, you have Java applications with SQLJ clauses that are translated by a Java precompiler and that produce modified Java code and a profile. Profiles are platform independent descriptions of the SQLJ clauses. The profile is then customized and produces a DB2 for OS/390 dependent DBRM that is bound into a package or plan. The application is executed via OS/390 JVM. The profiles are portable across platforms. Figure 12.1 shows the compilation process for SQLJ.

Figure 12.1. SQLJ and Compilation.

A profile generated on Oracle, compiled on NT against Oracle8, can be moved to DB2 for OS/390 on OS/390 and can customized and bound into a DB2 package or plan without modification of any source code. SQLJ applications have all the benefits of JDBC and also have the advantages of the static model SQLJ Compilation.


130 - Stored Procedure Builder

Chapter 12. Development Considerations

Stored Procedure Builder

The Stored Procedure Builder is a graphical application Windows-based process that runs on Windows NT, 95, or 98 that supports the rapid development of DB2 stored procedures written in either Java or SQL (the SQL procedural language). It can work as a Microsoft Visual Studio plug-in, or with Microsoft Visual Basic, IBM VisualAge for Java, or stand alone. For the SQL Procedure Language, it is used to generate a C program which will then be precompiled, compiled, link-edited, and bound. For Java, it invokes the necessary processes to build a Java stored procedure.

The Stored Procedure Builder (SPB) supports the entire DB2 family from a single development environment. Using SPB, you can perform the following tasks:

  • Create new stored procedures

  • Build stored procedures on local and remote DB2 servers

  • Modify and rebuild existing stored procedures

  • Run stored procedures to test the execution of installed stored procedures

The SPB is implemented with Java and all database connections are managed by using JDBC, so you can connect to any local DB2 alias or any other database for which you can specify a host name, port, and database name.

There is a stored procedure SmartGuide and SQL Assistant to help you develop stored procedures using the SPB. You launch the stored procedure SmartGuide from the Project window of the SPB. But the most important feature is that you can create stored procedures in Java and SQL. Creating stored procedures in Java and SQL produces stored procedures that are highly portable. You can use the code editor to modify the stored procedure.

Note

Using the SPB with the SQL Procedure Language is also a nice way to develop a C program very rapidly that can then be maintained and developed further simply as C source code.

When creating a stored procedure, you can choose to return a single result set, multiple result sets, or output parameters only. You can use the stored procedure SmartGuide to define input and output parameters for a stored procedure so that it receives values for host variables from the client application. When you finished building a stored procedure on a target database, the SPB process will drive the create procedure process in DB2 so there is nothing more to do other than use the SPB as shown in Figure 12.2.

Figure 12.2. Stored Procedure Builder main interface.

After successfully building a stored procedure, the SPB will allow you to work with existing stored procedures in order to modify, rebuild, run, or test a procedure. When you open a stored procedure, the stored procedure source code is displayed in the editor pane on the right side. The editor pane has a language-sensitive editor for stored procedures. The color in the editor display window designates between user code, syntax, reserved words, and so on.

To test a stored procedure using this tool, you can run it from within the SPB to make sure that it is correctly installed. Testing stored procedures this way makes programming a lot easier and you know that the stored procedure is correctly installed on the DB2 database server. You can also drop a stored procedure or copy it to another database connection from within the SPB.


131 - Summary

Chapter 12. Development Considerations

Summary

In this chapter we have examined additional features of the DB2 development environment. One of the most important features for rapid application development and control is the Stored Procedure Builder tool, which can easily create and maintain stored procedures written in either the SQL procedure language or Java.

Java is becoming widely accepted as the next universally used language, and to interact with DB2, there is both dynamic SQL via the JDBC API, and static SQL using the SQLJ. SQLJ lets the application programming use normal SQL without having to learn the complexity of JDBC.

For the other applications that require portability, we looked at the continually emerging DB2 ODBC application-programming interface. This is a combination of all the features in both ODBC and CLI. Programs with DB2 ODBC can be moved to any platform with relative ease.

DB2 has implemented object-relational support with large objects and their supporting components. But the basics are just large objects that are either character or binary, and have no object description. By using the extenders, these large objects take on their unique characteristics such as video and audio. The extenders provide an API library, user-defined functions, and user-defined distinct datatypes, as well as some specific stored procedures. One of the most important additions to the extenders has been the implementation of XML, which is a key component in the interchange of documents in the e-business world.


132 - Additional Resources

Chapter 12. Development Considerations

Additional Resources

  • IBM DB2 UDB Application Programming Guide and Reference for JAVA, SC26-9032.

  • IBM DB2 UDB ODBC Guide and Reference, SC26-9941.


133 - Chapter 13. Advanced Functions

Part IV. Developing Applications

Chapter 13. Advanced Functions

  • Stored Procedures

  • Triggers

  • User-Defined Data Types (Distinct Types)

  • User-Defined Functions

  • Object-Relational Extensions (Large Objects)

This chapter will cover some of the more powerful features of DB2's SQL language. The extended programming features, stored procedures, and user-defined functions are discussed along with triggers and user-defined data types. In addition, the object-relational extensions are examined.


134 - Stored Procedures

Chapter 13. Advanced Functions

Stored Procedures

A stored procedure is a compiled program which is stored at a local or remote DB2 server. This procedure can execute SQL statements, as well as connect to other DB2 servers and retrieve data from non-database systems. A stored procedure typically contains two or more SQL statements and some manipulative or logical processing in a host language. A client application program uses the SQL CALL statement to invoke a stored procedure.

EXEC SQL CALL MYSP (parm1, parm2, parm3)

A thread invoked SQL to call the stored
procedure passing it a series of parameters
that match the stored procedure definition

One of the main reasons for using stored procedures is to reduce the network traffic in a distributed environment caused by client application programs that execute many remote SQL statements. Remote SQL statements can create many network send and receive operations to the server. A stored procedure can encapsulate many of the application's SQL statements into a single message to DB2, thus reducing network traffic.

Another reason for using stored procedures is to encapsulate a business function into a single module, coded once, and stored at the server for all processes to use. Figure 13.1 shows how using stored procedures can significantly reduce traffic across the network.

Figure 13.1. Stored Procedure Network Improvements.

Additional Benefits

Some of the additional benefits of stored procedures include:

  • Simplifies development and maintenance

  • Removes client dependency on database design at server

  • Allows for the ability to dynamically change application programs

  • Code is changed and refreshed at the server

  • Client code does not need any change

  • Client code can continue to run while changes made

  • Most of application can exist at server, not client

  • Less code needs to be globally changed at all client locations

  • Code is reusable

  • Improves security

  • Eliminates the need for end-user table authority

  • Moves processing away from end-users

  • Accesses and updates data that is not stored in DB2, such as VSAM or IMS

Language Support

The program in the stored procedure can be written in several different languages:

  • C, C++ (LE/370 1.4)

  • COBOL, OO COBOL (LE/370 1.5)

  • PL/1

  • Assembler

  • REXX

  • Java

  • SQL procedure language

The calling program and the procedure can be in different languages. For example, the client code could be COBOL and the stored procedure could be C. The connection is through thread; individual coding does not matter. The stored procedure code is compiled and stored at the server. It can contain static SQL and dynamic SQL, as well as IFI calls, and can issue DB2 commands.

LE/370

The LE/370 product libraries are used to load and execute the stored procedures in the stored procedures address space. When you create a stored procedure (by the insert process in V5 or the CREATE DDL in V6), the run-time information is passed for LE/370 when the stored procedure is executed. LE/370 is used because it establishes a common run-time environment for the many different languages used for building stored procedures. It provides a consistent set of interfaces for run-time services. When using LE/370, you do not have to specify the language-specific libraries in the JCL procedure of the stored procedures address space.

LE/370 performs several functions for DB2:

  • Hides the differences among programming languages

  • Provides the ability to make a stored procedure resident in the stored procedures address space

  • Supports a large number of run-time options, including the possibility to debug your stored procedures

Note

The LE/370 run-time options can be used to invoke the CODE/370 debugger or use the VisualDebugger.

Creating/Altering Stored Procedures

The following statement shows an example of creating a stored procedure. There are many options when creating stored procedures, but some of the main ones include a parameter list, the language, and its external program name. The actual definition is stored in the catalog table SYSROUITNES, and the parameter definitions are stored in SYSPARMS. A stored procedure is created by the CREATE PROCEDURE statement, and its definition can be altered by the ALTER PROCEDURE statement.

CREATE PROCEDURE SYSPROC.MYSP1 (PARM1  SMALLINT IN,
      PARM2 CHAR(10) INOUT,
      CODE INTEGER OUT)
   EXTERNAL NAME 'CERTSP1'     -- Load module name
   LANGUAGE COBOL              -- language
   PARAMETER STYLE GENERAL     -- type of parameter list
   COLLID DB2SPS               -- collection ID
   ASUTIME LIMIT 2000          -- maximum amount of SUs allowed
   STAY RESIDENT YES           -- make memory resident
   DYNAMIC RESULT SETS 3       -- 3 result sets returned
   NOT DETERMINISTIC           -- different results with each call
   MODIFIES SQL DATA           -- Issues SQL
   WLM ENVIRONMENT MYAPPS      -- requires a WLM environment
   PROGRAM TYPE MAIN           -- main, not sub-program
   RUN OPTIONS('TRAP(ON)')     -- specifies the LE run options
   COMMIT ON RETURN NO         -- do not commit on exit

ALTER PROCEDURE SYSPROC.MYSP1 STAY RESIDENT NO

The alter command will take effect as long as the stored procedure is not currently active in the system. If the stored procedure is active, it will have to be stopped and restarted. If active, there will be different commands for this depending on whether it is currently under the control of DB2 or WLM (see SPAS and WLM environments below).

Schema Qualification

The stored procedure name is implicitly or explicitly qualified by a schema. If no schema name is used to qualify the name, then it is qualified by the following rules:

  • If the statement is embedded in a program:

    • The schema name is the authorization ID in the QUALIFIER bind option when the plan or package was created or last rebound. If the QUALIFIER was not specified, the schema name is the owner of the plan or package.

  • If the statement is dynamically prepared:

    • The schema name is the SQL authorization ID in the CURRENT SQLID special register.

There is more on the schema implementation in the section on schemas in this chapter.

Removing Stored Procedures

Procedures are removed from the system by using the DROP PROCEDURE command, as in:

DROP PROCEDURE MYSPAP RESTRICT

This will drop the stored procedure unless the rules identified by the required keyword RESTRICT hold true. RESTRICT prevents the procedure from being dropped if a trigger definition contains a CALL statement with the name of the procedure.

When a procedure is directly or indirectly dropped, all privileges on the procedure are also dropped. In addition, any plans or packages that are dependent on the procedure are made inoperative.

SPAS and WLM Execution Environments

Stored Procedure Address Space (SPAS)

All of the stored procedures that are associated with the DB2 established address space (SPAS) will be executing in a single address space. If one procedure fails, it can effect the others that are also executing in that address space.

All of the stored procedures in this environment will execute at the priority of the address space. There is also no support for two-phase commit when updating non-relational data.

Administration of stored procedures, when being used in a DB2 managed stored procedure address space, includes:

  • START PROCEDURE/STOP PROCEDURE ACTION()

    • Allows queuing option for callers to dynamically start and stop the ability to use/execute the stored procedure

    • It is possible to START or STOP procedures with SCOPE(GROUP) option for data sharing

  • DISPLAY procedures

    • Ability to DISPLAY the status of a stored procedure

  • Display thread updated

    • Information on threads waiting for stored procedures

    • Information on threads executing within stored procedures

The START and STOP can be issued for an individual stored procedure unless they are in a WLM managed address space (see below) where they operate differently, and control of the environment is by WLM.

The DISPLAY PROCEDURE command will yield information about the status of any stored procedure currently started or executing in the address space, but will not display any information on the individual threads using the stored procedure. For that, the DISPLAY THREAD is required.

Work Load Manager (WLM)

While stored procedures can run in a DB2-establish stored procedure address space (SPAS), it is highly recommended that they execute in a WLM (Workload Manager) goal mode application environment. For more information on the SPAS and WLM refer to Chapter 4, Architecture and Subsystem. The reason for this is that WLM can support multiple address spaces. This gives the ability to incur a failure of a routine without affecting other routines in another WLM environment.

Administration of stored procedures, when being used in a WLM managed address space, is under the control of MVS WLM. The START command is the same but just works differently. But in general, you would use WLM commands to control the stored procedures, which are invoked as MVS VARY WLM commands.

  • To refresh a stored procedure

    • Use command VARY WLM,APPLENV=name,REFRESH

    • Use this to refresh the environment (LE) to load a new version of a stored procedure

    • Affects ALL STORED PROCEDURES within the WLM application environment

  • To stop a stored procedure:

    • Use command VARY WLM,APPLENV=name,QUIESCE

    • Use this to stop all stored procedure address spaces within the WLM application environment

  • To start a stored procedure:

    • Use command VARY WLM,APPLENV=name,RESUME

    • Use this to start all stored procedure address spaces associated with a WLM application environment

The priorities of the stored procedures in this environment will be inherited from the caller of the stored procedure.

Two-phase commit is supported for any access to non-relational data updates if the data is managed by RRS. For more information on RRS, refer to Chapter 4.


135 - Triggers

Chapter 13. Advanced Functions

Triggers

A trigger is a set of actions that will be executed when a defined event occurs. The triggering events can be the following SQL statements:

  • INSERT

  • UPDATE

  • DELETE

Triggers are defined for a specific table and once defined, a trigger is automatically active. A table can have multiple triggers defined for it, and if multiple triggers are defined for a given table, the order of trigger activation is based on the trigger creation timestamp (the order in which the triggers were created). Trigger definitions are stored in the DB2 catalog tables.

  • SYSIBM.SYSTRIGGERS

    • One row for each trigger

    • TEXT column contains full text of CREATE TRIGGER

  • SYSIBM.SYSPACKAGE

    • One row for each trigger package

    • TYPE column set to "T" to indicate a trigger package

Trigger Usage

Some of the uses of a trigger include:

  • Data validation

    • Ensures that a new data value is within the proper range. This is similar to table-check constraints, but it is a more flexible data validation mechanism.

  • Data conditioning

    • Implemented using triggers that fire before data record modification. This allows the new data value to be modified or conditioned to a predefined value.

  • Data integrity

    • Can be used to ensure that cross-table dependencies are maintained.

The triggered action could involve updating data records in related tables. This is similar to referential integrity, but it is a more flexible alternative.

Trigger Activation

A trigger can be defined to fire (be activated) in one of two ways:

  • A before trigger

    • Will fire for each row in the set of affected rows before the triggering SQL statement executes. Therefore, the trigger body is seeing the new data values prior to its being inserted or updated into the table.

  • An after trigger

    • Will fire for each row in the set of affected rows or after the statement has successfully completed (depending on the defined granularity). Therefore, the trigger body is seeing the table as being in a consistent state. (All transactions have been completed.)

Another important feature about triggers is that they can fire other triggers (or the same trigger) or other constraints. These are known as cascading triggers.

During the execution of a trigger, the new and old data values can be accessible to the trigger depending on the nature of the trigger (before or after).

By using triggers you can:

  • Reduce the amount of application development and make development faster.

    • Since triggers are stored in DB2 itself and are processed by DB2, you do not need to code the triggers or their actions into your applications.

  • Provide a global environment for your business rules.

    • Since the triggers have to be defined only once and are then stored in the database, they are available to all applications executing against the database.

  • Reduce the maintenance of your applications.

    • Since the trigger is handled by DB2 and is stored in the database itself, any changes to the trigger due to changes in your environment have to occur in only one application not multiple.

The best method of understanding the usage of triggers is to see some in action. The DB2CERT database contains many relationships that can be maintained using triggers.

After Trigger

In the following example, a trigger is defined to set the value of the PASS_FAIL column for each of the tests taken by a candidate. (Note that we add this column for this scenario.)

The trigger has been given the name PassFail (no relationship with the column called PASS_FAIL). Once the trigger has been created, it is active.

The PassFail trigger is an AFTER, INSERT, and FOR EACH ROW trigger. Every time there is a row inserted into the test_taken table, this trigger will fire. The trigger body section will perform an UPDATE statement to set the value of the PASS_FAIL column for the newly inserted row. The column is populated with either the value P (representing a passing grade) or the value F (representing a failing grade).

Note

Remember that a trigger defined against one table can modify other tables in the trigger body.

CREATE TRIGGER PassFail
  AFTER INSERT ON db2cert.test_taken
  REFERENCING NEW AS n
  FOR EACH ROW MODE DB2SQL
UPDATE db2cert.test_taken
  SET PASS_FAIL =
    CASE
      WHEN n.score >=
        (SELECT cut_score FROM db2cert.test
         WHERE number = n.number)
      THEN'P'
      WHEN n.score <
        (SELECT cut_score FROM db2cert.test
         WHERE number = n.number)
         THEN'F'
         END
   WHERE n.cid = cid
     AND n.tcid = tcid
     AND n.number = number
     AND n.date_taken = date_taken

Before Trigger

A before trigger will be activated before the trigger operation has completed. The triggering operation can be an INSERT, UPDATE, or DELETE statement. This type of trigger is very useful for three purposes:

  • To condition data

  • To provide default values

  • To enforce data value constraints dynamically

There are three before trigger examples shown below that are used in the DB2 Certification application.

All three of these triggers have been implemented to avoid seat conflicts for test candidates. The triggers will fire during an insert of each new candidate for a test.

** Example 1 **
CREATE TRIGGER pre9 NO CASCADE BEFORE
   INSERT ON db2cert.test_taken
   REFERENCING NEW AS n
   FOR EACH ROW MODE DB2SQL
WHEN (n.start_time <'09:00:00')
   SIGNAL SQLSTATE'70003'
   ('Cannot assign seat before 09:00:00!')

** Example 2 **
CREATE TRIGGER aft5 NO CASCADE BEFORE
   INSERT ON db2cert.test_taken
   REFERENCING NEW AS n
   FOR EACH ROW MODE DB2SQL
WHEN (n.start_time +
   (SELECT SMALLINT(length) FROM db2cert.test
   WHERE number = n.number) MINUTES
      >'17:00:00')
   SIGNAL SQLSTATE'70004'
   ('Cannot assign seat after 17:00:00!')

** Example 3 **
CREATE TRIGGER start NO CASCADE BEFORE
   INSERT ON db2cert.test_taken
   REFERENCING NEW AS n
   FOR EACH ROW MODE DB2SQL
WHEN (
   EXISTS (SELECT cid FROM db2cert.test_taken
   WHERE seat_no = n.seat_no
   AND tcid = n.tcid
   AND date_taken = n.date_taken
   AND n.start_time BETWEEN
      start_time AND finish_time))
   SIGNAL SQLSTATE'70001' ('Start Time Conflict!')

If the conditions are encountered, an SQL error will be flagged using the SQL function called SIGNAL. A different SQLSTATE value will be provided when the triggered conditions are encountered.

The pre9 trigger, shown above, is used to ensure that a test candidate is not scheduled to take a test before 9:00 a.m. The aft5 trigger is used to ensure that a test candidate is not scheduled to take a test after 5:00 p.m. The start trigger is used to avoid conflicts during a testing day.

Row and Statement Triggers

In order to understand the concept of trigger granularity, it is necessary to understand the rows affected by the triggering operations. The set of affected rows contains all rows that are deleted, inserted, or updated by the triggering operations. Figure 13.2 shows the different types of triggers.

Figure 13.2. Types of triggers.

Row Triggers

The keyword FOR EACH ROW is used to activate the trigger as many times as the number of rows in the set of affected rows. The previous example shows a row trigger.

Statement Triggers

The keyword FOR EACH STATEMENT is used to activate the trigger once for the triggering operation.

Transition Tables and Variables

Transition Variables

Transition variables allow row triggers to access row data in order to see row data as it existed before the triggering operation and see row data as it existed after the triggering operation. These variables are implemented by a REFERENCING clause in the definition.

REFERENCING OLD AS OLD_ACCOUNTS
                 NEW AS NEW_ACCOUNTS
Transition Tables

Transition tables allow after triggers to access a set of affected rows and see how they were before the triggering operation and then see all rows after the triggering operation. Transition tables are also implemented using the REFERENCING clause in the trigger definition.

REFERENCING OLD_TABLE AS OLD_ACCT_TABL
NEW_TABLE AS NEW_ACCT_TABLE

Note

Transition tables are materialized in DSNDB07.

Allowable Combinations

While there are many different combinations of trigger options available, not all are compatible. Table 13.1 below shows the valid combinations for trigger options.

Table 13.1. Triggers Types and Usage

Granularity

Activation Time

Trigger Operation

Transition Variables Allowed

Transition Tables Allowed

ROW

BEFORE

INSERT

UPDATE

DELETE

NEW

OLD, NEW

OLD

none

AFTER

INSERT

NEW

NEW_TABLE

UPDATE

OLD, NEW

OLD_TABLE, NEW_TABLE

DELETE

OLD

OLD_TABLE

STATEMENT

BEFORE

Invalid

none

Invalid

AFTER

INSERT

NEW_TABLE

UPDATE

OLD_TABLE, NEW_TABLE

DELETE

OLD_TABLE

Trigger Packages

When a trigger is created, DB2 creates a trigger package. This package is different from packages that you created for an application program (for more information on packages refer to Chapter 11, Application Development). Trigger pages can be rebound locally, but you cannot bind them (this is done automatically during creation). The package can be rebound only with the REBIND TRIGGER PACKAGE command and this will allow you to change the subset of default bind options (CURRENTDATA, EXPLAIN, FLAG, ISOLATION, RELEASE). For more information on the bind options, refer to Chapter 13.

Trigger packages cannot be copied, freed, or dropped. In order to delete a trigger package, the DROP TRIGGER SQL statement will need to be issued.

Note

Rebinding trigger packages after creation is also useful for picking up new access paths.

The qualifier of trigger name determines package collection. For static SQL, the Authorization ID of QUALIFIER bind option will be the qualifier and for dynamic whatever the CURRENT SQLID is will be the qualifier. For more information on qualifiers, refer to Chapter 4, Controlling Data Access.


136 - Object-Relational Extensions

Chapter 13. Advanced Functions

Object-Relational Extensions

With the object extensions of DB2, you can incorporate object-oriented concepts and methodologies into your relational database by extending DB2 with richer sets of data types and functions. With those extensions, you can store instances of object-oriented data types in columns of tables and operate on them using functions in SQL statements. In addition, you can control the types of operations that users can perform on those data types.

The object extensions that DB2 provides are:

  • Large objects (LOBs)

  • Distinct types

  • User-defined functions

Schemas

Schemas are qualifiers used for the object-relational extensions as well as stored procedures. They are used to qualify:

  • User-defined distinct types

  • User-defined functions

  • Stored procedures

  • Triggers

All of the objects qualified by the same schema name can be thought of as a group of related objects. A schema name has a maximum length of 8 bytes. The schema name SYSIBM is used for built-in data types and built-in functions, and SYSPROC is used for some stored procedures delivered by IBM in support of the Control Center, as well as Visual Explain.

Schema Names

The schema name can be specified explicitly when the object is referenced in the CREATE, ALTER, DROP, or COMMENT ON statement. If the object is unqualified and the statement is dynamically prepared, the SQL authorization ID contained in the CURRENT SQLID special register is used for the schema name of the objects.

Schema Privileges

There are certain authorities associated with schemas. Schema privileges include CREATIN, ALTERIN, and DROPIN. This allows you to create, alter, or drop objects in the identified schema. If the schema name is an authorization ID, then that authorization ID has those privileges implicitly.

PATH Bind Option

The PATH bind option is applicable to BIND PLAN, BIND PACKAGE, REBIND PLAN, and REBIND PACKAGE. The list of schemas specified is used to resolve unqualified references to user-defined distinct types and user-defined functions in static SQL statements. It is also used to resolve unqualified stored procedure names when the SQL CALL statement specifies a literal for the procedure name. It specifies an ordered list of schemas to be searched to resolve these unqualified references.

CURRENT PATH Special Register

There is also a corresponding special register for the PATH. The SET CURRENT PATH statement changes the value of the PATH special register. This PATH special register is used in the same way as the PATH bind option—to resolve unqualified references in dynamic SQL. It can also be used to resolve unqualified stored procedure names in CALL host-variable statements. The PATH bind option is used to set the initial value for this special register.

SYSIBM and SYSPROC do not need to be specified as part of the PATH—it is implicitly assumed as the first schema.

User-Defined Data Types

We looked briefly at what user-defined distinct types (UDTs) are and how they are implemented using DDL in Chapter 5, Data Structures. In this section, we will take a closer look at UDTs and some of their benefits.

By using UDTs, you can avoid some excess code in order to support data typing that is not included in the DB2 product. By using UDTs, you enable DB2 to do strong-typing, which says that only functions and operators defined on the UDT can be applied to instances of the UDT. This is beneficial for application so that you do not have to code for comparison errors.

Once the UDT is defined, column definitions can reference that type during the issuing of the CREATE or ALTER statement, the same as they would any DB2 built-in data type. If a distinct type is specified without a schema name, the distinct type is resolved by searching the schemas in the current path.

UDTs allow you to use DB2 built-in data types in special ways. UDTs are built from the DB2 built-in types. UDTs allow us to extend these types and declare specialized usage on them. DB2 then enforces these rules by performing only the kinds of computations and comparisons that you have defined for the data type.

Defining

In order to use UDT, you must first create it. The UDTs are created by using one of the DB2 built-in types as a base. You create them using the CREATE DISTINCT TYPE statement as follows:

CREATE DISTINCT TYPE distinct-type-name
   AS source-data-type
   WITH COMPARISONS

The name of the distinct type is a two-part name, which must be unique within the DB2 subsystem. The qualifier is a schema name. The distinct type shares a common internal representation with its source data type. However, the distinct type is considered to be an independent data type that is distinct from the others.

Note

LONG VARHCAR and LONG VARGRAPHIC cannot be used as source types.

An instance of a distinct type can be compared only with another instance of the same distinct type. The WITH COMPARISONS clause is to allow for comparison only between the same distinct types. This phrase is required if the source data type is not a large object type such as BLOB, CLOD, or DBCLOB. Comparisons for these types are not allowed.

Note

If you specify the WITH COMPARISONS on a distinct type with a large object source type, you will get a warning message, but the comparisons are still not allowed.

Casting

There are two operations that are allowed on distinct types: comparisons and casting. You can compare the values of distinct type (non-LOB), or you can cast between the distinct type and the source type.

Note

Character and arithmetic operators that are used in built-in functions that are used on source type are NOT automatically inherited by the distinct type. These operators and functions need to be created explicitly.

Comparison operators such as those listed next are allowed on UDTs.

= | >= | ¬> | IN
<> | <= | BETWEEN | NOT IN
> | ¬= | NOT BETWEEN | IS NOT NULL
< | ¬< | IS NULL |

Note

LIKE and NOT LIKE are not supported.

Casting functions are used to convert instances of source data types into instances of different data types. These functions have the name of the target data type and will have a single parameter, which is the source data type. They will return the target data type. Two cast functions are generated by DB2 when the CREATE DISTINCT TYPE is issued. These are used to convert between the distinct type and its source type. They will be created in the same schema as the distinct type.

The following shows an example of creating a UDT and then using it both with and without casting:

CREATE DISTINCT TYPE EURO AS DECIMAL (9,2) WITH COMPARISONS
   EURO(DECIMAL)
   -- where EURO is the target and DECIMAL is the source
   DECIMAL (EURO)
   -- where DECIMAL is the target and EURO is the source

Without casting – using the function name
   SELECT ITEM
   FROM INVENTORY
   WHERE COST > EURO (1000.00)

With casting – using cast function
   SELECT ITEM
   FROM INVENTORY
   WHERE COST > CAST (1000.00 AS EURO)

Note

Constants are always considered to be source-type values.

If you want to find all items that have a cost of greater than 1000.00 Euros, you will have to cast because you cannot compare the data of type EURO with data of the source data type of EURO—which is DECIMAL. You will need to use the cast function to cast data from DECIMAL to EURO. You can also use the cast function, DECIMAL, to cast from EURO to DECIMAL and cast the column COST to type DECIMAL. Depending on the way you choose to cast—to or from the UDT—you can use the function name notation, data-type(argument), or the cast notation, CAST(argument AS data-type).

Built-In Functions for UDTs

The built-in data types come with a collection of built-in functions that operate on them. Some of these functions implement operators such as the arithmetic operators on numeric data types and the substring operators on character data types. Other functions include scalar and column functions, which are discussed in Chapter 6, Advanced SQL.

When you create a UDT, you can also specify that it inherit some of the functions that operate on the corresponding source type. This done by creating new functions called sourced-functions that will operate on the UDT and duplicate the semantics of the built-in functions that work on the source type. The example below shows how to create a sourced function.

CREATE FUNCTION '+' (EURO, EURO) RETURNS EURO
        SOURCE SYSIBM.'+' (DECIMAL(9,2), DECIMAL(9,2))

You can also give your UDTs distinct semantics of their own by creating external functions that you write in a host language, which will operate on your UDT.

Privileges

You need to have privileges granted in order to use UDTs. The GRANT USAGE ON DISTINCT TYPE is used in order to GRANT privileges to use the UDT as a column data type in a CREATE or ALTER statement, or to use the UDT as a parameter in a stored procedure or user-defined function. The GRANT EXECUTE ON can be used to allow users to be able to cast functions on a UDT.

Catalog Information

There is information about the UDTs stored in the DB2 catalog. The following tables contain information about UDTs:

  • SYSIBM.SYSDATATYPES— row for each UDT

  • SYSIBM.SYSROUTINES— row for each cast function

  • SYSIBM.SYSROUTINEAUTH— authorizations for EXECUTE privilege

  • SYSIBM.SYSRESTAUTH— authorizations for USAGE privilege

User-Defined Functions

User-defined functions (UDFs) form the basis of object-relational extensions to the SQL language along with user-defined types (UDTs) and LOBs. Fundamentally, a database function is a relationship between a set of input data values and a result value. DB2 Universal Database comes with many built-in functions; however, it is possible to create your own column, scalar, and table functions.

While there are three types of functions in DB2, there are only two user-defined functions: scalar and table. There is no concept of a column function being user-defined.

Note

The built-in column and scalar functions are cover in Chapter 5, Manipulating Database Objects.

In DB2, you can create your own scalar or table functions. A UDF can be written in a high-level programming language such as COBOL, C, C++, or Java or you can use a single SQL statement.

There are two types of UDFs:

  • Sourced

    • A user-defined function based on another UDF or built-in function. This concept is similar to overloading classes in object-oriented programming.

  • External

    • A UDF that you write in a supported language

External UDFs

External UDFs, once written and generated as dynamically loadable libraries or classes, must be registered with the database. An external function is defined to DB2 with a reference to an OS/390 load module that DB2 should load when the function is invoked. The OS/390 load module contains the object code for the application program that contains the logic of the external function. If the program contains SQL statements, then there is an associated package that contains the DBRM. External functions cannot be column functions.

The congrat function shown in below is registered using the CREATE FUNCTION statement.

CREATE FUNCTION congrat(VARCHAR(30),VARCHAR(40))
   RETURNS CLOB(1K)
   EXTERNAL NAME 'congrat.a!congrat'
   LANGUAGE C
   PARAMETER STYLE DB2SQL
   DETERMINISTIC
   FENCED
   NO SQL
   NO EXTERNAL ACTION
   DISALLOW PARALLEL;
Sourced UDFs

Sourced UDFs are registered in the same way, simply by specifying the DB2 built-in source function. Sourced functions can be scalar functions or column functions, but cannot be table functions. Sourced functions are often helpful when there is a need to allow for the use of a built-in function on a UDT (see the previous section in this chapter for information on UDTs).

This example allows us to create an AVG function for the SCORE data type:

CREATE FUNCTION AVG (SCORE)
   RETURNS SCORE
   SOURCE SYSIBM.AVG(DECIMAL);

These CREATE FUNCTION statements place an entry for each UDF in the SYSIBM.SYSROUTINES and the parameters are recorded in SYSIBM.SYSPARMS. These catalog tables can be queried for information about the UDFs.

Table Functions

With DB2, you can also use another type of user-defined function called a table function. A table function is a UDF that returns a table to the SQL statement that calls it. This means that a table function can be referenced only in the FROM clause of a SELECT statement. The table function provides a means of including external data in SQL queries. Table functions can read non-DB2 data, for instance, a file on the operating system or over the World Wide Web, tabularize it, and return the data to DB2 as a relational table that can subsequently be treated like any other relational table. For example, the APPFORM table function in the next example takes in a candidate application form, processes it, and returns the data in an appropriate format to be inserted in the CANDIDATE table (except for the candidate ID, which is generated):

CREATE FUNCTION APPFORM(VARCHAR(30))
RETURNS TABLE
  (LNAME VARCHAR(30),
   FNAME VARCHAR(30),
   INITIAL CHAR(1),
   HPHONE PHONE,
   WPHONE PHONE,
   STREETNO VARCHAR(8),
   STREETNAME VARCHAR(20),
   CITY VARCHAR(20),
   PROV_STATE VARCHAR(30),
   CODE CHAR(6),
   COUNTRY VARCHAR(20))
EXTERNAL NAME APPFORM
LANGUAGE C
PARAMETER STYLE DB2SQL
NO SQL
DETERMINISTIC
NO EXTERNAL ACTION
FINAL CALL
DISALLOW PARALLEL
CARDINALITY 20;

If we wanted to insert a new candidate into the CANDIDATE table based on his or her application form, we could use the following SELECT statement:

INSERT INTO CANDIDATE
   SELECT CID,
      LNAME, FNAME, INITIAL,
      HPHONE, WPHONE,
      STREETNO, STREETNAME, CITY,
      PROV_STATE, CODE, COUNTRY
   FROM TABLE(APPFORM('D:\DOCS\NEWFORM.TXT')) AS AP
UDF Execution

The UDF programs execute in an OS/390 Workload Manager (WLM) environment. The WLM environment is supported by one or more WLM address spaces. The WLM keywords that name the WLM environment in which to execute should be specified when creating the function, else the program defaults to the WLM environment specified at installation time. This default environment can be seen in the SYSIBM.SYSROUTINES catalog table.

UDFs execute under the same thread as the invoking program and will run at the same priority.

Large Objects

LOB Datatypes

Three new datatypes have been added to DB2:

  • BLOB datatype (binary large object)

    • Binary strings (not associated to a CCSID)

    • Good for storing image, voice and sound data

  • CLOB datatype (character large object)

    • Strings made of single-byte characters or single/double byte characters with an associated CCSID

    • Use if data is larger than VARCHAR allows

  • DBCLOB datatype (double-byte character large object)

    • Strings made of double-byte characters with an associated CCSID

Each of these can contain up to two gigabytes of data, although in most cases the amount of storage for individual columns will be considerably less (depending on the type of data stored). There is a large use today of the 32-KB byte long varchar column, which has limitations in both size and functionality. Most of this use will probably be replaced by using LOBs in the future, certainly for new applications and functions.

LOB Implementation

LOBs are implemented with structures that are different from normal tables and table spaces. A LOB table space must be created for each column (or each column of each partition) of a base LOB table. This table space contains the auxiliary table and this table must have an auxiliary index associated with it. (For more information on LOB storage, refer to Chapter 8, Data Storage and Memory Management.) The LOB table space has a different recovery scheme, optional logging, and different locking options. For information on LOB recovery, refer to Chapter 8, Database Recovery, and for LOB lock information, refer to Chapter 14, Locking and Concurrency.

If a table contains a LOB column and the plan or package is bound with SQLRULES(STD), then DB2 will implicitly create the LOB table space, the auxiliary table, and the auxiliary index. DB2 will choose the name and characteristics for these implicitly created objects.

Note

It is better to develop naming standards beforehand for these objects and control placement of them. This is critical for management and performance.

INSERTing and LOADing LOBs

LOB loading and inserting is also different from processes for non-LOB data. The methods here are also entirely different depending on whether extenders are used or not (for more information on extenders, refer to Chapter 12, Development Considerations). Without extenders, there are some real limitations that need to be addressed when inserting LOB data, primarily the 32-KB limit and logging impacts. If the total length of the LOB column and the base table row is less than 32 KB, then the LOAD utility can insert the LOB column. When the limits of LOAD are exceeded, then SQL INSERT or UPDATE statements need to be used. But the SQL INSERT has its own limitations in that enough memory needs to be available to hold the entire value of the LOB. The limitation here is the amount of memory available to the application, and the amount of memory that can be addressed by the language used. If the LOB are all small, then it is not as much of an issue due to the fact that memory and language constructs are not a limiting factor. But when dealing the very large LOBs, the differences can be seen easily when comparing the C language construct with COBOL.

C Language for a LOB host variable:

SQL TYPE IS CLOB(20000K) my_clob;

is generated by DB2 as:

   struct { unsigned long length;
            Char data[20960000];
          } my_clob;

COBOL Language for a LOB host variable:

01 MY-CLOB     USAGE IS SQL TYPE IS CLOB(20000K).

is generated by DB2 as:

01  MY-CLOB.
    02  MY-CLOB-LENGTH    PIC 9(9) COMP.
    02 MY-CLOB-DATA.
       49  FILLER     PIC  X(32767).
       49  FILLER     PIC  X(32767).
       49  FILLER     PIC  X(32767).
--Repeated 622 times

This is another area where extenders assist in solving the problem. When a table and column are enabled for an extender, the whole process changes. An INSERT statement can be used in the program to contain extender functions (UDFs) that allow the image to be loaded into the database directly from an external file. Actually, with the image extender for example, the image content is inserted into an administrative support table, and another record is then inserted into another administrative table, describing the attributes of the image such as number of colors, thumbnail-sized version, and format characteristics (JPEG, TIFF, etc). The extenders do require WLM to be installed in support of the extender UDFs and stored procedures, and specifically using WLM in goal mode for performance reasons.

LOBs and User-Defined Types

Being able to store LOBs and manipulate them through extenders is only part of the story. The capability also exists to define new distinct data types based on the needs of particular applications. A user-defined type (UDT), also known as a distinct type, provides a way to differentiate one LOB from another LOB, even of the same base type such as BLOB or CLOB. A UDT is not limited to objects and can be created for standard data types as well as LOBs.

Even though stored as LOBs (binary or character), image, video, spatial, XML, and audio objects are treated as types distinct from BLOBs and CLOBs and distinct from each other. For example, suppose an application that processes spatial data features needs a polygon data type. You can create a distinct type named polygon for polygon objects as follows:

CREATE DISTINCT TYPE polygon AS BLOB (512K)

The polygon-type object is treated as a distinct type of object even though internally it is represented as a 512-kilobyte binary object (BLOB). UDTs are used like SQL built-in types to describe the data stored in columns of tables, as shown in the example below. The extenders create distinct data types for the type of object they process, such as image, audio, and video, which makes it easier for applications to incorporate these types of objects.

CREATE TABLE DB2_DEMO
   (GEO_ID CHAR(6),
    EURO_ANNUAL    EURO_DOLLAR,
    OZ_ANNUAL      AUSTRALIAN_DOLLAR,
    US_ANNUAL      US_DOLLAR,
    DEMO_GRAPHIC   POLYGON)

Casting functions allow operations between different data types—for example, comparing a slide from a video UDT to an expression. You must CAST the expression to a UDT type (video, in this example) in order for the comparison to work. There are casting functions supplied with DB2 (CHAR, INTEGER, and so on) and there are others created automatically whenever a UDT is created with the CREATE DISTINCT TYPE statement.


137 - Summary

Chapter 13. Advanced Functions

Summary

In this chapter, we have talked about very powerful DB2 and SQL features, such as stored procedure and triggers. These features that DB2 provides can help reduce the amount of application development time and maintenance. It can help to encapsulate our code into one place for use by many applications.

DB2 has implemented some object-relational functions, such as large object (LOB) support, user-defined distinct types, user-defined functions, and so on. We have discussed user-defined data types and user-defined functions. These features also allow us great flexibility and power in our applications. LOBs, of course, give us the ability to stored large amounts of data and different types of data such as multi-media, giving us the ability to develop very sophisticated applications.


138 - Additional Resources

Chapter 13. Advanced Functions

Additional Resources

  • IBM DB2 UDB for OS/390 Image, Audio and Video Extenders Administration and Programming, SC26–9947.

  • IBM DB2 UDB for OS/390 Text Extender Administration and Programming, SC26–9948.

  • IBM DB2 UDB for OS/390 XML Extender Administration and Programming, SC27–9949.

  • IBM DB2 UDB for OS/390 Administration Guide, SC26–9931.

  • IBM DB2 UDB for OS/390 Application Programming and SQL Guide, SC26–9933.


139 - Chapter 14. Locking and Concurrency

Part IV. Developing Applications

Chapter 14. Locking and Concurrency

  • Locking Data

  • Lock Attributes

  • Avoiding Locks

  • Claims and Drains

  • Escalation and Promotion

It would be easy to have one process simply lock all the data while using it, but that, of course, would lead to other applications being unable to access the data during this time. Concurrency, the ability for multiple applications to access the same data at the same time, needs to be allowed but also controlled so that we prevent lost updates, access to uncommitted data, or have unrepeatable reads (data changing between reads).

A balance must be achieved for best concurrency of all processes. There are many controls in DB2 to allow it to achieve maximum concurrency while maintaining the integrity of our data. This ranges from the parameters we bind our programs with, to options of the DDL for the creation of objects, to subsystem level parameters.

DB2 uses locks to control currency within a database, that is, simultaneous access to a resource by more than one user (also referred to as serialization). Locks also prevent access to uncommitted data, which prevents valid data from becoming lost (part of data integrity), and allows a user to see the same data, without the data ever changing, within a period of processing called a commit scope. From a performance standpoint, everything done in DB2 has a trade-off. In locking, there is a trade-off between concurrency and performance, as more concurrency comes at a higher cost of CPU usage due to lock management. There are also cases where DB2 will override the locking strategy designed, because processes hold lock resources exceeding the site established thresholds. However, only the use of certain combinations of parameters cause this to occur.

We will take a look at the various different components of locking and how to use them effectively to meet concurrency requirements.


140 - Locking Data

Chapter 14. Locking and Concurrency

Locking Data

DB2 uses transaction locking (using the IRLM), latches, and other non-IRLM mechanisms to control the concurrency and access of SQL statements and utilities. These mechanisms are there to associate a resource with a process so that other processes cannot access the same resource when it would cause lost updates and access to uncommitted data. Generally, a process will hold a lock on manipulated data until it has completed its work. This way it ensures that other processes do not get hold of data that has been changed and not committed. Another use is for repeatable read, when an application has the need to reread data that must still be in the same state as it was when it was initially read.

There are also user options to avoid locking and allow for access to uncommitted data, and system settings and mechanisms to provide for lock avoidance when it would not cause a data integrity problem.

Not all objects need to have locks acquired on them to establish concurrency. The following items can get locked:

  • User tables

  • Catalog tables

  • Directory tables

Indexes are not locked as serialization is controlled by using latches, and concurrency is controlled by data locking. There are also drains and claims used to control utilities and DB2 commands. Draining allows utilities and commands to acquire partial or full control of a needed object with the least interruption to concurrent access.

Many times DB2 might need to access data in the catalog, and the data being read or updated will be locked. The following locks can be used:

  • During binding

    • SKCT (Skeleton Cursor Table or plan)

    • SKPT (Skeleton Package Table or package)

  • During DDL activity

    • DBD (Database Descriptor)

Lock Attributes

Locks should be viewed as having:

  • Size

    • Determines how much data is locked

  • Mode

    • Determines whether or not the lock allows others to read and/or update the locked object

  • Duration

    • Determines how long the lock is held

Each plays a role in the acquisition and release of a lock, and understanding these attributes can help us use them wisely and avoid contention.

More than one type of lock can exist for a specific user for a particular table space. This will depend on the mode of the lock. For example, locks might be held on the table space, table, page, and row simultaneously for a particular table space. Each lock size will have a mode during its existence.

We will take a look at all the attributes of locks and how they work together in detail in the following sections.

Lock Sizes

Locks can be taken on various objects in DB2. The size of the lock will determine how much of the total object will be locked. There are several objects that can be locked in DB2, determining the lock size:

  • Non-LOB data

    • Table space

      • Lock the entire table space

    • Table

      • Only segmented table spaces allow for an individual table to be locked

    • Partition

      • Lock only a partition

    • Page

      • Lock an individual page in a table space

    • Row

      • Lock an individual row in the table

  • LOB data

    • LOB locks

Table space locks are the most encompassing lock, but of course this allows for the least amount of concurrency. It would seem that for maximum concurrency we should use row locking; however, locks are overhead and row level lock can cause excessive overhead in terms of processing time and storage. Page locking is generally the accepted lock level for the majority of situations. There are always exceptions, however, such as table locking (discussed later).

Note

It is important to understand that in the hierarchy of locks, row locks and page locks occur at the same level. It either a page lock or a row lock is escalated, it moves to the same higher level, and a row lock can never escalate to a page lock.

Lock Table Statement

There are some situations that may call for the ability to exclusively lock a table or partition. There is a single SQL statement that can override locking any time.

You can do this by using the LOCK TABLE IN EXCLUSIVE MODE statement. Some of those reasons include if you want to take a "snapshot" of data and be able to access a table without any other concurrent processes changing data. You can allow other processes to access the table by using the use LOCK TABLE IN SHARE MODE. This is different from the repeatable read(RR) isolation because RR locks out only changes from rows or pages that you have already accessed.

The LOCK TABLE/TABLESPACE statement can be used only to get a table/table space lock, in either share (S) mode or exclusive (X) mode. The lock is acquired when the SQL statement is executed. Its release is dependent upon the previous settings of the ACQUIRE/RELEASE parameters. The table/table space lock is released at thread deallocation if RELEASE(DEALLOCATE) was specified in the options when the program was bound. The lock also can be released at either COMMIT or ROLLBACK time if the RELEASE(COMMIT) option was specified at bind time. Despite the type of locking held when the SQL lock table statement was issued, when the COMMIT or ROLLBACK is issued, the system falls back to page mode locking, unless another SQL lock table statement is issued. A user/application must constantly reissue the SQL lock table statement over the COMMIT or ROLLBACK when RELEASE(COMMIT) has been specified, if locking is still needed at the table/table space level.

The LOCK TABLE IN EXCLUSIVE statement can also be used to avoid additional overhead if you are updating a large portion of a table. This can be more efficient because by preventing concurrent accesses to the data, you can avoid the overhead to lock and unlock each page during updates.

Timeouts can also be prevented by using the LOCK TABLE IN EXCLUSIVE or IN SHARE MODE statement. This will allow you to process the data quickly without the possibility of having to contend for locks with out applications.

Selective Partition Locking

By using LOCKPART(YES) for a partitioned table space, only the individual partitions are locked when they are accessed. This is known as Selective Partition Locking (SPL). It also gives us the ability to tell DB2 not to escalate locks to the table space level, but rather just escalate locks at the partition level. This will allow for less propagation of locks to the coupling facility in a data sharing environment and better concurrency for application accessing data in various partitions. This can be especially useful if usage against the partitions is spread over multiple members using affinity routing (see Chapter 9 for more information on data sharing).

However, selective partition locking will help only in a non-data sharing environment if an agent actually escalates. If you are not having escalation problems, then you may not want to consider using SPL. Without SPL, the parent intent lock is taken on the last partition, regardless of which ones you access. These locks are almost always intent locks and therefore almost never cause a problem. With SPL, the parent intent lock is taken on whichever part you access. If escalation then occurs with SPL, the lock escalation occurs just for the part (or parts) that you hit too many locks on. Without SPL, since the parent lock is only on the last one, we escalate the entire page set and access is prevented to all partitions.

LOB Locking

LOB locks have different characteristics from the regular locks. There is a new lock mode for LOBs called a LOB lock. There is no concept of row or page locking with a LOB. The locking for LOBs is not at all like the traditional transaction level locking. Since LOBs are in an associated object, concurrency between the base table and the LOB must be maintained at all times, even if the base table is using UR (uncommitted read). A LOB lock still needs to be held on the LOB to provide consistency and, most importantly, to maintain space in the LOB table space. LOB locks avoid conflicts between readers and deletes/updates. A SELECT or DELETE will acquire a share mode LOB lock and doesn't prevent the DELETE, but it will prevent the reuse of the deallocated pages until all of the LOB locks are released. A shadow copy of the LOB will exist until the LOB locks are released. INSERTs will acquire exclusive LOB locks on new LOBs (an UPDATE is basically a DELETE followed by an INSERT). The LOCK TABLE statement, which produces a gross lock on the object, can still be used and will not require individual LOB locks to be acquired.

The ACQUIRE option of BIND has no effect on when the table space lock on the LOB table space is taken. Locks on LOB table spaces are acquired when they are needed.

The table space lock is released according to the value specified on the RELEASE option of BIND with a couple of exceptions: when a cursor is defined WITH HOLD or a LOB locator is held. When a cursor is defined WITH HOLD, LOB locks are held through commit operations. When a LOB value is assigned to a LOB locator, the lock acquired will remain until the application commits. If the application uses HOLD LOCATOR, the locator and the LOB lock must be held and cannot be freed until certain conditions exist. The locks would be held until the first commit operation after a FREE LOCATOR statement is issued, or until the thread is deallocated. Using LOB columns will generally require an increase in the number of locks held by a process. As a starting point for preventing unforeseen problems, LOCKMAX should be non-zero on the LOB table spaces. An even better strategy would be to use the gross locks during heavy inserting processes, which would prevent most of the locking difficulties.

Lock Modes

The following lists the allowable lock modes in DB2:

  • IS— intent share

  • S— share

  • IX— intent exclusive

  • U— update

  • SIX— share with intent exclusive

  • X— exclusive

Modes are easy to understand with a simple formula. If the table/table space lock has an "I" in it, which stands for INTENT, then row or page locks are in use on the individual pages. In all other cases, the table/table space lock will be the only lock used. The intent locks act as an indicator to DB2 to identify what is occurring within the table/table space. Some locking is beyond the control of the programmer and designer, while the other locks can be controlled to a degree. The following Table 14.1 shows the compatibility of the different lock modes.

Table or Table Space Locks

When using page or row locks, the table or table space lock will be:

  • IS— intent share

    • Want to read-only (may lock pages/rows)

    • Allow read of my pages/rows and read/update of all other pages/rows

    Table 14.1. Lock Modes

    Table Space, Table and Partition

    Lock Mode

    IS

    IX

    S

    U

    SIX

    X

    IS (Intent Share)

    yes

    yes

    yes

    yes

    yes

    no

    IX (Intent Exclusive)

    yes

    yes

    no

    no

    no

    no

    S (Share)

    yes

    no

    yes

    yes

    no

    no

    U (Update)

    yes

    no

    yes

    no

    no

    no

    SIX (Share w/Intent Exclusive)

    yes

    no

    no

    no

    no

    no

    X (Exclusive)

    no

    no

    no

    no

    no

    no

    Page and Row

    Lock Mode

    S (Share)

    U (Update)

    X (Exclusive)

    S (Share)

    yes

    yes

    no

    U (Update)

    yes

    no

    no

    X (Exclusive)

    no

    no

    no

    • The lock owner has SHARE (S) locks on the pages, which allows other users to read and update.

  • IX— intent exclusive

    • Want to update (may lock pages/rows)

    • Allow read/update of all other pages/rows

      • The lock owner can have SHARE (S), UPDATE (U), or EXCLUSIVE (X) locks on the pages. Other users can still read and update.

  • SIX— share with intent exclusive

    • Want to read (will not lock pages/rows)

    • Want to update (may lock pages/rows)

    • Allow read of all other pages/rows

      • The lock owner can have SHARE (S), UPDATE (U), or EXCLUSIVE (X) locks on the pages. Other users can still read, but no other user can update.

Note

SIX is not a common lock. This mode that usually occurs when an IX lock already exists and a SHARE(S) has to be acquired.

When not using row or page locks, the table/table space lock will be:

  • S— share

    • Want to read-only

    • Allow others read-only

      • No one can update

  • U— update

    • Want to read with possibility of update

    • Allow others read-only

      • No other user can update

  • X— exclusive

    • Want to update

    • Not allow any other access

      • User needs exclusive use of the table/table space. No other user is allowed.

Page or Row Locks

Page locks can be only one of three types:

  • S— share

    • Restricts other users to read-only use of this page. S or U locks can be acquired by other users. Using SELECT or FETCH without UPDATE intent will acquire this lock.

  • U— update

    • This allows the user to go to the actual UPDATE step if required. Other applications can get S locks, but nothing else.

  • X— exclusive

    • User has updated the data on the page and no other user can get any page lock on the page.

The lock mode is determined by the DML statement according to the following:

SELECT… | S
SELECT…FOR UPDATE OF… | U
UPDATE/DELETE/INSERT | X

DB2 does not allow update of uncommitted changes (UPDATE/ INSERT/DELETE) by other users. Reading of uncommitted data is allowed only if ISOLATION level of UR is specified in the SQL WITH clause or as a BIND option.

Lock Durations

Table, Partition, and Table Space Lock Durations

Table space and table lock duration is determined by the BIND ACQUIRE/ RELEASE parameters.

  • ACQUIRE— determines when the table or table space or partition locks are taken

    • ALLOCATE

      • When the first SQL statement is issued, the maximum required lock is take on all the objects in the plan or package.

    • USE

      • When a SQL statement is issued, the required lock is taken on the involved objects.

  • RELEASE – determines when the table or table space or partition locks are released

    • DEALLOCATE

      • At the end of the program

    • COMMIT

      • At COMMIT time

There is an exception that applies when the cursor has been defined using the WITH HOLD option. The locks needed to maintain position of the cursor are then held until the next commit after the cursor is closed (RELEASE COMMIT) or until the application deallocation (RELEASE DEALLOCATE).

Page or Row Lock Durations

The duration of the lock defines the actual length of time that the lock will be held (period between acquiring and releasing the lock). We can control how long locks are held through various means such as our defined commit scopes, program bind parameters, and system parameters. We of course want to maximize concurrency, so we do not want to hold the lock unnecessarily long (using a large lock for the duration of the process), but we do not want to accumulate too much overhead by taking an excessive number of smaller locks (such as taking several row locks over a period of time for the same process).

Isolation levels are set when a program is bound or when used on an individual SQL statement using the WITH ISOLATION clause. The isolation levels directly effect the duration that locks are held. The isolation levels that can be used either at bind time or with an individual SQL statement are:

  • CS— Cursor Stability

  • RS— Read Stability

  • RR— Repeatable Read

  • UR— Uncommitted Read

Of these, cursor stability, read stability, and repeatable read have to do with lock durations, whereas uncommitted read has to do with overriding locks, which is discussed in the next section.

Cursor Stability

This level of program isolation will hold a lock on the row or page only (depending on the lock size defined) if the cursor is actually positioned on that row or page. The lock will be released when the cursor moves to a new row or page, except the lock will be held until a commit is issued if changes are being made to the data. This option allows for the maximum concurrency for applications that are accessing the same data that cannot allow for uncommitted reads.

Read Stability

This option will hold locks on all rows or pages qualified by stage 1 predicates for the application until the commit is issued. Non-qualified rows or pages, even though touched, are not locked as with the RR option. Uncommitted changes of other applications cannot be read, but if an application issues the same query again, any data changed or inserted by other applications will be read, since RS allows other applications to insert new rows or update rows that could fall within the range of the query. This option is less restrictive but similar is function to repeatable read.

Repeatable Read

This option will hold a lock on all rows or pages touched by an application program since the last commit was issued, whether or not all those rows or pages satisfied the query. It will hold these locks until the next commit point, which ensures that if the application needs to read the data again, the values will be the same (no other process could update the locked data). This option is the most restrictive, in terms of concurrency of applications.

S—Share Lock Duration

Using SELECT or FETCH without UPDATE intent will acquire this lock. It is released for plans bound with CS when:

  • A TSO process issues an SQL COMMIT or ROLLBACK.

  • An IMS process issues the next SYNCH or ROLB.

  • A CICS synch point call occurs.

  • A non-cursor SQL statement completes without update.

  • The cursor position moves to a new page without update.

  • Closing a cursor.

Using the cursor hold option, will hold the locks over COMMIT or ROLLBACK processes in TSO and CAF. If there is no COMMIT used in the program, the locks will be held until the cursor is closed.

It is important to understand that the lock will be held on the previous page until DB2 gets the new lock on the new page.

U—Update Lock Duration

A FETCH with a cursor that contains FOR UPDATE OF acquires this lock. UPDATE and DELETE statements without a cursor also acquire this lock. UPDATE and DELETE statements cause an implicit SELECT to occur, first matching the conditions in the WHERE clause. During this SELECT, DB2 places the U lock on the page accessed, before the UPDATE or DELETE. DB2 releases the lock for plans bound with CS when:

  • A TSO process issues an SQL COMMIT or ROLLBACK.

  • An IMS process issues the next SYNCH or ROLB.

  • The next synch point call occurs in CICS.

  • A non-cursor SQL statement has completed without update.

  • The cursor position moves to a new page without update.

  • Closing a cursor without an update has occurred.

Use the cursor hold option in TSO and hold the lock over COMMIT and ROLLBACK. However, the COMMIT will demote the U LOCK back to an S LOCK. If an update is to occur, then the lock will be promoted to an X LOCK before changing the data page.

X—Exclusive Lock Duration

Exclusive locks are released when:

  • A TSO process issues an SQL COMMIT or ROLLBACK.

  • An IMS process issues the next SYNCH or ROLB.

  • The next synch point call occurs in CICS.

The lock will be held over COMMIT and ROLLBACK in TSO and CAF when using the cursor hold option. However, the COMMIT will demote the X LOCK back to an S LOCK.


141 - Overriding or Avoiding Locks

Chapter 14. Locking and Concurrency

Overriding or Avoiding Locks

Page and row locking can be avoided at execution time by letting the system use lock avoidance mechanisms. DB2 can test to see if a row or page has committed data on it. If it does, then no lock at all is required on the data. Lock avoidance is valid only for read-only or ambiguous cursors, and requires a combination of events and settings to occur. First, the statement must be a read-only or an ambiguous cursor along with the proper isolation level and the appropriate CURRENTDATA setting. The best setting for the most lock avoidance is using an ISOLATION level of CS, either as a bind parameter or on the individual SQL statement, and CURRENTDATA set to NO as a bind option.

Lock avoidance is normally the default for most use, as it removes a lot of locking overhead. DB2 executes a small instruction set to determine if an IRLM lock is truly needed, for read-only cursors or ambiguous cursors.

Page and row locking may also be overridden at bind time or execution time. Locking can be overridden for read-only cursors (not ambiguous cursors) by using the UR ISOLATION level, at bind time or on the individual statement level as shown below:

EXEC SQL SELECT ... WITH ISOLATION UR

A SQL statement is considered to be read-only when it uses any of the following:

  • JOIN

  • SELECT DISTINCT

  • GROUP BY

  • ORDER BY

  • UNION

  • UNION ALL

  • SELECT of a column function

  • FOR FETCH ONLY (FOR READ-ONLY)

  • SELECT FROM non-updateable catalog table

If the statement does not incorporate any of the above, it is said to be ambiguous, meaning that DB2 does not know for sure if dynamic DML will be issued against the cursor. The CURRENTDATA setting of NO, states to DB2 that the ambiguous cursor is read-only. If the statement is targeted by dynamic SQL, then you will get an error.

The following Table 14.2 shows when locking is avoided or overridden:

Table 14.2. Lock Avoidance

ISOLATION

CURRENTDATA

Cursor type

Avoid locks on returned data

Avoid locks on rejected data

UR

N/A

Read-only

N/A

N/A

CS

YES

Read-only

Updateable

Ambiguous

NO

YES

NO

Read-only

Updateable

Ambiguous

YES

NO

YES

RS

N/A

Read-only

Updateable

Ambiguous

NO

YES

RR

N/A

Read-only

Updateable

Ambiguous

NO

NO

Note

If your program is bound with ISOLATION(UR), mass deleters cannot run concurrently with UR program.


142 - System Parameters

Chapter 14. Locking and Concurrency

System Parameters

There are some system parameters (DSNZPARMs) that are used to control some aspects of locking.

RECURHL | The use of the RECURHL = YES DSNZPARM can help with concurrency because it will allow DB2 to release the locks that are held by a cursor defined WITH HOLD.
IRLMRWT | This is the number of seconds that a transaction will wait for a lock before a timeout is detected. The IRLM uses this for timeout and deadlock detection. Most shops take the default of 60 seconds, but in some high-performance situations where they wanted the detection to occur sooner (so the applications were not incurring excessive lock wait times), it has been set lower. If this was hit often, the applications were reviewed and tuned.
XLKUPDLT | This DSNZPARM allows you to specify the locking method to use when a searched UPDATE or DELETE is performed. The default is NO, best for concurrency, which says that DB2 will use an S or U lock when scanning qualifying rows and then will upgrade to an X lock when a qualifying row is found. The value of YES would be useful in a data sharing environment when the searches involve an index because it would take an X lock on qualifying rows/pages.
NUMLKTS | This represents the maximum number of locks on an object. If you are turning off lock escalation (LOCKMAX 0), you will need to increase this number. If you are using LOCKMAX SYSTEM, then the value here will be the value for SYSTEM.
NUMLKUS | This is the maximum number of page or row locks that a single application can have held concurrently on all tablespaces. This includes data pages, index pages, subpages, and rows. If you were to specify 0 then there will be no limit on the number of locks. You would want to be careful with 0 because if you are turning off lock escalation and not committing frequently enough you could run into storage problems (DB2 uses approximately 250 bytes for each lock).


143 - Claims and Drains

Chapter 14. Locking and Concurrency

Claims and Drains

Claims and drain locks are used to control currency between SQL processes and utilities, with partition independence being a major focus. Utilities and SQL can concurrently access and update different partitions, including different logical partitions of the non-partitioned indexes.

A logical partition refers to the set of index entries, which point to rows in a particular data partition. Logical partitions exist only in non-partitioned indexes of partitioned tables. An index entry belongs to one and only one logical partition.

Claims

When an application first accesses an object, within a unit of work, it makes a claim on the object. It releases the claim at the next commit point. Unlike a transaction lock, the claim cannot persist past the commit point. To access the object in the next unit of work, the application must make a new claim.

Claims can be acquired on:

  • Simple table space

  • Segmented table space

  • Index space

  • Data partition

  • Index partition

There are three different claim classes:

  • Write class

  • Repeatable read class

  • Cursor stability class

Claim Class | ISOLATION level | Allows reading | Allows updating | Allows inserting | Allows deleting
Write | Any | YES | YES | YES | YES
Repeatable read | RR | YES | NO | NO | NO
Cursor stability | CS | YES | NO | NO | NO

Claims are released at COMMIT except for utilities and cursors defined WITH HOLD still positioned on an object. All SQL processes are claimers, and only occasionally is a utility a claimer (e.g., online load resume).

Note

There is no limit in DB2 to the number of concurrent claimers.

Drains

Drain locks are used to serialize access to partitions and page sets among utilities, commands, and SQL applications. The drain is initiated at any time, but the actual take-over of an object occurs only when all access to the object has been quiesced. The drain process acquires a lock to prevent subsequent access from occurring until the lock is released.

To drain a resource, a utility or command first acquires a drain lock and then waits until all claimers of a particular class on the resource are released. When all claims on the resource in a claim class are released, the resource is considered drained. A utility trying to take over can time out if a long running SQL process does not release the claim quick enough.

A utility that needs only read-only access will drain on the write class, which will prevent any new updating claimers. A utility that needs to change data will drain all claim classes.


144 - Locking Issues and Problems

Chapter 14. Locking and Concurrency

Locking Issues and Problems

There are times when too much locking can be a problem. This possibly can result in suspensions, timeouts, or even deadlocks. We will take a look at how these situations can occur.

Timeouts and Deadlocks

While waiting for a lock, a process can exceed an allowable amount of wait time that has been established. A system-wide option determines the maximum time a process can wait before it gets a resource unavailable error. This value is called the resource timeout value and is calculated by the following formula:

  1. Divide RESOURCE TIMEOUT by DEADLOCK TIME.

  2. Round the result up to the next largest integer.

  3. Multiply this integer by the DEADLOCK TIME.

  4. Multiply the result by the appropriate factor.

Component | Factor | Notes
Transaction and queries | * 1 |
BIND | * 3 |
IMS BMP | * 4 |
IMS fast path | * 6 | Non-message processing
Utility | * Utility timeout | Defaults to 6

The result is the timeout value that is always larger or equal to the RESOURCE TIMEOUT. When the timeout value has been reached, the requesting task is informed of the unavailable resource. There are several conditions that can cause a timeout value to be exceeded by a process, and each of these returns different messages to different locations.

The deadlock time is determined by another system parameter, the DEADLOCK TIME (default 5 seconds). This is the time interval between two successive scans for a deadlock. Every deadlock interval, the IRLM will verify whether there are deadlock situations. In those cases, DB2 will inform the task that made the smallest number of changes (generally) of the fact that an error has occurred. The other tasks will continue without any problems.

The main causes of a timeout are the following:

  • A program holding a lock has not freed it soon enough.

  • A deadlock has occurred between DB2 and a non-DB2 resource.

The main causes for deadlocks are the following:

  • Two hot pages

  • Lock escalation

  • One hot page, promotion from S to X lock, and FOR UPDATE OF missing on the cursor

When a timeout/deadlock has occurred, error codes and messages are sent to the following:

  • To the application program

    • –911 = "THE CURRENT UNIT OF WORK HAS BEEN ROLLED BACK DUE TO DEADLOCK OR TIMEOUT, REASON reason-code, TYPE OF RESOURCE resource-type, AND RESOURCE NAME resource-name."

      • The reason code indicates whether a timeout or a deadlock has occurred.

      • The application is rolled back to the previous commit. On receipt of the –911 return code, the application should terminate or retry.

    • –913 = "UNSUCESSFUL EXECUTION CAUSED BY DEADLOCK OR TIMEOUT, REASON reason-code, TYPE OF RESOURCE resource-type, AND RESOURCE NAME resource-name."

      • The reason code indicates whether a timeout or a deadlock has occurred.

      • The current unit of work HAS NOT BEEN ROLLED BACK. The application should either commit or roll back to the previous COMMIT. Then the application should terminate or retry.

  • To the console

    • DSNT376I message

    • DSNT500I message

Displaying Locks

You can display the locks that are being held to held solve application con currency problems. By using the –DISPLAY DATABASE (LOCKS) command, you can see many properties about a given lock. Figure 14.1 shows an example of the output from a DISPLAY DATABASE LOCKS command.

Figure 14.1. Display Database Locks.

Lock Promotion and Escalation

Lock Escalation

Lock escalation can occur when the value for LOCKMAX is reached. At this time DB2 will release all of the locks held in favor of taking a more comprehensive lock (i.e., release several page or row locks to escalate to a table lock). DB2 tries to balance the number of locks on objects based upon the amount of concurrent access. The LOCKMAX option will then further determine when/if escalation occurs.

Note

Lock escalation can occur for objects defined with LOCKSIZE ANY or LOCKSIZE PAGE or ROW. ANY just allows DB2 the choice of lock to take initially.

The value of LOCKMAX is set on the CREATE TABLESPACE statement to define how many locks can be held simultaneously on an object. Locks will be escalated when this number is reached. You can specifically set this value to a number or if you leave it at the default of SYSTEM (use LOCKSIZE ANY; otherwise, the default is 0 for specific object locks such as page/row/table/table space), it will use the number set by the NUMLKTS DSNZPARM.

You can turn off lock escalation by using the LOCKMAX = 0 parameter on the CREATE TABLESPACE statement. If you choose to turn off lock escalation, you want to be sure your applications accessing those objects are committing frequently and you have adjusted the DSNZPARM NUMLKTS to allow for more locks to be taken, else you run the risk of hitting negative SQL codes when you reach the maximum number of locks (without escalation, you could hold more, smaller, locks). Lock escalation is there to protect you from taking excessive system resources, so if you turn it off you will need to control it. If you are CPU bound, it may not be a good idea to turn off lock escalation because it will take DB2 longer to traverse long chains of locks and it will take more CPU for IRLM latch activity.

Lock Promotion

There is a lock mode that will be used for any table, table space, or table space partition for a SQL statement, and it is reflected in the PLAN_TABLE populated by EXPLAIN in the column TSLOCKMODE if the isolation can be determined at bind time. This is the lock mode that the SQL statement would use for the table/table space lock if and only if it had not been raised by a preceding SQL statement. For example, a SQL select statement might have a TSLOCKMODE of IS, and yet during execution it could have been promoted to an IX and that is what the SELECT statement would use. This is called lock promotion, but it generally is not a concern, just a fact.


145 - Summary

Chapter 14. Locking and Concurrency

Summary

In this chapter, we have discussed how concurrency and serialization is controlled. To protect the data as it is being modified, rules are established and the changes and grouped into units of work. The data updated is made permanent by the COMMIT statement, or removed by the ROLLBACK statement. The rules of concurrency are determined by the isolation level, the lock size, and the rules of lock duration.

DB2 will implement the isolation level semantics of data access by implicitly acquiring locks on behalf of applications. Applications can decide to lock a resource for exclusive or share modes. The resources that can be locked are the row, page, table, table space, or partition.

If a requested lock is more restrictive, and another application already has the resource locked, a wait on the release of the lock will occur. The amount of time an application will wait is determined by system-wide parameters for both resource timeout and deadlocks.

If multiple applications require access to data that is held by other applications, a deadlock scenario can occur. DB2 will detect the occurrence of any deadlocks and force one of the transactions to ROLLBACK. Every lock requested requires memory in the IRLM, and the amount of lock storage is configurable using IRLM parameters.


146 - Additional Resources

Chapter 14. Locking and Concurrency

Additional Resources

  • IBM DB2 UDB for OS/390 Administration Guide, SC26-9931.

  • IBM DB2 UDB for OS/390 Application Programming and SQL Guide, SC26-9933.

  • IBM DB2 UDB for OS/390 Command Reference, SC26-9934.

  • IBM DB2 UDB for OS/390 SQL Reference, SC26-9944.


147 - Appendix A. Appendix

Appendix A. Appendix

This file will create the following DB2 objects for the DB2CERT database:

  • Stogroup, Database, Segmented Table Space, Partitioned Table Space, LOB Table Space, Tables, Indexes, Auxiliary Table, Auxiliary Index, User-Defined Types

CREATE STOGROUP CERTSTG
VOLUME(*) VCAT DB2USER1;

CREATE DATABASE CERTTS
STOGROUP CERTSTG
BUFFERPOOL BP7
INDEXBP BP8;

CREATE TABLESPACE CERTTS
IN DB2CERT
USING STOGROUP CERTSTG
   PRIQTY 52
   SECQTY 20
   ERASE NO
 LOCKSIZE PAGE
 BUFFERPOOL BP6
 CLOSE YES
 SEGSIZE 32;

CREATE TABLESPACE CERTTSPT
IN DB2CERT
USING STOGROUP CERTSTG
   PRIQTY 100
   SECQTY 120
   ERASE NO
NUMPARTS 2
(PART 1
 COMPRESS YES,
 PART 2
 FREEPAGE 20)
ERASE NO
LOCKSIZE PAGE
CLOSE NO;

CREATE LOB TABLESPACE CERTPIC
  IN DB2CERT
  USING STOGROUP CERTSTG
      PRIQTY 3200
      SECQTY 1600
  LOCKSIZE LOB
  BUFFERPOOL BP16K1
  GBPCACHE SYSTEM
  LOG NO
  CLOSE NO;

CREATE DISTINCT TYPE CANDIDATE_ID AS CHAR(9) WITH COMPARISONS;
CREATE DISTINCT TYPE TEST_ID AS CHAR(6) WITH COMPARISONS;
CREATE DISTINCT TYPE CENT AS CHAR(4) WITH COMPARISONS;
CREATE DISTINCT TYPE PHONE AS CHAR(10) WITH COMPARISONS;
CREATE DISTINCT TYPE SCORE AS DECIMAL(6,2) WITH COMPARISONS;
CREATE DISTINCT TYPE BITMAP AS BLOB(1M);
CREATE DISTINCT TYPE MINUTES AS SMALLINT WITH COMPARISONS;

CREATE TABLE DB2USER1.CANDIDATE
 (CID           CANDIDATE_ID    NOT NULL,
  LNAME         VARCHAR(30)     NOT NULL,
  FNAME         VARCHAR(30)     NOT NULL,
  INITIAL       CHAR(1),
  HPHONE        PHONE,
  WPHONE        PHONE,
  STREETNO      VARCHAR(8),
  STREETNAME    VARCHAR(20)     NOT NULL,
  CITY          VARCHAR(30)     NOT NULL,
  PROV_STATE    VARCHAR(30)     NOT NULL,
  CODE          CHAR(6)         NOT NULL,
  COUNTRY       VARCHAR(20)     NOT NULL,
  CERT_DBA      CHAR(1) NOT NULL WITH DEFAULT,
  CERT_APP      CHAR(1) NOT NULL WITH DEFAULT,
  PHOTO         BITMAP,
  PRIMARY KEY (CID)
  IN DB2CERT.CERTTS
 );

CREATE TABLE DB2USER1.TEST
(NUMBER         TEST_ID         NOT NULL,
 NAME           VARCHAR(50)     NOT NULL,
 TYPE           CHAR(1)         NOT NULL,
 CUT_SCORE      SCORE,
 LENGTH         MINUTES         NOT NULL,
 TOTALTAKEN     SMALLINT        NOT NULL WITH DEFAULT,
 TOTALPASSED    SMALLINT        NOT NULL WITH DEFAULT,
 PRIMARY KEY (NUMBER),
 CONSTRAINT test_type CHECK (TYPE in ('P','B'))
 IN DB2CERT.CERTTS
 );

-- Create Unique Clustering Index

CREATE UNIQUE INDEX DB2USER1.TESTIX
  ON DB2USER1.TEST
   (NUMBER ASC)
  USING STOGROUP CERTSTG
  PRIQTY 512
  SECQTY 64
  ERASE NO
  CLUSTER

 CREATE TABLE DB2USER1.TEST_CENTER
 (TCID          CENTER_ID       NOT NULL,
  NAME          VARCHAR(40)     NOT NULL,
  STREETNO      VARCHAR(8)      NOT NULL,
  STREETNAME    VARCHAR(20)     NOT NULL,
  CITY          VARCHAR(30)     NOT NULL,
  PROV_STATE    VARCHAR(30)     NOT NULL,
  COUNTRY       VARCHAR(20)     NOT NULL,
  CODE          CHAR(6)         NOT NULL,
  TYPE          CHAR(1)         NOT NULL,
  PHONE         PHONE           NOT NULL,
  NOSEATS       SMALLINT        NOT NULL,
  PRIMARY KEY (TCID)
  IN DB2CERT.CERTTSPT
 );

-- Create Partitioning Index

CREATE UNIQUE INDEX DB2USER1.TESTCNTX
  ON DB2USER1.TEST_CENTER
   (TCID ASC)
  USING STOGROUP CERTSTG
  PRIQTY 512
  SECQTY 64
  ERASE NO
  CLUSTER
    (PART 1 VALUES (300),
     PART 2 VALUES (500))
  BUFFERPOOL BP3
  CLOSE YES;

-- Create NPI with pieces

CREATE UNIQUE INDEX DB2USER1.TESTCN2X
  ON DB2USER1.TEST_CENTER
   (CODE ASC)
  USING STOGROUP CERTSTG
  PIECESIZE 512K;

 CREATE TABLE DB2USER1.TEST_TAKEN
( CID           CANDIDATE_ID    NOT NULL,
  TCID          CENTER_ID       NOT NULL,
  NUMBER        TEST_ID         NOT NULL,
  DATE_TAKEN    DATE            NOT NULL WITH DEFAULT,
  START_TIME    TIME            WITH DEFAULT,
  FINISH_TIME   TIME            WITH DEFAULT,
  SCORE         SCORE,
  PASS_FAIL     CHAR(1),
  SEAT_NO       CHAR(2)         NOT NULL,
  PRIMARY KEY (TCID,CID,DATE_TAKEN),
  FOREIGN KEY (CID)
        REFERENCES CANDIDATE ON DELETE CASCADE,
  FOREIGN KEY (TCID)
        REFERENCES TEST_CENTER ON DELETE CASCADE,
  FOREIGN KEY (NUMBER)
        REFERENCES TEST ON DELETE RESTRICT
                               ON UPDATE RESTRICT
  IN DB2CERT.CERTTS
);

CREATE AUX TABLE CAND_PHOTO
  IN DB2CERT.CERTPIC
  STORES DB2USER1.CANDIDATE
  COLUMN PHOTO;

CREATE UNIQUE INDEX DB2CERT.PHOTOIX
  ON DB2USER1.CAND_PHOTO
  USING VCAT DB2USER1
  COPY YES;


148 - Appendix B. DB2 UDB for OS/390 and z/OS Certification

Appendix B. DB2 UDB for OS/390 and z/OS Certification

If you are knowledgeable with DB2 Universal Database V7.1 and are capable of performing the intermediate to advanced skills required in the day-to-day administration of DB2 databases, you may benefit from this certification role.

To attain the IBM Certified Solutions Expert—DB2 UDB V7.1 Database Administration for OS/390 certification, candidates must pass 2 tests: Exam 512—DB2 Family Fundamentals and Exam 516—DB2 UDB V7.1 for OS/390 Database Administration. The exam objectives and sample questions for both tests are included in this appendix.


149 - Certification Test Objectives

Appendix B. DB2 UDB for OS/390 and z/OS Certification

Certification Test Objectives

The following text explains the objectives for each test that you must pass in order to become certified for DB2 UDB for OS/390.

DB2 Family Fundamentals (512)

IBM certifications in DB2 UDB V7.1 require an in-depth knowledge of basic SQL, DML, DDL, and an understanding of database objects (tables, views, etc.). Further, a basic understanding of the concepts and features of DCL, database creation, data types, stored procedures, programming, components and packaging (SDK, EEE, etc.), protocols supported, installation of products, GUI tools, CLP, basic isolation levels, and import/export is required.

Listed below are the primary objectives which the DB2 UDB V7.1 Family Fundamentals test covers. These objectives are listed here to assist you in your preparation for this test, which is a required component towards any DB2 UDB V7.1 certification role.

Installation and Planning

  • Ability to install DB2 Universal Database, DB2 clients & developers edition

  • Knowledge of and ability to use the DB2 UDB GUI tools and CLP

  • Knowledge of concepts of Data Warehouse and OLAP issues

  • Knowledge of new V7.1 tools

Security and Instances

  • Ability to describe the functionality of the Administration Server Instance (DAS)

  • Ability to provide users with authority on database objects

Creating and Accessing DB2 Databases

  • Ability to create a DB2 database

  • Ability to catalog a remote database or local database or DRDA

  • Ability to use the DB2 GUI tools to create, access, and manipulate DB2 objects

  • Ability to create and access basic DB2 objects

SQL Usage

  • Ability to start/end a unit of work

  • Given a DDL SQL statement, knowledge to identify results

  • Given a DML SQL statement, knowledge to identify results

  • Given a DCL SQL statement, knowledge to identify results

  • Ability to use SQL to SELECT data from tables

  • Ability to use SQL to sort or group data

  • Ability to use SQL to UPDATE, DELETE, or INSERT data

  • Knowledge to identify the effect of a COMMIT or ROLLBACK statement

  • Knowledge to identify the scope of a COMMIT or ROLLBACK

Database Objects

  • Ability to demonstrate usage of DB2 data types

  • Given a situation, ability to create a table

  • Given a situation, knowledge to identify when referential constraints are used

  • Knowledge to identify methods of data validation

  • Knowledge to identify characteristics of a table, view, or index

  • Knowledge of concepts of extenders

Database Concurrency

  • Knowledge to identify factors that influence locking

  • Ability to list the objects that locks can be obtained on

  • Knowledge to identify scope of different types of DB2 locks

  • Knowledge to identify factors affecting the number of locks that are used

  • Knowledge to identify factors affecting the type of locks obtained

  • Given a situation, knowledge to identify the isolation levels that should be used

DB2 UDB V7.1 for OS/390 Database Administration (516)

The certified DBA for OS/390 has extensive knowledge and ability to apply this knowledge in the OS/390 platform for the following critical areas:

Basic creating and managing DB2 objects (such as the ability to identify characteristics of an index, view, or table; interpret the contents of system catalogues and directories; create and alter table spaces; drop objects; add and drop columns in a table; identify the results of DDL SQL statements; and manage storage allocation); data recovery (such as the ability to understand the status of different objects, and backup and recovery considerations); performance and tuning; DB2 utilities (including the various DB2 commands) and security.

In addition, the certified individual is able to apply basic knowledge in the following areas:

Create aliases and synonyms for tables; describe activities associated with enabling stored procedures; describe the use of defer define; identify methods of data validation; check constraints, triggers, and UDFs; identify steps to prepare for a DB2 disaster recovery; convert logical design to physical design; capture and analyze DB2 trace data; determine if compression should be used; plan benefits of data sharing; describe considerations for distributed data, including security, backup, and recovery; describe the features that enable 7x24 availability; DB2 utilities including REPAIR; standalone utilities; run-time considerations for utilities; restart of a failed utility; RUNSTATS; REORG; TERM UTILITY command; LOAD options; DB2I; MODIFY; and CHECK DATA/INDEX/LOB utility.

Listed below are the primary objectives which the DB2 UDB V7.1 for OS/390 Database Administration test covers. These objectives are listed here to assist you in your preparation for this test, which is a required component towards the IBM Certified Solutions Expert—DB2 UDB V7.1 Database Administration for OS/390 certification role.

Create and Manage DB2 Objects

  • Identify the characteristics of an index, view, or table.

  • Interpret the contents of system catalog and directory.

  • Create and alter table spaces.

  • Drop objects.

  • Add and drop columns and change characteristics in a table.

  • Create aliases and synonyms for tables.

  • Describe activities associated with enabling stored procedures (Stored Procedure Builder).

  • Create all forms of referential integrity and determine the effects of each.

  • Given a DDL SQL statement, identify the results.

  • Manage storage allocation for example, using VSAM DELETE/DEFINEs or STOGROUPs.

  • Describe the use of Defer Define.

  • Identify methods of data validation, check constraints, triggers, and UDFs.

Data Recovery

  • Identify steps to prepare for a DB2 disaster recovery.

  • Understand the different object status (e.g., RECP, GRECP, LPL, RESTP).

  • Discuss backup and recovery considerations.

Performance and Tuning

  • Describe concurrency considerations.

  • Describe the effect of COMMIT frequency.

  • Convert logical design to physical design.

  • Analyze EXPLAIN/VISUAL EXPLAIN information.

  • Capture and analyze DB2 trace data.

  • Describe the difference between dynamic and static SQL.

  • Choose correct indexing columns and clustering index.

  • Determine correct index characteristics (FREEPAGE, PCTFREE, etc.).

  • Determine if compression should be used.

  • Determine when to REORG.

  • Describe BIND options.

Planning

  • Determine the benefits of data sharing.

  • Describe considerations for distributed data including security, backup, and recovery.

  • Describe the features that enable 7x24 availability.

Utilities

  • Describe the uses of the REPAIR utility.

  • Use various DB2 commands: DISPLAY, START, STOP, ALTER, RECOVER, and TERM UTILITY.

  • Describe the function and use of dynamic allocation and wildcarding.

  • Describe the features of the UNLOAD utility.

  • Describe the use of standalone utilities.

  • Identify the functions of the Control Center.

  • Describe the run-time considerations for utilities.

  • Demonstrate the restart of a failed utility.

  • Describe when it is necessary to run RUNSTATS.

  • Describe considerations for using online REORG.

  • Describe the implications of the TERM UTILITY command.

  • Explain the different LOAD options.

  • Describe DB2I and demonstrate its use.

  • Discuss the use of MODIFY utility.

  • Describe the function of the CHECK DATA/INDEX/LOB utility.

Security

  • Given a DCL SQL statement, identify the results.

  • Identify the use of GRANT and REVOKE.

Sample Questions (Fundamentals—Exam 512)

1:

A client application on OS/390 must access a DB2 server on Unix, Windows or OS/2. At a minimum, which of the following is required to be the DB2 server machine?

  1. DB2 Connect Enterprise Edition

  2. DB2 Universal Database Enterprise Edition

  3. DB2 Connect and DB2 Universal Database Workgroup Edition

  4. DB2 Connect and DB2 Universal Database Enterprise Edition
A1:

B

2:

Which of the following products is required to be installed in order to build an application on AIX, which will access a DB2 UDB for OS/390 database?

  1. DB2 Connect Personal Edition

  2. DB2 Personal Developer's Edition

  3. DB2 Universal Developer's Edition

  4. DB2 Universal Database Workgroup Edition
A2:

C

3:

Which of the following tools can be used to catalog a database?

  1. Journal

  2. Alert Center

  3. License Center

  4. Client Configuration Assistant
A3:

D

4:

Which of the following utilities would you run to order data and reclaim space from deleted rows in a table?

  1. reorg

  2. db2look

  3. db2move

  4. runstats
A4:

A

5:

Which of the following processes is NOT performed by DB2 Warehouse Manager?

  1. Query

  2. Loading

  3. Extraction

  4. Transformation
A5:

A

6:

Which of the following DB2 components allows reference to Oracle and DB2 databases in a single query?

  1. DB2 Query Patroller

  2. DB2 Warehouse Manager

  3. DB2 Relational Connect

  4. DB2 Connect Enterprise Edition
A6:

C

7:

The purpose of the USE privilege is to:

  1. query data in a table.

  2. load data into a table.

  3. create tables within a table space.

  4. create table spaces within a database.
A7:

C

8:

Which of the following database authorities is required to add a new package?

  1. BINDADD

  2. CREATETAB

  3. CREATEPKG

  4. PACKAGEADD
A8:

A

9:

Which two of the following authorities can create a database?

  1. DBADM

  2. SYSADM

  3. DBCTRL

  4. SYSCTRL

  5. SYSMAINT
A9:

B & D

10:

Cataloging a remote database is:

  1. performed on a PC or UNIX machine to identify the server the DB2 database manager is on.

  2. performed on a PC or UNIX machine to identify the DB2 database to users and applications.

  3. never performed in DB2, as only one database per node is allowed, so cataloging a node automatically catalogs the database at that node.

  4. performed on a PC or UNIX machine to open the catalogs in the DB2 database and present a user with a list of all accessible tables in that database.
A10:

B

11:

Given the create statements:

CREATE DISTINCT TYPE kph AS INTEGER WITH COMPARISONS
CREATE DISTINCT TYPE mph AS INTEGER WITH COMPARISONS

CREATE TABLE speed_limits
  (route_num SMALLINT,
   canada_sl KPH NOT NULL,
   us_sl MPH NOT NULL)

Which of the following is a valid query?

  1. SELECT route_num FROM speed_limits WHERE canada_sl > 80

  2. SELECT route_num FROM speed_limits WHERE canada_sl > kph

  3. SELECT route_num FROM speed_limits WHERE canada_sl > us_sl

  4. SELECT route_num FROM speed_limits WHERE canada_sl > kph(80)
A11:

D

12:

Given the tables:

EMPLOYEE
   emp_num emp_name    dept
   1         Adams     1
   2         Jones     1
   3         Smith     2
   4         Williams  1

DEPT
   dept_id dept_name
   1          Planning
   2          Support

and the statement:

ALTER TABLE employee
ADD FOREIGN KEY (dept) REFERENCES (dept_id)
ON DELETE CASCADE

How many units of work will be needed to process this statement?

DELETE FROM dept WHERE dept_id=1

  1. 0

  2. 1

  3. 2

  4. 3

  5. 4

  6. 6
A12:

B

13:

Given the tables:

COUNTRY
ID NAME       PERSON CITIES
1 Argentina   1      10
2 Canada      2      20
3 Cuba        2      10
4 Germany     1      0
5 France      7      5

STAFF
ID LASTNAME
1  Jones
2  Smith

The statement:

INSERT INTO staff SELECT person, 'Greyson' FROM country WHERE person > 1

will insert how many rows into the STAFF table?

  1. 0

  2. 1

  3. 2

  4. 3
A13:

D

14:

Which of the following statements eliminates all but one of each set of duplicate rows in the final result table?

  1. SELECT UNIQUE * FROM t1

  2. SELECT DISTINCT * FROM t1

  3. SELECT * FROM DISTINCT T1

  4. SELECT UNIQUE (*) FROM t1

  5. SELECT DISTINCT (*) FROM t1
A14:

B

15:

Given the following table definition and SQL statements:

CREATE TABLE table1 (col1 INT, col2 CHAR(40), col3 INT)
GRANT INSERT, UPDATE, SELECT, REFERENCES ON TABLE table1 TO
  USER usera

Which of the following SQL statements will revoke privileges for user USERA on COL1 and COL2?

  1. REVOKE UPDATE ON TABLE table1 FROM USER usera

  2. REVOKE ALL PRIVILEGES ON TABLE table1 FROM USER usera

  3. REVOKE ALL PRIVILEGES ON TABLE table1 COLUMNS (col1, col2) FROM USERA

  4. REVOKE REFERENCES ON TABLE table1 COLUMNS (col1, col2) FROM USER usera
A15:

B

16:

If, for a given table, the Control Center does not show the choice Generate DDL, which of the following describes the reason?

  1. The table is a system object.

  2. The table is a summary table.

  3. The table is in LOAD PENDING.

  4. The table is a replicated table.

  5. The table was created by a different user.
A16:

A

17:

How many indexes will be created by the following statement?

Create table mytab
  (
  Col1 int not null primary key,
  Col2 char(64),
  Col3 char(32),
  Col4 int not null,
  constraint c4 unique (Col4,Col1)
  )

  1. 0

  2. 1

  3. 2

  4. 3

  5. 4
A17:

C

18:

Given the two table definitions:

ORG
  deptnumb       INTEGER
  deptname       CHAR(30)
  manager        INTEGER
  division       CHAR(30)
  location       CHAR(30)

STAFF
  id             INTEGER
  name           CHAR(30)
  dept           INTEGER
  job            CHAR(20)
  years          INTEGER
  salary         DECIMAL(10, 2)
  comm           DECIMAL(10, 2)

Which of the following statements will display each department, alphabetically by name, and the name of the manager of the department?

  1. SELECT a.deptname, b.name FROM org a, staff b WHERE a.manager=b.id

  2. SELECT a.deptname, b.name FROM org a, staff b WHERE b.manager=a.id

  3. SELECT a.deptname, b.name FROM org a, staff b WHERE a.manager=b.id GROUP BY a.deptname, b.name

  4. SELECT a.deptname, b.name FROM org a, staff b, WHERE b.manager=a.id GROUP BY a.deptname, b.name
A18:

C

19:

Given the following tables:

NAMES
Name                      Number
Wayne Gretzky                 99
Jaromir Jagr                  68
Bobby Orr                      4
Bobby Hull                    23
Brett Hull                    16
Mario Lemieux                 66
Steve Yzerman                 19
Claude Lemieux                19
Mark Messier                  11
Mats Sundin 13

POINTS
Name                      Points
Wayne Gretzky                244
Jaromir Jagr                 168
Bobby Orr                    129
Bobby Hull                    93
Brett Hull                   121
Mario Lemieux                189

PIM
Name                         PIM
Mats Sundin                   14
Jaromir Jagr                  18
Bobby Orr                     12
Mark Messier                  32
Brett Hull                    66
Mario Lemieux                 23
Joe Sakic                     94

Which of the following statements will display the player's names, numbers, points, and PIM for all players with an entry in all three tables?

  1. SELECT names.name, names.number, points.points, pim.pim FROM names INNER JOIN points ON names.name=points.name INNER JOIN pim ON pim.name=names.name

  2. SELECT names.name, names.number, points.points, pim.pim FROM names OUTER JOIN points ON names.name=points.name OUTER JOIN pim ON pim.name=names.name

  3. SELECT names.name, names.number, points.points, pim.pim FROM names LEFT OUTER JOIN points ON names.name=points.name LEFT OUTER JOIN pim ON pim.name=names.name

  4. SELECT names.name, names.number, points.points, pim.pim FROM names RIGHT OUTER JOIN points ON names.name=points.name RIGHT OUTER JOIN pim ON pim.name=names.name
A19:

A

20:

Given the following table definitions:

DEPARTMENT
  deptno             CHAR(3)
  deptname           CHAR(30)
  mgrno              INTEGER
  admrdept           CHAR(3)

EMPLOYEE
  empno              INTEGER
  firstname          CHAR(30)
  midinit            CHAR
  lastname           CHAR(30)
  workdept           CHAR(3)

Which of the following statements will list every employee's number and last name with the employee number and last name of his or her manager, including employees without a manager?

  1. SELECT e.empno, e.lastname, m.empno, m.lastname FROM employee e LEFT INNER JOIN department INNER JOIN employee m ON mgrno = m.empno ON e.workdept = deptno

  2. SELECT e.empno, e.lastname, m.empno, m.lastname, FROM employee e LEFT OUTER JOIN department INNER JOIN employee m ON mgrno = m.empno ON e.workdept = deptno

  3. SELECT e.empno, e.lastname, m.empno, m.lastname FROM employee e RIGHT OUTER JOIN department INNER JOIN employee m ON mgrno = m.empno ON e.workdept = deptno

  4. SELECT e.empno, e.lastname, m.empno, m.lastname FROM employee e RIGHT INNER JOIN department INNER JOIN employee m ON mgrno = m.empno ON e.workdept = deptno
A20:

B

21:

Given the table:

STAFF
  ID LASTNAME
    Jones
    Smith

When issuing the query SELECT * FROM staff, the row return order will be based on which of the following?

  1. An ambiguous order

  2. The primary key order

  3. The order that the rows were inserted into the table

  4. The values for the ID column, then the LASTNAME column
A21:

A

22:

Given the table:

STAFF
  ID LASTNAME
  1 Jones
  2 Smith
  <null>

Which of the following statements removes all rows from the table where there is a NULL value for LASTNAME?

  1. DELETE FROM staff WHERE lastname IS NULL

  2. DELETE FROM staff WHERE lastname = 'NULL'

  3. DELETE ALL FROM staff WHERE lastname IS NULL

  4. DELETE ALL FROM staff WHERE lastname = 'NULL'
A22:

A

23:

Given the tables:

COUNTRY
  ID NAME           PERSON  CITIES
  1 Argentina       1       10
  2 Canada          2       20
  3 Cuba            2       10
  4 Germany         1        0
  5 France          7        5
STAFF
  ID LASTNAME
  1  Jones
  2  Smith

Which of the following statements removes the rows from the COUNTRY table that have PERSONS in the STAFF table?

  1. DELETE FROM country WHERE id IN (SELECT id FROM staff)

  2. DELETE FROM country WHERE id IN (SELECT person FROM staff)

  3. DELETE FROM country WHERE person IN (SELECT id FROM staff)

  4. DELETE FROM country WHERE person IN (SELECT person FROM staff)
A23:

C

24:

The table STOCK has the following column definitions:

type          CHAR (1)
status        CHAR(1)
quantity      INTEGER
price         DEC (7, 2)

Items are indicated to be out of stock by setting STATUS to NULL and QUANTITY and PRICE to zero. Which of the following statements updates the STOCK table to indicate that all the items except for those with TYPE of "S" are temporarily out of stock?

  1. UPDATE stock SET status='NULL', quantity=0, price=0 WHERE type <> 'S'

  2. UPDATE stock SET (status, quantity, price) = (NULL, 0, 0) WHERE type <> 'S'

  3. UPDATE stock SET (status, quantity, price) = ('NULL', 0, 0) WHERE type <>'S'

  4. UPDATE stock SET status = NULL, SET quantity=0, SET price = 0 WHERE type <>'S'
A24:

B

25:

Which of the following CANNOT have an autocommit setting?

  1. Embedded SQL

  2. The Command Center

  3. The Command Line Processor

  4. The DB2 Call Level Interface
A25:

A

26:

Given the successfully executed embedded SQL:

INSERT INTO staff VALUES (1, 'Colbert', 'Dorchester', 1)
COMMIT
INSERT INTO staff VALUES (6, 'Anders', 'Cary', 6)
INSERT INTO staff VALUES (3, 'Gaylord', 'Geneva', 8)
ROLLBACK WORK

Which of the following indicates the number of new rows that would be in the STAFF table?

  1. 0

  2. 1

  3. 2

  4. 3
A26:

B

27:

Given: CREATE TABLE t1 (c1 CHAR(4) NOT NULL). Which of the following can be inserted into this table?

  1. 4

  2. NULL

  3. 'abc'

  4. 'abcde'
A27:

C

28:

Given the requirements to store names, employee numbers, and when the employees were hired, which of the following DB2 data types CANNOT be used to contain the day the employee was hired?

  1. CLOB

  2. TIME

  3. VARCHAR

  4. TIMESTAMP
A28:

B

29:

Which of the following DDL statements creates a table where employee ids are unique?

  1. CREATE TABLE t1 (employid INTEGER)

  2. CREATE TABLE t1 (employid UNIQUE INTEGER)

  3. CREATE TABLE t1 (employid INTEGER NOT NULL)

  4. CREATE TABLE t1 (employid INTEGER NOT NULL, primary key (employid))
A29:

D

30:

Given the transaction:

"CREATE TABLE t1 (id INTEGER,CONSTRAINT chkid CHECK (id<100))"
"INSERT INTO t1 VALUES (100)"
"COMMIT"

Which of the following results from the transaction?

  1. The row is inserted with a NULL value.

  2. The row is inserted with a value of 100.

  3. The row insertion with a value of 100 is rejected.

  4. The trigger called chkid is fired to validate the data.
A30:

C

31:

Which of the following delete rules will not allow a row to be deleted from the parent table if a row with the corresponding key value still exists in the child table?

  1. DELETE

  2. CASCADE

  3. RESTRICT

  4. SET NULL
A31:

C

32:

If a table is defined with a check constraint for one or more columns, which of the following will perform the data validation after the table is loaded with the load utility?

  1. Reorg

  2. Check

  3. Runstats

  4. Image Copy

  5. Set Integrity
A32:

E

33:

Given the statements and operations:

"CREATE TABLE t1 (c1 CHAR(1))"
Six rows are inserted with values of: a, b, c, d, e and f
"SET CONSTRAINTS FOR t1 OFF"
"ALTER TABLE t1 ADD CONSTRAINT con1 CHECK (c1 ='a')"
"SET CONSTRAINTS FOR t1 IMMEDIATE CHECKED FOR EXCEPTION IN t1
 USE t1exp"

Which of the following describes what happens to the rows with values of b, c, d, e, and f?

  1. deleted from T1 only

  2. deleted from T1 and written into the t1exp file

  3. deleted from T1 and inserted into the table t1exp

  4. deleted from T1 and written into the db2diag.log file

  5. deleted from T1 and inserted into the table syscat.checks
A33:

C

34:

A declared temporary table is used for which of the following purposes?

  1. backup purposes

  2. storing intermediate results

  3. staging area for the load utility

  4. sharing result sets between applications
A34:

B

35:

Given the statement:

CREATE VIEW v1 AS SELECT c1 FROM t1 WHERE c1='a'
WITH CHECK OPTION

Which of the following SQL statements will insert data into the table?

  1. INSERT INTO v1 VALUES (a)

  2. INSERT INTO v1 VALUES (b)

  3. INSERT INTO v1 VALUES ('b')

  4. INSERT INTO v1 VALUES ('a')

  5. INSERT INTO v1 VALUES ('ab')
A35:

D

36:

Which of the following products can be used to store image data in a DB2 database?

  1. Net.Data

  2. Net Search

  3. DB2 AVI Extenders

  4. DB2 XML Extenders

  5. DB2 Text Extenders
A36:

C

37:

For which of the following database objects can locks NOT be obtained?

  1. A row

  2. A table

  3. A column

  4. An index key
A37:

C

38:

An update lock gets released by an application using the repeatable read isolation level during which of the following?

  1. If the cursor accessing the row is closed

  2. If the transaction issues a ROLLBACK statement

  3. If the cursor accessing the row is moved to the next row

  4. If the transaction changes are made via an UPDATE statement
A38:

B

39:

Which of the following isolation levels is most likely to acquire a table level lock during an index scan?

  1. Read Stability

  2. Repeatable Read

  3. Cursor Stability

  4. Uncommitted Read
A39:

B

40:

Which of the following processing can occur for a unit of work using an isolation level of Uncommitted Read and scanning through the table more than once within the unit of work?

  1. Access uncommitted changes made by other processes

  2. Update uncommitted changes made by other processes

  3. Update rows of a return set and have those updates changed by other processes from one scan to the next

  4. Update rows of a return set and have those updates committed by other processes from one scan to the next
A40:

A

41:

Which of the following releases a lock by an application using the cursor stability isolation level?

  1. If the cursor accessing the row is moved to the next row

  2. If the cursor accessing the row is used to update the row

  3. If the application's current row is deleted by the application

  4. If the application's current row needs to be updated by another application
A41:

A

42:

Given the requirement of providing a read-only database, applications accessing the database should be run with which of the following isolation levels to allow for the most read concurrency?

  1. Read stability

  2. Repeatable read

  3. Cursor stability

  4. Uncommitted read
A42:

D

Sample Questions (Administration—Exam 516)

1:

Which two of the following properties describe the following index?

CREATE INDEX emp1
    ON emp (cola, colb)

  1. Unique

  2. Composite

  3. Non-Unique

  4. Clustering

  5. Descending

  6. Partitioning
A1:

B, C

2:

Which of the following represent mutually exclusive keywords on the CREATE INDEX statement?

  1. CLOSE and DEFER

  2. VCAT and STOGROUP

  3. FREEPAGE and PCTFREE

  4. CLUSTER and PIECESIZE
A2:

B

3:

RUNSTATS was executed on the EMPLOYEE table and its indexes, choosing the option to update the DB2 catalog with the statistics gathered. Immediately following the RUNSTATS execution, the following statistics were retrieved from the DB2 catalog:

The non-partitioned EMPLOYEE table has a value of 3,000,000 in the CARDF column of SYSIBM.SYSTABLEPART. The EMP1 index of the EMPLOYEE table has a value of 2 in the FIRSTKEYCARD column of SYSIBM.SYSINDEXES and 2,000,000 in the FULLKEYCARD column of SYSIBM. SYSINDEXES.

How many index entries are there?

  1. 2

  2. 3,000,000

  3. 4,000,000

  4. 2,000,000
A3:

D

4:

Which of the following CREATE TABLESPACE statements will create a table space TS1 in database DB1 that can hold more than 256 GB of non-LOB data?

  1. CREATE TABLESPACE ts1 IN db1 ...

  2. CREATE BIG TABLESPACE ts1 IN db1 ...

  3. CREATE TABLESPACE ts1 IN db1 NUMPARTS 180 DSSIZE 4G...

  4. CREATE SEGMENTED TABLESPACE ts1 IN db1 NUMPARTS 20 DSSIZE 4G ...
A4:

C

5:

Which of the following CREATE TABLESPACE statements will create a segmented table space TS1 in database DB1 with 16 pages to a segment?

  1. CREATE TABLESPACE ts1 IN db1 ...

  2. CREATE TABLESPACE ts1 IN db1 SEGSIZE 16

  3. CREATE PARTITIONED TABLESPACE ts1 IN db1 SEGSIZE 16...

  4. CREATE SEGMENTED TABLESPACE ts1 IN db1 SEGMENTSIZE 16...
A5:

B

6:

For a LOB column, if the table space containing the base table is partitioned, which of the following must also occur?

  1. Partition the LOB table space with the same number of partitions.

  2. Create a LOB table space and an auxiliary table for each partition.

  3. Create an auxiliary table for each partition in only one LOB table space.

  4. Create a table space with DSSIZE and an auxiliary table for each partition.
A6:

B

7:

In order to remove the WITH RESTRICT ON DROP specification from a table, which of the following SQL statements can be used?

  1. DROP

  2. ALTER

  3. DELETE

  4. REVOKE
A7:

B

8:

Which two of the following can be done with an ALTER TABLE statement?

  1. Make a column NOT NULL.

  2. Add a column constraint.

  3. Change the length of a VARCHAR column.

  4. Add a column to the middle of the table.

  5. Change the data type from INTEGER to FLOAT.
A8:

B, C

9:

User SMITH who has SYSADM authority is asked to create a synonym on table TEST.PARTS for user JONES. Which of the following statements will define a synonym on table TEST.PARTS for JONES?

  1. CREATE SYNONYM parts FOR test.parts;

  2. SET CURRENT SQLID = 'JONES' CREATE SYNONYM parts FOR parts;

  3. SET CURRENT SQLID = 'TEST' CREATE SYNONYM parts FOR test.parts;

  4. SET CURRENT SQLID = 'JONES' CREATE SYNONYM parts FOR test.parts;
A9:

D

10:

Given that the WLM application environment is already set up in OS/390 and active in DB2. Which of the following tasks must be completed before a stored procedure can run on an OS/390 server in a WLM controlled stored procedure address space?

    1. Precompile and compile the application.

    2. Link-edit the application.

    3. Bind the DBRM to DB2 using the command BIND PLAN.

    4. Define the stored procedure to WLM.

    5. Use GRANT EXECUTE to authorize the appropriate users to use the stored procedure.

    1. Precompile and compile the application.

    2. Link-edit the application.

    3. Bind the DBRM to DB2 using the command BIND PACKAGE.

    4. Define the stored procedure to DB2.

    5. Use GRANT EXECUTE to authorize the appropriate users to use the stored procedure.

    1. Precompile and compile the application.

    2. Link-edit the application.

    3. Bind the DBRM to DB2 using the command BIND PACKAGE.

    4. Define the stored procedure to WLM.

    5. Use GRANT EXECUTE to authorize the appropriate users to use the stored procedure.

    1. Precompile and compile the application.

    2. Link-edit the application.

    3. Bind the DBRM to DB2 using the command BIND PROCEDURE.

    4. Define the stored procedure to DB2.

    5. Use GRANT EXECUTE to authorize the appropriate users to use the stored procedure.
A10:

B

11:

The following data exists in the EMPLOYEE and DEPARTMENT tables:

DEPARTMENT
DEPTNO | DEPTNAME
01 | SALES
02 | MARKETING
03 | HUMAN RESOURCES

EMPLOYEE
EMPNO | WORKDEPT
10 | 01
20 | 01
30 | 04

DEPTNO is the primary key of DEPARTMENT. EMPNO is the primary key of EMPLOYEE.

The DBA performs the following DDL:

ADD FOREIGN KEY empdept (workdept) REFERENCES department.deptno ON DELETE RESTRICT

Which of the following actions will occur?

  1. The DDL will fail.

  2. The EMPLOYEE table will be put in CHECK PENDING.

  3. The DEPARTMENT table will be put in CHECK PENDING.

  4. The row from EMPLOYEE where EMPNO=30 will be deleted.
A11:

B

12:

A referential integrity (RI) constraint is added with special register CURRENT RULES set to 'STD'. No delete rule is specified. Which of the following delete rules will the RI constraint default to?

  1. CASCADE

  2. RESTRICT

  3. SET NULL

  4. NO ACTION
A12:

D

13:

A table has been created using the following DDL statements:

CREATE TABLE dev.t1
(
c1 DECIMAL(15,0) GENERATED BY DEFAULT AS IDENTITY
, c2 VARCHAR(200)
);
CREATE UNIQUE INDEX i1 ON TABLE dev.t1(c1);

Which of the following errors can occur if two applications, one providing values for column C1 and C2, the other providing values for C2 only, are inserting data into table T1?

  1. Duplicate values for column c1, no SQLERROR

  2. SQLCODE -817, the SQL statement cannot be executed because the statement will result in a prohibited update operation

  3. SQLCODE -803, an inserted or updated value is invalid because the index in index space i1 constrains columns of the table so no two rows can contain duplicate values in those columns

  4. SQLCODE -811, the result of an embedded select statement or a subselect in the set clause of an update statement is a table of more than one row, or the result of a subquery of a basic predicate is more than one value
A13:

C

14:

Table space DB1.TS1 is storage group managed with PRIQTY of 7200 and SECQTY of 720. Which of the following scenarios represent the minimum requirement for successfully altering table space DB1.TS1 to have a SECQTY of 7200 and have the new SECQTY take affect on the next secondary extent?

  1. ALTER TABLESPACE db1.ts1 SECQTY 7200

  2. ALTER TABLESPACE db1.ts1 SECQTY 7200 REORG TABLESPACE db1.ts1 . . .

  3. -STOP DB(db1) SPACENAME(ts1) ALTER TABLESPACE db1.ts1 SECQTY 7200 -START DB(db1) SPACENAME(ts1)

  4. -STOP DB(db1) SPACENAM(ts1) ALTER TABLESPACE db1.ts1 SECQTY 7200 -START DB(db1) SPACENAM(ts1) REORG TABLESPACE db1.ts1 . . .
A14:

A

15:

Which of the following columns and column values in the DB2 catalog identifies an unpopulated index that has been created with DEFINE NO?

  1. 'N' in CREATED in SYSIBM.SYSINDEXES

  2. 0 in SPACE in SYSIBM.SYSINDEXES

  3. 'N' in DEFINE in SYSIBM.SYSINDEXES

  4. -1 in SPACE in SYSIBM.SYSINDEXPART
A15:

D

16:

Which two of the following are valid ALTER TABLE capabilities?

  1. Add an IDENTITY column.

  2. Change the name of the table's table space.

  3. Add a column to SYSIBM.LOCATIONS.

  4. Increase the length of an existing integer column.

  5. Disassociate an existing validation proc from a table.
A16:

A, E

17:

Which of the following are the minimum requirements to be able to recover at an off-site location?

  1. BSDS only

  2. BSDS and all logs

  3. BSDS and archive logs

  4. Image copies and archive logs
A17:

D

18:

After a program updates DB2 data without committing, a -CANCEL THREAD NOBACKOUT command is issued against the executing program. Which of the following table space exception statuses would the updated object have?

  1. COPY

  2. RECP

  3. RESTP

  4. DEFER
A18:

C

19:

Which of the following table space or index space statuses prevent any access by an application program:

RECP

RO

UT

ICOPY

LSTOP

  1. I and V

  2. I and III

  3. II and IV

  4. I, III and V

  5. III, IV and V
A19:

B

20:

The STATE table is joined in a referential set to the EMPLOYEE table. The EMPLOYEE table gets 150 changes a week by a batch job that runs at 9:00 a.m. every morning. The batch job is the only way the EMPLOYEE table is updated. An image copy of the EMPLOYEE table is taken every night at 6:00 p.m. An image copy of the STATE table is taken on the first day of every month. There have been no changes to the STATE table since the last image copy. On Friday morning at 10:00 a.m. on the 30th of June, the programmer informs the DBA that the last batch run was incorrect due to programming changes.

Which of the following can a DBA do to ensure all tables in the referential set are brought to a consistent state?

  1. Recover EMPLOYEE to the June 29th image copy; rebuild the EMPLOYEE indexes

  2. Recover the STATE table to the June 1st image copy; recover EMPLOYEE to the June 29th image copy

  3. Recover the STATE table to the June 1st image copy; recover EMPLOYEE to the June 29th image copy. Recover STATE and EMPLOYEE indexes

  4. Recover the STATE table to the June 1st image copy; recover the STATE indexes. Have the programmer write an update program to remove the incorrect changes in the EMPLOYEE table
A20:

A

21:

Given the following scenario:

Table space TSA with one table—TABA, and one index on that table—IXA. Following are five events that occur in sequence:

Time T1: TSA and IXA are quiesced

Time T2: There are updates to table TABA and the IXA index

Time T3: TSA and IXA are copied SHARE LEVEL REFERENCE

Time T4: Updates to table TABA and the IXA index are allowed

Time T5: TSA and IXA are Copied SHARE LEVEL CHANGE

Time T6: Issue the command -ARCHIVE LOG

Which two of the following recovery scenarios produce consistent data between TABA and IXA?

  1. Recover to time T1

  2. Recover to time T2

  3. Recover to time T3

  4. Recover to time T4

  5. Recover to time T5
A21:

A, C

22:

Table space TSP1 contains only one table, EMP. EMP has a unique index on the EMPNO column. A full IC on just table space TSP1 is taken on Sunday and a disk crash is experienced on Thursday. A point-in-time recovery is performed to the last full IC taken on Sunday on TSP1. The index, however, is still in a restricted state. Which of the following utilities can be run to ensure the integrity of the index and reset its status?

  1. CHECK INDEX

  2. REORG INDEX

  3. REPAIR INDEX

  4. REBUILD INDEX
A22:

D

23:

Which of the following isolation levels will ensure that the same rows are retrieved for successive executions of the same SQL statement?

  1. Read Stability

  2. Repeatable Read

  3. Uncommitted Read

  4. Cursor Stability
A23:

B

24:

An application needs access from batch programs and online transactions at the same time. Currently the batch program updates DB2 data and issues no commits. The online transactions also do updates and consequently receive many timeouts and deadlocks. Which of the following would be the BEST solution to correct this deadlock and timeout problem?

  1. BIND the batch programs with ACQUIRE(USE).

  2. BIND the batch programs with RELEASE(COMMIT).

  3. Add more frequent commit points within the batch job.

  4. Change the batch program to issue a LOCK TABLE against the table being updated.
A24:

C

25:

During logical design, the entities ORDER, PART, and CUSTOMER were developed. The primary keys are ORD_NUM, PART_NUM, and CUST_NUM respectively. CUST_NUM is a foreign key on ORDER and PART_NUM is a foreign key on ORDER. The customer's name and address are on CUSTOMER. The order number, order date, and part number are on ORDER. The part description is on PART. A critical transaction for the company is to produce an order form for each customer order that contains the customer's name and address, order number, order date, and part numbers and part descriptions for all parts ordered. There are over 1,000,000 of these produced daily.

Which of the following can the DBA do during physical design to improve performance of this critical query?

  1. Denormalize the PART and ORDER tables.

  2. Denormalize the PART and CUSTOMER tables.

  3. Build an index on ORD_NUM on the ORDER table.

  4. Create a view joining the ORDER and PART tables.

  5. Create a view joining ORDER, PART, and CUSTOMER tables.
A25:

A

26:

How many unique keys can a table have?

  1. 255

  2. 750

  3. More than one

  4. Only one: the primary key

  5. Two: the primary and a unique key
A26:

C

27:

Given the following output from the PLAN_TABLE:

TABNO  PLANNO  TNAME  ACCESSTYPE  MATCHCOLS  INDEXONLY METHOD
   1       2   TABLEY      I          1         Y        1
   2       1   TABLEX      I          0         N        0

Which of the following describes what is happening?

  1. First, TABLEX is accessed via a non-matching index scan. Then a nested loop join is used to access TABLEY via a 1 column matching index scan.

  2. First, TABLEX is accessed via a matching index scan. Then a nested loop join is used to access TABLEY via a 1 column matching index-only scan.

  3. First, TABLEX is accessed via a non-matching index scan. Then a nested loop join is used to access TABLEY via a 1 column matching index-only scan.

  4. First, TABLEY is accessed via a non-matching index scan. Then a nested loop join is used to access TABLEX via a 1 column matching index-only scan.
A27:

C

28:

Given the following:

TABLEX
COLA VARCHAR(10)
COLC DECIMAL (15,2)

There is an index on TABLEX on COLA.

TABLEY
COLB CHAR(5)
COLD DECIMAL (13,2)

There is an index on TABLEY on COLB,COLD.

SELECT X.COLA, X.COLC, Y.COLB, Y.COLD
  FROM TABLEX X, TABLEY Y
 WHERE X.COLA = Y.COLB
   AND X.COLC = Y.COLD
   AND X.COLA = 'abc'

PLANNO TNAME    ACCESSTYPE    MATCHCOLS     INDEXONLY
  1    TABLEX        I              1            N
  2    TABLEY        I              1            Y

Why does MATCHCOLS indicate 1 for TABLEY?

  1. X.COLA and Y.COLB have different lengths.

  2. X.COLC and Y.COLD have different lengths.

  3. X.COLA and Y.COLB have different data types.

  4. MATCHCOLS indicates only the first column used in an index.
A28:

B

29:

Given the following fields from a DB2 Accounting Long Report:

Class 1 Elapsed Time : 1:01.610942

Class 1 CPU Time : 56.879690

Class 2 Elapsed Time : 01.002753

Class 2 CPU Time : 01.001654

Class 3 Elapsed Time : 0.359907

Which of the following scenarios are TRUE?

  1. Most of the time is being spent in DB2 and not the application.

  2. Most of the time is being spent in DB2 but waiting on resources.

  3. Most of the time is being spent in the application and not in DB2.

  4. Unable to determine with just these fields from the accounting report.
A29:

C

30:

What category (type) of DB2 trace data is needed to determine the number of times that certain SQL statements are issued?

  1. Audit

  2. Statistics

  3. Accounting

  4. Performance
A30:

C

31:

Which of the following is TRUE for dynamic SQL?

  1. All of the following are true.

  2. Dynamic SQL is more flexible but more complex than static SQL.

  3. For dynamic SQL, security is checked at execution time when the access path is created.

  4. Dynamic SQL must be used if the type of SQL statement is unknown at program preparation time.
A31:

A

32:

The following SQL will be run nightly on a 10-million row table. It must complete within the batch window. COLZ has a cardinality of 9 million, COLD has a cardinality of 2 million, and COLA has a cardinality of 1 million.

SELECT COLA, COLB, COLC, COLD, COLE
  FROM TABLEX
  WHERE COLZ = ?
  ORDER BY COLD, COLA

In addition, 5 million important customers will be running the following SQL query. It is an important business criterion that these customers complete their queries in subsecond response time. COLY is the primary key of this table.

SELECT COLA, COLB, COLZ
  FROM TABLEX
  WHERE COLY = ?
  ORDER BY COLA

Which of the following should be the clustering index?

  1. COLY, COLA

  2. COLZ, COLD, COLA

  3. COLZ, COLA, COLD

  4. COLY, COLD, COLA
A32:

B

33:

Given the EMPLOYEE table with an index on DEPT, DEPT is a column which allows NULLS. The DBA defines an index on DEPT as UNIQUE WHERE NOT NULL before any data is loaded. As data is loaded, which of the following will happen to any row with NULL in DEPT?

  1. It will be rejected.

  2. It will be accepted.

  3. It will be accepted only if the rest of the row is unique.

  4. It will be rejected if it is the second row inserted with a NULL value.
A33:

B

34:

A table that contains phone book information: last name, first name, address, and phone number. The table has a clustering index on last name, first name, and the table has a high rate of change activity (>20% of volume inserted or updated) in random order.

Index page splitting can BEST be reduced for this table through the use of which two of the following used together?

  1. Frequent reorganizations

  2. Index with FREEPAGE 0; PCTFREE 0

  3. Index with FREEPAGE 20; PCTFREE 20

  4. Table space with FREEPAGE 20; PCTFREE 20

  5. Index with FREEPAGE 0; PCTFREE 10 (default values)
A34:

A, C

35:

Which of the following table spaces is most likely to benefit from compression?

  1. A small table space

  2. A table space containing LOB data

  3. A table space containing encrypted data

  4. A table space containing data with many repeated strings
A35:

D

36:

Which of the following queries helps determine when to REORG an index?

  1. SELECT IXNAME, IXCREATOR FROM SYSIBM.SYSINDEXPART WHERE CARDF > 250;

  2. SELECT IXNAME, IXCREATOR FROM SYSIBM.SYSINDEXPART WHERE LEAFDIST > 250;

  3. SELECT PERCDROP FROM SYSIBM.SYSTABLEPART WHERE DBNAME = 'XXX' AND TSNAME = 'YYY';

  4. SELECT CARD, NEARINDREF, FARINDREF FROM SYSIBM. SYSTABLEPART WHERE DBNAME = 'XXX' AND TSNAME = 'YYY';
A36:

B

37:

Given the following: The EMPLOYEE table is copied using a full image copy with a share level of reference. Data is inserted into the EMPLOYEE table. The EMPLOYEE table is reorged with a share level of reference and no logging. The REPAIR utility is used to reset the copy pending flag.

Which of the following is TRUE about the EMPLOYEE table?

  1. The table is in read-only status.

  2. The table is in copy pending status.

  3. The table can be used with no problem.

  4. The table is not recoverable to currency.
A37:

D

38:

A user needs to bind a package which references tables to which she does not have explicit access. Which of the following is the minimum level of authorization that she needs?

  1. SYSADM

  2. PACKADM

  3. BINDADD

  4. BINDAGENT
A38:

D

39:

Which of the following columns in SYSPLAN can be used to determine the data currency option that a plan was bound with?

  1. ISOLATION

  2. DEFERPREP

  3. CURRENTDATA

  4. EXPREDICATE
A39:

D

40:

Which of the following ISOLATION levels allows an application to read a row locked by another process?

  1. UR

  2. CS

  3. RR

  4. RS
A40:

A

41:

Which of the following statements are TRUE regarding DB2 data sharing?

  1. The same application can be run on more than one DB2 subsystem to achieve transaction rates that are higher than possible on a single subsystem using the same DB2 data.

  2. A new DB2 member can be added on another server and access the same data through the new DB2.

  3. All DB2 members in the data-sharing group have concurrent read/write access to the same DB2 data.

  4. All DB2 members in the data-sharing group use a single DB2 catalog and directory.

    1. II and IV

    2. II and III

    3. I, II and III

    4. II, III and IV

    5. I, II, III and IV
A41:

E

42:

Which of the following TCP/IP components is NOT required to connect to DB2 UDB for OS/390?

  1. DNS

  2. DB2 location name

  3. Network TCP port number

  4. Network communications mode
A42:

D

43:

Many are using packaged applications (for example, ERPs) with the following characteristics:

Dynamic SQL is extensively used by the application and there is a large number of tables and indexes that have never contained data.

Which two of the following DB2 functions enhance applications with the above characteristics?

  1. DB2 Governor

  2. 8KB buffer pools

  3. Dynamic statement caching

  4. SQL GROUP BY within a view

  5. Dual active and archive logging

  6. Deferred definition of table space/index data sets

  7. DB2 Unload utility
A43:

C, F

44:

Which of the following utility statements does NOT support concurrent processing?

  1. LOAD RESUME

  2. LOAD REPLACE

  3. COPY SHRLEVEL CHANGE

  4. REORG SHRLEVEL CHANGE
A44:

B

45:

An ID with installation SYSOPR authority can only execute REPAIR on table spaces in which of the following databases?

  1. DSNDB01

    DSNDB06

  2. User databases

  3. DSNDB01

    DSNDB07

    User databases

  4. DSNDB01

    DSNDB06

    DSNDB07

    User databases
A45:

A

46:

Which of the following is NOT an option of repairing an index with the REPAIR utility?

  1. No copy pending (REPAIR SET INDEX....NOCOPYPEND)

  2. No rebuild pending (REPAIR SET INDEX....NORBDPEND)

  3. No auxiliary warning (REPAIR SET INDEX....NOAUXWARN)

  4. No recovery pending (REPAIR SET INDEX....NORCVRPEND)
A46:

C

47:

Given the following situation: BP7 is currently 10,000 pages and is used for DSNDB07. A new application requires extensive sorting, and test results indicate that BP7 should be at least 80,000 pages. Which of the following is the correct command to expand buffer pool BP7 to 80,000 pages?

  1. -ALTER BUFFERPOOL(BP7) SIZE(80000)

  2. -ALTER BUFFERPOOL(BP7) HPSIZE(80000)

  3. -ALTER BUFFERPOOL(BP7) VPSIZE(80000)

  4. -ALTER BUFFERPOOL(BP7) VPPSEQT(80000)
A47:

C

48:

Which of the following can break in on locks held by a RELEASE (DEALLOCATE) plan or package?

  1. REBIND PACKAGE

  2. -STOP PROCEDURE

  3. -RECOVER INDOUBT ACTION(COMMIT)

  4. -STOP DB(dbname) SPACE(spacename)

  5. -STOP DB(dbname) SPACE(spacename) AT(COMMIT)
A48:

E

49:

Which of the following values for the UTPROC parameter of DSNUPROC causes the utility to perform a syntax check on the control statements?

  1. EXPAND

  2. PREVIEW

  3. TEMPLATE

  4. GENERATE
A49:

B

50:

The use of TEMPLATE control statements simplifies your utility JCL. Which of the following is NOT true when using a TEMPLATE statement?

  1. Use of a template eliminates most data set DD cards.

  2. Utilities submitted using templates are not restartable.

  3. The DB2 Control Center can be used to create and update utility templates.

  4. DB2 utilities dynamically allocate the data sets that are required based on your allocation information.
A50:

B

51:

Which two of the following are NOT allowed with the UNLOAD utility?

  1. SHRLEVEL REFERENCE

  2. SHRLEVEL CHANGE ISOLATION RR

  3. SHRLEVEL CHANGE ISOLATION UR

  4. SHRLEVEL CHANGE ISOLATION CS

  5. SHRLEVEL CHANGE ISOLATION RS
A51:

B, E

52:

In order to format the contents of the recovery log for display, DSN1LOGP can be run. If the BSDS is not available, which two of the following DD statements are used to point to the DB2 LOGs?

  1. ACTIVEn

  2. ARCHIVE

  3. ACTLOGn

  4. ARCHLOG

  5. SYSCOPY

  6. SYSOUTn

  7. DSNLOGn
A52:

A, B

53:

The DB2 Control Center can assist in determining when a table space or index should be reorganized on OS/390. Using the Query Catalog panel for clustering level problems on an index, which of the following statements is NOT provided?

  1. % of rows with percent free

  2. % of rows on pages far away

  3. % or rows in clustering order

  4. % of rows on pages near to optimal page
A53:

A

54:

Which of the following two options are NOT available when restarting DB2 Utilities within the DB2 Control Center?

  1. Restart from an image copy

  2. Restart from a point in time

  3. Restart from the last committed phase

  4. Restart from the last committed point (current)

  5. Restart of any utility started within the DB2 Control Center
A54:

A, B

55:

Which of the following levels of DB2 authorization is required to use DB2 Control Center actions?

  1. SYSADMIN

  2. SYSADMIN or DBADMIN

  3. No additional authorities are needed beyond those needed to execute the same actions without DB2 Control Center.

  4. Any DB2 authorization level can view output from DB2 Control Center, but only SYSADMIN can perform functions that get executed.
A55:

C

56:

Which of the following utilities are NOT supported for LOB table spaces?

  1. LOAD, with LOG NO option

  2. COPY, with CONCURRENT option

  3. CHECK LOB, with EXCEPTIONS option

  4. REORG, with SHRLEVEL REFERENCE option
A56:

D

57:

During the RELOAD phase due to an operator's accidentally canceling the job, a REORG utility with utility ID of MYREORG has failed. Which of the following UTPROC parameters restarts the utility?

  1. UTPROC='MYREORG,RESTART'

  2. UTPROC='MYREORG,RESTART(CURRENT)'

  3. UTPROC='MYREORG,RESTART(PHASE)'

  4. UTPROC='MYREORG,RESTART(FAILED)'

Note

A TERM UTILITY was NOT performed.

  1. I only is correct.

  2. IV only is correct.

  3. III only is correct.

  4. I and III are correct.

  5. I, II and III are correct.
A57:

E

58:

After running an EXPLAIN on a SPUFI SELECT statement, the output from the DSN_STATEMNT_TABLE is as follows:

STMT      COST
TYPE      CATEGORY     PROCMS     PROCSU     REASON
----      --------     ------     ------     ------
SELECT      B          53         225        TABLE CARDINALITY

What action may be taken to obtain a more accurate cost estimate for this statement?

  1. No action.

  2. Reorganize the table space.

  3. Run RUNSTATS on the table space.

  4. Add an appropriate index to the table.
A58:

C

59:

Given the following series of events:

  1. Table EMPLOYEE and associated indexes EMPIX1 and EMPIX2 are created

    --> Point 'A' <--

  2. Data is loaded into the EMPLOYEE table with the LOAD utility

    --> Point 'B' <--

  3. A BIND is performed for program PROG1, which accesses the EMPLOYEE table using only static SQL

    --> Point 'C' <--

  4. Program PROG1 is executed

    --> Point 'D' <--

  5. A LOAD REPLACE is performed for the EMPLOYEE table

    --> Point 'E' <--

  6. Program PROG1 is executed

At what point(s) would an execution of the RUNSTATS utility affect the access path for program PROG1?

  1. B only

  2. A only

  3. B and E

  4. A and D

  5. C and E
A59:

A

60:

The RUNSTATS utility does NOT provide statistics information related to which of the following?

  1. Table space data

  2. Table and index data

  3. Database access paths

  4. Column data value distribution
A60:

C

61:

Which of the following statements are NOT true about the DB2 online REORG utility?

  1. There are times during the REORG when both reading and writing are prohibited.

  2. The online REORG function added three new phases to the utility: LOG, SWITCH, and BUILD2.

  3. The inline image copy produced during online REORG may be either SHARELEVEL: CHANGE or REFERENCE.

  4. When reorganizing a table space or partition, the associated indexes may be either type 1 or type 2.

  5. The reorganization is done to shadow copies of the table space data set and associated index data sets.
A61:

C

62:

In a data-sharing environment, which of the following is TRUE?

  1. An active utility cannot be terminated in a data-sharing environment.

  2. A stopped utility can be terminated only from the DB2 on which it ran.

  3. An active utility can be terminated only from the DB2 on which it is running.
A62:

C

63:

Which of the following keywords for the LOAD utility eliminates having to write intermediate work files when constructing the indexes?

  1. REUSE

  2. SORTKEYS

  3. WORKFILES(NO)

  4. SHRLEVEL(NOSORT)
A63:

B

64:

Which of the following is NOT provided by Option 8 (Utilities) on the DB2I menu?

  1. Automatic generation of JCL for standalone utilities

  2. Ability to display or terminate the generated utility job

  3. Restart of a utility at the beginning of a phase or at current position
A64:

A

65:

When using DB2I under TSO, which of the following is NOT a valid command or subcommand?

  1. Run plan or package

  2. BIND plan or package

  3. FREE plan or package

  4. ALLOC plan or package
A65:

D

66:

In order to improve performance for processes that access data from SYSIBM.SYSCOPY and SYSIBM.SYSLGRNX, which of the following utilities should be run?

  1. MODIFY

  2. REPAIR

  3. MERGECOPY

  4. COPY TABLE

  5. COPY INDEX
A66:

A

67:

Given the following scenario:

Table space TS1 in database DB1 contains table T1. Table T1 has one index, I1, defined.

The following utility statements are executed (assume that data set names and DD names are the same):

COPY TABLESPACE db1.ts1 COPYDDN(ddts1)
COPY INDEXSPACE db1.i1 COPYDDN(ddi1)

RECOVER TABLESPACE db1.ts1 TOCOPY ddts1
RECOVER INDEXSPACE db1.i1 TOCOPY ddi1

What is the status of the index space after the utilities have been executed?

  1. RW

  2. COPY

  3. RECP

  4. CHKP

  5. RDBP
A67:

D

68:

Given the following sequence of DCL statements, identify the result of the final REVOKE:

Bob: | GRANT SYSADM TO WADE;
Claire: | GRANT SELECT ON EMP TO WADE WITH GRANT OPTION;
Wade: | GRANT SELECT ON EMP TO JOHN;
Claire: | REVOKE SELECT ON EMP FROM WADE;

  1. Bob cannot do a SELECT against EMP.

  2. John cannot do a SELECT against EMP.

  3. Wade cannot do a SELECT against EMP.

  4. John and Wade cannot do a SELECT against EMP.
A68:

B

69:

John is able to create a VIEW on your table. With just DBADM authority, you want to deny only John's access to your data through this VIEW. Which of the following is the best way to do this?

  1. DROP your table.

  2. DROP John's view.

  3. REVOKE CREATE VIEW from John.

  4. REVOKE SELECT on your table from John.
A69:

D


150 -

Bibliography

DB2 Universal Database Server for OS/390 Version 7 Product Libraries:


151 -

Bibliography

DB2 Universal Database for OS/390

• DB2 Administration Guide, SC26-9931

• DB2 Application Programming and SQL Guide, SC26-9933

• DB2 Application Programming Guide and Reference for Java™, SC26-9032

• DB2 Command Reference, SC26-9934

• DB2 Data Sharing: Planning and Administration, SC26-9935

• DB2 Data-Sharing Quick Reference Card, SX26-3846

• DB2 Diagnosis Guide and Reference, LY37-3740

• DB2 Diagnostic Quick Reference Card, LY37-3741

• DB2 Image, Audio, and Video Extenders Administration and Programming, SC26-9947

• DB2 Installation Guide, GC26-9936

• DB2 Licensed Program Specifications, GC26-9938

• DB2 Master Index, SC26-9939

• DB2 Messages and Codes, GC26-9940

• DB2 ODBC Guide and Reference, SC26-9941

• DB2 Program Directory, GI10-8182

• DB2 Reference for Remote DRDA Requesters and Servers, SC26-9942

• DB2 Reference Summary, SX26-3847

• DB2 Release Planning Guide, SC26-9943

• DB2 SQL Reference, SC26-9944

• DB2 Text Extender Administration and Programming, SC26-9948

• DB2 Universal Database for OS/390 XML Extender Administration and Programming, SC27-9949

• DB2 Utility Guide and Reference, SC26-9945

• DB2 What's New?, GC26-9946


152 -

Bibliography

DB2 Administration Tool

• DB2 Administration Tool for OS/390 User's Guide, SC26-9847


153 -

Bibliography

DB2 Buffer Pool Tool

• DB2 Buffer Pool Tool for OS/390 User's Guide and Reference, SC26-9306


154 -

Bibliography

DB2 DataPropagator

• DB2 Replication Guide and Reference, SC26-9642


155 -

Bibliography

Net.Data for OS/390: The following books are available at http://www.ibm.com/software/net.data/library.html:

• Net.Data Library: Administration and Programming Guide for OS/390

• Net.Data Library: Language Environment Interface Reference

• Net.Data Library: Messages and Codes Net.Data Library: Reference


156 -

Bibliography

DB2 PM for OS/390

• DB2 PM for OS/390 Batch User's Guide, SC26-9167

• DB2 PM for OS/390 Command Reference, SC26-9166

• DB2 PM for OS/390 Data Collector Application Programming Interface Guide, SC27-0861-00

• DB2 PM for OS/390 General Information, GC26-9172

• DB2 PM for OS/390 Installation and Customization, SC26-9171

• DB2 PM for OS/390 Messages, SC26-9169

• DB2 PM for OS/390 Online Monitor User's Guide, SC26-9168

• DB2 PM for OS/390 Program Directory, GI10-8183

• DB2 PM for OS/390 Report Reference, Volume 1, SC26-9164

• DB2 PM for OS/390 Report Reference, Volume 2, SC26-9165

• DB2 PM for OS/390 Using the Workstation Online Monitor, SC26-9170


157 -

Bibliography

Query Management Facility

• Query Management Facility: Developing QMF Applications, SC26-9579

• Query Management Facility: Getting Started with QMF on Windows, SC26-9582

• Query Management Facility: High Performance Option User's Guide for OS/390, SC26-9581

• Query Management Facility: Installing and Managing QMF on OS/390, GC26-9575

• Query Management Facility: Installing and Managing QMF on Windows, GC26-9583

• Query Management Facility: Introducing QMF, GC26-9576

• Query Management Facility: Messages and Codes, GC26-9580

• Query Management Facility: Reference, SC26-9577

• Query Management Facility: Using QMF, SC26-9578


158 -

Bibliography

Ada/370

• IBM Ada/370 Language Reference, SC09-1297

• IBM Ada/370 Programmer's Guide, SC09-1414

• IBM Ada/370 SQL Module Processor for DB2 Database Manager User's Guide, SC09-1450


159 -

Bibliography

APL2

• APL2 Programming Guide, SH21-1072

• APL2 Programming: Language Reference, SH21-1061

• APL2 Programming: Using Structured Query Language (SQL), SH21-1057


160 -

Bibliography

AS/400

• DB2 for OS/400 SQL Programming, SC41-4611

• DB2 for OS/400 SQL Reference, SC41-4612


161 -

Bibliography

BASIC

• IBM BASIC/MVS Language Reference, GC26-4026

• IBM BASIC/MVS Programming Guide, SC26-4027


162 -

Bibliography

BookManager READ/MVS

• BookManager READ/MVS V1R3: Installation Planning and Customization, SC38-2035


163 -

Bibliography

C/370

• IBM SAA AD/Cycle C/370 Programming Guide, SC09-1841

• IBM SAA AD/Cycle C/370 Programming Guide for Language Environment/370, SC09-1840

• IBM SAA AD/Cycle C/370 User's Guide, SC09-1763

• SAA CPI C Reference, SC09-1308


164 -

Bibliography

Character Data Representation Architecture

• Character Data Representation Architecture Overview, GC09-2207

• Character Data Representation Architecture Reference and Registry, SC09-2190


165 -

Bibliography

CICS/ESA

• CICS/ESA Application Programming Guide, SC33-1169

• CICS for MVS/ESA Application Programming Reference, SC33-1170

• CICS for MVS/ESA CICS-RACF Security Guide, SC33-1185

• CICS for MVS/ESA CICS-Supplied Transactions, SC33-1168

• CICS for MVS/ESA Customization Guide, SC33-1165

• CICS for MVS/ESA Data Areas, LY33-6083

• CICS for MVS/ESA Installation Guide, SC33-1163

• CICS for MVS/ESA Intercommunication Guide, SC33-1181

• CICS for MVS/ESA Messages and Codes, GC33-1177

• CICS for MVS/ESA Operations and Utilities Guide, SC33-1167

• CICS/ESA Performance Guide, SC33-1183

• CICS/ESA Problem Determination Guide,SC33-1176

• CICS for MVS/ESA Resource Definition Guide, SC33-1166

• CICS for MVS/ESA System Definition Guide, SC33-1164

• CICS for MVS/ESA System Programming Reference, GC33-1171


166 -

Bibliography

CICS/MVS

• CICS/MVS Application Programmer's Reference, SC33-0512

• CICS/MVS Facilities and Planning Guide, SC33-0504

• CICS/MVS Installation Guide, SC33-0506

• CICS/MVS Operations Guide, SC33-0510

• CICS/MVS Problem Determination Guide, SC33-0516

• CICS/MVS Resource Definition (Macro), SC33-0509

• CICS/MVS Resource Definition (Online), SC33-0508


167 -

Bibliography

CICS Transaction Server for OS/390

• CICS Application Programming Guide, SC33-1687


168 -

Bibliography

IBM C/C++ for MVS/ESA

• IBM C/C++ for MVS/ESA Library Reference, SC09-1995

• IBM C/C++ for MVS/ESA Programming Guide, SC09-1994


169 -

Bibliography

IBM COBOL

• Conversion Guide IMS-DB and DB2 Migration and Coexistence Guide, GH21-1083

• IBM COBOL Language Reference, SC26-4769

• IBM COBOL for MVS and VM Programming Guide, SC26-4767

• IBM COBOL for OS/390 and VM Programming Guide, SC26-9049


170 -

Bibliography

Cooperative Development Environment

• Cooperative Development Environment/370: Debug Tool, SC09-1623


171 -

Bibliography

Data Extract (DXT)

• Data Extract Version 2: General Information, GC26-4666

• Data Extract Version 2: Planning and Administration Guide, SC26-4631


172 -

Bibliography

DataPropagator NonRelational

• DataPropagator NonRelational MVS/ESA Administration Guide, SH19-5036

• DataPropagator NonRelational MVS/ESA Reference, SH19-5039


173 -

Bibliography

Data Facility Data Set Services

• Data Facility Data Set Services: User's Guide and Reference, SC26-4388


174 -

Bibliography

Database Design

• DB2 High-Performance Design, R. Yevich and S. Lawson, ISBN 0-13-203795-5

• Handbook of Relational Database Design, C. Fleming and B. Von Halle, Addison Wesley, ISBN 0-20111-434-8


175 -

Bibliography

DataHub

• IBM DataHub General Information, GC26-4874


176 -

Bibliography

DB2 Connect

• DB2 Connect Enterprise Edition for OS/2 and Windows NT: Quick Beginnings, GC09-2828

• DB2 Connect Personal Edition Quick Beginnings, GC09-2830

• DB2 Connect User's Guide, SC09-2838


177 -

Bibliography

DB2 Universal Database (DB2 UDB)

• DB2 UDB Administration Guide, Volume 1: Design and Implementation, SC09-2839

• DB2 UDB Administration Guide, Volume 2: Performance, SC09-2840

• DB2 UDB Administrative API Reference, SC09-2841

• DB2 UDB Application Building Guide, SC09-2842

• DB2 UDB Application Development Guide, SC09-2845

• DB2 UDB Call Level Interface Guide and Reference, SC09-2843

• DB2 UDB SQL Getting Started, SC09-2856

• DB2 UDB SQL Reference, Volume 1, SC09-2847

• DB2 UDB SQL Reference, Volume 2, SC09-2848


178 -

Bibliography

Device Support Facilities

• Device Support Facilities User's Guide and Reference, GC35-0033


179 -

Bibliography

DFSMS/MVS

• DFSMS/MVS: Access Method Services for the Integrated Catalog, SC26-4906

• DFSMS/MVS: Access Method Services for VSAM Catalogs, SC26-4905

• DFSMS/MVS: Administration Reference for DFSMSdss, SC26-4929

• DFSMS/MVS: DFSMShsm Managing Your Own Data, SH21-1077

• DFSMS/MVS: Diagnosis Reference for DFSMSdfp, LY27-9606

• DFSMS/MVS Storage Management Library: Implementing System-Managed Storage, SC26–3123

• DFSMS/MVS: Macro Instructions for Data Sets, SC26-4913

• DFSMS/MVS: Managing Catalogs, SC26-4914

• DFSMS/MVS: Program Management, SC26-4916

• DFSMS/MVS: Storage Administration Reference for DFSMSdfp, SC26-4920

• DFSMS/MVS: Using Advanced Services, SC26-4921

• DFSMS/MVS: Utilities, SC26-4926

• MVS/DFP: Using Data Sets, SC26-4749


180 -

Bibliography

DFSORT

• DFSORT Application Programming: Guide, SC33-4035


181 -

Bibliography

Distributed Relational Database Data

• IBM SQL Reference, SC26-8416

• Open Group Technical Standard (the Open Group presently makes the following books available through its Web site at http://www.opengroup.org):

• –DRDA Volume 1: Distributed Relational Database Architecture (DRDA), Document number C911

• –DRDA Version, Volume 2: Formatted Data Object Content Architecture, Document number C912

• –DRDA, Volume 3: Distributed Database Management (DDM) Architecture, Document number C913

• Stream and OPA Reference, SC31-6806


182 -

Bibliography

Domain Name System

• DNS and BIND, Third Edition, Paul Albitz and Cricket Liu, O'Reilly, ISBN 1-56592-512-2


183 -

Bibliography

Education

• IBM Dictionary of Computing, McGraw-Hill, ISBN 0-07031-489-6

• 1999 IBM All-in-One Education and Training Catalog, GR23-8105


184 -

Bibliography

Enterprise System/9000 and Enterprise System/3090

• Enterprise System/9000 and Enterprise System/3090 Processor Resource/System, Manager Planning Guide, GA22-7123


185 -

Bibliography

High-Level Assembler

• High-Level Assembler for MVS and VM and VSE Language Reference, SC26-4940

• High-Level Assembler for MVS and VM and VSE Programmer's Guide, SC26-4941


186 -

Bibliography

Parallel Sysplex Library

• OS/390 Parallel Sysplex Application Migration, GC28-1863

• OS/390 Parallel Sysplex Overview: An Introduction to Data Sharing and Parallelism, GC28-1860

• OS/390 Parallel Sysplex Systems Management, GC28-1861

• OS/390 Parallel Sysplex Test Report, GC28-1963

• System/390 9672/9674 System Overview, GA22-7148

• System/390 MVS Sysplex Hardware and Software Migration, GC28-1862


187 -

Bibliography

sICSF/MVS

• ICSF/MVS General Information, GC23-0093 IMS/ESA

• IMS Batch Terminal Simulator General Information, GH20-5522

• IMS/ESA Administration Guide: System, SC26-8013

• IMS/ESA Administration Guide: Transaction Manager, SC26-8731

• IMS/ESA Application Programming: Database Manager, SC26-8727

• IMS/ESA Application Programming: Design Guide, SC26-8016

• IMS/ESA Application Programming: Transaction Manager, SC26-8729

• IMS/ESA Customization Guide, SC26-8020

• IMS/ESA Installation, Volume 1: Installation and Verification, SC26-8023

• IMS/ESA Installation, Volume 2: System Definition and Tailoring, SC26-8024

• IMS/ESA Messages and Codes, SC26-8028

• IMS/ESA Operator's Reference, SC26-8030

• IMS/ESA Utilities Reference: System, SC26-8035


188 -

Bibliography

ISPF

• ISPF V4 Dialog Developer's Guide and Reference, SC34-4486

• ISPF V4 Messages and Codes, SC34-4450

• ISPF V4 Planning and Customizing, SC34-4443

• ISPF V4 User's Guide, SC34-4484


189 -

Bibliography

Language Environment

• Debug Tool User's Guide and Reference, SC09-2137


190 -

Bibliography

National Language Support

• National Language Support Reference, Volume 2, SE09-8002


191 -

Bibliography

NetView

• NetView Installation and Administration Guide, SC31-8043

• NetView User's Guide, SC31-8056


192 -

Bibliography

ODBC

• Microsoft ODBC 3.0 Software Development Kit and Programmer's Reference, Microsoft Press, ISBN 1-57231-516-4


193 -

Bibliography

OS/390

• OS/390 C/C++ Programming Guide, SC09-2362

• OS/390 C/C++ Run-Time Library Reference, SC28-1663

• OS/390 C/C++ User's Guide, SC09-2361

• OS/390 DCE Administration Guide, SC28-1584

• OS/390 DCE Introduction, GC28-1581

• OS/390 DCE Messages and Codes, SC28-1591

• OS/390 eNetwork Communications Server: IP Configuration, SC31-8513

• OS/390 Hardware Configuration Definition Planning, GC28-1750

• OS/390 Information Roadmap, GC28-1727

• OS/390 Introduction and Release Guide, GC28-1725

• OS/390 JES2 Initialization and Tuning Guide, SC28-1791

• OS/390 JES3 Initialization and Tuning Guide, SC28-1802

• OS/390 Language Environment for OS/390 and VM Concepts Guide, GC28-1945

• OS/390 Language Environment for OS/390 and VM Customization, SC28-1941

• OS/390 Language Environment for OS/390 and VM Debugging Guide, SC28-1942

• OS/390 Language Environment for OS/390 and VM Programming Guide, SC28-1939

• OS/390 Language Environment for OS/390 and VM Programming Reference, SC28-1940

• OS/390 MVS Diagnosis: Procedures, LY28-1082

• OS/390 MVS Diagnosis: Reference, SY28-1084

• OS/390 MVS Diagnosis: Tools and Service Aids, LY28-1085

• OS/390 MVS Initialization and Tuning Guide, SC28-1751

• OS/390 MVS Initialization and Tuning Reference, SC28-1752

• OS/390 MVS Installation Exits, SC28-1753

• OS/390 MVS JCL Reference, GC28-1757

• OS/390 MVS JCL User's Guide, GC28-1758

• OS/390 MVS Planning: Global Resource Serialization, GC28-1759

• OS/390 MVS Planning: Operations, GC28-1760

• OS/390 MVS Planning: Workload Management, GC28-1761

• OS/390 MVS Programming: Assembler Services Guide, GC28-1762

• OS/390 MVS Programming: Assembler Services Reference, GC28-1910

• OS/390 MVS Programming: Authorized Assembler Services Guide, GC28-1763

• OS/390 MVS Programming: Authorized Assembler Services Reference, Volumes 1-4, GC28-1764, GC28-1765, GC28-1766,GC28-1767

• OS/390 MVS Programming: Callable Services for High-Level Languages, GC28-1768

• OS/390 MVS Programming: Extended Addressability Guide, GC28-1769

• OS/390 MVS Programming: Sysplex Services Guide, GC28-1771

• OS/390 MVS Programming: Sysplex Services Reference, GC28-1772

• OS/390 MVS Programming: Workload Management Services, GC28-1773

• OS/390 MVS Routing and Descriptor Codes, GC28-1778

• OS/390 MVS Setting Up a Sysplex, GC28-1779

• OS/390 MVS System Codes, GC28-1780

• OS/390 MVS System Commands, GC28-1781

• OS/390 MVS System Messages, Volume 1, GC28-1784

• OS/390 MVS System Messages, Volume 2, GC28-1785

• OS/390 MVS System Messages, Volume 3, GC28-1786

• OS/390 MVS System Messages, Volume 4, GC28-1787

• OS/390 MVS System Messages, Volume 5, GC28-1788

• OS/390 MVS Using the Subsystem Interface, SC28-1789

• OS/390 RMF User's Guide, SC28-1949

• OS/390 Security Server (RACF) Auditor's Guide, SC28-1916

• OS/390 Security Server (RACF) Command Language Reference, SC28-1919

• OS/390 Security Server (RACF) General User's Guide, SC28-1917

• OS/390 Security Server (RACF) Introduction, GC28-1912

• OS/390 Security Server (RACF) Macros and Interfaces, SK2T-6700 (OS/390 Collection Kit), SK27-2180 (OS/390 Security Server Information Package)

• OS/390 Security Server (RACF) Security Administrator's Guide, SC28-1915

• OS/390 Security Server (RACF) System Programmer's Guide, SC28-1913

• OS/390 SMP/E Reference, SC28-1806

• OS/390 SMP/E User's Guide, SC28-1740

• OS/390 TSO/E CLISTS, SC28-1973

• OS/390 TSO/E Command Reference, SC28-1969

• OS/390 TSO/E Customization, SC28-1965

• OS/390 TSO/E Messages, GC28-1978

• OS/390 TSO/E Programming Guide, SC28-1970

• OS/390 TSO/E Programming Services, SC28-1971

• OS/390 TSO/E REXX Reference, SC28-1975

• OS/390 TSO/E User's Guide, SC28-1968

• OS/390 UNIX System Services Command Reference, SC28-1892

• OS/390 UNIX System Services Messages and Codes, SC28-1908

• OS/390 UNIX System Services Planning, SC28-1890

• OS/390 UNIX System Services User's Guide, SC28-1891

• OS/390 UNIX System Services Programming: Assembler Callable Services Reference, SC28-1899


194 -

Bibliography

PL/I for MVS &VM

• IBM PL/I MVS and VM Language Reference, SC26-3114

• IBM PL/I MVS and VM Programming Guide, SC26-3113


195 -

Bibliography

OS PL/I

• OS PL/I Programming Guide, SC26-4307

• OS PL/I Programming Language Reference, SC26-4308


196 -

Bibliography

Prolog

• IBM SAA AD/Cycle Prolog/MVS and VM Programmer's Guide, SH19-6892


360 -

Bibliography

RAMAC

• Enterprise Storage Server Introduction and Planning, GC26-7294

• Peer-to-Peer Remote Copy, SG24-5338

• IBM RAMAC Virtual Array, SG24-4951

• RAMAC Virtual Array: Implementing


361 -

Bibliography

Remote Recovery Data Facility

• Remote Recovery Data Facility Program Description and Operations, LY37-3710


362 -

Bibliography

Storage Management

• DFSMS/MVS Storage Management Library: Implementing System-Managed Storage, SC26-3123

• MVS/ESA Storage Management Library: Leading a Storage Administration Group, SC26-3126

• MVS/ESA Storage Management Library: Managing Data, SC26-3124


363 -

Bibliography

MVS/ESA Storage Management Library

• Managing Storage Groups, SC26-3125

• MVS Storage Management Library: Storage Management Subsystem Migration Planning Guide, SC26-4659


364 -

Bibliography

System/370 and System/390

• ESA/370 Principles of Operation, SA22-7200

• ESA/390 Principles of Operation, SA22-7201

• System/390 MVS Sysplex Hardware and Software Migration, GC28-1210


365 -

Bibliography

System Network Architecture (SNA)

• SNA Formats, GA27-3136

• SNA LU 6.2 Peer Protocols Reference, SC31-6808

• SNA/Management Services Alert Implementation Guide, GC31-6809

• SNA Transaction Programmer's Reference Manual for LU Type 6.2, GC30-3084


366 -

Bibliography

TCP/IP

• IBM TCP/IP for MVS: Customization and Administration Guide, SC31-7134

• IBM TCP/IP for MVS: Diagnosis Guide, LY43-0105

• IBM TCP/IP for MVS: Messages and Codes, SC31-7132

• IBM TCP/IP for MVS: Planning and Migration Guide, SC31-7189


367 -

Bibliography

VS COBOL II

• VS COBOL II Application Programming Guide for MVS and CMS, SC26-4045

• VS COBOL II Application Programming: Language Reference, GC26-4047

• VS COBOL II Installation and Customization for MVS, SC26-4048


368 -

Bibliography

VS FORTRAN

• VS FORTRAN Version 2: Language and Library Reference, SC26-4221

• VS FORTRAN Version 2: Programming Guide for CMS and MVS, SC26-4222


369 -

Bibliography

VTAM

• Planning for NetView, NCP, and VTAM, SC31-8063

• VTAM for MVS/ESA Diagnosis, LY43-0069

• VTAM for MVS/ESA Messages and Codes, SC31-6546

• VTAM for MVS/ESA Network Implementation Guide, SC31-6548

• VTAM for MVS/ESA Operation, SC31-6549

• VTAM for MVS/ESA Programming, SC31-6550

• VTAM for MVS/ESA Programming for LU 6.2, SC31-6551

• VTAM for MVS/ESA Resource Definition Reference, SC31-6552